Command Line Usage

This contains the command line usage for each of the standard scripts we release. These are each included in parlai/scripts.

data_stats

Count and display statistics of the data.

Examples

parlai data_stats -t convai2 -dt train:ordered

CLI help

usage: parlai data_stats [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                         [-dt DATATYPE] [-nt NUMTHREADS] [-bs BATCHSIZE]
                         [-dynb {batchsort,full,None}] [-dp DATAPATH]
                         [-n NUM_EXAMPLES] [-ltim LOG_EVERY_N_SECS]
                         [--agent {0,1}] [--new-line-new-utt NEW_LINE_NEW_UTT]
                         [--ignore-tokens IGNORE_TOKENS]
                         [--bpe-vocab BPE_VOCAB] [--bpe-merge BPE_MERGE]

Compute data statistics

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  -n, -ne, --num-examples NUM_EXAMPLES
  -ltim, --log-every-n-secs LOG_EVERY_N_SECS
  --agent {0,1}
        Use teacher (agent 0) or model (agent 1) (default: 0)
  --new-line-new-utt NEW_LINE_NEW_UTT
        New lines treat substrings as separate utterances. (default: False)
  --ignore-tokens IGNORE_TOKENS
        ignore tokens containings these substrings (comma-separated) (default:
        )

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train:ordered)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

BPEHelper Arguments:
  --bpe-vocab BPE_VOCAB
        path to pre-trained tokenizer vocab (default: None)
  --bpe-merge BPE_MERGE
        path to pre-trained tokenizer merge (default: None)

profile_interactive

Basic script which allows to profile interaction with a model using repeat_query to avoid human interaction (so we can time it, only).

CLI help

usage: parlai profile_interactive [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                                  [-dt DATATYPE] [-nt NUMTHREADS]
                                  [-bs BATCHSIZE]
                                  [-dynb {batchsort,full,None}] [-dp DATAPATH]
                                  [-m MODEL] [-mf MODEL_FILE] [-im INIT_MODEL]
                                  [-d DISPLAY_EXAMPLES] [-ne NUM_EXAMPLES]
                                  [--display-prettify DISPLAY_PRETTIFY]
                                  [--display-ignore-fields DISPLAY_IGNORE_FIELDS]
                                  [-it INTERACTIVE_TASK]

Interactive chat with a model

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  -d, --display-examples DISPLAY_EXAMPLES
  -ne, --num-examples NUM_EXAMPLES
  --display-prettify DISPLAY_PRETTIFY
        Set to use a prettytable when displaying examples with text candidates
        (default: False)
  --display-ignore-fields DISPLAY_IGNORE_FIELDS
        Do not display these fields (default:
        label_candidates,text_candidates)
  -it, --interactive-task INTERACTIVE_TASK
        Create interactive version of task (default: True)

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: interactive)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

multiprocessing_train

Main launch script for single-host, multi-GPU training.

This is a drop-in replacement for train_model.py. This script will launch N subprocess, each which runs the full training loop independently.

Uses torch.nn.parallel.DistributedDataParallel for its main uses. Agents must specifically implement the wrapper of DistributedDatParallel, but all TorchRankerAgents and TorchGeneratorAgents support this.

Examples

parlai multiprocessing_train -m transformer/generator -bs 16 -t convai2 -mf /tmp/mymodel

CLI help

usage: parlai multiprocessing_train [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                                    [-dt DATATYPE] [-nt NUMTHREADS]
                                    [-bs BATCHSIZE]
                                    [-dynb {batchsort,full,None}]
                                    [-dp DATAPATH] [-m MODEL] [-mf MODEL_FILE]
                                    [-im INIT_MODEL] [-et EVALTASK]
                                    [-eps NUM_EPOCHS] [-ttim MAX_TRAIN_TIME]
                                    [-vtim VALIDATION_EVERY_N_SECS]
                                    [-stim SAVE_EVERY_N_SECS]
                                    [-sval SAVE_AFTER_VALID]
                                    [-veps VALIDATION_EVERY_N_EPOCHS]
                                    [-vp VALIDATION_PATIENCE]
                                    [-vmt VALIDATION_METRIC] [-vmm {max,min}]
                                    [-mcs METRICS] [-micro AGGREGATE_MICRO]
                                    [-tblog TENSORBOARD_LOG]
                                    [--bpe-vocab BPE_VOCAB]
                                    [--bpe-merge BPE_MERGE]
                                    [--distributed-world-size DISTRIBUTED_WORLD_SIZE]

Train a model

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

Training Loop Arguments:
  -et, --evaltask EVALTASK
        task to use for valid/test (defaults to the one used for training)
        (default: None)
  -eps, --num-epochs NUM_EPOCHS
  -ttim, --max-train-time MAX_TRAIN_TIME
  -vtim, --validation-every-n-secs VALIDATION_EVERY_N_SECS
        Validate every n seconds. Saves model to model_file (if set) whenever
        best val metric is found (default: -1)
  -stim, --save-every-n-secs SAVE_EVERY_N_SECS
        Saves the model to model_file.checkpoint after every n seconds
        (default -1, never). (default: -1)
  -sval, --save-after-valid SAVE_AFTER_VALID
        Saves the model to model_file.checkpoint after every validation
        (default False).
  -veps, --validation-every-n-epochs VALIDATION_EVERY_N_EPOCHS
        Validate every n epochs. Saves model to model_file (if set) whenever
        best val metric is found (default: -1)
  -vp, --validation-patience VALIDATION_PATIENCE
        number of iterations of validation where result does not improve
        before we stop training (default: 10)
  -vmt, --validation-metric VALIDATION_METRIC
        key into report table for selecting best validation (default:
        accuracy)
  -vmm, --validation-metric-mode {max,min}
        how to optimize validation metric (max or min) (default: None)
  -mcs, --metrics METRICS
        list of metrics to show/compute, e.g. all, default,or give a list
        split by , like ppl,f1,accuracy,hits@1,rouge,bleuthe rouge metrics
        will be computed as rouge-1, rouge-2 and rouge-l (default: default)
  -micro, --aggregate-micro AGGREGATE_MICRO
        Report micro-averaged metrics instead of macro averaged metrics.
        (default: False)

Tensorboard Arguments:
  -tblog, --tensorboard-log TENSORBOARD_LOG
        Tensorboard logging of metrics, default is False

BPEHelper Arguments:
  --bpe-vocab BPE_VOCAB
        path to pre-trained tokenizer vocab (default: None)
  --bpe-merge BPE_MERGE
        path to pre-trained tokenizer merge (default: None)

Distributed Training:
  --distributed-world-size DISTRIBUTED_WORLD_SIZE
        Number of workers. (default: 0)

convert_data_to_fasttext_format

Convert a dataset into the ParlAI text format.

Examples

parlai convert_data_to_fasttext_format -t babi:task1k:1 --outfile /tmp/dump

CLI help

usage: parlai convert_data_to_fasttext_format [-h] [--helpall] [-o INIT_OPT]
                                              [-t TASK] [-dt DATATYPE]
                                              [-nt NUMTHREADS] [-bs BATCHSIZE]
                                              [-dynb {batchsort,full,None}]
                                              [-dp DATAPATH] [-n NUM_EXAMPLES]
                                              [-of OUTFILE]
                                              [-ltim LOG_EVERY_N_SECS]

Convert data for ingestion in fastText

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  -n, --num-examples NUM_EXAMPLES
        Total number of exs to convert, -1 to convert all examples (default:
        -1)
  -of, --outfile OUTFILE
        Output file where to save, by default will be created in /tmp
        (default: None)
  -ltim, --log-every-n-secs LOG_EVERY_N_SECS

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train:ordered)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

multiprocessing_eval

Main launch script for single-host, multi-GPU evaluation.

This is a drop-in replacement for eval_model. This script will launch N subprocess, each which runs the full eval loop independently.

Uses torch.nn.parallel.DistributedDataParallel for its main uses. Agents must specifically implement the wrapper of DistributedDataParallel, but all TorchRankerAgents and TorchGeneratorAgents support this.

Examples

parlai multiprocessing_eval -mf "zoo:tutorial_transformer_generator/model" -bs 16 -t convai2

CLI help

usage: parlai multiprocessing_eval [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                                   [-dt DATATYPE] [-nt NUMTHREADS]
                                   [-bs BATCHSIZE]
                                   [-dynb {batchsort,full,None}]
                                   [-dp DATAPATH] [-m MODEL] [-mf MODEL_FILE]
                                   [-im INIT_MODEL] [-rf REPORT_FILENAME]
                                   [--save-world-logs SAVE_WORLD_LOGS]
                                   [--save-format {conversations,parlai}]
                                   [-ne NUM_EXAMPLES] [-d DISPLAY_EXAMPLES]
                                   [-ltim LOG_EVERY_N_SECS] [-mcs METRICS]
                                   [-micro AGGREGATE_MICRO]
                                   [--log-keep-fields LOG_KEEP_FIELDS]
                                   [-tblog TENSORBOARD_LOG]
                                   [--distributed-world-size DISTRIBUTED_WORLD_SIZE]

Evaluate a model

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  -rf, --report-filename REPORT_FILENAME
        Saves a json file of the evaluation report either as an extension to
        the model-file (if begins with a ".") or a whole file path. Set to the
        empty string to not save at all. (default: )
  --save-world-logs SAVE_WORLD_LOGS
        Saves a jsonl file containing all of the task examples and model
        replies. Must also specify --report-filename. (default: False)
  --save-format {conversations,parlai}
  -ne, --num-examples NUM_EXAMPLES
  -d, --display-examples DISPLAY_EXAMPLES
  -ltim, --log-every-n-secs LOG_EVERY_N_SECS
  -mcs, --metrics METRICS
        list of metrics to show/compute, e.g. all, default,or give a list
        split by , like ppl,f1,accuracy,hits@1,rouge,bleuthe rouge metrics
        will be computed as rouge-1, rouge-2 and rouge-l (default: default)
  -micro, --aggregate-micro AGGREGATE_MICRO
        Report micro-averaged metrics instead of macro averaged metrics.
        (default: False)

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        valid)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

World Logging:
  --log-keep-fields LOG_KEEP_FIELDS
        Fields to keep when logging. Should be a comma separated list
        (default: all)

Tensorboard Arguments:
  -tblog, --tensorboard-log TENSORBOARD_LOG
        Tensorboard logging of metrics, default is False

Distributed Training:
  --distributed-world-size DISTRIBUTED_WORLD_SIZE
        Number of workers. (default: 0)

convo_render

CLI help

usage: parlai convo_render [-h] [--helpall] [--init-opt INIT_OPT] [-t TASK]
                           [-dt DATATYPE] [-nt NUMTHREADS] [-bs BATCHSIZE]
                           [-dynb {batchsort,full,None}] [-dp DATAPATH]
                           [--input INPUT] [--output OUTPUT] [--width WIDTH]
                           [--height HEIGHT] [--user-icon USER_ICON]
                           [--alt-icon ALT_ICON] [--num-examples NUM_EXAMPLES]

Render data as HTML

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.

Main ParlAI Arguments:
  --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

Conversation Rendering Arguments:
  --input, -i INPUT
        Input file to read conversations from (default: None)
  --output, -o OUTPUT
        Output file to write conversations to. One of [.pdf, .png, .html] only
        (default: None)
  --width, -wd WIDTH
        Width of output file (default: 8)
  --height, -ht HEIGHT
        Height of output file (default: 10)
  --user-icon, -uic USER_ICON
        Absolute Path/URL to user image icon (default: https://emojipedia-
        us.s3.dualstack.us-
        west-1.amazonaws.com/thumbs/160/apple/76/woman_1f469.png)
  --alt-icon, -aic ALT_ICON
        Absolute Path/URL to alternate image icon (default:
        https://emojipedia-us.s3.dualstack.us-
        west-1.amazonaws.com/thumbs/160/facebook/230/parrot_1f99c.png)
  --num-examples, -ne NUM_EXAMPLES
        Number of conversations to render (default: 10)

profile_train

Run the python or pytorch profiler and prints the results.

Examples

To make sure that bAbI task 1 (1k exs) loads one can run and to see a few of them:

parlai profile_train -t babi:task1k:1 -m seq2seq -e 0.1 --dict-file /tmp/dict

CLI help

usage: parlai profile_train [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                            [-dt DATATYPE] [-nt NUMTHREADS] [-bs BATCHSIZE]
                            [-dynb {batchsort,full,None}] [-dp DATAPATH]
                            [-m MODEL] [-mf MODEL_FILE] [-im INIT_MODEL]
                            [-et EVALTASK] [-eps NUM_EPOCHS]
                            [-ttim MAX_TRAIN_TIME]
                            [-vtim VALIDATION_EVERY_N_SECS]
                            [-stim SAVE_EVERY_N_SECS] [-sval SAVE_AFTER_VALID]
                            [-veps VALIDATION_EVERY_N_EPOCHS]
                            [-vp VALIDATION_PATIENCE] [-vmt VALIDATION_METRIC]
                            [-vmm {max,min}] [-mcs METRICS]
                            [-micro AGGREGATE_MICRO] [-tblog TENSORBOARD_LOG]
                            [--bpe-vocab BPE_VOCAB] [--bpe-merge BPE_MERGE]
                            [--torch TORCH] [--torch-cuda TORCH_CUDA]
                            [--debug DEBUG]

cProfile a training run

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

Training Loop Arguments:
  -et, --evaltask EVALTASK
        task to use for valid/test (defaults to the one used for training)
        (default: None)
  -eps, --num-epochs NUM_EPOCHS
  -ttim, --max-train-time MAX_TRAIN_TIME
  -vtim, --validation-every-n-secs VALIDATION_EVERY_N_SECS
        Validate every n seconds. Saves model to model_file (if set) whenever
        best val metric is found (default: -1)
  -stim, --save-every-n-secs SAVE_EVERY_N_SECS
        Saves the model to model_file.checkpoint after every n seconds
        (default -1, never). (default: -1)
  -sval, --save-after-valid SAVE_AFTER_VALID
        Saves the model to model_file.checkpoint after every validation
        (default False).
  -veps, --validation-every-n-epochs VALIDATION_EVERY_N_EPOCHS
        Validate every n epochs. Saves model to model_file (if set) whenever
        best val metric is found (default: -1)
  -vp, --validation-patience VALIDATION_PATIENCE
        number of iterations of validation where result does not improve
        before we stop training (default: 10)
  -vmt, --validation-metric VALIDATION_METRIC
        key into report table for selecting best validation (default:
        accuracy)
  -vmm, --validation-metric-mode {max,min}
        how to optimize validation metric (max or min) (default: None)
  -mcs, --metrics METRICS
        list of metrics to show/compute, e.g. all, default,or give a list
        split by , like ppl,f1,accuracy,hits@1,rouge,bleuthe rouge metrics
        will be computed as rouge-1, rouge-2 and rouge-l (default: default)
  -micro, --aggregate-micro AGGREGATE_MICRO
        Report micro-averaged metrics instead of macro averaged metrics.
        (default: False)

Tensorboard Arguments:
  -tblog, --tensorboard-log TENSORBOARD_LOG
        Tensorboard logging of metrics, default is False

BPEHelper Arguments:
  --bpe-vocab BPE_VOCAB
        path to pre-trained tokenizer vocab (default: None)
  --bpe-merge BPE_MERGE
        path to pre-trained tokenizer merge (default: None)

Profiler Arguments:
  --torch TORCH
        If true, use the torch profiler. Otherwise use cProfile. (default:
        False)
  --torch-cuda TORCH_CUDA
        If true, use the torch cuda profiler. Otherwise use cProfile.
        (default: False)
  --debug DEBUG
        If true, enter debugger at end of run. (default: False)

display_data

Basic example which iterates through the tasks specified and prints them out. Used for verification of data loading and iteration.

For example, to make sure that bAbI task 1 (1k exs) loads one can run and to see a few of them:

Examples

parlai display_data -t babi:task1k:1

CLI help

usage: parlai display_data [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                           [-dt DATATYPE] [-nt NUMTHREADS] [-bs BATCHSIZE]
                           [-dynb {batchsort,full,None}] [-dp DATAPATH]
                           [-m MODEL] [-mf MODEL_FILE] [-im INIT_MODEL]
                           [-n NUM_EXAMPLES] [-mdl MAX_DISPLAY_LEN]
                           [--display-ignore-fields DISPLAY_IGNORE_FIELDS]
                           [-v]

Display data from a task

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  -n, -ne, --num-examples NUM_EXAMPLES
  -mdl, --max-display-len MAX_DISPLAY_LEN
  --display-ignore-fields DISPLAY_IGNORE_FIELDS
  -v, --display-verbose
        If false, simple converational view, does not show other message
        fields.

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train:ordered)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

interactive_web

Talk with a model using a web UI.

Examples

parlai interactive_web -mf "zoo:tutorial_transformer_generator/model"

CLI help

usage: parlai interactive_web [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                              [-dt DATATYPE] [-nt NUMTHREADS] [-bs BATCHSIZE]
                              [-dynb {batchsort,full,None}] [-dp DATAPATH]
                              [-m MODEL] [-mf MODEL_FILE] [-im INIT_MODEL]
                              [-d DISPLAY_EXAMPLES]
                              [--display-prettify DISPLAY_PRETTIFY]
                              [--display-ignore-fields DISPLAY_IGNORE_FIELDS]
                              [-it INTERACTIVE_TASK] [--outfile OUTFILE]
                              [--save-format {conversations,parlai}]
                              [-fixedCands LOCAL_HUMAN_CANDIDATES_FILE]
                              [--single-turn SINGLE_TURN]
                              [--log-keep-fields LOG_KEEP_FIELDS]

Interactive chat with a model on the command line

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  -d, --display-examples DISPLAY_EXAMPLES
  --display-prettify DISPLAY_PRETTIFY
        Set to use a prettytable when displaying examples with text candidates
        (default: False)
  --display-ignore-fields DISPLAY_IGNORE_FIELDS
        Do not display these fields (default:
        label_candidates,text_candidates)
  -it, --interactive-task INTERACTIVE_TASK
        Create interactive version of task (default: True)
  --outfile OUTFILE
        Saves a jsonl file containing all of the task examples and model
        replies. Set to the empty string to not save at all (default: )
  --save-format {conversations,parlai}
        Format to save logs in. conversations is a jsonl format, parlai is a
        text format. (default: conversations)

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: interactive)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

Local Human Arguments:
  -fixedCands, --local-human-candidates-file LOCAL_HUMAN_CANDIDATES_FILE
        File of label_candidates to send to other agent (default: None)
  --single-turn SINGLE_TURN
        If on, assumes single turn episodes. (default: False)

World Logging:
  --log-keep-fields LOG_KEEP_FIELDS
        Fields to keep when logging. Should be a comma separated list
        (default: all)

interactive

Basic script which allows local human keyboard input to talk to a trained model.

Examples

parlai interactive -m drqa -mf "models:drqa/squad/model"

When prompted, enter something like: Bob is Blue.\nWhat is Bob?

Input is often model or task specific, but in drqa, it is always context '\n' question.

CLI help

usage: parlai interactive [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                          [-dt DATATYPE] [-nt NUMTHREADS] [-bs BATCHSIZE]
                          [-dynb {batchsort,full,None}] [-dp DATAPATH]
                          [-m MODEL] [-mf MODEL_FILE] [-im INIT_MODEL]
                          [-d DISPLAY_EXAMPLES]
                          [--display-prettify DISPLAY_PRETTIFY]
                          [--display-ignore-fields DISPLAY_IGNORE_FIELDS]
                          [-it INTERACTIVE_TASK] [--outfile OUTFILE]
                          [--save-format {conversations,parlai}]
                          [-fixedCands LOCAL_HUMAN_CANDIDATES_FILE]
                          [--single-turn SINGLE_TURN]
                          [--log-keep-fields LOG_KEEP_FIELDS]

Interactive chat with a model on the command line

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  -d, --display-examples DISPLAY_EXAMPLES
  --display-prettify DISPLAY_PRETTIFY
        Set to use a prettytable when displaying examples with text candidates
        (default: False)
  --display-ignore-fields DISPLAY_IGNORE_FIELDS
        Do not display these fields (default:
        label_candidates,text_candidates)
  -it, --interactive-task INTERACTIVE_TASK
        Create interactive version of task (default: True)
  --outfile OUTFILE
        Saves a jsonl file containing all of the task examples and model
        replies. Set to the empty string to not save at all (default: )
  --save-format {conversations,parlai}
        Format to save logs in. conversations is a jsonl format, parlai is a
        text format. (default: conversations)

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: interactive)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

Local Human Arguments:
  -fixedCands, --local-human-candidates-file LOCAL_HUMAN_CANDIDATES_FILE
        File of label_candidates to send to other agent (default: None)
  --single-turn SINGLE_TURN
        If on, assumes single turn episodes. (default: False)

World Logging:
  --log-keep-fields LOG_KEEP_FIELDS
        Fields to keep when logging. Should be a comma separated list
        (default: all)

convert_data_to_parlai_format

Convert a dataset into the ParlAI text format.

Examples

parlai convert_data_to_parlai_format -t babi:task1k:1 --outfile /tmp/dump

CLI help

usage: parlai convert_data_to_parlai_format [-h] [--helpall] [-o INIT_OPT]
                                            [-t TASK] [-dt DATATYPE]
                                            [-nt NUMTHREADS] [-bs BATCHSIZE]
                                            [-dynb {batchsort,full,None}]
                                            [-dp DATAPATH] [-n NUM_EXAMPLES]
                                            [-of OUTFILE] [-if IGNORE_FIELDS]
                                            [-ltim LOG_EVERY_N_SECS]

Dump a task to a standardized format

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  -n, --num-examples NUM_EXAMPLES
        Total number of exs to convert, -1 to convert all examples (default:
        -1)
  -of, --outfile OUTFILE
        Output file where to save, by default will be created in tmp (default:
        None)
  -if, --ignore-fields IGNORE_FIELDS
        Ignore these fields from the message (returned with .act() ) (default:
        id)
  -ltim, --log-every-n-secs LOG_EVERY_N_SECS

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train:stream)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

eval_wordstat

This helper script can be used alone with modelfile and task: the output will contain the word statistics of the model outputs. One can also use the function defined here in other places in order to get such statistic for any agent given the agent object (with corr. dict) and a sequence.

Additionally provides function get_word_stats that can be used in other parts of runtime code since it depends only on the agent object. For example:

from parlai.scripts.eval_wordstat import get_word_stats
reqs, cnt = get_word_stats(predictions.tolist(), self.dict)

Examples

parlai eval_wordstat -mf data/model -t convai2:self --freq-bins 10,100,1000

CLI help

usage: parlai eval_wordstat [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                            [-dt DATATYPE] [-nt NUMTHREADS] [-bs BATCHSIZE]
                            [-dynb {batchsort,full,None}] [-dp DATAPATH]
                            [-m MODEL] [-mf MODEL_FILE] [-im INIT_MODEL]
                            [--bpe-vocab BPE_VOCAB] [--bpe-merge BPE_MERGE]
                            [-ne NUM_EXAMPLES] [-ltim LOG_EVERY_N_SECS]
                            [-ed EXTERNAL_DICT] [-fb FREQ_BINS]
                            [-dup DUMP_PREDICTIONS_PATH] [-cun COMPUTE_UNIQUE]
                            [-tblog TENSORBOARD_LOG]

Compute statistics from model predictions

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  -ne, --num-examples NUM_EXAMPLES
  -ltim, --log-every-n-secs LOG_EVERY_N_SECS
  -ed, --external-dict EXTERNAL_DICT
        External dictionary for stat computation (default: None)
  -fb, --freq-bins FREQ_BINS
        Bins boundaries for rare words stat (default: 0,100,1000,10000)
  -dup, --dump-predictions-path DUMP_PREDICTIONS_PATH
        Dump predictions into file (default: None)
  -cun, --compute-unique COMPUTE_UNIQUE
        Compute % of unique responses from the model (default: True)

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        valid)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

BPEHelper Arguments:
  --bpe-vocab BPE_VOCAB
        path to pre-trained tokenizer vocab (default: None)
  --bpe-merge BPE_MERGE
        path to pre-trained tokenizer merge (default: None)

Tensorboard Arguments:
  -tblog, --tensorboard-log TENSORBOARD_LOG
        Tensorboard logging of metrics, default is False

train_model

Training script for ParlAI.

The standard way to train a model. After training, also computes validation and test error.

The user must provide a model (with --model) and a task (with --task).

Examples

parlai train_model -m ir_baseline -t dialog_babi:Task:1 -mf /tmp/model
parlai train_model -m seq2seq -t babi:Task10k:1 -mf '/tmp/model' -bs 32 -lr 0.5 -hs 128
parlai train_model -m drqa -t babi:Task10k:1 -mf /tmp/model -bs 10

CLI help

usage: parlai train_model [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                          [-dt DATATYPE] [-nt NUMTHREADS] [-bs BATCHSIZE]
                          [-dynb {batchsort,full,None}] [-dp DATAPATH]
                          [-m MODEL] [-mf MODEL_FILE] [-im INIT_MODEL]
                          [-et EVALTASK] [-eps NUM_EPOCHS]
                          [-ttim MAX_TRAIN_TIME]
                          [-vtim VALIDATION_EVERY_N_SECS]
                          [-stim SAVE_EVERY_N_SECS] [-sval SAVE_AFTER_VALID]
                          [-veps VALIDATION_EVERY_N_EPOCHS]
                          [-vp VALIDATION_PATIENCE] [-vmt VALIDATION_METRIC]
                          [-vmm {max,min}] [-mcs METRICS]
                          [-micro AGGREGATE_MICRO] [-tblog TENSORBOARD_LOG]
                          [--bpe-vocab BPE_VOCAB] [--bpe-merge BPE_MERGE]

Train a model

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

Training Loop Arguments:
  -et, --evaltask EVALTASK
        task to use for valid/test (defaults to the one used for training)
        (default: None)
  -eps, --num-epochs NUM_EPOCHS
  -ttim, --max-train-time MAX_TRAIN_TIME
  -vtim, --validation-every-n-secs VALIDATION_EVERY_N_SECS
        Validate every n seconds. Saves model to model_file (if set) whenever
        best val metric is found (default: -1)
  -stim, --save-every-n-secs SAVE_EVERY_N_SECS
        Saves the model to model_file.checkpoint after every n seconds
        (default -1, never). (default: -1)
  -sval, --save-after-valid SAVE_AFTER_VALID
        Saves the model to model_file.checkpoint after every validation
        (default False).
  -veps, --validation-every-n-epochs VALIDATION_EVERY_N_EPOCHS
        Validate every n epochs. Saves model to model_file (if set) whenever
        best val metric is found (default: -1)
  -vp, --validation-patience VALIDATION_PATIENCE
        number of iterations of validation where result does not improve
        before we stop training (default: 10)
  -vmt, --validation-metric VALIDATION_METRIC
        key into report table for selecting best validation (default:
        accuracy)
  -vmm, --validation-metric-mode {max,min}
        how to optimize validation metric (max or min) (default: None)
  -mcs, --metrics METRICS
        list of metrics to show/compute, e.g. all, default,or give a list
        split by , like ppl,f1,accuracy,hits@1,rouge,bleuthe rouge metrics
        will be computed as rouge-1, rouge-2 and rouge-l (default: default)
  -micro, --aggregate-micro AGGREGATE_MICRO
        Report micro-averaged metrics instead of macro averaged metrics.
        (default: False)

Tensorboard Arguments:
  -tblog, --tensorboard-log TENSORBOARD_LOG
        Tensorboard logging of metrics, default is False

BPEHelper Arguments:
  --bpe-vocab BPE_VOCAB
        path to pre-trained tokenizer vocab (default: None)
  --bpe-merge BPE_MERGE
        path to pre-trained tokenizer merge (default: None)

detect_offensive_language

Basic example which iterates through the tasks specified and checks them for offensive language.

Examples

parlai detect_offensive_language -t "convai_chitchat" --display-examples True

CLI help

usage: parlai detect_offensive_language [-h] [--helpall] [-o INIT_OPT]
                                        [-t TASK] [-dt DATATYPE]
                                        [-nt NUMTHREADS] [-bs BATCHSIZE]
                                        [-dynb {batchsort,full,None}]
                                        [-dp DATAPATH] [-m MODEL]
                                        [-mf MODEL_FILE] [-im INIT_MODEL]
                                        [-ltim LOG_EVERY_N_SECS]
                                        [-d DISPLAY_EXAMPLES]
                                        [--safety {classifier,all,string_matcher}]

Check task for offensive language

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  -ltim, --log-every-n-secs LOG_EVERY_N_SECS
  -d, --display-examples DISPLAY_EXAMPLES
  --safety {classifier,all,string_matcher}
        Type of safety detector to apply to messages (default: all)

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train:ordered)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: repeat_query)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

display_model

Basic example which iterates through the tasks specified and runs the given model on them.

Examples

parlai display_model -t babi:task1k:1 -m "repeat_label"
parlai display_model -t "#MovieDD-Reddit" -m "ir_baseline" -mp "-lp 0.5" -dt test

CLI help

usage: parlai display_model [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                            [-dt DATATYPE] [-nt NUMTHREADS] [-bs BATCHSIZE]
                            [-dynb {batchsort,full,None}] [-dp DATAPATH]
                            [-m MODEL] [-mf MODEL_FILE] [-im INIT_MODEL]
                            [-n NUM_EXAMPLES]
                            [--display-ignore-fields DISPLAY_IGNORE_FIELDS]

Display model predictions.

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  -n, -ne, --num-examples NUM_EXAMPLES
  --display-ignore-fields DISPLAY_IGNORE_FIELDS

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        valid)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

safe_interactive

Script for safety protected interaction between a local human keyboard input and a trained model.

CLI help

usage: parlai safe_interactive [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                               [-dt DATATYPE] [-nt NUMTHREADS] [-bs BATCHSIZE]
                               [-dynb {batchsort,full,None}] [-dp DATAPATH]
                               [-m MODEL] [-mf MODEL_FILE] [-im INIT_MODEL]
                               [-d DISPLAY_EXAMPLES]
                               [--display-prettify DISPLAY_PRETTIFY]
                               [--display-ignore-fields DISPLAY_IGNORE_FIELDS]
                               [-it INTERACTIVE_TASK]
                               [--safety {classifier,all,none,string_matcher}]
                               [-fixedCands LOCAL_HUMAN_CANDIDATES_FILE]
                               [--single-turn SINGLE_TURN]

Like interactive, but adds a safety filter

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  -d, --display-examples DISPLAY_EXAMPLES
  --display-prettify DISPLAY_PRETTIFY
        Set to use a prettytable when displaying examples with text candidates
        (default: False)
  --display-ignore-fields DISPLAY_IGNORE_FIELDS
        Do not display these fields (default:
        label_candidates,text_candidates)
  -it, --interactive-task INTERACTIVE_TASK
        Create interactive version of task (default: True)

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: interactive)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

Safe Local Human Arguments:
  --safety {classifier,all,none,string_matcher}
        Apply safety filtering to messages (default: all)

Local Human Arguments:
  -fixedCands, --local-human-candidates-file LOCAL_HUMAN_CANDIDATES_FILE
        File of label_candidates to send to other agent (default: None)
  --single-turn SINGLE_TURN
        If on, assumes single turn episodes. (default: False)

build_dict

Generates a dictionary file from the training data.

Examples

# learn the vocabulary from one task, then train on another task.
parlai build_dict -t convai2 --dict-file premade.dict
parlai train_model -t squad --dict-file premade.dict -m seq2seq

CLI help

usage: parlai build_dict [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                         [-dt DATATYPE] [-nt NUMTHREADS] [-bs BATCHSIZE]
                         [-dynb {batchsort,full,None}] [-dp DATAPATH]
                         [-m MODEL] [-mf MODEL_FILE] [-im INIT_MODEL]
                         [--bpe-vocab BPE_VOCAB] [--bpe-merge BPE_MERGE]

Build a dictionary.

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

BPEHelper Arguments:
  --bpe-vocab BPE_VOCAB
        path to pre-trained tokenizer vocab (default: None)
  --bpe-merge BPE_MERGE
        path to pre-trained tokenizer merge (default: None)

self_chat

Allows a model to self-chat on a given task.

CLI help

usage: parlai self_chat [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                        [-dt DATATYPE] [-nt NUMTHREADS] [-bs BATCHSIZE]
                        [-dynb {batchsort,full,None}] [-dp DATAPATH]
                        [-m MODEL] [-mf MODEL_FILE] [-im INIT_MODEL]
                        [--seed SEED] [-d DISPLAY_EXAMPLES]
                        [--display-ignore-fields DISPLAY_IGNORE_FIELDS]
                        [-st SELFCHAT_TASK] [--num-self-chats NUM_SELF_CHATS]
                        [--selfchat-max-turns SELFCHAT_MAX_TURNS]
                        [--seed-messages-from-task] [--outfile OUTFILE]
                        [--save-format {conversations,parlai}]
                        [-pmf PARTNER_MODEL_FILE]
                        [--partner-opt-file PARTNER_OPT_FILE]
                        [--log-keep-fields LOG_KEEP_FIELDS]

Generate self-chats of a model

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  --seed SEED
  -d, --display-examples DISPLAY_EXAMPLES
  --display-ignore-fields DISPLAY_IGNORE_FIELDS
        Do not display these fields (default:
        label_candidates,text_candidates)
  -st, --selfchat-task SELFCHAT_TASK
        Create a self chat version of the task (default: True)
  --num-self-chats NUM_SELF_CHATS
        Number of self chats to run (default: 1)
  --selfchat-max-turns SELFCHAT_MAX_TURNS
        The number of dialogue turns before self chat ends (default: 6)
  --seed-messages-from-task
        Automatically seed conversation with messages from task dataset.
  --outfile OUTFILE
        File to save self chat logs (default: None)
  --save-format {conversations,parlai}
        Format to save logs in. conversations is a jsonl format, parlai is a
        text format. (default: conversations)
  -pmf, --partner-model-file PARTNER_MODEL_FILE
        Define a different partner for self chat (default: None)
  --partner-opt-file PARTNER_OPT_FILE
        Path to file containing opts to override for partner (default: None)

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: self_chat)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

World Logging:
  --log-keep-fields LOG_KEEP_FIELDS
        Fields to keep when logging. Should be a comma separated list
        (default: all)

extract_image_feature

Basic example which iterates through the tasks specified and load/extract the image features.

For more options, check parlai.core.image_featurizers

Examples

To extract the image feature of COCO images:

parlai extract_image_feature -t vqa_v1 -im resnet152

CLI help

usage: parlai extract_image_feature [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                                    [-dt DATATYPE] [-nt NUMTHREADS]
                                    [-bs BATCHSIZE]
                                    [-dynb {batchsort,full,None}]
                                    [-dp DATAPATH]

Load/extract image features

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

build_candidates

Build the candidate responses for a retrieval model.

Examples

parlai build_candidates -t convai2 --outfile /tmp/cands.txt

CLI help

usage: parlai build_candidates [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                               [-dt DATATYPE] [-nt NUMTHREADS] [-bs BATCHSIZE]
                               [-dynb {batchsort,full,None}] [-dp DATAPATH]
                               [-n NUM_EXAMPLES] [-of OUTFILE]
                               [-ltim LOG_EVERY_N_SECS]

Build the candidate responses for a retrieval model

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  -n, --num-examples NUM_EXAMPLES
        Total number of exs to convert, -1 to convert all examples (default:
        -1)
  -of, --outfile OUTFILE
        Output file where to save, by default will be created in /tmp
        (default: None)
  -ltim, --log-every-n-secs LOG_EVERY_N_SECS

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train:evalmode)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

eval_model

Basic example which iterates through the tasks specified and evaluates the given model on them.

Examples

parlai eval_model -t "babi:Task1k:2" -m "repeat_label"
parlai eval_model -t "#CornellMovie" -m "ir_baseline" -mp "-lp 0.5"

CLI help

usage: parlai eval_model [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                         [-dt DATATYPE] [-nt NUMTHREADS] [-bs BATCHSIZE]
                         [-dynb {batchsort,full,None}] [-dp DATAPATH]
                         [-m MODEL] [-mf MODEL_FILE] [-im INIT_MODEL]
                         [-rf REPORT_FILENAME]
                         [--save-world-logs SAVE_WORLD_LOGS]
                         [--save-format {conversations,parlai}]
                         [-ne NUM_EXAMPLES] [-d DISPLAY_EXAMPLES]
                         [-ltim LOG_EVERY_N_SECS] [-mcs METRICS]
                         [-micro AGGREGATE_MICRO]
                         [--log-keep-fields LOG_KEEP_FIELDS]
                         [-tblog TENSORBOARD_LOG]

Evaluate a model

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  -rf, --report-filename REPORT_FILENAME
        Saves a json file of the evaluation report either as an extension to
        the model-file (if begins with a ".") or a whole file path. Set to the
        empty string to not save at all. (default: )
  --save-world-logs SAVE_WORLD_LOGS
        Saves a jsonl file containing all of the task examples and model
        replies. Must also specify --report-filename. (default: False)
  --save-format {conversations,parlai}
  -ne, --num-examples NUM_EXAMPLES
  -d, --display-examples DISPLAY_EXAMPLES
  -ltim, --log-every-n-secs LOG_EVERY_N_SECS
  -mcs, --metrics METRICS
        list of metrics to show/compute, e.g. all, default,or give a list
        split by , like ppl,f1,accuracy,hits@1,rouge,bleuthe rouge metrics
        will be computed as rouge-1, rouge-2 and rouge-l (default: default)
  -micro, --aggregate-micro AGGREGATE_MICRO
        Report micro-averaged metrics instead of macro averaged metrics.
        (default: False)

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        valid)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

World Logging:
  --log-keep-fields LOG_KEEP_FIELDS
        Fields to keep when logging. Should be a comma separated list
        (default: all)

Tensorboard Arguments:
  -tblog, --tensorboard-log TENSORBOARD_LOG
        Tensorboard logging of metrics, default is False

verify_data

Verify data doesn’t have basic mistakes, like empty text fields or empty label candidates.

Examples

parlai verify_data -t convai2 -dt train:ordered

CLI help

usage: parlai verify_data [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                          [-dt DATATYPE] [-nt NUMTHREADS] [-bs BATCHSIZE]
                          [-dynb {batchsort,full,None}] [-dp DATAPATH]
                          [-m MODEL] [-mf MODEL_FILE] [-im INIT_MODEL]
                          [-ltim LOG_EVERY_N_SECS] [-d DISPLAY_EXAMPLES]

Check tasks for common errors

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  -ltim, --log-every-n-secs LOG_EVERY_N_SECS
  -d, --display-examples DISPLAY_EXAMPLES

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train:ordered)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

distributed_train

Distributed training script. NOT MEANT TO BE CALLED DIRECTLY BY USER.

This script is meant to be in conjunction with SLURM <https://slurm.schedmd.com/>, which provides environmental variables describing the environment.

An example sbatch script is below, for a 2-host, 8-GPU setup (16 total gpus):

#!/bin/sh
#SBATCH --job-name=distributed_example
#SBATCH --output=/path/to/savepoint/stdout.%j
#SBATCH --error=/path/to/savepoint/stderr.%j
#SBATCH --partition=priority
#SBATCH --nodes=2
#SBATCH --time=0:10:00
#SBATCH --signal=SIGINT
#SBATCH --gres=gpu:8
#SBATCH --ntasks-per-node=8
#SBATCH --mem=64G
#SBATCH --cpus-per-task=10
srun python -u -m parlai.scripts.distributed_train     -m seq2seq -t convai2 --dict-file /path/to/dict-file

CLI help

usage: parlai distributed_train [-h] [--helpall] [-o INIT_OPT] [-t TASK]
                                [-dt DATATYPE] [-nt NUMTHREADS]
                                [-bs BATCHSIZE] [-dynb {batchsort,full,None}]
                                [-dp DATAPATH] [-m MODEL] [-mf MODEL_FILE]
                                [-im INIT_MODEL] [-et EVALTASK]
                                [-eps NUM_EPOCHS] [-ttim MAX_TRAIN_TIME]
                                [-vtim VALIDATION_EVERY_N_SECS]
                                [-stim SAVE_EVERY_N_SECS]
                                [-sval SAVE_AFTER_VALID]
                                [-veps VALIDATION_EVERY_N_EPOCHS]
                                [-vp VALIDATION_PATIENCE]
                                [-vmt VALIDATION_METRIC] [-vmm {max,min}]
                                [-mcs METRICS] [-micro AGGREGATE_MICRO]
                                [-tblog TENSORBOARD_LOG]
                                [--bpe-vocab BPE_VOCAB]
                                [--bpe-merge BPE_MERGE]
                                [--distributed-world-size DISTRIBUTED_WORLD_SIZE]
                                [--port PORT]

Train a model

optional arguments:
  -h, --help
        show this help message and exit
  --helpall
        Show usage, including advanced arguments.
  --port PORT
        TCP port number (default: 61337)

Main ParlAI Arguments:
  -o, --init-opt INIT_OPT
        Path to json file of options. Note: Further Command-line arguments
        override file-based options. (default: None)
  -t, --task TASK
        ParlAI task(s), e.g. "babi:Task1" or "babi,cbt" (default: None)
  -dt, --datatype DATATYPE
        choose from: train, train:ordered, valid, test. to stream data add
        ":stream" to any option (e.g., train:stream). by default train is
        random with replacement, valid is ordered, test is ordered. (default:
        train)
  -nt, --numthreads NUMTHREADS
        number of threads. Used for hogwild if batchsize is 1, else for number
        of threads in threadpool loading, (default: 1)
  -bs, --batchsize BATCHSIZE
        batch size for minibatch training schemes (default: 1)
  -dynb, --dynamic-batching {batchsort,full,None}
        Use dynamic batching (default: None)
  -dp, --datapath DATAPATH
        path to datasets, defaults to {parlai_dir}/data (default: None)

ParlAI Model Arguments:
  -m, --model MODEL
        the model class name. can match parlai/agents/<model> for agents in
        that directory, or can provide a fully specified module for `from X
        import Y` via `-m X:Y` (e.g. `-m
        parlai.agents.seq2seq.seq2seq:Seq2SeqAgent`) (default: None)
  -mf, --model-file MODEL_FILE
        model file name for loading and saving models (default: None)
  -im, --init-model INIT_MODEL
        load model weights and dict from this file (default: None)

Training Loop Arguments:
  -et, --evaltask EVALTASK
        task to use for valid/test (defaults to the one used for training)
        (default: None)
  -eps, --num-epochs NUM_EPOCHS
  -ttim, --max-train-time MAX_TRAIN_TIME
  -vtim, --validation-every-n-secs VALIDATION_EVERY_N_SECS
        Validate every n seconds. Saves model to model_file (if set) whenever
        best val metric is found (default: -1)
  -stim, --save-every-n-secs SAVE_EVERY_N_SECS
        Saves the model to model_file.checkpoint after every n seconds
        (default -1, never). (default: -1)
  -sval, --save-after-valid SAVE_AFTER_VALID
        Saves the model to model_file.checkpoint after every validation
        (default False).
  -veps, --validation-every-n-epochs VALIDATION_EVERY_N_EPOCHS
        Validate every n epochs. Saves model to model_file (if set) whenever
        best val metric is found (default: -1)
  -vp, --validation-patience VALIDATION_PATIENCE
        number of iterations of validation where result does not improve
        before we stop training (default: 10)
  -vmt, --validation-metric VALIDATION_METRIC
        key into report table for selecting best validation (default:
        accuracy)
  -vmm, --validation-metric-mode {max,min}
        how to optimize validation metric (max or min) (default: None)
  -mcs, --metrics METRICS
        list of metrics to show/compute, e.g. all, default,or give a list
        split by , like ppl,f1,accuracy,hits@1,rouge,bleuthe rouge metrics
        will be computed as rouge-1, rouge-2 and rouge-l (default: default)
  -micro, --aggregate-micro AGGREGATE_MICRO
        Report micro-averaged metrics instead of macro averaged metrics.
        (default: False)

Tensorboard Arguments:
  -tblog, --tensorboard-log TENSORBOARD_LOG
        Tensorboard logging of metrics, default is False

BPEHelper Arguments:
  --bpe-vocab BPE_VOCAB
        path to pre-trained tokenizer vocab (default: None)
  --bpe-merge BPE_MERGE
        path to pre-trained tokenizer merge (default: None)

Distributed Training:
  --distributed-world-size DISTRIBUTED_WORLD_SIZE
        Number of workers. (default: None)