Model Zoo

This is a list of pretrained ParlAI models. Some are meant to be used as components in larger systems, while others may be used by themselves.

Pretrained Embeddings

Some models support using Pretrained Embeddings, via torchtext. As of writing, this includes:

Example invocation:

python -m parlai.scripts.train_model -t convai2 -m seq2seq -emb fasttext_cc

Adding ‘-fixed’ to the name e.g. ‘twitter-fixed’ means backprop will not go through this (i.e. they will remain unchanged).

BERT

BERT is in the model zoo and is automatically used for initialization of bert bi-ranker and cross-rankers.

Example invocation:

python -m parlai.scripts.train_model -t convai2 -m bert_ranker/bi_encoder_ranker

Kvmemnn Convai2 Model

KvMemNN trained on the ConvAI2 task, used as a baseline in the competition.

Example invocation:

python -m parlai.scripts.interactive -mf models:convai2/kvmemnn/model
...

Seq2Seq Convai2 Model

SeqSeq trained on the ConvAI2 task, used as a baseline in the competition.

Example invocation:

python -m parlai.scripts.interactive -mf models:convai2/seq2seq/convai2_self_seq2seq_model -m legacy:seq2seq:0
...

Seq2Seq Twitter Model

Seq2Seq conversational model trained on the Twitter task

Example invocation:

python -m parlai.scripts.eval_model --model legacy:seq2seq:0 --task twitter -mf models:twitter/seq2seq/twitter_seq2seq_model
...
{'exs': 10405, 'accuracy': 0.001538, 'f1': 0.07537, 'bleu': 0.002304, 'loss': 3.93, 'ppl': 50.9}

Drqa Squad Model

drqa reader trained on SQuAD

Example invocation:

python -m parlai.scripts.eval_model --model drqa --task squad -mf models:drqa/squad/model
...
{'exs': 10570, 'accuracy': 0.6886, 'f1': 0.7821, 'hits@1': 0.689, 'hits@5': 0.689, 'hits@10': 0.689, 'hits@100': 0.689, 'bleu': 0.1364, 'train_loss': 0}

Wikipedia Retriever (Used For Open Squad)

Retrieval over Wikipedia dump, used for DrQA on the open squad dataset. This is the dump from the original paper, used for replicating results.

Example invocation:

python -m parlai.scripts.interactive --model tfidf_retriever -mf models:wikipedia_2016-12-21/tfidf_retriever/drqa_docs
...
Enter Your Message: Yann LeCun
[candidate_scores]: [507.05804682 390.18244433 279.24033928 269.60377042 214.00140589]
[SparseTfidfRetrieverAgent]:
Deep learning (also known as deep structured learning, hierarchical learning or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high level abstractions in data. In a simple case, you could have two sets of neurons: ones that receive an input signal and ones that send an output signal. When the input layer receives an input it passes on a modified version of the input to the next layer. In a deep network, there are many layers between the input and output (and the layers are not made of neurons but it can help to think of it that way), allowing the algorithm to use multiple processing layers, composed of multiple linear and non-linear transformations.

Deep learning is part of a broader family of machine learning methods based on ...
to commonsense reasoning which operates on concepts in terms of production rules of the grammar, and is a basic goal of both human language acquisition and AI. (See also Grammar induction.)

Wikipedia Retriever (Used For Wizard Of Wikipedia)

Retrieval over Wikipedia dump, used for DrQA on the open squad dataset.

Example invocation:

python -m parlai.scripts.interactive --model tfidf_retriever -mf models:wikipedia_full/tfidf_retriever/model
...
Enter Your Message: Yann LeCun
[candidate_scores]: [454.74038503 353.88863708 307.31353203 280.4501096  269.89960432]
[SparseTfidfRetrieverAgent]:
Yann LeCun (; born 1960) is a computer scientist with contributions in machine learning, computer vision, mobile robotics and computational neuroscience. He is well known for his work on optical character recognition and computer vision using convolutional neural networks (CNN), and is a founding father of convolutional nets. He is also one of the main creators of the DjVu image compression technology (together with Léon Bottou and Patrick Haffner). He co-developed the Lush programming language with Léon Bottou.

Yann LeCun was born near Paris, France, in 1960. He received a Diplôme d'Ingénieur from the Ecole Superieure d'Ingénieur en Electrotechnique et Electronique (ESIEE), Paris in 1983, and a PhD in Computer Science from Université Pierre et Marie Curie in 1987 during which he ...
of Science and Technology in Saudi Arabia because he was considered a terrorist in the country in view of his atheism.

In 2018 Yann LeCun picked a fight with a robot to support Facebook AI goals.

Wizard Of Wikipedia (End To End Generator)

End2End Generative model for Wizard of Wikipedia

Example invocation:

python examples/display_model.py -t wizard_of_wikipedia:generator -mf models:wizard_of_wikipedia/end2end_generator/model -n 1 --display-ignore-fields knowledge_parsed
...
[chosen_topic]: Gardening
[knowledge]: no_passages_used __knowledge__ no_passages_used
Gardening __knowledge__ Gardening is the practice of growing and cultivating plants as part of horticulture.
Gardening __knowledge__ In gardens, ornamental plants are often grown for their flowers, foliage, or overall appearance; useful plants, such as root vegetables, leaf vegetables, fruits, and herbs, are grown for consumption, for use as dyes, or for medicinal or cosmetic use.
Gardening __knowledge__ Gardening is considered by many people to be a relaxing activity.
Gardening __knowledge__ Gardening ranges in scale from fruit orchards, to long boulevard plantings with one or more different types of shrubs, trees, and herbaceous plants, to residential yards including lawns and foundation plantings, to plants in large or small containers ...
there had been several other notable gardening magazines in circulation, including the "Gardeners' Chronicle" and "Gardens Illustrated", but these were tailored more for the professional gardener.

[title]: Gardening
[checked_sentence]: Gardening is considered by many people to be a relaxing activity.
[eval_labels_choice]: I live on a farm, we garden all year long, it is very relaxing.
[checked_sentence_parsed]: Gardening __knowledge__ Gardening is considered by many people to be a relaxing activity.
[WizTeacher]: Gardening
I like Gardening, even when I've only been doing it for a short time.
[eval_labels: I live on a farm, we garden all year long, it is very relaxing.]
[TorchAgent]: i love gardening , it is considered a relaxing activity .

Wizard Of Wikipedia (Full Dialogue Retrieval Model)

Full Dialogue Retrieval Model for Wizard of Wikipedia

Example invocation:

python examples/display_model.py -t wizard_of_wikipedia -mf models:wizard_of_wikipedia/full_dialogue_retrieval_model/model -m projects:wizard_of_wikipedia:wizard_transformer_ranker --n-heads 6 --ffn-size 1200 --embeddings-scale False --delimiter ' __SOC__ ' --n-positions 1000 --legacy True
...
[chosen_topic]: Gardening
[knowledge]: Gardening Gardening is the practice of growing and cultivating plants as part of horticulture.
Gardening In gardens, ornamental plants are often grown for their flowers, foliage, or overall appearance; useful plants, such as root vegetables, leaf vegetables, fruits, and herbs, are grown for consumption, for use as dyes, or for medicinal or cosmetic use.
Gardening Gardening is considered by many people to be a relaxing activity.
Gardening Gardening ranges in scale from fruit orchards, to long boulevard plantings with one or more different types of shrubs, trees, and herbaceous plants, to residential yards including lawns and foundation plantings, to plants in large or small containers grown inside or outside.
Gardening Gardening may be very specialized, with only one type of plant grown, ...
there had been several other notable gardening magazines in circulation, including the "Gardeners' Chronicle" and "Gardens Illustrated", but these were tailored more for the professional gardener.

[title]: Gardening
[checked_sentence]: Gardening is considered by many people to be a relaxing activity.
[eval_labels_choice]: I live on a farm, we garden all year long, it is very relaxing.
[wizard_of_wikipedia]: Gardening
I like Gardening, even when I've only been doing it for a short time.
[label_candidates: OK what's the history?|Right, thats cool. I had no idea they still did the DVD thing, What is Netflix's highest rated show? do you know? |I will definitely check his first album out as he sounds interesting.|I don't know a whole lot about it. I was raised Catholic but don't practice anything now.|Well , this was a good conversation. |...and 95 more]
[eval_labels: I live on a farm, we garden all year long, it is very relaxing.]
[TorchAgent]: I live on a farm, we garden all year long, it is very relaxing.

Light Bert-Biranker Dialogue Model

LIGHT Dialogue task, replicating the numbers from the paper.

Example invocation:

python examples/eval_model.py -t light_dialog -mf models:light/biranker_dialogue/model
...
{'exs': 6623, 'accuracy': 0.7586, 'f1': 0.7802, 'hits@1': 0.759, 'hits@5': 0.965,'hits@10': 0.994, 'hits@100': 1.0, 'bleu': 0.7255, 'lr': 5e-05, 'num_updates': 15050,'examples': 6623, 'loss': 5307.0, 'mean_loss': 0.8013, 'mean_rank': 1.599, 'train_accuracy': 0}

Twitter Conversational Model

Generic conversational model trained on the twitter task

Example invocation:

python -m parlai.scripts.eval_model --model legacy:seq2seq:0 --task twitter -mf models:twitter/seq2seq/twitter_seq2seq_model
...
{'exs': 10405, 'accuracy': 0.001538, 'f1': 0.07537, 'bleu': 0.002304, 'loss': 3.93, 'ppl': 50.9}

Controllable Dialogue Pretrained Models

Generic conversational model trained on the twitter task

Example invocation:

python -m parlai.scripts.eval_model --model projects.controllable_dialogue.controllable_seq2seq.controllable_seq2seq:ControllableSeq2seqAgent --task projects.controllable_dialogue.tasks.agents -mf models:controllable_dialogue/convai2_finetuned_baseline
...

Personality-Captions Pretrained Model

Transresnet Model pretrained on the Personality-Captions task

Example invocation:

python examples/eval_model.py -t personality_captions -mf models:personality_captions/transresnet/model --num-test-labels 5 -dt test
...
{'exs': 10000, 'accuracy': 0.5113, 'f1': 0.5951, 'hits@1': 0.511, 'hits@5': 0.816, 'hits@10': 0.903, 'hits@100': 0.998, 'bleu': 0.4999, 'hits@1/100': 1.0, 'loss': -0.002, 'med_rank': 1.0}