core.utils

File for miscellaneous utility functions and constants.

parlai.utils.misc.maintain_dialog_history(history, observation, reply='', historyLength=1, useReplies='label_else_model', dict=None, useStartEndIndices=True, splitSentences=False)

Keep track of dialog history, up to a truncation length.

Either includes replies from the labels, model, or not all using param ‘replies’.

DEPRECATED. USE PARLAI.CORE.TORCH_AGENT INSTEAD.

parlai.utils.misc.load_cands(path, lines_have_ids=False, cands_are_replies=False)

Load global fixed set of candidate labels that the teacher provides.

Every example will include these as candidates. The true labels for a specific example are also added to this set, so that it’s possible to get the right answer.

class parlai.utils.misc.Opt(*args, **kwargs)

Bases: dict

Class for tracking options.

Functions like a dict, but allows us to track the history of arguments as they are set.

__init__(*args, **kwargs)

Initialize self. See help(type(self)) for accurate signature.

display_deepcopies()

Display all deepcopies.

display_history(key)

Display the history for an item in the dict.

parlai.utils.misc.load_opt_file(optfile: str) → parlai.utils.misc.Opt

Load an Opt from disk.

class parlai.utils.misc.Predictor(args=None, **kwargs)

Bases: object

Wrapper to set up running version of model and request predictions.

Note that this maintains no World state (does not use a World), merely providing the observation directly to the model and getting a response.

This is limiting when it comes to certain use cases, but allows for quick model deployment.

__init__(args=None, **kwargs)

Initialize the predictor, setting up opt automatically if needed.

Args is expected to be in the same format as sys.argv: e.g. a list in the form [‘–model’, ‘seq2seq’, ‘-hs’, 128, ‘-lr’, 0.5].

kwargs is interpreted by appending ‘–’ to it and replacing underscores with hyphens, so ‘dict_file=/tmp/dict.tsv’ would be interpreted as ‘–dict-file /tmp/dict.tsv’.

predict(observation)

From a ParlAI-standard message dict, get model prediction.

class parlai.utils.misc.Timer

Bases: object

Computes elapsed time.

__init__()

Initialize timer.

reset()

Reset timer to zero.

resume()

Resume timer.

stop()

Pause timer.

time()

Get current timer time.

class parlai.utils.misc.TimeLogger

Bases: object

Class for logging time progress against a goal.

__init__()

Set up timer.

total_time()

Return time elapsed at last log call.

time()

Return current timer time.

log(done, total, report=None)

Log report, time elapsed, and percentage progress towards goal.

Parameters
  • done – number of examples completed so far

  • total – total number of elements to be completed. if total > 0, calculates the time remaining and percentage complete.

  • report – dict of pairs to log

Returns

tuple log string, log dict log string contains time elapsed and string representation of the log dict log dict contains pairs of all items to log, which includes percentage complete and projected time left if total > 0

class parlai.utils.misc.AttrDict(*args, **kwargs)

Bases: dict

Helper class to have a dict-like object with dot access.

For example, instead of d = {‘key’: ‘value’} use d = AttrDict(key=’value’). To access keys, instead of doing d[‘key’] use d.key.

While this has some limitations on the possible keys (for example, do not set the key items or you will lose access to the items() method), this can make some code more clear.

__init__(*args, **kwargs)

Initialize AttrDict using input dict.

class parlai.utils.misc.NoLock

Bases: object

Empty lock.

Does nothing when you enter or exit.

parlai.utils.misc.round_sigfigs(x: Union[float, torch.Tensor], sigfigs=4) → float

Round value to specified significant figures.

Parameters
  • x – input number

  • sigfigs – number of significant figures to return

Returns

float number rounded to specified sigfigs

parlai.utils.misc.no_lock()

Build a nolock for other classes to use for no-op locking.

class parlai.utils.misc.PaddingUtils

Bases: object

Helps with padding input and target tensors.

DEPRECATED. USE PARLAI.CORE.TORCH_AGENT INSTEAD.

classmethod pad_text(observations, dictionary, end_idx=None, null_idx=0, dq=False, eval_labels=True, truncate=None)

Pad observations to max width.

We check that examples are valid, pad with zeros, and sort by length so that we can use the pack_padded function. The list valid_inds keeps track of which indices are valid and the order in which we sort the examples.

dq – whether we should use deque or list eval_labels – whether or not we want to consider eval labels truncate – truncate input and output lengths

DEPRECATED. USE PARLAI.CORE.TORCH_AGENT INSTEAD.

classmethod map_predictions(predictions, valid_inds, batch_reply, observations, dictionary, end_idx, report_freq=0.1, labels=None, answers=None, ys=None)

Match predictions to original index in the batch.

Predictions are mapped back to appropriate indices in the batch_reply using valid_inds.

report_freq – how often we report predictions

DEPRECATED. USE PARLAI.CORE.TORCH_AGENT INSTEAD.

parlai.utils.misc.clip_text(text, max_len)

Clip text to max length, adding ellipses.

parlai.utils.misc.display_messages(msgs: List[Dict[str, Any]], prettify: bool = False, ignore_fields: str = '', max_len: int = 1000) → Optional[str]

Return a string describing the set of messages provided.

If prettify is true, candidates are displayed using prettytable. ignore_fields provides a list of fields in the msgs which should not be displayed.

parlai.utils.misc.str_to_msg(txt, ignore_fields='')

Convert formatted string to ParlAI message dict.

Parameters
  • txt – formatted string to convert. String format is tab-separated fields, with colon separating field name and contents.

  • ignore_fields – (default ‘’) comma-separated field names to not include in the msg dict even if they’re in the string.

parlai.utils.misc.msg_to_str(msg, ignore_fields='')

Convert ParlAI message dict to string.

Parameters
  • msg – dict to convert into a string.

  • ignore_fields – (default ‘’) comma-separated field names to not include in the string even if they’re in the msg dict.

parlai.utils.misc.set_namedtuple_defaults(namedtuple, default=None)

Set all of the fields for a given nametuple to a singular value.

Additionally removes the default docstring for each field. Modifies the tuple in place, but returns it anyway.

More info: https://stackoverflow.com/a/18348004

Parameters
  • namedtuple – A constructed collections.namedtuple

  • default – The default value to set.

Returns

the modified namedtuple

parlai.utils.misc.warn_once(msg: str, warningtype=None) → None

Raise a warning, but only once.

Parameters
  • msg (str) – Message to display

  • warningtype (Warning) – Type of warning, e.g. DeprecationWarning