Model Database's logo

Evaluate documentation

Hub methods

Model Database's logo
Join the Model Database community

and get access to the augmented documentation experience

to get started

Hub methods

Methods for using the Model Database Hub:

Push to hub

evaluate.push_to_hub

< >

( model_id: str task_type: str dataset_type: str dataset_name: str metric_type: str metric_name: str metric_value: float task_name: str = None dataset_config: str = None dataset_split: str = None dataset_revision: str = None dataset_args: typing.Dict[str, int] = None metric_config: str = None metric_args: typing.Dict[str, int] = None overwrite: bool = False )

Parameters

  • model_id (str) — Model id from https://hf.co/models.
  • task_type (str) — Task id, refer to https://github.com/huggingface/evaluate/blob/main/src/evaluate/config.py#L154 for allowed values.
  • dataset_type (str) — Dataset id from https://hf.co/datasets.
  • dataset_name (str) — Pretty name for the dataset.
  • metric_type (str) — Metric id from https://hf.co/metrics.
  • metric_name (str) — Pretty name for the metric.
  • metric_value (float) — Computed metric value.
  • task_name (str, optional) — Pretty name for the task.
  • dataset_config (str, optional) — Dataset configuration used in datasets.load_dataset(). See huggingface/datasets docs for more info: https://huggingface.co/docs/datasets/package_reference/loading_methods#datasets.load_dataset.name
  • dataset_split (str, optional) — Name of split used for metric computation.
  • dataset_revision (str, optional) — Git hash for the specific version of the dataset.
  • dataset_args (dict[str, int], optional) — Additional arguments passed to datasets.load_dataset().
  • metric_config (str, optional) — Configuration for the metric (e.g. the GLUE metric has a configuration for each subset)
  • metric_args (dict[str, int], optional) — Arguments passed during Metric.compute().
  • overwrite (bool, optional, defaults to False) — If set to True an existing metric field can be overwritten, otherwise attempting to overwrite any existing fields will cause an error.

Pushes the result of a metric to the metadata of a model repository in the Hub.

Example:

>>> push_to_hub(
...     model_id="huggingface/gpt2-wikitext2",
...     metric_value=0.5
...     metric_type="bleu",
...     metric_name="BLEU",
...     dataset_name="WikiText",
...     dataset_type="wikitext",
...     dataset_split="test",
...     task_type="text-generation",
...     task_name="Text Generation"
... )
←Saving methods Evaluator classes→