Dataset Preview
Viewer
The full dataset viewer is not available (click to read why). Only showing a preview of the rows.
Job manager was killed while running this job (job exceeded maximum duration).
Error code:   JobManagerExceededMaximumDurationError

Need help to make the dataset viewer work? Open a discussion for direct support.

template
string
id
string
question
string
image
image
label
int64
"adversarial"
"CLEVR_train_000000.png"
"Subtract all brown blocks. Subtract all red balls. How many blocks are left?"
3
"adversarial"
"CLEVR_train_000000.png"
"Subtract all cyan cubes. How many red spheres are left?"
0
"adversarial"
"CLEVR_train_000000.png"
"Add 5 big browns. How many large greens exist?"
0
"subtraction-multihop"
"CLEVR_train_000000.png"
"Subtract all green metallic cylinders. Subtract all cyan blocks. How many objects are left?"
4
"addition"
"CLEVR_train_000000.png"
"Add 4 metallic balls. How many metallic balls are left?"
5
"addition"
"CLEVR_train_000000.png"
"Add 1 tiny rubber objects. How many tiny rubber objects exist?"
3
"addition"
"CLEVR_train_000000.png"
"Add 3 green cylinders. How many objects exist?"
9
"subtraction"
"CLEVR_train_000000.png"
"Subtract all brown cylinders. How many cylinders are left?"
1
"subtraction"
"CLEVR_train_000000.png"
"Subtract all small cyan rubber cubes. How many cubes are left?"
2
"subtraction"
"CLEVR_train_000000.png"
"Subtract 0 green blocks. How many objects are left?"
6
"subtraction"
"CLEVR_train_000001.png"
"Subtract all balls. How many objects are left?"
4
"subtraction"
"CLEVR_train_000001.png"
"Subtract 1 cylinders. How many cylinders are left?"
0
"adversarial"
"CLEVR_train_000001.png"
"Subtract all purple cylinders. Subtract all blue blocks. How many cylinders are left?"
1
"adversarial"
"CLEVR_train_000001.png"
"Subtract all red cylinders. How many brown cubes are left?"
1
"subtraction-multihop"
"CLEVR_train_000001.png"
"Subtract all green spheres. Subtract all big blue objects. How many objects are left?"
7
"addition"
"CLEVR_train_000001.png"
"Add 5 tiny yellow shiny things. How many tiny yellow shiny things are left?"
6
"addition"
"CLEVR_train_000001.png"
"Add 4 brown rubber objects. How many brown rubber objects exist?"
5
"addition"
"CLEVR_train_000001.png"
"Add 1 brown cubes. How many objects exist?"
10
"subtraction"
"CLEVR_train_000001.png"
"Subtract all green spheres. How many spheres are left?"
4
"subtraction"
"CLEVR_train_000001.png"
"Subtract all small green rubber spheres. How many spheres are left?"
4
"subtraction"
"CLEVR_train_000002.png"
"Subtract 1 yellow spheres. How many objects are left?"
2
"subtraction"
"CLEVR_train_000002.png"
"Subtract all balls. How many objects are left?"
1
"subtraction"
"CLEVR_train_000002.png"
"Subtract 1 spheres. How many spheres are left?"
1
"adversarial"
"CLEVR_train_000002.png"
"Subtract all yellow cubes. Subtract all yellow cylinders. How many cubes are left?"
1
"adversarial"
"CLEVR_train_000002.png"
"Subtract all red cylinders. How many cyan balls are left?"
0
"subtraction-multihop"
"CLEVR_train_000002.png"
"Subtract all blue rubber cubes. Subtract all cylinders. How many objects are left?"
2
"addition"
"CLEVR_train_000002.png"
"Add 2 blue objects. How many blue objects are left?"
3
"addition"
"CLEVR_train_000002.png"
"Add 2 blue rubber blocks. How many blue rubber blocks exist?"
3
"addition"
"CLEVR_train_000002.png"
"Add 3 blue blocks. How many objects exist?"
6
"subtraction"
"CLEVR_train_000002.png"
"Subtract all yellow balls. How many balls are left?"
1
"subtraction"
"CLEVR_train_000003.png"
"Subtract all purple cubes. How many were subtracted if there are1purple cubes left?"
1
"subtraction"
"CLEVR_train_000003.png"
"Subtract all small brown matte blocks. How many blocks are left?"
4
"subtraction"
"CLEVR_train_000003.png"
"Subtract 2 purple blocks. How many objects are left?"
8
"subtraction"
"CLEVR_train_000003.png"
"Subtract all cylinders. How many objects are left?"
8
"subtraction"
"CLEVR_train_000003.png"
"Subtract 1 blocks. How many blocks are left?"
4
"adversarial"
"CLEVR_train_000003.png"
"Subtract all yellow cylinders. Subtract all purple blocks. How many cylinders are left?"
2
"adversarial"
"CLEVR_train_000003.png"
"Subtract all brown spheres. How many cyan cylinders are left?"
1
"subtraction-multihop"
"CLEVR_train_000003.png"
"Subtract all purple cubes. Subtract all green things. How many objects are left?"
7
"addition"
"CLEVR_train_000003.png"
"Add 8 purple rubber objects. How many purple rubber objects are left?"
10
"addition"
"CLEVR_train_000003.png"
"Add 1 big gray metal cylinders. How many big gray metal cylinders exist?"
2
"addition"
"CLEVR_train_000004.png"
"Add 2 shiny cylinders. How many objects exist?"
5
"subtraction"
"CLEVR_train_000004.png"
"Subtract 0 gray cylinders. How many objects are left?"
3
"subtraction"
"CLEVR_train_000004.png"
"Subtract all balls. How many objects are left?"
2
"subtraction"
"CLEVR_train_000004.png"
"Subtract 1 spheres. How many spheres are left?"
0
"adversarial"
"CLEVR_train_000004.png"
"Subtract all yellow cubes. How many yellow balls are left?"
0
"subtraction-multihop"
"CLEVR_train_000004.png"
"Subtract all big yellow matte spheres. Subtract all cubes. How many objects are left?"
2
"addition"
"CLEVR_train_000004.png"
"Add 1 cyan cylinders. How many cyan cylinders are left?"
2
"addition"
"CLEVR_train_000004.png"
"Add 3 purple spheres. How many purple spheres exist?"
4
"subtraction"
"CLEVR_train_000005.png"
"Subtract all gray balls. How many balls are left?"
2
"subtraction"
"CLEVR_train_000005.png"
"Subtract all big purple cubes. How many cubes are left?"
3
"adversarial"
"CLEVR_train_000005.png"
"Subtract all purple balls. Subtract all gray cylinders. How many balls are left?"
3
"addition"
"CLEVR_train_000005.png"
"Add 1 large cyan shiny things. How many objects exist?"
10
"subtraction"
"CLEVR_train_000005.png"
"Subtract 0 green cylinders. How many objects are left?"
9
"subtraction"
"CLEVR_train_000005.png"
"Subtract all spheres. How many objects are left?"
6
"subtraction"
"CLEVR_train_000005.png"
"Subtract 2 spheres. How many spheres are left?"
1
"adversarial"
"CLEVR_train_000005.png"
"Subtract all green blocks. How many brown cylinders are left?"
1
"subtraction-multihop"
"CLEVR_train_000005.png"
"Subtract all spheres. Subtract all blue cylinders. How many objects are left?"
5
"addition"
"CLEVR_train_000005.png"
"Add 5 rubber cubes. How many rubber cubes are left?"
7
"adversarial"
"CLEVR_train_000006.png"
"Subtract all yellow blocks. Subtract all brown spheres. How many blocks are left?"
1
"addition"
"CLEVR_train_000006.png"
"Add 1 cyan rubber cylinders. How many cyan rubber cylinders exist?"
1
"addition"
"CLEVR_train_000006.png"
"Add 2 tiny green blocks. How many objects exist?"
7
"subtraction"
"CLEVR_train_000006.png"
"Subtract 0 red cylinders. How many objects are left?"
5
"subtraction"
"CLEVR_train_000006.png"
"Subtract all cylinders. How many objects are left?"
3
"subtraction"
"CLEVR_train_000006.png"
"Subtract 1 cylinders. How many cylinders are left?"
1
"adversarial"
"CLEVR_train_000006.png"
"Subtract all purple cubes. How many purple cylinders are left?"
0
"subtraction-multihop"
"CLEVR_train_000006.png"
"Subtract all large brown things. Subtract all rubber cylinders. How many objects are left?"
3
"addition"
"CLEVR_train_000006.png"
"Add 5 big brown matte cylinders. How many big brown matte cylinders are left?"
6
"subtraction"
"CLEVR_train_000007.png"
"Subtract all purple blocks. How many were subtracted if there are1purple blocks left?"
2
"subtraction"
"CLEVR_train_000007.png"
"Subtract all purple blocks. How many blocks are left?"
2
"subtraction"
"CLEVR_train_000007.png"
"Subtract all purple blocks. How many blocks are left?"
2
"adversarial"
"CLEVR_train_000007.png"
"Subtract all brown spheres. Subtract all brown cylinders. How many spheres are left?"
2
"addition"
"CLEVR_train_000007.png"
"Add 3 big yellow shiny objects. How many big yellow shiny objects exist?"
4
"addition"
"CLEVR_train_000007.png"
"Add 2 rubber objects. How many objects exist?"
9
"subtraction"
"CLEVR_train_000007.png"
"Subtract 0 cyan balls. How many objects are left?"
7
"subtraction"
"CLEVR_train_000007.png"
"Subtract all blocks. How many objects are left?"
2
"subtraction"
"CLEVR_train_000007.png"
"Subtract 2 spheres. How many spheres are left?"
0
"adversarial"
"CLEVR_train_000007.png"
"Subtract all purple cubes. How many yellow balls are left?"
1
"subtraction"
"CLEVR_train_000008.png"
"Subtract all yellow spheres. How many spheres are left?"
2
"subtraction"
"CLEVR_train_000008.png"
"Subtract all brown balls. How many balls are left?"
2
"adversarial"
"CLEVR_train_000008.png"
"Subtract all brown cubes. Subtract all brown cylinders. How many cubes are left?"
2
"subtraction-multihop"
"CLEVR_train_000008.png"
"Subtract all large objects. Subtract all yellow objects. How many objects are left?"
4
"addition"
"CLEVR_train_000008.png"
"Add 2 rubber cubes. How many rubber cubes are left?"
3
"addition"
"CLEVR_train_000008.png"
"Add 2 blue spheres. How many blue spheres exist?"
3
"addition"
"CLEVR_train_000008.png"
"Add 3 tiny cyan objects. How many objects exist?"
9
"subtraction"
"CLEVR_train_000008.png"
"Subtract 1 cyan cylinders. How many objects are left?"
5
"subtraction"
"CLEVR_train_000008.png"
"Subtract all cylinders. How many objects are left?"
5
"subtraction"
"CLEVR_train_000008.png"
"Subtract 1 cylinders. How many cylinders are left?"
0
"subtraction"
"CLEVR_train_000009.png"
"Subtract all rubber cylinders. How many cylinders are left?"
1
"adversarial"
"CLEVR_train_000009.png"
"Subtract all cyan cubes. Subtract all red cylinders. How many cubes are left?"
1
"adversarial"
"CLEVR_train_000009.png"
"Subtract all yellow cubes. How many yellow balls are left?"
0
"subtraction-multihop"
"CLEVR_train_000009.png"
"Subtract all matte cylinders. Subtract all large green shiny things. How many objects are left?"
3
"addition"
"CLEVR_train_000009.png"
"Add 4 big gray cubes. How many big gray cubes are left?"
5
"addition"
"CLEVR_train_000009.png"
"Add 4 big cyan objects. How many big cyan objects exist?"
5
"addition"
"CLEVR_train_000009.png"
"Add 5 small cyan cylinders. How many objects exist?"
10
"subtraction"
"CLEVR_train_000009.png"
"Subtract 1 cyan balls. How many objects are left?"
4
"subtraction"
"CLEVR_train_000009.png"
"Subtract all balls. How many objects are left?"
4
"subtraction"
"CLEVR_train_000010.png"
"Subtract all green cylinders. How many were subtracted if there are1green cylinders left?"
1
"subtraction"
"CLEVR_train_000010.png"
"Subtract all green blocks. How many blocks are left?"
3
"subtraction"
"CLEVR_train_000010.png"
"Subtract all green cylinders. How many cylinders are left?"
1
"subtraction"
"CLEVR_train_000010.png"
"Subtract 1 cubes. How many cubes are left?"
4
End of preview (truncated to 100 rows)

Dataset Card for CLEVR-Math

Dataset Summary

Dataset for compositional multimodal mathematical reasoning based on CLEVR.

Loading the data, preprocessing text with CLIP

from transformers import CLIPPreprocessor
from datasets import load_dataset, DownloadConfig


dl_config = DownloadConfig(resume_download=True,
        num_proc=8,
        force_download=True)

# Load 'general' instance of dataset
dataset = load_dataset('dali-does/clevr-math', download_config=dl_config)

# Load version with only multihop in test data
dataset_multihop = load_dataset('dali-does/clevr-math', 'multihop',
                                download_config=dl_config)


model_path = "openai/clip-vit-base-patch32"
extractor = CLIPProcessor.from_pretrained(model_path)
def transform_tokenize(e):
  e['image'] = [image.convert('RGB') for image in e['image']]
  return extractor(text=e['question'],
                           images=e['image'],
                           padding=True)

dataset = dataset.map(transform_tokenize,
                      batched=True,
                      num_proc=8,
                      padding='max_length')

dataset_subtraction = dataset.filter(lambda e:
        e['template'].startswith('subtraction'), num_proc=4)

Supported Tasks and Leaderboards

Leaderboard will be announced at a later date.

Languages

The dataset is currently only available in English. To extend the dataset to other languages, the CLEVR templates must be rewritten in the target language.

Dataset Structure

Data Instances

  • general containing the default version with multihop questions in train and test
  • multihop containing multihop questions only in test data to test generalisation of reasoning

Data Fields

features = datasets.Features(
  {
    "template": datasets.Value("string"),
    "id": datasets.Value("string"),
    "question": datasets.Value("string"),
    "image": datasets.Image(),
    "label": datasets.Value("int64")
  }
)

Data Splits

train/val/test

Dataset Creation

Data is generated using code provided with the CLEVR-dataset, using blender and templates constructed by the dataset curators.

Considerations for Using the Data

Other Known Limitations

[More Information Needed]

Additional Information

Dataset Curators

Adam Dahlgren Lindström - [email protected]

Licensing Information

Licensed under Creative Commons Attribution Share Alike 4.0 International (CC-by 4.0).

Citation Information

[More Information Needed]

@misc{https://doi.org/10.48550/arxiv.2208.05358,
  doi = {10.48550/ARXIV.2208.05358},
  url = {https://arxiv.org/abs/2208.05358},
  author = {Lindström, Adam Dahlgren and Abraham, Savitha Sam},
  keywords = {Machine Learning (cs.LG), Computation and Language (cs.CL), Computer Vision and Pattern Recognition (cs.CV), FOS: Computer and information sciences, FOS: Computer and information sciences, I.2.7; I.2.10; I.2.6; I.4.8; I.1.4},
  title = {CLEVR-Math: A Dataset for Compositional Language, Visual, and Mathematical Reasoning},
  publisher = {arXiv},
  year = {2022},
  copyright = {Creative Commons Attribution Share Alike 4.0 International}
}

Contributions

Thanks to @dali-does for adding this dataset.

Downloads last month
6
Edit dataset card
Evaluate models HF Leaderboard