Participate

How it works

1. Fill in the registration form and accept the dataset usage terms.
2. Download the tracks you want from Hugging Face. MENA train and dev data is out now; the other tracks follow the timeline.
3. Build a system for Task 1 (spoken), Task 2 (textual), or both, in whichever language tracks you choose.
4. Check your scores locally with the scorer, and validate your file with the format checker.
5. Upload your predictions during the evaluation phase using the submission platform for your task.
6. Describe your system in a paper for the SemEval 2027 proceedings and present it at the workshop.

Code

The code repository holds everything you need to run and check a system:

• data download the tracks from Hugging Face
• baselines a runnable starting point
• scorer the official ranking metric, so you can reproduce your score
• format checker validate a submission file before you upload it

Submission

Upload your predictions through the corresponding task page:

• Task 1: Spoken Visual QA
• Task 2: Textual Visual QA

Note: Submissions are currently available for Arabic and English. Other languages are coming soon.

Rules

The full rules ship with the code repository. The essentials:

• Enter any subset. Systems are ranked per task and per track.
• The official ranking metric is BERTScore F1, described under Evaluation.
• Pretrained models and public external data are permitted; document what you used. Any limits will be stated at the data release.
• The data is licensed for non-commercial research use; see the dataset page.

Questions?

Ask in the community Slack or contact the organizers.