MMCultureQA 2027

Participate

Participation is open and free to individuals and teams from academia and industry. Enter one task or both, in any of the language tracks.

Register your team Join the Slack

How it works

  1. Fill in the registration form and accept the dataset usage terms.
  2. Download the tracks you want from Hugging Face. MENA train and dev data is out now; the other tracks follow the timeline.
  3. Build a system for Task 1 (spoken), Task 2 (textual), or both, in whichever language tracks you choose.
  4. Check your scores locally with the evaluation script that ships with the data.
  5. Upload your predictions to the CodaBench competition during the evaluation phase.
  6. Describe your system in a paper for the SemEval 2027 proceedings and present it at the workshop.

Submission

Submissions run through CodaBench. The competition link and the submission format will be posted here when the evaluation phase opens. A starter kit with data loaders, baselines, and the scorer ships with the training data.

Rules

The full rules ship with the starter kit. The essentials:

  • Enter any subset. Systems are ranked per task and per track.
  • The official ranking metric is BERTScore F1, described under Evaluation.
  • Pretrained models and public external data are permitted; document what you used. Any limits will be stated at the data release.
  • The data is licensed for non-commercial research use; see the dataset page.

Questions?

Ask in the community Slack or contact the organizers.