Participate
Participation is open and free to individuals and teams from academia and industry. Enter one task or both, in any of the language tracks.
How it works
- Register your team. Fill in the registration form and accept the dataset usage terms.
- Get the data. Start with the sample set on Hugging Face to preview the format; the full training and development data arrive on the schedule in the timeline.
- Build your system. Target Task 1 (spoken), Task 2 (textual), or both, in whichever language tracks you choose.
- Evaluate locally. Reproduce the official BERTScore F1 ranking with the evaluation script that ships with the data.
- Submit on CodaBench. During the evaluation phase, upload your predictions to the official competition.
- Write it up. Describe your system in a paper for the SemEval 2027 proceedings and present it at the workshop.
Submission
Submissions run through CodaBench. The competition link and the exact submission format will be posted here when the evaluation phase opens. A starter kit with data loaders, baseline systems, and the scorer is released together with the training data.
Rules
The full rules ship with the starter kit. The essentials:
- Enter any subset: one task or both, one language track or all of them. Systems are ranked per task and per track.
- The official ranking metric is BERTScore F1, described under Evaluation.
- Pretrained models and public external data are generally permitted; document what you used in your system paper. Any limits will be stated at the data release.
- The data is licensed for non-commercial research use; see the dataset page.
Questions?
Ask in the community Slack, where announcements about data, baselines, and submission land first, or contact the organizers.