MMCultureQA 2027

SemEval 2027 · Shared Task

MMCultureQA

Given an image and a question, spoken or written, systems produce a short open-ended answer. The catch: many answers require local cultural knowledge, not just what is visible in the picture.

Register your team Get the dataset Join the Slack

  • Speech and text questions
  • 18 language tracks
  • Ranked by BERTScore F1
  • Built on the OASIS dataset

The task

Vision alone is not enough

Questions cover food, places, customs, and everyday objects, so the right answer often depends on cultural knowledge rather than plain visual recognition. Every question exists in two forms, audio and text, which gives the shared task its two tasks:

Task 1

Spoken Visual QA

Image + spoken questionShort text answer

Answer a question asked as audio, testing multimodal understanding directly from speech.

Task details →

Task 2

Textual Visual QA

Image + written questionShort text answer

Answer the same question presented as text, with speech recognition taken out of the picture.

Task details →

Each task runs as a separate track per language, and you can enter any subset. See Tasks & Evaluation for the full definition and the OASIS dataset the task is built on.

Languages

Eighteen language tracks

  • العربية الفصحى Modern Standard Arabic
  • العربية الشامية Levantine Arabic
  • العربية المصرية Egyptian Arabic
  • English English
  • বাংলা Bangla
  • اردو Urdu
  • हिन्दी Hindi
  • অসমীয়া Assamese
  • ગુજરાતી Gujarati
  • मराठी Marathi
  • Italiano Italian
  • Español Spanish
  • Português Portuguese
  • Türkçe Turkish
  • አማርኛ Amharic
  • Afaan Oromoo Oromo
  • Soomaali Somali
  • ትግርኛ Tigrinya

Every language is a separate track, and you can enter any subset. How tracks open and how they are scored is on Tasks & Evaluation.

Timeline

Important dates

All dates are tentative and will be confirmed at the data release.

  1. 15 July 2026 · Sample data releaseAvailable →

    A small sample set to preview format and build a pipeline.

  2. 10 January 2027 · Evaluation phase begins

    Test inputs released; submissions open on CodaBench.

  3. 31 January 2027 · Evaluation phase ends

    Final deadline for system submissions.

  4. February 2027 · System description papers due

    Submit your system description paper.

  5. March 2027 · Notification to authors

  6. April 2027 · Camera-ready papers due

  7. Summer 2027 · SemEval 2027 workshop

    Co-located with a major NLP conference.

News

Latest updates

  • 14 September 2026
    Eighteen language tracks planned

    Track coverage now spans Arabic varieties, English, six South Asian languages, Italian, Spanish, Portuguese, Turkish, and four languages of the Horn of Africa. The organizing team has grown to twenty researchers across nine institutions.

  • 14 September 2026
    Training data release date

    The training and development release is being finalised and the date will be announced here and on Slack.

  • 30 June 2026
    Join us on Slack

    We now have a community Slack for announcements, questions, and discussion. Everyone is welcome. Join the Slack.

  • 14 June 2026
    Registration is open

    Teams can now sign up to take part in MMCultureQA. Register here.

  • 13 June 2026
    Shared task website is live

    Explore the task definition, the two tasks, the OASIS dataset, the evaluation plan, important dates, and the organizers.

  • 13 June 2026
    Save the date: sample data on 15 July 2026

    A sample set will be released first so teams can preview the format. Training data follows on 1 September 2026.