How do we detect language bias in an AI answer?
AI systems can answer differently depending on the language of the question. This challenge asks you to design a small, fair comparison to see whether answers in two languages differ in quality, accuracy or tone.
Why it matters
If a system serves some languages better than others, speakers of those languages receive unequal access to knowledge. Careful testing helps users and developers notice and report such gaps.
Steps
- Write five factual questions with verifiable answers, in two languages, with the same meaning.
- Ask the same AI tool each question in both languages under the same conditions and record the answers.
- Check each answer against a reliable source and score accuracy, completeness and tone.
- Summarise the differences and note how many questions would be needed for a stronger conclusion.
Expected output
A comparison table with questions, answers, sources and scores, and a cautious summary of what the small sample shows.
Assessment criteria
- The question pairs are equivalent in meaning.
- Accuracy is checked against cited sources.
- Conclusions acknowledge the small sample size.
Safety and ethics
Avoid questions about real private individuals and do not share answers that could spread misinformation without correction.
