There are thousands of languages in the world, but AI systems work well in only a small fraction of them. Language models, speech recognition and translation tools are strongest in English and a handful of widely spoken languages with abundant digital text. For billions of people, this creates an AI language divide: tools that are less accurate, less safe and less useful in their own languages.

Why the gap exists

AI learns from data. The amount of digital text varies enormously between languages:

  • High-resource languages — English, Chinese, Spanish, French, German and others — have vast amounts of web text, books and labeled datasets.
  • Low-resource languages — including many spoken by millions of people in Africa, South Asia and elsewhere — have far less digitized text, few annotated datasets and limited speech data.

A 2020 study by Pratik Joshi and colleagues classified the world’s languages by available resources and found that most languages fall into categories with very little data and attention from the research community.

The consequences

  • Lower quality: models make more errors, misunderstand context and produce awkward text.
  • Safety gaps: safety training and content moderation often work less well outside major languages, so harmful outputs may slip through and jailbreaks may be easier.
  • Higher costs: tokenizers designed around English often split other languages into more tokens, making the same content more expensive to process and slower.
  • Exclusion from services: as AI is integrated into education, healthcare and government, speakers of underserved languages risk worse services.
  • Cultural flattening: models may reflect English-speaking perspectives even when answering in other languages.

Efforts to close the gap

  • Large multilingual projects: Meta’s No Language Left Behind (2022) produced a translation model covering 200 languages, with new datasets and evaluation benchmarks for many low-resource languages.
  • Community-led research: grassroots networks such as Masakhane, focused on African languages, build datasets and models with native speakers.
  • National and regional models: governments and institutions fund models for local languages — part of sovereign AI strategies.
  • Speech data collection: open projects collect recordings from volunteers to improve speech recognition for more languages.
  • Better tokenizers and multilingual training in newer models.

What can help

  • Involve native speakers in data creation and evaluation — automatic translation of English benchmarks misses cultural context.
  • Fund the creation of openly licensed datasets.
  • Evaluate models per language and publish results.
  • Respect communities’ rights over their language data.

What individuals and organizations can do

  • Test tools in your language before relying on them, especially for health, legal or financial information, and compare answers with those in a major language.
  • Contribute data responsibly: community projects collect speech and text in underrepresented languages with consent and fair credit, such as open speech datasets built by volunteers.
  • Support local developers and researchers building models and benchmarks for regional languages.
  • Ask vendors which languages they evaluate, and how performance differs.
  • Use human translators for important communications where AI quality is uncertain.

Closing the language gap is not only a technical problem: it requires funding, local ownership of data and models, and evaluation by native speakers who understand cultural context.

Related: sovereign AI explained and AI for smallholder farmers, where local languages matter.

Bottom line

Language access is a question of fairness: better data, local ownership and native-speaker evaluation can close the gap.

What to watch next

  • Coverage and quality of major models in African, South Asian and Indigenous languages.
  • Open datasets created with speaker communities.
  • Tokenizer improvements that reduce cost disparities across languages.
  • Language-specific safety evaluations.

Key takeaways

  • AI performs best in a few high-resource languages with abundant digital text.
  • Most of the world’s languages have little data and research attention.
  • Speakers of underserved languages get lower quality, weaker safety and higher costs.
  • Projects like No Language Left Behind and Masakhane work to close the gap.
  • Native-speaker involvement and per-language evaluation are essential.

Frequently asked questions

How many languages do AI models support?

Leading models handle dozens of languages well and many more with varying quality. Coverage drops sharply for low-resource languages.

Why do some languages cost more to process?

Tokenizers trained mostly on English may break other languages into more tokens, increasing the cost and reducing the effective context length.

Can translation solve the problem?

Translation helps but loses nuance and cultural context; models trained with native data serve speakers better.