Tips
Which languages can on-device summary AI handle?
We asked Apple Intelligence, local MLX models and cloud models to summarize and translate an English meeting into 12 languages, and checked whether each answer came back in the language we asked for.
These are measurements made by the Otonota team, inside Otonota. Results vary with app and engine versions, recording equipment, distance and room, and with how people speak and what they talk about. Treat them as a reference, not a guarantee.
The short answer
- Starting from an English meeting, every model we tried answered in the requested language almost every time. The few misses were keyword lists left in English.
- Apple Intelligence does not support Arabic, Russian or Thai as output languages, so it simply cannot produce those. For the other nine languages it was reliable.
- Direction matters. The same small model that slipped 4 times out of 72 with English input slipped 27 times when the meeting was in Japanese. If your meetings are not in English, see “When the meeting is not in English” below.
What we checked
Otonota can summarize a meeting in a different language from the one being spoken: an English meeting summarized in Vietnamese, a Japanese one in English. Small language models sometimes ignore the requested language and answer in the language of the input instead.
We gave each model a fictional product meeting in English (about 360 words) and asked for three things in each of 12 output languages: a full translation, a one-line summary, and a list of keywords. Each was done twice, once with just the first three lines of the meeting and once with the whole meeting: 6 requests per language, 72 per model. We only checked whether the answer came back in the requested language, not how good the summary was.
iPhone and iPad
On iPhone and iPad you can summarize with Apple Intelligence (free) or the MLX models Qwen3.5 2B and 4B (Pro).
| Model | Edition | Translation | One-line summary | Keywords | Total / requests | Unsupported output languages |
|---|---|---|---|---|---|---|
| Apple Intelligence | Free, default | 0 | 0(best) | 3 | 3 / 54 | Russian, Arabic, Thai |
| MLX Qwen3.5 2B | Pro (6 GB memory or more) | 0 | 1 | 3 | 4 / 72 | none |
| MLX Qwen3.5 4B | Pro (8 GB memory or more) | 0 | 0(best) | 0(best) | 0 / 72 | none |
A miss is an answer in another language, an answer mixing in another script, or an empty answer. Each request type has 24 requests, minus those in unsupported output languages.
| Output language | Apple Intelligence | Qwen3.5 2B | Qwen3.5 4B |
|---|---|---|---|
| Japanese | 0 | 1 | 0 |
| English | 0 | 0 | 0 |
| Chinese | 2 | 0 | 0 |
| Korean | 0 | 0 | 0 |
| German | 0 | 0 | 0 |
| French | 0 | 0 | 0 |
| Spanish | 0 | 0 | 0 |
| Portuguese | 0 | 1 | 0 |
| Russian | n/a | 0 | 0 |
| Arabic | n/a | 0 | 0 |
| Thai | n/a | 0 | 0 |
| Vietnamese | 1 | 2 | 0 |
“n/a” means the model does not support that output language and did not run.
The misses were almost all keyword lists that stayed in English. Here is one, next to a model that got it right.
Apple Intelligence
Otonota, beta test results, testers, keep using the app
MLX Qwen3.5 4B
早安, 每周产品同步会, Onota, 贝塔测试结果, 一百二十名测试者, 百分之八十七保留使用意愿
Keywords are short noun phrases taken from the input, which makes them the easiest place for the input language to leak through. The 4B list is in Chinese, though it misspells the product name as “Onota”.
Mac
On the Mac you can also use MLX 9B, 14B and 27B (Pro, on Macs with enough memory) and the cloud models Claude, ChatGPT and Gemini (Pro, with your own API key). MLX Qwen3.5 2B is free on the Mac.
| Model | Edition | Translation | One-line summary | Keywords | Total / requests | Unsupported output languages |
|---|---|---|---|---|---|---|
| Apple Intelligence | Free, default | 0 | 0(best) | 3 | 3 / 54 | Russian, Arabic, Thai |
| MLX Qwen3.5 2B | Free | 0 | 1 | 3 | 4 / 72 | none |
| MLX Qwen3.5 4B | Pro | 0 | 0(best) | 0(best) | 0 / 72 | none |
| MLX Qwen3.5 9B | Pro (16 GB or more) | 0 | 0(best) | 0(best) | 0 / 72 | none |
| MLX Qwen3 14B | Pro (24 GB or more) | 0 | 0(best) | 0(best) | 0 / 72 | none |
| MLX Qwen3.8 27B | Pro (32 GB or more) | 0 | 0(best) | 0(best) | 0 / 72 | none |
| Claude Haiku 4.5 | Pro, own API key | 0 | 0(best) | 0(best) | 0 / 72 | none |
| Claude Sonnet 5 | Pro, own API key | 0 | 0(best) | 0(best) | 0 / 72 | none |
| Claude Opus 5 | Pro, own API key | 0 | 0(best) | 0(best) | 0 / 72 | none |
| ChatGPT gpt-5.6 Luna | Pro, own API key | 0 | 0(best) | 0(best) | 0 / 72 | none |
| ChatGPT gpt-5.6 Terra | Pro, own API key | 0 | 0(best) | 0(best) | 0 / 72 | none |
| ChatGPT gpt-5.6 Sol | Pro, own API key | 0 | 0(best) | 0(best) | 0 / 72 | none |
| Gemini 3.6 Flash | Pro, own API key | 0 | 0(best) | 1 | 1 / 72 | none |
| Gemini 3.7 Flash | Pro, own API key | 0 | 0(best) | 0(best) | 0 / 72 | none |
With cloud models, the transcript being summarized is sent to that provider.
With English input, every on-device model from 4B up and every cloud model except Gemini 3.6 Flash answered in the requested language every time (figure 2). Gemini 3.6 Flash returned one broken Thai keyword list.
The highlighted bar is the default, Apple Intelligence: share of the 54 requests in the nine output languages it supports.
Show the numbers
| Item | Value |
|---|---|
| Apple Intelligence | 5.6% |
| MLX Qwen3.5 2B | 5.6% |
| MLX Qwen3.5 4B | 0.0% |
| MLX Qwen3.5 9B | 0.0% |
| MLX Qwen3 14B | 0.0% |
| MLX Qwen3.8 27B | 0.0% |
| Claude Haiku 4.5 | 0.0% |
| Claude Sonnet 5 | 0.0% |
| Claude Opus 5 | 0.0% |
| ChatGPT gpt-5.6 Luna | 0.0% |
| ChatGPT gpt-5.6 Terra | 0.0% |
| ChatGPT gpt-5.6 Sol | 0.0% |
| Gemini 3.6 Flash | 1.4% |
| Gemini 3.7 Flash | 0.0% |
| Output language | Apple Intelligence | Qwen3.5 2B | Qwen3.5 4B | Qwen3.5 9B | Qwen3 14B | Qwen3.8 27B |
|---|---|---|---|---|---|---|
| Japanese | 0 | 1 | 0 | 0 | 0 | 0 |
| English | 0 | 0 | 0 | 0 | 0 | 0 |
| Chinese | 2 | 0 | 0 | 0 | 0 | 0 |
| Korean | 0 | 0 | 0 | 0 | 0 | 0 |
| German | 0 | 0 | 0 | 0 | 0 | 0 |
| French | 0 | 0 | 0 | 0 | 0 | 0 |
| Spanish | 0 | 0 | 0 | 0 | 0 | 0 |
| Portuguese | 0 | 1 | 0 | 0 | 0 | 0 |
| Russian | n/a | 0 | 0 | 0 | 0 | 0 |
| Arabic | n/a | 0 | 0 | 0 | 0 | 0 |
| Thai | n/a | 0 | 0 | 0 | 0 | 0 |
| Vietnamese | 1 | 2 | 0 | 0 | 0 | 0 |
Cloud models are left out; table 3 shows they almost never missed.
When the meeting is not in English
We ran the same test with the meeting in Japanese. There the small models had a much harder time: MLX Qwen3.5 2B answered in Japanese, or mixed Japanese in, 27 times out of 72, mostly for one-line summaries and keywords in French, Spanish, Portuguese, Arabic, Thai and Vietnamese. Larger models slipped less: 4B and 9B missed 8 and 7 times, 14B and 27B only 3 and 2. Apple Intelligence stayed reliable in the languages it supports. If you summarize non-English meetings into a third language, prefer one of those.
Which one to choose
iPhone and iPad
- Summaries in English and other major languages — the default Apple Intelligence is enough, and free
- Summaries in Arabic, Russian or Thai — Apple Intelligence cannot write these. Switch to an MLX model (Pro); with English meetings both 2B and 4B answered in the requested language every time
Mac
- Summaries in English and other major languages — the default Apple Intelligence is enough, and free
- Summaries in Arabic, Russian or Thai — use an MLX model (Pro) or a cloud model (Pro, your own API key)
- A Mac without Apple Intelligence — the free MLX Qwen3.5 2B fills in. It is fine for English meetings; for meetings in other languages, a 9B or larger model is safer
Summary language and format settings are covered in Summaries and translation (Mac) andSummaries and the glossary (iPhone and iPad); model sizes and memory inChoosing engines. For speech recognition, see Which transcription engine is most accurate for meetings?
For our recommendation by device and situation on one page, see Recommended settings at a glance.
How we measured
- Input
- A fictional launch-preparation meeting for a product called “Otonota” (359 words, 2,126 characters). Either the first three lines (239 characters) or the whole meeting
- Requests
- Full translation, one-line summary and keywords (up to 10) × 12 output languages (Japanese, English, Chinese, Korean, German, French, Spanish, Portuguese, Russian, Arabic, Thai, Vietnamese)
- Models
- On device: Apple Intelligence (Apple Foundation Models); MLX Qwen3.5 2B / 4B / 9B, Qwen3 14B and Qwen3.8 27B, all 4-bit. Cloud: Claude, ChatGPT and Gemini models. Prompts as in Otonota 1.1.4
- Judgement
- We counted the share of each script (kana, Han, Hangul, Cyrillic, Arabic, Thai, Latin) in the answer. An answer missed if the requested script was not dominant, if another script made up more than 5% (8% for kana in Chinese or Korean; a product name left in its original script is allowed), or if it was empty. Between Latin-script languages (Portuguese requested, English returned) we used Apple's language identifier, NLLanguageRecognizer
- Hardware
- MacBook Pro (M4 Max, 36 GB, macOS 27.3)
- Measured
- 20 November 2026
In the Japanese-input run, one full translation by MLX Qwen3.8 27B did not finish within the 120-second limit and is not counted. We judged language only. A summary in the right language can still get a number wrong or leave something out. One script and one topic; other meetings will give different counts.
Questions and answers
- Can Apple Intelligence summarize an English meeting in another language?
- Yes. In the 9 of our 12 output languages that Apple Intelligence supports, it answered in the requested language in all but 3 of 54 requests (keyword lists left in English). It does not support Arabic, Russian or Thai.
- How do I get summaries in a language Apple Intelligence does not support?
- Switch to a local MLX model (Pro). With English meetings, both MLX Qwen3.5 2B and 4B answered in Arabic, Russian and Thai every time. On the Mac you can also use a cloud model (Claude, ChatGPT or Gemini) with your own API key.
- Do small local language models answer in the wrong language?
- Rarely with English input: MLX Qwen3.5 2B missed 4 of 72 requests. With a Japanese meeting as input the same model missed 27 of 72, mostly by answering in Japanese. For non-English meetings, larger models are safer.
- Do ChatGPT, Claude and Gemini get the language right?
- With English input, all cloud models answered in the requested language every time except Gemini 3.6 Flash, which returned one broken Thai keyword list. Cloud models are available only in the Mac app with Pro and your own API key.
- Is translation more reliable than summaries?
- Yes. Full translations came back in the requested language from every model. The misses were one-line summaries and keyword lists, which reuse words from the input and so let the input language leak through.
- Did you check whether the summaries were correct?
- No. We checked only whether each answer came back in the requested language. An answer in the right language can still get a number wrong or leave something out.
