Otonota

Tips

Which languages can on-device summary AI handle?

We asked Apple Intelligence, local MLX models and cloud models to summarize and translate an English meeting into 12 languages, and checked whether each answer came back in the language we asked for.

Published Measured:

These are measurements made by the Otonota team, inside Otonota. Results vary with app and engine versions, recording equipment, distance and room, and with how people speak and what they talk about. Treat them as a reference, not a guarantee.

The short answer

  • Starting from an English meeting, every model we tried answered in the requested language almost every time. The few misses were keyword lists left in English.
  • Apple Intelligence does not support Arabic, Russian or Thai as output languages, so it simply cannot produce those. For the other nine languages it was reliable.
  • Direction matters. The same small model that slipped 4 times out of 72 with English input slipped 27 times when the meeting was in Japanese. If your meetings are not in English, see “When the meeting is not in English” below.

What we checked

Otonota can summarize a meeting in a different language from the one being spoken: an English meeting summarized in Vietnamese, a Japanese one in English. Small language models sometimes ignore the requested language and answer in the language of the input instead.

We gave each model a fictional product meeting in English (about 360 words) and asked for three things in each of 12 output languages: a full translation, a one-line summary, and a list of keywords. Each was done twice, once with just the first three lines of the meeting and once with the whole meeting: 6 requests per language, 72 per model. We only checked whether the answer came back in the requested language, not how good the summary was.

iPhone and iPad

On iPhone and iPad you can summarize with Apple Intelligence (free) or the MLX models Qwen3.5 2B and 4B (Pro).

Table 1Models on iPhone and iPad: answers not in the requested language, English meeting summarized and translated into 12 languages
ModelEditionTranslationOne-line summaryKeywordsTotal / requestsUnsupported output languages
Apple IntelligenceFree, default00(best)33 / 54Russian, Arabic, Thai
MLX Qwen3.5 2BPro (6 GB memory or more)0134 / 72none
MLX Qwen3.5 4BPro (8 GB memory or more)00(best)0(best)0 / 72none

A miss is an answer in another language, an answer mixing in another script, or an empty answer. Each request type has 24 requests, minus those in unsupported output languages.

Table 2Models on iPhone and iPad: misses by output language (English meeting as input, out of 6 each)
Output languageApple IntelligenceQwen3.5 2BQwen3.5 4B
Japanese010
English000
Chinese200
Korean000
German000
French000
Spanish000
Portuguese010
Russiann/a00
Arabicn/a00
Thain/a00
Vietnamese120

“n/a” means the model does not support that output language and did not run.

The misses were almost all keyword lists that stayed in English. Here is one, next to a model that got it right.

Apple Intelligence

Otonota, beta test results, testers, keep using the app

MLX Qwen3.5 4B

早安, 每周产品同步会, Onota, 贝塔测试结果, 一百二十名测试者, 百分之八十七保留使用意愿

Fig. 1Keywords requested in Chinese, from the first three lines of the English meeting

Keywords are short noun phrases taken from the input, which makes them the easiest place for the input language to leak through. The 4B list is in Chinese, though it misspells the product name as “Onota”.

Mac

On the Mac you can also use MLX 9B, 14B and 27B (Pro, on Macs with enough memory) and the cloud models Claude, ChatGPT and Gemini (Pro, with your own API key). MLX Qwen3.5 2B is free on the Mac.

Table 3Models on the Mac: answers not in the requested language, English meeting summarized and translated into 12 languages
ModelEditionTranslationOne-line summaryKeywordsTotal / requestsUnsupported output languages
Apple IntelligenceFree, default00(best)33 / 54Russian, Arabic, Thai
MLX Qwen3.5 2BFree0134 / 72none
MLX Qwen3.5 4BPro00(best)0(best)0 / 72none
MLX Qwen3.5 9BPro (16 GB or more)00(best)0(best)0 / 72none
MLX Qwen3 14BPro (24 GB or more)00(best)0(best)0 / 72none
MLX Qwen3.8 27BPro (32 GB or more)00(best)0(best)0 / 72none
Claude Haiku 4.5Pro, own API key00(best)0(best)0 / 72none
Claude Sonnet 5Pro, own API key00(best)0(best)0 / 72none
Claude Opus 5Pro, own API key00(best)0(best)0 / 72none
ChatGPT gpt-5.6 LunaPro, own API key00(best)0(best)0 / 72none
ChatGPT gpt-5.6 TerraPro, own API key00(best)0(best)0 / 72none
ChatGPT gpt-5.6 SolPro, own API key00(best)0(best)0 / 72none
Gemini 3.6 FlashPro, own API key00(best)11 / 72none
Gemini 3.7 FlashPro, own API key00(best)0(best)0 / 72none

With cloud models, the transcript being summarized is sent to that provider.

With English input, every on-device model from 4B up and every cloud model except Gemini 3.6 Flash answered in the requested language every time (figure 2). Gemini 3.6 Flash returned one broken Thai keyword list.

Models on the Mac: share of answers not in the requested language (English meeting as input, lower is better)Apple Intelligence: 5.6%; MLX Qwen3.5 2B: 5.6%; MLX Qwen3.5 4B: 0.0%; MLX Qwen3.5 9B: 0.0%; MLX Qwen3 14B: 0.0%; MLX Qwen3.8 27B: 0.0%; Claude Haiku 4.5: 0.0%; Claude Sonnet 5: 0.0%; Claude Opus 5: 0.0%; ChatGPT gpt-5.6 Luna: 0.0%; ChatGPT gpt-5.6 Terra: 0.0%; ChatGPT gpt-5.6 Sol: 0.0%; Gemini 3.6 Flash: 1.4%; Gemini 3.7 Flash: 0.0%Apple Intelligence: 5.6%Apple Intelligence5.6%MLX Qwen3.5 2B: 5.6%MLX Qwen3.5 2B5.6%MLX Qwen3.5 4B: 0.0%MLX Qwen3.5 4B0.0%MLX Qwen3.5 9B: 0.0%MLX Qwen3.5 9B0.0%MLX Qwen3 14B: 0.0%MLX Qwen3 14B0.0%MLX Qwen3.8 27B: 0.0%MLX Qwen3.8 27B0.0%Claude Haiku 4.5: 0.0%Claude Haiku 4.50.0%Claude Sonnet 5: 0.0%Claude Sonnet 50.0%Claude Opus 5: 0.0%Claude Opus 50.0%ChatGPT gpt-5.6 Luna: 0.0%ChatGPT gpt-5.6 Luna0.0%ChatGPT gpt-5.6 Terra: 0.0%ChatGPT gpt-5.6 Terra0.0%ChatGPT gpt-5.6 Sol: 0.0%ChatGPT gpt-5.6 Sol0.0%Gemini 3.6 Flash: 1.4%Gemini 3.6 Flash1.4%Gemini 3.7 Flash: 0.0%Gemini 3.7 Flash0.0%0%2%4%6%8%10%Models on the Mac: share of answers not in the requested language (English meeting as input, lower is better)Apple Intelligence: 5.6%; MLX Qwen3.5 2B: 5.6%; MLX Qwen3.5 4B: 0.0%; MLX Qwen3.5 9B: 0.0%; MLX Qwen3 14B: 0.0%; MLX Qwen3.8 27B: 0.0%; Claude Haiku 4.5: 0.0%; Claude Sonnet 5: 0.0%; Claude Opus 5: 0.0%; ChatGPT gpt-5.6 Luna: 0.0%; ChatGPT gpt-5.6 Terra: 0.0%; ChatGPT gpt-5.6 Sol: 0.0%; Gemini 3.6 Flash: 1.4%; Gemini 3.7 Flash: 0.0%Apple Intelligence: 5.6%Apple Intelligence5.6%MLX Qwen3.5 2B: 5.6%MLX Qwen3.5 2B5.6%MLX Qwen3.5 4B: 0.0%MLX Qwen3.5 4B0.0%MLX Qwen3.5 9B: 0.0%MLX Qwen3.5 9B0.0%MLX Qwen3 14B: 0.0%MLX Qwen3 14B0.0%MLX Qwen3.8 27B: 0.0%MLX Qwen3.8 27B0.0%Claude Haiku 4.5: 0.0%Claude Haiku 4.50.0%Claude Sonnet 5: 0.0%Claude Sonnet 50.0%Claude Opus 5: 0.0%Claude Opus 50.0%ChatGPT gpt-5.6 Luna: 0.0%ChatGPT gpt-5.6 Luna0.0%ChatGPT gpt-5.6 Terra: 0.0%ChatGPT gpt-5.6 Terra0.0%ChatGPT gpt-5.6 Sol: 0.0%ChatGPT gpt-5.6 Sol0.0%Gemini 3.6 Flash: 1.4%Gemini 3.6 Flash1.4%Gemini 3.7 Flash: 0.0%Gemini 3.7 Flash0.0%0%2%4%6%8%10%Models on the Mac: share of answers not in the requested language (English meeting as input, lower is better)Apple Intelligence: 5.6%; MLX Qwen3.5 2B: 5.6%; MLX Qwen3.5 4B: 0.0%; MLX Qwen3.5 9B: 0.0%; MLX Qwen3 14B: 0.0%; MLX Qwen3.8 27B: 0.0%; Claude Haiku 4.5: 0.0%; Claude Sonnet 5: 0.0%; Claude Opus 5: 0.0%; ChatGPT gpt-5.6 Luna: 0.0%; ChatGPT gpt-5.6 Terra: 0.0%; ChatGPT gpt-5.6 Sol: 0.0%; Gemini 3.6 Flash: 1.4%; Gemini 3.7 Flash: 0.0%Apple Intelligence: 5.6%Apple Intelligence5.6%MLX Qwen3.5 2B: 5.6%MLX Qwen3.5 2B5.6%MLX Qwen3.5 4B: 0.0%MLX Qwen3.5 4B0.0%MLX Qwen3.5 9B: 0.0%MLX Qwen3.5 9B0.0%MLX Qwen3 14B: 0.0%MLX Qwen3 14B0.0%MLX Qwen3.8 27B: 0.0%MLX Qwen3.8 27B0.0%Claude Haiku 4.5: 0.0%Claude Haiku 4.50.0%Claude Sonnet 5: 0.0%Claude Sonnet 50.0%Claude Opus 5: 0.0%Claude Opus 50.0%ChatGPT gpt-5.6 Luna: 0.0%ChatGPT gpt-5.6 Luna0.0%ChatGPT gpt-5.6 Terra: 0.0%ChatGPT gpt-5.6 Terra0.0%ChatGPT gpt-5.6 Sol: 0.0%ChatGPT gpt-5.6 Sol0.0%Gemini 3.6 Flash: 1.4%Gemini 3.6 Flash1.4%Gemini 3.7 Flash: 0.0%Gemini 3.7 Flash0.0%0%2%4%6%8%10%
Fig. 2Models on the Mac: share of answers not in the requested language (English meeting as input, lower is better)

The highlighted bar is the default, Apple Intelligence: share of the 54 requests in the nine output languages it supports.

Show the numbers
ItemValue
Apple Intelligence5.6%
MLX Qwen3.5 2B5.6%
MLX Qwen3.5 4B0.0%
MLX Qwen3.5 9B0.0%
MLX Qwen3 14B0.0%
MLX Qwen3.8 27B0.0%
Claude Haiku 4.50.0%
Claude Sonnet 50.0%
Claude Opus 50.0%
ChatGPT gpt-5.6 Luna0.0%
ChatGPT gpt-5.6 Terra0.0%
ChatGPT gpt-5.6 Sol0.0%
Gemini 3.6 Flash1.4%
Gemini 3.7 Flash0.0%
Table 4On-device models on the Mac: misses by output language (English meeting as input, out of 6 each)
Output languageApple IntelligenceQwen3.5 2BQwen3.5 4BQwen3.5 9BQwen3 14BQwen3.8 27B
Japanese010000
English000000
Chinese200000
Korean000000
German000000
French000000
Spanish000000
Portuguese010000
Russiann/a00000
Arabicn/a00000
Thain/a00000
Vietnamese120000

Cloud models are left out; table 3 shows they almost never missed.

When the meeting is not in English

We ran the same test with the meeting in Japanese. There the small models had a much harder time: MLX Qwen3.5 2B answered in Japanese, or mixed Japanese in, 27 times out of 72, mostly for one-line summaries and keywords in French, Spanish, Portuguese, Arabic, Thai and Vietnamese. Larger models slipped less: 4B and 9B missed 8 and 7 times, 14B and 27B only 3 and 2. Apple Intelligence stayed reliable in the languages it supports. If you summarize non-English meetings into a third language, prefer one of those.

Which one to choose

iPhone and iPad

  • Summaries in English and other major languages — the default Apple Intelligence is enough, and free
  • Summaries in Arabic, Russian or Thai — Apple Intelligence cannot write these. Switch to an MLX model (Pro); with English meetings both 2B and 4B answered in the requested language every time

Mac

  • Summaries in English and other major languages — the default Apple Intelligence is enough, and free
  • Summaries in Arabic, Russian or Thai — use an MLX model (Pro) or a cloud model (Pro, your own API key)
  • A Mac without Apple Intelligence — the free MLX Qwen3.5 2B fills in. It is fine for English meetings; for meetings in other languages, a 9B or larger model is safer

Summary language and format settings are covered in Summaries and translation (Mac) andSummaries and the glossary (iPhone and iPad); model sizes and memory inChoosing engines. For speech recognition, see Which transcription engine is most accurate for meetings?

For our recommendation by device and situation on one page, see Recommended settings at a glance.

How we measured

Input
A fictional launch-preparation meeting for a product called “Otonota” (359 words, 2,126 characters). Either the first three lines (239 characters) or the whole meeting
Requests
Full translation, one-line summary and keywords (up to 10) × 12 output languages (Japanese, English, Chinese, Korean, German, French, Spanish, Portuguese, Russian, Arabic, Thai, Vietnamese)
Models
On device: Apple Intelligence (Apple Foundation Models); MLX Qwen3.5 2B / 4B / 9B, Qwen3 14B and Qwen3.8 27B, all 4-bit. Cloud: Claude, ChatGPT and Gemini models. Prompts as in Otonota 1.1.4
Judgement
We counted the share of each script (kana, Han, Hangul, Cyrillic, Arabic, Thai, Latin) in the answer. An answer missed if the requested script was not dominant, if another script made up more than 5% (8% for kana in Chinese or Korean; a product name left in its original script is allowed), or if it was empty. Between Latin-script languages (Portuguese requested, English returned) we used Apple's language identifier, NLLanguageRecognizer
Hardware
MacBook Pro (M4 Max, 36 GB, macOS 27.3)
Measured
20 November 2026

In the Japanese-input run, one full translation by MLX Qwen3.8 27B did not finish within the 120-second limit and is not counted. We judged language only. A summary in the right language can still get a number wrong or leave something out. One script and one topic; other meetings will give different counts.

Questions and answers

Can Apple Intelligence summarize an English meeting in another language?
Yes. In the 9 of our 12 output languages that Apple Intelligence supports, it answered in the requested language in all but 3 of 54 requests (keyword lists left in English). It does not support Arabic, Russian or Thai.
How do I get summaries in a language Apple Intelligence does not support?
Switch to a local MLX model (Pro). With English meetings, both MLX Qwen3.5 2B and 4B answered in Arabic, Russian and Thai every time. On the Mac you can also use a cloud model (Claude, ChatGPT or Gemini) with your own API key.
Do small local language models answer in the wrong language?
Rarely with English input: MLX Qwen3.5 2B missed 4 of 72 requests. With a Japanese meeting as input the same model missed 27 of 72, mostly by answering in Japanese. For non-English meetings, larger models are safer.
Do ChatGPT, Claude and Gemini get the language right?
With English input, all cloud models answered in the requested language every time except Gemini 3.6 Flash, which returned one broken Thai keyword list. Cloud models are available only in the Mac app with Pro and your own API key.
Is translation more reliable than summaries?
Yes. Full translations came back in the requested language from every model. The misses were one-line summaries and keyword lists, which reuse words from the input and so let the input language leak through.
Did you check whether the summaries were correct?
No. We checked only whether each answer came back in the requested language. An answer in the right language can still get a number wrong or leave something out.