Bangla culture is richly expressed through region, dialect, history, food, politics, media, and everyday visual life, yet it remains underrepresented in multimodal evaluation. To address this gap, we introduce BanglaVerse, a culturally grounded benchmark for evaluating multilingual visionâlanguage models (VLMs) on Bengali culture across historically linked languages and regional dialects. Built from 1,152 manually curated images across nine domains, the benchmark supports visual question answering and captioning, and is expanded into four languages and five Bangla dialects, yielding âŧ32.3K artifacts. Our experiments show that evaluating only standard Bangla overestimates true model capability: performance drops under dialectal variation, especially for caption generation, while historically linked languages such as Hindi and Urdu retain some cultural meaning but remain weaker for structured reasoning. Across domains, the main bottleneck is missing cultural knowledge rather than visual grounding alone, with knowledge-intensive categories. These findings position BanglaVerse as a more realistic test bed for measuring culturally grounded multimodal understanding under linguistic variation.
| Category | Type | Count | Category | Type | Count | Category | Type | Count |
|---|---|---|---|---|---|---|---|---|
| Cult. | Img. | 114 | M&M | Img. | 150 | Pers. | Img. | 150 |
| Cap. | 1,026 | Cap. | 1,350 | Cap. | 1,350 | |||
| VQA | 2,052 | VQA | 2,592 | VQA | 2,700 | |||
| Total | 3,192 | Total | 4,092 | Total | 4,200 | |||
| Food | Img. | 150 | Nat. Achv. | Img. | 75 | Pol. | Img. | 150 |
| Cap. | 1,350 | Cap. | 675 | Cap. | 1,350 | |||
| VQA | 2,709 | VQA | 1,332 | VQA | 2,754 | |||
| Total | 4,209 | Total | 2,082 | Total | 4,254 | |||
| Hist. | Img. | 150 | Nature | Img. | 150 | Sports | Img. | 63 |
| Cap. | 1,350 | Cap. | 1,350 | Cap. | 576 | |||
| VQA | 2,772 | VQA | 2,691 | VQA | 1,125 | |||
| Total | 4,272 | Total | 4,191 | Total | 1,764 | |||
| Total Images | 1,152 | |||||||
| Total Captions Across 9 Varieties | 10,377 | |||||||
| Total VQA Across 9 Varieties | 20,727 | |||||||
| Grand Total Artifacts | 32,256 | |||||||
| Dialect/Language | Caption | VQA |
|---|---|---|
| English | The farmer in the picture is plowing the land with an ox-drawn plow, a tradition of rural agriculture. |
Question: What work is being done using the cow in the picture? Options: A) Paddy threshing B) Land cultivation C) Grain reserve D) Cattle feed preparation Answer: Land cultivation |
| Bangla | āĻāĻŦāĻŋāϤ⧠āĻā§āώāĻ āĻāϰā§āϰ āĻšāĻžāϞ āĻĻāĻŋāϝāĻŧā§ āĻāĻŽāĻŋ āĻāĻžāώ āĻāϰāĻā§āύ, āϝāĻž āĻā§āϰāĻžāĻŽā§āĻŖ āĻā§āώāĻŋāϰ āĻāϤāĻŋāĻšā§āϝāĨ¤ |
Question: āĻāĻŦāĻŋāϤ⧠āĻāϰ⧠āĻŦā§āϝāĻŦāĻšāĻžāϰ āĻāϰ⧠āĻā§ āĻāĻžāĻ āĻāϰāĻž āĻšāĻā§āĻā§? Options: A) āϧāĻžāύ āĻŽāĻžāĻĄāĻŧāĻžāĻ B) āĻāĻŽāĻŋ āĻāĻžāώ C) āĻļāϏā§āϝ āĻŽāĻā§āĻĻ D) āĻā§-āĻāĻžāĻĻā§āϝ āĻĒā§āϰāϏā§āϤā§āϤ Answer: āĻāĻŽāĻŋ āĻāĻžāώ |
| Hindi | ā¤ā¤ŋ⤤āĨ⤰ ā¤ŽāĨ⤠ā¤ā¤ŋā¤¸ā¤žā¤¨ ā¤ŦāĨ⤞ ā¤šā¤˛ ⤏āĨ ā¤ā¤ŧā¤ŽāĨ⤍ ā¤āĨ⤤ ā¤°ā¤šāĨ ā¤šāĨā¤, ā¤āĨ ā¤āĨā¤°ā¤žā¤ŽāĨ⤪ ā¤āĨ⤎ā¤ŋ ā¤āĨ ā¤Ē⤰ā¤ā¤Ēā¤°ā¤ž ā¤šāĨāĨ¤ |
Question: ā¤ā¤ŋ⤤āĨ⤰ ā¤ŽāĨ⤠ā¤ŦāĨ⤞ ā¤ā¤ž ā¤ā¤Ē⤝āĨ⤠ā¤ā¤°ā¤āĨ ā¤āĨ⤍ ā¤¸ā¤ž ā¤ā¤žā¤°āĨ⤝ ā¤ā¤ŋā¤¯ā¤ž ā¤ā¤ž ā¤°ā¤šā¤ž ā¤šāĨ? Options: A) ā¤§ā¤žā¤¨ ā¤āĨ ā¤Žā¤Ąā¤ŧā¤žā¤ B) ā¤āĨ⤤ ā¤āĨā¤¤ā¤¨ā¤ž C) ā¤ ā¤¨ā¤žā¤ ā¤ā¤ā¤Ąā¤žā¤°ā¤Ŗ D) ā¤āĨ-ā¤ā¤šā¤žā¤° ⤤āĨā¤¯ā¤žā¤° ā¤ā¤°ā¤¨ā¤ž Answer: ā¤āĨ⤤ ā¤āĨā¤¤ā¤¨ā¤ž |
| Urdu | ØĒØĩŲÛØą Ų ÛÚē ÚŠØŗØ§Ų Ø¨ÛŲŲÚē ØŗÛ ÛŲ ÚŲØ§ ÚŠØą Ø˛Ų ÛŲ ØĒÛØ§Øą ÚŠØą ØąÛØ§ ÛÛØ ØŦŲ Ø¯ÛÛÛ Ø˛ØąØ§ØšØĒ ڊا اÛÚŠ ØąŲØ§ÛØĒÛ ØˇØąÛŲÛ ÛÛÛ |
Question: ØĒØĩŲÛØą Ų
ÛÚē گاØĻÛ ÚŠØ§ ÚŠÛØ§ Ø§ØŗØĒØšŲ
Ø§Ų ÛŲ ØąÛØ§ ÛÛØ Options: A) Ø¯ÚžØ§Ų ÚŠÛ Ú¯ÛØ§ØĻÛ B) ڊاشØĒŲ Ø˛Ų ÛŲ C) ØēŲÛ ÚŠØ§ Ø°ØŽÛØąÛ D) Ú¯ÚžØ§Øŗ ÚŠÛ ØĒÛØ§ØąÛ Answer: ڊاشØĒŲ Ø˛Ų ÛŲ |
| Barishal | āĻāĻŦāĻŋāĻĄāĻžā§ āĻāĻāĻā§āĻāĻž āĻāĻŋāώāĻžāύ āĻāϰā§āϰ āĻšāĻžāϞ āĻĻāĻŋā§āĻž āĻā§āĻāĻ āĻāώāϤ⧠āĻāĻā§, āϝā§āĻāĻĄāĻž āĻā§āϰāĻžāĻŽā§āϰ āĻā§āώāĻŋāϰ āĻĒā§āϰāĻžāύ āύāĻŋā§āĻŽāĨ¤ |
Question: āĻāĻŦāĻŋāĻĄāĻžā§ āĻāϰ⧠āĻĻāĻŋā§āĻž āĻā§ āĻāĻžāĻŽ āĻāϰāĻž āĻ
āĻāϤā§āĻā§? Options: A) āϧāĻžāύ āĻŽāĻžā§āĻžāĻ B) āĻā§āĻāĻ āĻāώāĻž C) āĻļāϏā§āϝ āĻĨā§āĻā§āĻž D) āĻāϰā§āϰ āĻāĻžāĻāύ āĻŦāĻžāύāĻžāύ⧠Answer: B) āĻā§āĻāĻ āĻāώāĻž |
| Chittagong | āĻāĻŦāĻŋāϤ āĻāĻā§āĻā§ā§āĻž āĻāĻŋāώāĻžāύ āĻāϰā§āϰ āĻšāĻžāϞ āĻĻāĻŋā§ā§āύ⧠āĻāĻāĻŽ āĻāώā§āϰ, āϝāĻŋā§āĻžāύ āĻāĻžāĻā§āϰ āĻā§āώāĻŋāϰ āĻĒā§āϰāĻžāύāĻž āύāĻŋā§āĻŽāĨ¤ |
Question: āĻāĻŦāĻŋāϤ āĻāϰ⧠āĻĻāĻŋā§ā§āύ⧠āĻā§ āĻāĻžāĻŽ āĻāϰā§āϰ? Options: A) āϧāĻžāύ āĻŽāĻžā§āĻžāĻ B) āĻāĻāĻŽ āĻāώāĻž C) āĻļāϏā§āϝ āϤāĻšāύ D) āĻāϰā§āϰ āĻšāĻžāύāĻž āĻŦāĻžāύāĻžāύ Answer: B) āĻāĻāĻŽ āĻāώāĻž |
| Noakhali | āĻāĻŦāĻŋāĻĄāĻžāϤ āĻāĻāĻāύ āĻāĻŋāώāĻžāύ āĻāϰā§āϰ āĻšāĻžāϞ āĻĻāĻŋ āĻāĻŽāĻŋ āĻāĻžāώ āĻāϰā§āϰ, āĻšā§āĻāĻĄāĻž āĻā§āϰāĻžāĻŽā§āϰ āĻā§āώāĻŋāϰ āĻĒā§āϰāĻžāύāĻž āύāĻŋā§āĻŽāĨ¤ |
Question: āĻāĻŦāĻŋāĻĄāĻžāϤ āĻāϰ⧠āĻĻāĻŋ āĻā§ āĻāĻžāĻŽ āĻāϰāĻž āĻšāύā§āύā§āϰ? Options: A) āϧāĻžāύ āĻŽāĻžā§āĻžāĻ B) āĻāĻŽāĻŋ āĻāĻžāώ C) āĻļāϏā§āϝ āϰāĻžāĻšāύ D) āĻāϰā§āϰ āĻāĻžāύāĻž āĻŦāĻžāύāĻžāύ Answer: B) āĻāĻŽāĻŋ āĻāĻžāώ |
| Rangpur | āĻāĻŦāĻŋāϤ āĻāĻāύāĻž āĻāĻŋāώāĻžāύ āĻāϰā§āϰ āĻšāĻžāϞ āĻĻāĻŋā§āĻž āĻāĻŽāĻŋāϤ āĻšāĻžāϞ āĻŦāĻžāĻāĻŦāĻžāϰ āύāĻžāĻāĻā§, āϝā§āĻāĻāĻž āĻāĻžāĻā§ā§āϰ āĻā§āώāĻŋāϰ āĻĒā§āϰāĻžāύāĻž āύāĻŋā§āĻŽāĨ¤ |
Question: āĻāĻŦāĻŋāϤ āĻāϰ⧠āĻĻāĻŋā§āĻž āĻā§ āĻāĻžāĻŽ āĻāϰāĻž āĻšāĻžāĻāĻŦāĻžāϰ āύāĻžāĻāĻā§? Options: A) āϧāĻžāύ āĻŽāĻžā§āĻžāĻ B) āĻāĻŽāĻŋāϤ āĻšāĻžāϞ āĻŦāĻžāĻā§āĻž C) āĻļāϏā§āϝ āĻĨā§āĻā§āĻž D) āĻāϰā§āϰ āĻāĻžāĻŦāĻžāϰ āĻŦāĻžāύāĻžāύ Answer: B) āĻāĻŽāĻŋāϤ āĻšāĻžāϞ āĻŦāĻžāĻā§āĻž |
| Sylhet | āĻāĻŦāĻŋāϤ āĻāĻāĻāύ āĻāĻŋāώāĻžāύ āĻāϰā§āϰ āĻšāĻžāϞ āĻĻāĻŋā§āĻž āĻā§āϤ āĻāĻžāώ āĻāϰāϰāĻž, āϝā§āϤāĻž āĻāĻžāĻāϰ āĻā§āώāĻŋāϰ āĻĒā§āϰāĻžāύāĻž āύāĻŋā§āĻŽāĨ¤ |
Question: āĻāĻŦāĻŋāϤ āĻāϰ⧠āĻĻāĻŋā§āĻž āĻāĻŋāϤāĻž āĻāĻžāĻŽ āĻāϰāĻž āĻ
āϰ? Options: A) āϧāĻžāύ āĻŽāĻžā§āĻžāĻ B) āĻā§āϤ āĻāĻžāώ C) āĻļāϏā§āϝ āϤāĻā§āĻž D) āĻāϰā§āϰ āĻāĻžāύāĻŋ āĻŦāĻžāύāĻžāύ Answer: B) āĻā§āϤ āĻāĻžāώ |
| Models | Domain | Zero-Shot | Few-Shot | CoT VQA | ||||
|---|---|---|---|---|---|---|---|---|
| Caption | VQA | Caption | VQA | |||||
| B-F1 | LLM | Accuracy (%) | B-F1 | LLM | Accuracy (%) | |||
Gemma3-4B |
Cult. | 0.69 | 0.53 | 0.63 | 0.69 | 0.52 | 0.59 | 0.52 |
| Food | 0.70 | 0.52 | 0.74 | 0.70 | 0.53 | 0.63 | 0.78 | |
| Hist. | 0.69 | 0.54 | 0.47 | 0.68 | 0.47 | 0.37 | 0.44 | |
| M&M | 0.66 | 0.50 | 0.30 | 0.84 | 0.52 | 0.32 | 0.38 | |
| Nat. Achv. | 0.70 | 0.56 | 0.31 | 0.67 | 0.50 | 0.29 | 0.36 | |
| Nature | 0.70 | 0.58 | 0.66 | 0.69 | 0.54 | 0.74 | 0.59 | |
| Pers. | 0.65 | 0.45 | 0.61 | 0.69 | 0.49 | 0.62 | 0.49 | |
| Pol. | 0.67 | 0.50 | 0.39 | 0.69 | 0.48 | 0.40 | 0.39 | |
| Sports | 0.68 | 0.52 | 0.44 | 0.68 | 0.51 | 0.26 | 0.48 | |
Gemma3-12B |
Cult. | 0.70 | 0.55 | 0.68 | 0.69 | 0.52 | 0.70 | 0.72 |
| Food | 0.71 | 0.53 | 0.83 | 0.71 | 0.52 | 0.82 | 0.89 | |
| Hist. | 0.69 | 0.54 | 0.62 | 0.68 | 0.48 | 0.63 | 0.75 | |
| M&M | 0.66 | 0.51 | 0.40 | 0.78 | 0.52 | 0.32 | 0.33 | |
| Nat. Achv. | 0.69 | 0.56 | 0.49 | 0.66 | 0.49 | 0.50 | 0.59 | |
| Nature | 0.70 | 0.58 | 0.63 | 0.69 | 0.56 | 0.59 | 0.66 | |
| Pers. | 0.64 | 0.49 | 0.67 | 0.68 | 0.48 | 0.70 | 0.74 | |
| Pol. | 0.66 | 0.51 | 0.55 | 0.68 | 0.48 | 0.56 | 0.61 | |
| Sports | 0.68 | 0.57 | 0.53 | 0.67 | 0.48 | 0.47 | 0.46 | |
Gemma3-27B |
Cult. | 0.67 | 0.44 | 0.72 | 0.70 | 0.53 | 0.73 | 0.73 |
| Food | 0.71 | 0.58 | 0.87 | 0.72 | 0.55 | 0.89 | 0.89 | |
| Hist. | 0.70 | 0.56 | 0.69 | 0.69 | 0.51 | 0.69 | 0.74 | |
| M&M | 0.67 | 0.47 | 0.36 | 0.82 | 0.53 | 0.32 | 0.42 | |
| Nat. Achv. | 0.71 | 0.61 | 0.51 | 0.67 | 0.50 | 0.51 | 0.78 | |
| Nature | 0.70 | 0.57 | 0.66 | 0.70 | 0.59 | 0.72 | 0.78 | |
| Pers. | 0.65 | 0.51 | 0.78 | 0.70 | 0.52 | 0.83 | 0.76 | |
| Pol. | 0.67 | 0.52 | 0.56 | 0.69 | 0.49 | 0.50 | 0.62 | |
| Sports | 0.70 | 0.58 | 0.69 | 0.70 | 0.52 | 0.60 | 0.74 | |
Qwen2.5-VL-7B |
Cult. | 0.69 | 0.46 | 0.55 | 0.66 | 0.48 | 0.47 | 0.59 |
| Food | 0.70 | 0.42 | 0.68 | 0.64 | 0.55 | 0.54 | 0.68 | |
| Hist. | 0.70 | 0.51 | 0.60 | 0.67 | 0.57 | 0.52 | 0.52 | |
| M&M | 0.70 | 0.44 | 0.22 | 0.76 | 0.50 | 0.27 | 0.17 | |
| Nat. Achv. | 0.69 | 0.46 | 0.69 | 0.69 | 0.51 | 0.71 | 0.54 | |
| Nature | 0.70 | 0.49 | 0.42 | 0.55 | 0.38 | 0.64 | 0.39 | |
| Pers. | 0.65 | 0.44 | 0.47 | 0.69 | 0.48 | 0.46 | 0.56 | |
| Pol. | 0.66 | 0.46 | 0.39 | 0.69 | 0.48 | 0.32 | 0.41 | |
| Sports | 0.68 | 0.47 | 0.48 | 0.72 | 0.52 | 0.48 | 0.40 | |
Qwen3-VL-8B |
Cult. | 0.67 | 0.44 | 0.68 | 0.67 | 0.42 | 0.68 | 0.62 |
| Food | 0.68 | 0.42 | 0.70 | 0.69 | 0.47 | 0.66 | 0.66 | |
| Hist. | 0.67 | 0.44 | 0.69 | 0.67 | 0.39 | 0.69 | 0.78 | |
| M&M | 0.67 | 0.36 | 0.40 | 0.67 | 0.37 | 0.28 | 0.38 | |
| Nat. Achv. | 0.66 | 0.40 | 0.78 | 0.65 | 0.37 | 0.63 | 0.70 | |
| Nature | 0.68 | 0.43 | 0.46 | 0.64 | 0.54 | 0.47 | 0.52 | |
| Pers. | 0.63 | 0.41 | 0.51 | 0.65 | 0.45 | 0.52 | 0.57 | |
| Pol. | 0.64 | 0.40 | 0.45 | 0.68 | 0.43 | 0.48 | 0.45 | |
| Sports | 0.66 | 0.41 | 0.49 | 0.64 | 0.34 | 0.60 | 0.45 | |
GPT-4.1-mini |
Cult. | 0.71 | 0.57 | 0.65 | 0.71 | 0.57 | 0.56 | 0.77 |
| Food | 0.72 | 0.56 | 0.88 | 0.73 | 0.54 | 0.81 | 0.94 | |
| Hist. | 0.71 | 0.58 | 0.66 | 0.72 | 0.57 | 0.60 | 0.72 | |
| M&M | 0.70 | 0.54 | 0.49 | 0.89 | 0.61 | 0.42 | 0.46 | |
| Nat. Achv. | 0.71 | 0.62 | 0.58 | 0.71 | 0.60 | 0.57 | 0.78 | |
| Nature | 0.71 | 0.60 | 0.57 | 0.72 | 0.61 | 0.56 | 0.65 | |
| Pers. | 0.66 | 0.51 | 0.65 | 0.71 | 0.53 | 0.60 | 0.75 | |
| Pol. | 0.66 | 0.50 | 0.50 | 0.69 | 0.52 | 0.50 | 0.61 | |
| Sports | 0.69 | 0.58 | 0.53 | 0.71 | 0.55 | 0.48 | 0.69 | |
Full results by model and domain for Bangla Language.
| Models | Domain | Zero-Shot | Few-Shot | CoT VQA | ||||
|---|---|---|---|---|---|---|---|---|
| Caption | VQA | Caption | VQA | |||||
| B-F1 | LLM | Accuracy (%) | B-F1 | LLM | Accuracy (%) | |||
Gemma3-4B |
Cult. | 0.87 | 0.65 | 0.72 | 0.86 | 0.56 | 0.58 | 0.50 |
| Food | 0.85 | 0.61 | 0.75 | 0.86 | 0.54 | 0.70 | 0.68 | |
| Hist. | 0.86 | 0.61 | 0.43 | 0.84 | 0.44 | 0.40 | 0.44 | |
| M&M | 0.85 | 0.55 | 0.26 | 0.92 | 0.55 | 0.34 | 0.42 | |
| Nat. Achv. | 0.87 | 0.63 | 0.41 | 0.87 | 0.54 | 0.34 | 0.33 | |
| Nature | 0.86 | 0.66 | 0.55 | 0.86 | 0.59 | 0.55 | 0.39 | |
| Pers. | 0.83 | 0.55 | 0.57 | 0.84 | 0.51 | 0.52 | 0.56 | |
| Pol. | 0.84 | 0.56 | 0.41 | 0.83 | 0.49 | 0.43 | 0.44 | |
| Sports | 0.86 | 0.60 | 0.54 | 0.84 | 0.43 | 0.42 | 0.55 | |
Gemma3-12B |
Cult. | 0.87 | 0.62 | 0.69 | 0.86 | 0.55 | 0.72 | 0.71 |
| Food | 0.85 | 0.59 | 0.73 | 0.86 | 0.54 | 0.77 | 0.73 | |
| Hist. | 0.85 | 0.60 | 0.62 | 0.85 | 0.51 | 0.60 | 0.74 | |
| M&M | 0.85 | 0.53 | 0.44 | 0.90 | 0.59 | 0.30 | 0.40 | |
| Nat. Achv. | 0.87 | 0.63 | 0.57 | 0.86 | 0.48 | 0.50 | 0.60 | |
| Nature | 0.86 | 0.64 | 0.53 | 0.85 | 0.56 | 0.58 | 0.64 | |
| Pers. | 0.83 | 0.53 | 0.62 | 0.85 | 0.50 | 0.71 | 0.65 | |
| Pol. | 0.83 | 0.54 | 0.52 | 0.85 | 0.50 | 0.52 | 0.63 | |
| Sports | 0.85 | 0.58 | 0.61 | 0.85 | 0.50 | 0.56 | 0.62 | |
Gemma3-27B |
Cult. | 0.87 | 0.66 | 0.77 | 0.86 | 0.57 | 0.75 | 0.80 |
| Food | 0.85 | 0.62 | 0.88 | 0.86 | 0.54 | 0.84 | 0.82 | |
| Hist. | 0.85 | 0.65 | 0.62 | 0.87 | 0.57 | 0.62 | 0.73 | |
| M&M | 0.86 | 0.53 | 0.41 | 0.92 | 0.54 | 0.36 | 0.45 | |
| Nat. Achv. | 0.87 | 0.62 | 0.65 | 0.86 | 0.55 | 0.50 | 0.86 | |
| Nature | 0.86 | 0.67 | 0.60 | 0.85 | 0.55 | 0.61 | 0.68 | |
| Pers. | 0.83 | 0.55 | 0.81 | 0.85 | 0.50 | 0.79 | 0.77 | |
| Pol. | 0.83 | 0.55 | 0.51 | 0.84 | 0.51 | 0.47 | 0.49 | |
| Sports | 0.87 | 0.66 | 0.70 | 0.85 | 0.50 | 0.70 | 0.61 | |
Qwen2.5-VL-7B |
Cult. | 0.87 | 0.69 | 0.62 | 0.84 | 0.59 | 0.65 | 0.58 |
| Food | 0.85 | 0.60 | 0.77 | 0.81 | 0.62 | 0.72 | 0.75 | |
| Hist. | 0.86 | 0.58 | 0.62 | 0.83 | 0.52 | 0.54 | 0.60 | |
| M&M | 0.86 | 0.53 | 0.32 | 0.90 | 0.54 | 0.30 | 0.37 | |
| Nat. Achv. | 0.88 | 0.64 | 0.60 | 0.87 | 0.62 | 0.51 | 0.61 | |
| Nature | 0.87 | 0.68 | 0.49 | 0.77 | 0.56 | 0.40 | 0.48 | |
| Pers. | 0.83 | 0.54 | 0.51 | 0.84 | 0.51 | 0.52 | 0.60 | |
| Pol. | 0.83 | 0.55 | 0.49 | 0.84 | 0.54 | 0.42 | 0.56 | |
| Sports | 0.85 | 0.56 | 0.59 | 0.86 | 0.60 | 0.49 | 0.59 | |
Qwen3-VL-8B |
Cult. | 0.87 | 0.63 | 0.71 | 0.86 | 0.50 | 0.66 | 0.64 |
| Food | 0.85 | 0.62 | 0.78 | 0.83 | 0.58 | 0.73 | 0.71 | |
| Hist. | 0.83 | 0.62 | 0.64 | 0.86 | 0.49 | 0.61 | 0.63 | |
| M&M | 0.87 | 0.56 | 0.37 | 0.91 | 0.48 | 0.31 | 0.45 | |
| Nat. Achv. | 0.87 | 0.73 | 0.65 | 0.88 | 0.60 | 0.56 | 0.69 | |
| Nature | 0.85 | 0.61 | 0.46 | 0.81 | 0.43 | 0.38 | 0.47 | |
| Pers. | 0.83 | 0.54 | 0.64 | 0.85 | 0.49 | 0.64 | 0.63 | |
| Pol. | 0.82 | 0.52 | 0.45 | 0.85 | 0.48 | 0.45 | 0.42 | |
| Sports | 0.85 | 0.59 | 0.60 | 0.86 | 0.58 | 0.55 | 0.65 | |
GPT-4.1-mini |
Cult. | 0.87 | 0.64 | 0.72 | 0.87 | 0.66 | 0.68 | 0.76 |
| Food | 0.86 | 0.64 | 0.87 | 0.87 | 0.67 | 0.78 | 0.92 | |
| Hist. | 0.84 | 0.61 | 0.64 | 0.87 | 0.54 | 0.66 | 0.70 | |
| M&M | 0.85 | 0.43 | 0.51 | 0.95 | 0.66 | 0.36 | 0.51 | |
| Nat. Achv. | 0.88 | 0.66 | 0.68 | 0.88 | 0.69 | 0.55 | 0.77 | |
| Nature | 0.86 | 0.66 | 0.65 | 0.86 | 0.63 | 0.58 | 0.70 | |
| Pers. | 0.83 | 0.53 | 0.83 | 0.86 | 0.58 | 0.76 | 0.82 | |
| Pol. | 0.83 | 0.54 | 0.40 | 0.85 | 0.50 | 0.43 | 0.58 | |
| Sports | 0.86 | 0.55 | 0.56 | 0.87 | 0.61 | 0.46 | 0.75 | |
Full results by model and domain for English Language.
| Models | Domain | Zero-Shot | Few-Shot | CoT VQA | ||||
|---|---|---|---|---|---|---|---|---|
| Caption | VQA | Caption | VQA | |||||
| B-F1 | LLM | Accuracy (%) | B-F1 | LLM | Accuracy (%) | |||
Gemma3-4B |
Cult. | 0.71 | 0.57 | 0.72 | 0.69 | 0.52 | 0.53 | 0.58 |
| Food | 0.71 | 0.53 | 0.70 | 0.71 | 0.52 | 0.57 | 0.65 | |
| Hist. | 0.70 | 0.55 | 0.52 | 0.69 | 0.47 | 0.32 | 0.52 | |
| M&M | 0.67 | 0.49 | 0.26 | 0.84 | 0.52 | 0.25 | 0.32 | |
| Nat. Achv. | 0.70 | 0.56 | 0.29 | 0.69 | 0.53 | 0.30 | 0.38 | |
| Nature | 0.70 | 0.58 | 0.66 | 0.71 | 0.56 | 0.61 | 0.34 | |
| Pers. | 0.65 | 0.48 | 0.57 | 0.68 | 0.47 | 0.45 | 0.45 | |
| Pol. | 0.67 | 0.50 | 0.36 | 0.69 | 0.45 | 0.41 | 0.40 | |
| Sports | 0.68 | 0.54 | 0.47 | 0.69 | 0.46 | 0.37 | 0.44 | |
Gemma3-12B |
Cult. | 0.71 | 0.57 | 0.61 | 0.70 | 0.51 | 0.62 | 0.68 |
| Food | 0.70 | 0.53 | 0.73 | 0.71 | 0.52 | 0.67 | 0.77 | |
| Hist. | 0.70 | 0.54 | 0.62 | 0.69 | 0.48 | 0.59 | 0.71 | |
| M&M | 0.70 | 0.53 | 0.40 | 0.81 | 0.51 | 0.34 | 0.35 | |
| Nat. Achv. | 0.71 | 0.58 | 0.52 | 0.69 | 0.50 | 0.37 | 0.68 | |
| Nature | 0.70 | 0.57 | 0.46 | 0.71 | 0.56 | 0.57 | 0.56 | |
| Pers. | 0.65 | 0.49 | 0.54 | 0.69 | 0.50 | 0.57 | 0.62 | |
| Pol. | 0.67 | 0.51 | 0.43 | 0.69 | 0.45 | 0.49 | 0.47 | |
| Sports | 0.68 | 0.54 | 0.47 | 0.69 | 0.49 | 0.45 | 0.52 | |
Gemma3-27B |
Cult. | 0.72 | 0.58 | 0.71 | 0.71 | 0.53 | 0.73 | 0.73 |
| Food | 0.71 | 0.56 | 0.84 | 0.71 | 0.54 | 0.81 | 0.87 | |
| Hist. | 0.70 | 0.54 | 0.67 | 0.70 | 0.50 | 0.63 | 0.72 | |
| M&M | 0.69 | 0.52 | 0.38 | 0.86 | 0.53 | 0.33 | 0.42 | |
| Nat. Achv. | 0.73 | 0.61 | 0.54 | 0.70 | 0.53 | 0.55 | 0.82 | |
| Nature | 0.71 | 0.59 | 0.61 | 0.70 | 0.57 | 0.57 | 0.62 | |
| Pers. | 0.66 | 0.50 | 0.66 | 0.70 | 0.50 | 0.68 | 0.68 | |
| Pol. | 0.68 | 0.52 | 0.49 | 0.69 | 0.49 | 0.49 | 0.45 | |
| Sports | 0.71 | 0.58 | 0.56 | 0.69 | 0.51 | 0.53 | 0.53 | |
Qwen2.5-VL-7B |
Cult. | 0.70 | 0.47 | 0.57 | 0.64 | 0.43 | 0.56 | 0.60 |
| Food | 0.71 | 0.41 | 0.64 | 0.65 | 0.36 | 0.67 | 0.63 | |
| Hist. | 0.70 | 0.50 | 0.58 | 0.68 | 0.38 | 0.48 | 0.55 | |
| M&M | 0.70 | 0.39 | 0.29 | 0.80 | 0.43 | 0.22 | 0.36 | |
| Nat. Achv. | 0.70 | 0.48 | 0.75 | 0.69 | 0.47 | 0.58 | 0.47 | |
| Nature | 0.71 | 0.49 | 0.41 | 0.65 | 0.48 | 0.50 | 0.45 | |
| Pers. | 0.65 | 0.43 | 0.33 | 0.69 | 0.48 | 0.43 | 0.45 | |
| Pol. | 0.67 | 0.46 | 0.40 | 0.69 | 0.47 | 0.34 | 0.41 | |
| Sports | 0.69 | 0.48 | 0.48 | 0.72 | 0.54 | 0.42 | 0.46 | |
Qwen3-VL-8B |
Cult. | 0.71 | 0.58 | 0.60 | 0.68 | 0.40 | 0.52 | 0.64 |
| Food | 0.71 | 0.53 | 0.72 | 0.67 | 0.34 | 0.65 | 0.75 | |
| Hist. | 0.71 | 0.50 | 0.66 | 0.68 | 0.43 | 0.71 | 0.70 | |
| M&M | 0.67 | 0.49 | 0.37 | 0.78 | 0.41 | 0.29 | 0.37 | |
| Nat. Achv. | 0.71 | 0.61 | 0.66 | 0.69 | 0.47 | 0.70 | 0.76 | |
| Nature | 0.70 | 0.60 | 0.36 | 0.60 | 0.38 | 0.40 | 0.47 | |
| Pers. | 0.65 | 0.49 | 0.48 | 0.69 | 0.50 | 0.48 | 0.49 | |
| Pol. | 0.66 | 0.51 | 0.43 | 0.69 | 0.47 | 0.45 | 0.43 | |
| Sports | 0.69 | 0.55 | 0.53 | 0.72 | 0.50 | 0.50 | 0.47 | |
GPT-4.1-mini |
Cult. | 0.72 | 0.59 | 0.64 | 0.72 | 0.61 | 0.61 | 0.71 |
| Food | 0.72 | 0.57 | 0.85 | 0.73 | 0.59 | 0.70 | 0.83 | |
| Hist. | 0.71 | 0.55 | 0.70 | 0.72 | 0.55 | 0.59 | 0.68 | |
| M&M | 0.69 | 0.52 | 0.38 | 0.87 | 0.58 | 0.37 | 0.42 | |
| Nat. Achv. | 0.72 | 0.61 | 0.59 | 0.72 | 0.56 | 0.64 | 0.73 | |
| Nature | 0.71 | 0.62 | 0.49 | 0.71 | 0.59 | 0.38 | 0.53 | |
| Pers. | 0.65 | 0.50 | 0.60 | 0.71 | 0.53 | 0.50 | 0.72 | |
| Pol. | 0.68 | 0.53 | 0.46 | 0.69 | 0.52 | 0.53 | 0.49 | |
| Sports | 0.69 | 0.56 | 0.44 | 0.74 | 0.54 | 0.44 | 0.53 | |
Full results by model and domain for Hindi Language.
| Models | Domain | Zero-Shot | Few-Shot | CoT VQA | ||||
|---|---|---|---|---|---|---|---|---|
| Caption | VQA | Caption | VQA | |||||
| B-F1 | LLM | Accuracy (%) | B-F1 | LLM | Accuracy (%) | |||
Gemma3-4B |
Cult. | 0.70 | 0.48 | 0.57 | 0.69 | 0.48 | 0.34 | 0.48 |
| Food | 0.70 | 0.46 | 0.63 | 0.70 | 0.47 | 0.52 | 0.56 | |
| Hist. | 0.69 | 0.49 | 0.51 | 0.70 | 0.44 | 0.37 | 0.45 | |
| M&M | 0.69 | 0.45 | 0.19 | 0.85 | 0.52 | 0.24 | 0.28 | |
| Nat. Achv. | 0.69 | 0.49 | 0.31 | 0.69 | 0.46 | 0.26 | 0.38 | |
| Nature | 0.70 | 0.49 | 0.67 | 0.69 | 0.51 | 0.57 | 0.41 | |
| Pers. | 0.65 | 0.45 | 0.52 | 0.68 | 0.48 | 0.43 | 0.47 | |
| Pol. | 0.67 | 0.47 | 0.37 | 0.70 | 0.47 | 0.37 | 0.41 | |
| Sports | 0.68 | 0.47 | 0.39 | 0.70 | 0.48 | 0.34 | 0.52 | |
Gemma3-12B |
Cult. | 0.71 | 0.54 | 0.56 | 0.69 | 0.50 | 0.64 | 0.60 |
| Food | 0.69 | 0.51 | 0.67 | 0.70 | 0.51 | 0.65 | 0.68 | |
| Hist. | 0.70 | 0.53 | 0.59 | 0.70 | 0.48 | 0.59 | 0.63 | |
| M&M | 0.72 | 0.52 | 0.26 | 0.83 | 0.53 | 0.28 | 0.32 | |
| Nat. Achv. | 0.70 | 0.55 | 0.55 | 0.70 | 0.50 | 0.43 | 0.64 | |
| Nature | 0.70 | 0.56 | 0.36 | 0.70 | 0.56 | 0.47 | 0.46 | |
| Pers. | 0.66 | 0.49 | 0.47 | 0.69 | 0.50 | 0.61 | 0.55 | |
| Pol. | 0.68 | 0.53 | 0.41 | 0.70 | 0.47 | 0.50 | 0.50 | |
| Sports | 0.68 | 0.54 | 0.42 | 0.70 | 0.50 | 0.52 | 0.50 | |
Gemma3-27B |
Cult. | 0.71 | 0.58 | 0.61 | 0.70 | 0.52 | 0.68 | 0.68 |
| Food | 0.71 | 0.56 | 0.72 | 0.71 | 0.53 | 0.77 | 0.78 | |
| Hist. | 0.71 | 0.58 | 0.65 | 0.70 | 0.49 | 0.64 | 0.70 | |
| M&M | 0.73 | 0.51 | 0.32 | 0.87 | 0.52 | 0.32 | 0.35 | |
| Nat. Achv. | 0.71 | 0.61 | 0.58 | 0.70 | 0.54 | 0.57 | 0.83 | |
| Nature | 0.70 | 0.59 | 0.41 | 0.70 | 0.56 | 0.57 | 0.51 | |
| Pers. | 0.66 | 0.49 | 0.56 | 0.69 | 0.51 | 0.61 | 0.59 | |
| Pol. | 0.68 | 0.54 | 0.45 | 0.71 | 0.49 | 0.46 | 0.48 | |
| Sports | 0.71 | 0.60 | 0.61 | 0.73 | 0.49 | 0.60 | 0.61 | |
Qwen2.5-VL-7B |
Cult. | 0.69 | 0.43 | 0.58 | 0.65 | 0.42 | 0.54 | 0.55 |
| Food | 0.69 | 0.34 | 0.66 | 0.64 | 0.32 | 0.56 | 0.60 | |
| Hist. | 0.70 | 0.46 | 0.58 | 0.69 | 0.41 | 0.54 | 0.54 | |
| M&M | 0.66 | 0.35 | 0.24 | 0.80 | 0.41 | 0.29 | 0.35 | |
| Nat. Achv. | 0.68 | 0.40 | 0.67 | 0.71 | 0.46 | 0.65 | 0.55 | |
| Nature | 0.69 | 0.43 | 0.49 | 0.56 | 0.32 | 0.24 | 0.46 | |
| Pers. | 0.65 | 0.38 | 0.43 | 0.69 | 0.45 | 0.51 | 0.44 | |
| Pol. | 0.68 | 0.45 | 0.42 | 0.70 | 0.47 | 0.35 | 0.36 | |
| Sports | 0.67 | 0.46 | 0.44 | 0.72 | 0.44 | 0.29 | 0.50 | |
Qwen3-VL-8B |
Cult. | 0.70 | 0.53 | 0.53 | 0.67 | 0.40 | 0.59 | 0.57 |
| Food | 0.69 | 0.49 | 0.66 | 0.66 | 0.37 | 0.60 | 0.64 | |
| Hist. | 0.71 | 0.54 | 0.66 | 0.67 | 0.38 | 0.60 | 0.68 | |
| M&M | 0.68 | 0.37 | 0.25 | 0.82 | 0.47 | 0.39 | 0.26 | |
| Nat. Achv. | 0.71 | 0.57 | 0.69 | 0.69 | 0.50 | 0.50 | 0.70 | |
| Nature | 0.69 | 0.52 | 0.37 | 0.59 | 0.35 | 0.46 | 0.38 | |
| Pers. | 0.64 | 0.43 | 0.42 | 0.69 | 0.51 | 0.45 | 0.44 | |
| Pol. | 0.67 | 0.47 | 0.39 | 0.69 | 0.47 | 0.41 | 0.44 | |
| Sports | 0.69 | 0.58 | 0.51 | 0.72 | 0.47 | 0.46 | 0.49 | |
GPT-4.1-mini |
Cult. | 0.72 | 0.57 | 0.64 | 0.72 | 0.57 | 0.60 | 0.69 |
| Food | 0.71 | 0.54 | 0.69 | 0.72 | 0.56 | 0.61 | 0.78 | |
| Hist. | 0.72 | 0.55 | 0.60 | 0.71 | 0.56 | 0.56 | 0.63 | |
| M&M | 0.70 | 0.50 | 0.43 | 0.87 | 0.55 | 0.35 | 0.43 | |
| Nat. Achv. | 0.71 | 0.60 | 0.58 | 0.71 | 0.58 | 0.56 | 0.73 | |
| Nature | 0.71 | 0.62 | 0.41 | 0.71 | 0.59 | 0.35 | 0.56 | |
| Pers. | 0.66 | 0.49 | 0.50 | 0.69 | 0.50 | 0.45 | 0.73 | |
| Pol. | 0.68 | 0.52 | 0.39 | 0.70 | 0.51 | 0.46 | 0.56 | |
| Sports | 0.70 | 0.57 | 0.47 | 0.74 | 0.55 | 0.45 | 0.56 | |
Full results by model and domain for Urdu Language.
| Models | Domain | Zero-Shot | Few-Shot | CoT VQA | ||||
|---|---|---|---|---|---|---|---|---|
| Caption | VQA | Caption | VQA | |||||
| B-F1 | LLM | Accuracy (%) | B-F1 | LLM | Accuracy (%) | |||
Gemma3-4B |
Cult. | 0.68 | 0.49 | 0.61 | 0.67 | 0.48 | 0.38 | 0.48 |
| Food | 0.69 | 0.45 | 0.74 | 0.69 | 0.53 | 0.53 | 0.75 | |
| Hist. | 0.65 | 0.47 | 0.48 | 0.66 | 0.43 | 0.33 | 0.48 | |
| M&M | 0.64 | 0.43 | 0.29 | 0.75 | 0.47 | 0.29 | 0.29 | |
| Nat. Achv. | 0.65 | 0.45 | 0.26 | 0.65 | 0.49 | 0.25 | 0.36 | |
| Nature | 0.68 | 0.52 | 0.67 | 0.68 | 0.56 | 0.60 | 0.45 | |
| Pers. | 0.63 | 0.44 | 0.56 | 0.64 | 0.45 | 0.46 | 0.52 | |
| Pol. | 0.63 | 0.45 | 0.41 | 0.66 | 0.45 | 0.37 | 0.40 | |
| Sports | 0.64 | 0.48 | 0.42 | 0.64 | 0.47 | 0.44 | 0.63 | |
Gemma3-12B |
Cult. | 0.69 | 0.47 | 0.64 | 0.68 | 0.46 | 0.63 | 0.70 |
| Food | 0.70 | 0.44 | 0.86 | 0.70 | 0.49 | 0.76 | 0.95 | |
| Hist. | 0.67 | 0.42 | 0.63 | 0.67 | 0.41 | 0.55 | 0.75 | |
| M&M | 0.66 | 0.39 | 0.32 | 0.80 | 0.49 | 0.37 | 0.35 | |
| Nat. Achv. | 0.66 | 0.44 | 0.49 | 0.67 | 0.46 | 0.50 | 0.42 | |
| Nature | 0.69 | 0.46 | 0.46 | 0.67 | 0.47 | 0.51 | 0.52 | |
| Pers. | 0.64 | 0.42 | 0.67 | 0.66 | 0.41 | 0.65 | 0.67 | |
| Pol. | 0.64 | 0.42 | 0.51 | 0.67 | 0.42 | 0.55 | 0.50 | |
| Sports | 0.65 | 0.42 | 0.48 | 0.66 | 0.42 | 0.52 | 0.47 | |
Gemma3-27B |
Cult. | 0.69 | 0.47 | 0.70 | 0.69 | 0.50 | 0.72 | 0.69 |
| Food | 0.70 | 0.51 | 0.88 | 0.72 | 0.54 | 0.87 | 0.90 | |
| Hist. | 0.66 | 0.41 | 0.67 | 0.70 | 0.47 | 0.64 | 0.71 | |
| M&M | 0.64 | 0.42 | 0.36 | 0.78 | 0.48 | 0.31 | 0.39 | |
| Nat. Achv. | 0.66 | 0.44 | 0.51 | 0.68 | 0.48 | 0.53 | 0.75 | |
| Nature | 0.68 | 0.45 | 0.61 | 0.69 | 0.51 | 0.63 | 0.75 | |
| Pers. | 0.63 | 0.42 | 0.82 | 0.67 | 0.45 | 0.80 | 0.75 | |
| Pol. | 0.63 | 0.39 | 0.53 | 0.68 | 0.45 | 0.51 | 0.58 | |
| Sports | 0.66 | 0.44 | 0.65 | 0.66 | 0.44 | 0.65 | 0.80 | |
Qwen2.5-VL-7B |
Cult. | 0.68 | 0.38 | 0.48 | 0.66 | 0.34 | 0.45 | 0.51 |
| Food | 0.69 | 0.34 | 0.61 | 0.64 | 0.33 | 0.51 | 0.67 | |
| Hist. | 0.69 | 0.44 | 0.57 | 0.65 | 0.34 | 0.44 | 0.61 | |
| M&M | 0.68 | 0.34 | 0.21 | 0.78 | 0.41 | 0.26 | 0.31 | |
| Nat. Achv. | 0.67 | 0.30 | 0.66 | 0.69 | 0.44 | 0.68 | 0.55 | |
| Nature | 0.68 | 0.36 | 0.40 | 0.55 | 0.31 | 0.66 | 0.35 | |
| Pers. | 0.64 | 0.36 | 0.36 | 0.68 | 0.45 | 0.42 | 0.52 | |
| Pol. | 0.65 | 0.38 | 0.38 | 0.69 | 0.45 | 0.32 | 0.51 | |
| Sports | 0.68 | 0.35 | 0.45 | 0.70 | 0.44 | 0.39 | 0.53 | |
Qwen3-VL-8B |
Cult. | 0.67 | 0.43 | 0.59 | 0.66 | 0.34 | 0.59 | 0.70 |
| Food | 0.68 | 0.40 | 0.72 | 0.69 | 0.42 | 0.73 | 0.74 | |
| Hist. | 0.67 | 0.46 | 0.71 | 0.67 | 0.38 | 0.71 | 0.65 | |
| M&M | 0.67 | 0.35 | 0.36 | 0.69 | 0.37 | 0.37 | 0.28 | |
| Nat. Achv. | 0.66 | 0.41 | 0.67 | 0.65 | 0.35 | 0.59 | 0.57 | |
| Nature | 0.68 | 0.43 | 0.53 | 0.65 | 0.34 | 0.31 | 0.48 | |
| Pers. | 0.63 | 0.42 | 0.52 | 0.65 | 0.43 | 0.54 | 0.52 | |
| Pol. | 0.64 | 0.45 | 0.45 | 0.66 | 0.42 | 0.47 | 0.46 | |
| Sports | 0.65 | 0.47 | 0.48 | 0.64 | 0.38 | 0.53 | 0.43 | |
GPT-4.1-mini |
Cult. | 0.70 | 0.46 | 0.64 | 0.70 | 0.50 | 0.59 | 0.75 |
| Food | 0.72 | 0.45 | 0.84 | 0.73 | 0.54 | 0.73 | 0.92 | |
| Hist. | 0.69 | 0.44 | 0.56 | 0.70 | 0.50 | 0.54 | 0.68 | |
| M&M | 0.68 | 0.37 | 0.45 | 0.90 | 0.56 | 0.41 | 0.45 | |
| Nat. Achv. | 0.69 | 0.42 | 0.61 | 0.69 | 0.52 | 0.51 | 0.85 | |
| Nature | 0.69 | 0.41 | 0.48 | 0.71 | 0.51 | 0.51 | 0.62 | |
| Pers. | 0.65 | 0.36 | 0.59 | 0.69 | 0.52 | 0.53 | 0.73 | |
| Pol. | 0.65 | 0.37 | 0.49 | 0.69 | 0.45 | 0.43 | 0.56 | |
| Sports | 0.66 | 0.43 | 0.52 | 0.71 | 0.45 | 0.53 | 0.71 | |
Full results by model and domain for Barishal Dialect.
| Models | Domain | Zero-Shot | Few-Shot | CoT VQA | ||||
|---|---|---|---|---|---|---|---|---|
| Caption | VQA | Caption | VQA | |||||
| B-F1 | LLM | Accuracy (%) | B-F1 | LLM | Accuracy (%) | |||
Gemma3-4B | Cult. | 0.67 | 0.45 | 0.65 | 0.68 | 0.45 | 0.42 | 0.62 |
| Food | 0.69 | 0.48 | 0.75 | 0.69 | 0.51 | 0.62 | 0.77 | |
| Hist. | 0.65 | 0.40 | 0.47 | 0.68 | 0.45 | 0.33 | 0.41 | |
| M&M | 0.65 | 0.42 | 0.25 | 0.78 | 0.51 | 0.25 | 0.30 | |
| Nat. Achv. | 0.65 | 0.44 | 0.28 | 0.67 | 0.43 | 0.27 | 0.31 | |
| Nature | 0.69 | 0.51 | 0.63 | 0.69 | 0.53 | 0.57 | 0.49 | |
| Pers. | 0.63 | 0.40 | 0.52 | 0.66 | 0.42 | 0.44 | 0.51 | |
| Pol. | 0.63 | 0.41 | 0.38 | 0.67 | 0.41 | 0.35 | 0.53 | |
| Sports | 0.63 | 0.45 | 0.45 | 0.65 | 0.45 | 0.27 | 0.61 | |
Gemma3-12B | Cult. | 0.68 | 0.45 | 0.65 | 0.68 | 0.45 | 0.61 | 0.69 |
| Food | 0.70 | 0.48 | 0.85 | 0.71 | 0.51 | 0.71 | 0.83 | |
| Hist. | 0.67 | 0.40 | 0.62 | 0.68 | 0.40 | 0.59 | 0.74 | |
| M&M | 0.66 | 0.42 | 0.32 | 0.83 | 0.51 | 0.37 | 0.28 | |
| Nat. Achv. | 0.66 | 0.47 | 0.54 | 0.67 | 0.45 | 0.49 | 0.62 | |
| Nature | 0.68 | 0.43 | 0.51 | 0.68 | 0.46 | 0.45 | 0.59 | |
| Pers. | 0.64 | 0.41 | 0.69 | 0.66 | 0.41 | 0.64 | 0.83 | |
| Pol. | 0.64 | 0.41 | 0.47 | 0.68 | 0.40 | 0.49 | 0.54 | |
| Sports | 0.65 | 0.45 | 0.52 | 0.66 | 0.45 | 0.42 | 0.50 | |
Gemma3-27B | Cult. | 0.68 | 0.40 | 0.68 | 0.69 | 0.47 | 0.73 | 0.77 |
| Food | 0.70 | 0.44 | 0.90 | 0.71 | 0.50 | 0.88 | 0.85 | |
| Hist. | 0.66 | 0.38 | 0.67 | 0.69 | 0.46 | 0.67 | 0.77 | |
| M&M | 0.66 | 0.35 | 0.34 | 0.79 | 0.50 | 0.30 | 0.34 | |
| Nat. Achv. | 0.66 | 0.45 | 0.52 | 0.68 | 0.43 | 0.46 | 0.69 | |
| Nature | 0.69 | 0.43 | 0.59 | 0.70 | 0.50 | 0.57 | 0.70 | |
| Pers. | 0.63 | 0.35 | 0.78 | 0.67 | 0.46 | 0.82 | 0.80 | |
| Pol. | 0.64 | 0.36 | 0.55 | 0.68 | 0.46 | 0.48 | 0.48 | |
| Sports | 0.66 | 0.38 | 0.66 | 0.67 | 0.41 | 0.66 | 0.71 | |
Qwen2.5-VL-7B | Cult. | 0.67 | 0.36 | 0.55 | 0.63 | 0.34 | 0.49 | 0.60 |
| Food | 0.69 | 0.31 | 0.68 | 0.65 | 0.29 | 0.49 | 0.66 | |
| Hist. | 0.69 | 0.42 | 0.62 | 0.67 | 0.37 | 0.57 | 0.50 | |
| M&M | 0.67 | 0.35 | 0.18 | 0.79 | 0.45 | 0.19 | 0.18 | |
| Nat. Achv. | 0.67 | 0.34 | 0.66 | 0.70 | 0.39 | 0.60 | 0.43 | |
| Nature | 0.68 | 0.37 | 0.51 | 0.54 | 0.28 | 0.38 | 0.37 | |
| Pers. | 0.63 | 0.34 | 0.36 | 0.67 | 0.41 | 0.44 | 0.54 | |
| Pol. | 0.65 | 0.39 | 0.36 | 0.69 | 0.44 | 0.31 | 0.29 | |
| Sports | 0.67 | 0.37 | 0.52 | 0.69 | 0.45 | 0.43 | 0.58 | |
Qwen3-VL-8B | Cult. | 0.67 | 0.45 | 0.60 | 0.67 | 0.42 | 0.57 | 0.67 |
| Food | 0.68 | 0.40 | 0.69 | 0.69 | 0.43 | 0.67 | 0.74 | |
| Hist. | 0.67 | 0.44 | 0.68 | 0.67 | 0.41 | 0.68 | 0.69 | |
| M&M | 0.66 | 0.36 | 0.29 | 0.68 | 0.34 | 0.25 | 0.28 | |
| Nat. Achv. | 0.65 | 0.41 | 0.59 | 0.66 | 0.42 | 0.53 | 0.59 | |
| Nature | 0.67 | 0.44 | 0.57 | 0.64 | 0.33 | 0.48 | 0.45 | |
| Pers. | 0.63 | 0.43 | 0.57 | 0.64 | 0.41 | 0.46 | 0.50 | |
| Pol. | 0.64 | 0.44 | 0.48 | 0.66 | 0.41 | 0.41 | 0.38 | |
| Sports | 0.64 | 0.42 | 0.54 | 0.66 | 0.39 | 0.44 | 0.56 | |
GPT-4.1-mini | Cult. | 0.69 | 0.41 | 0.70 | 0.70 | 0.45 | 0.58 | 0.73 |
| Food | 0.72 | 0.46 | 0.84 | 0.74 | 0.52 | 0.75 | 0.91 | |
| Hist. | 0.69 | 0.41 | 0.64 | 0.70 | 0.46 | 0.64 | 0.66 | |
| M&M | 0.68 | 0.42 | 0.48 | 0.89 | 0.61 | 0.42 | 0.43 | |
| Nat. Achv. | 0.69 | 0.46 | 0.59 | 0.70 | 0.54 | 0.57 | 0.80 | |
| Nature | 0.70 | 0.39 | 0.58 | 0.71 | 0.46 | 0.44 | 0.57 | |
| Pers. | 0.65 | 0.34 | 0.67 | 0.69 | 0.52 | 0.54 | 0.78 | |
| Pol. | 0.66 | 0.36 | 0.50 | 0.68 | 0.45 | 0.46 | 0.59 | |
| Sports | 0.67 | 0.41 | 0.50 | 0.70 | 0.48 | 0.44 | 0.66 | |
Full results by model and domain for Chittagong Dialect.
| Models | Domain | Zero-Shot | Few-Shot | CoT VQA | ||||
|---|---|---|---|---|---|---|---|---|
| Caption | VQA | Caption | VQA | |||||
| B-F1 | LLM | Accuracy (%) | B-F1 | LLM | Accuracy (%) | |||
Gemma3-4B | Cult. | 0.69 | 0.48 | 0.65 | 0.68 | 0.49 | 0.49 | 0.51 |
| Food | 0.69 | 0.45 | 0.74 | 0.69 | 0.50 | 0.61 | 0.72 | |
| Hist. | 0.66 | 0.40 | 0.46 | 0.68 | 0.42 | 0.37 | 0.51 | |
| M&M | 0.65 | 0.41 | 0.23 | 0.79 | 0.48 | 0.28 | 0.31 | |
| Nat. Achv. | 0.66 | 0.49 | 0.31 | 0.68 | 0.45 | 0.20 | 0.43 | |
| Nature | 0.70 | 0.56 | 0.63 | 0.69 | 0.56 | 0.69 | 0.49 | |
| Pers. | 0.64 | 0.45 | 0.51 | 0.68 | 0.47 | 0.50 | 0.53 | |
| Pol. | 0.65 | 0.45 | 0.40 | 0.68 | 0.43 | 0.41 | 0.46 | |
| Sports | 0.64 | 0.42 | 0.50 | 0.66 | 0.46 | 0.45 | 0.45 | |
Gemma3-12B | Cult. | 0.69 | 0.42 | 0.67 | 0.68 | 0.44 | 0.65 | 0.70 |
| Food | 0.71 | 0.44 | 0.86 | 0.71 | 0.49 | 0.77 | 0.88 | |
| Hist. | 0.67 | 0.37 | 0.63 | 0.68 | 0.42 | 0.60 | 0.62 | |
| M&M | 0.67 | 0.37 | 0.35 | 0.75 | 0.54 | 0.34 | 0.36 | |
| Nat. Achv. | 0.66 | 0.42 | 0.55 | 0.68 | 0.45 | 0.44 | 0.44 | |
| Nature | 0.69 | 0.43 | 0.48 | 0.68 | 0.45 | 0.46 | 0.60 | |
| Pers. | 0.64 | 0.41 | 0.68 | 0.65 | 0.41 | 0.67 | 0.69 | |
| Pol. | 0.65 | 0.40 | 0.54 | 0.67 | 0.41 | 0.56 | 0.60 | |
| Sports | 0.65 | 0.40 | 0.53 | 0.65 | 0.42 | 0.52 | 0.82 | |
Gemma3-27B | Cult. | 0.70 | 0.47 | 0.73 | 0.69 | 0.48 | 0.69 | 0.79 |
| Food | 0.71 | 0.47 | 0.90 | 0.72 | 0.52 | 0.91 | 0.90 | |
| Hist. | 0.68 | 0.44 | 0.66 | 0.69 | 0.45 | 0.69 | 0.70 | |
| M&M | 0.68 | 0.41 | 0.35 | 0.80 | 0.51 | 0.32 | 0.46 | |
| Nat. Achv. | 0.67 | 0.47 | 0.48 | 0.67 | 0.43 | 0.53 | 0.85 | |
| Nature | 0.69 | 0.47 | 0.62 | 0.70 | 0.51 | 0.68 | 0.67 | |
| Pers. | 0.65 | 0.39 | 0.80 | 0.68 | 0.46 | 0.85 | 0.83 | |
| Pol. | 0.65 | 0.39 | 0.51 | 0.69 | 0.46 | 0.47 | 0.50 | |
| Sports | 0.67 | 0.43 | 0.65 | 0.66 | 0.42 | 0.58 | 0.67 | |
Qwen2.5-VL-7B | Cult. | 0.68 | 0.37 | 0.51 | 0.69 | 0.36 | 0.49 | 0.53 |
| Food | 0.69 | 0.33 | 0.66 | 0.71 | 0.41 | 0.54 | 0.68 | |
| Hist. | 0.69 | 0.42 | 0.60 | 0.70 | 0.44 | 0.55 | 0.60 | |
| M&M | 0.68 | 0.35 | 0.21 | 0.83 | 0.53 | 0.29 | 0.23 | |
| Nat. Achv. | 0.67 | 0.41 | 0.69 | 0.70 | 0.42 | 0.48 | 0.56 | |
| Nature | 0.68 | 0.41 | 0.37 | 0.71 | 0.44 | 0.58 | 0.34 | |
| Pers. | 0.64 | 0.36 | 0.46 | 0.67 | 0.41 | 0.44 | 0.43 | |
| Pol. | 0.65 | 0.39 | 0.39 | 0.68 | 0.44 | 0.36 | 0.30 | |
| Sports | 0.67 | 0.40 | 0.44 | 0.72 | 0.49 | 0.42 | 0.41 | |
Qwen3-VL-8B | Cult. | 0.67 | 0.45 | 0.65 | 0.67 | 0.35 | 0.62 | 0.64 |
| Food | 0.69 | 0.40 | 0.73 | 0.68 | 0.40 | 0.69 | 0.74 | |
| Hist. | 0.68 | 0.43 | 0.69 | 0.66 | 0.35 | 0.63 | 0.72 | |
| M&M | 0.67 | 0.36 | 0.38 | 0.69 | 0.37 | 0.30 | 0.35 | |
| Nat. Achv. | 0.67 | 0.46 | 0.63 | 0.66 | 0.42 | 0.57 | 0.81 | |
| Nature | 0.68 | 0.43 | 0.38 | 0.64 | 0.30 | 0.35 | 0.43 | |
| Pers. | 0.64 | 0.42 | 0.50 | 0.65 | 0.42 | 0.44 | 0.46 | |
| Pol. | 0.64 | 0.42 | 0.43 | 0.66 | 0.39 | 0.44 | 0.34 | |
| Sports | 0.66 | 0.47 | 0.50 | 0.66 | 0.38 | 0.47 | 0.58 | |
GPT-4.1-mini | Cult. | 0.70 | 0.44 | 0.65 | 0.71 | 0.52 | 0.58 | 0.72 |
| Food | 0.71 | 0.41 | 0.85 | 0.73 | 0.51 | 0.73 | 0.88 | |
| Hist. | 0.69 | 0.44 | 0.61 | 0.71 | 0.50 | 0.57 | 0.68 | |
| M&M | 0.68 | 0.36 | 0.48 | 0.90 | 0.64 | 0.38 | 0.49 | |
| Nat. Achv. | 0.69 | 0.43 | 0.62 | 0.71 | 0.53 | 0.62 | 0.82 | |
| Nature | 0.70 | 0.39 | 0.49 | 0.71 | 0.50 | 0.51 | 0.59 | |
| Pers. | 0.65 | 0.37 | 0.57 | 0.70 | 0.51 | 0.57 | 0.74 | |
| Pol. | 0.67 | 0.38 | 0.42 | 0.69 | 0.49 | 0.51 | 0.57 | |
| Sports | 0.68 | 0.42 | 0.53 | 0.69 | 0.48 | 0.50 | 0.61 | |
Full results by model and domain for Noakhali Dialect.
| Models | Domain | Zero-Shot | Few-Shot | CoT VQA | ||||
|---|---|---|---|---|---|---|---|---|
| Caption | VQA | Caption | VQA | |||||
| B-F1 | LLM | Accuracy (%) | B-F1 | LLM | Accuracy (%) | |||
Gemma3-4B | Cult. | 0.68 | 0.49 | 0.63 | 0.68 | 0.49 | 0.50 | 0.59 |
| Food | 0.70 | 0.50 | 0.73 | 0.70 | 0.52 | 0.56 | 0.72 | |
| Hist. | 0.67 | 0.48 | 0.44 | 0.68 | 0.44 | 0.42 | 0.40 | |
| M&M | 0.66 | 0.44 | 0.27 | 0.80 | 0.52 | 0.29 | 0.30 | |
| Nat. Achv. | 0.67 | 0.52 | 0.35 | 0.66 | 0.44 | 0.27 | 0.45 | |
| Nature | 0.69 | 0.55 | 0.67 | 0.69 | 0.57 | 0.69 | 0.41 | |
| Pers. | 0.63 | 0.42 | 0.59 | 0.67 | 0.46 | 0.55 | 0.53 | |
| Pol. | 0.64 | 0.41 | 0.42 | 0.69 | 0.45 | 0.38 | 0.41 | |
| Sports | 0.65 | 0.48 | 0.48 | 0.66 | 0.47 | 0.37 | 0.46 | |
Gemma3-12B | Cult. | 0.69 | 0.45 | 0.62 | 0.68 | 0.44 | 0.62 | 0.82 |
| Food | 0.70 | 0.47 | 0.85 | 0.71 | 0.51 | 0.78 | 0.82 | |
| Hist. | 0.67 | 0.43 | 0.64 | 0.68 | 0.44 | 0.60 | 0.76 | |
| M&M | 0.65 | 0.41 | 0.37 | 0.82 | 0.51 | 0.32 | 0.44 | |
| Nat. Achv. | 0.65 | 0.45 | 0.52 | 0.68 | 0.46 | 0.52 | 0.71 | |
| Nature | 0.68 | 0.43 | 0.44 | 0.69 | 0.46 | 0.54 | 0.58 | |
| Pers. | 0.63 | 0.42 | 0.66 | 0.65 | 0.40 | 0.71 | 0.76 | |
| Pol. | 0.65 | 0.41 | 0.55 | 0.67 | 0.40 | 0.54 | 0.62 | |
| Sports | 0.64 | 0.44 | 0.50 | 0.66 | 0.44 | 0.52 | 0.54 | |
Gemma3-27B | Cult. | 0.70 | 0.52 | 0.69 | 0.68 | 0.49 | 0.72 | 0.64 |
| Food | 0.71 | 0.50 | 0.88 | 0.72 | 0.52 | 0.90 | 0.87 | |
| Hist. | 0.68 | 0.47 | 0.69 | 0.69 | 0.47 | 0.66 | 0.75 | |
| M&M | 0.67 | 0.43 | 0.33 | 0.84 | 0.52 | 0.34 | 0.43 | |
| Nat. Achv. | 0.68 | 0.48 | 0.52 | 0.69 | 0.43 | 0.55 | 0.73 | |
| Nature | 0.69 | 0.49 | 0.59 | 0.70 | 0.54 | 0.60 | 0.67 | |
| Pers. | 0.64 | 0.42 | 0.78 | 0.69 | 0.48 | 0.78 | 0.79 | |
| Pol. | 0.65 | 0.43 | 0.57 | 0.68 | 0.46 | 0.51 | 0.51 | |
| Sports | 0.67 | 0.48 | 0.66 | 0.68 | 0.45 | 0.53 | 0.84 | |
Qwen2.5-VL-7B | Cult. | 0.69 | 0.49 | 0.55 | 0.69 | 0.47 | 0.40 | 0.59 |
| Food | 0.69 | 0.43 | 0.68 | 0.72 | 0.46 | 0.57 | 0.68 | |
| Hist. | 0.70 | 0.51 | 0.60 | 0.71 | 0.52 | 0.51 | 0.50 | |
| M&M | 0.69 | 0.44 | 0.22 | 0.81 | 0.57 | 0.26 | 0.20 | |
| Nat. Achv. | 0.68 | 0.49 | 0.69 | 0.69 | 0.50 | 0.71 | 0.51 | |
| Nature | 0.70 | 0.51 | 0.42 | 0.71 | 0.50 | 0.70 | 0.40 | |
| Pers. | 0.65 | 0.45 | 0.47 | 0.68 | 0.47 | 0.39 | 0.53 | |
| Pol. | 0.66 | 0.47 | 0.39 | 0.67 | 0.49 | 0.36 | 0.33 | |
| Sports | 0.69 | 0.49 | 0.48 | 0.72 | 0.54 | 0.44 | 0.44 | |
Qwen3-VL-8B | Cult. | 0.66 | 0.43 | 0.57 | 0.66 | 0.35 | 0.65 | 0.66 |
| Food | 0.68 | 0.41 | 0.75 | 0.68 | 0.41 | 0.70 | 0.73 | |
| Hist. | 0.67 | 0.44 | 0.70 | 0.67 | 0.38 | 0.72 | 0.65 | |
| M&M | 0.66 | 0.34 | 0.32 | 0.67 | 0.38 | 0.27 | 0.38 | |
| Nat. Achv. | 0.65 | 0.41 | 0.66 | 0.64 | 0.42 | 0.55 | 0.65 | |
| Nature | 0.67 | 0.42 | 0.57 | 0.65 | 0.34 | 0.50 | 0.49 | |
| Pers. | 0.62 | 0.41 | 0.57 | 0.64 | 0.44 | 0.53 | 0.54 | |
| Pol. | 0.64 | 0.42 | 0.45 | 0.67 | 0.45 | 0.47 | 0.44 | |
| Sports | 0.64 | 0.44 | 0.48 | 0.65 | 0.36 | 0.42 | 0.61 | |
GPT-4.1-mini | Cult. | 0.69 | 0.42 | 0.66 | 0.70 | 0.50 | 0.58 | 0.78 |
| Food | 0.72 | 0.44 | 0.86 | 0.73 | 0.54 | 0.68 | 0.89 | |
| Hist. | 0.69 | 0.45 | 0.63 | 0.69 | 0.48 | 0.62 | 0.72 | |
| M&M | 0.69 | 0.39 | 0.53 | 0.91 | 0.63 | 0.39 | 0.43 | |
| Nat. Achv. | 0.69 | 0.45 | 0.57 | 0.70 | 0.52 | 0.58 | 0.74 | |
| Nature | 0.69 | 0.41 | 0.55 | 0.71 | 0.53 | 0.49 | 0.69 | |
| Pers. | 0.64 | 0.39 | 0.59 | 0.70 | 0.51 | 0.46 | 0.72 | |
| Pol. | 0.66 | 0.35 | 0.46 | 0.69 | 0.46 | 0.46 | 0.51 | |
| Sports | 0.67 | 0.43 | 0.53 | 0.71 | 0.53 | 0.50 | 0.74 | |
Full results by model and domain for Rangpur Dialect.
| Models | Domain | Zero-Shot | Few-Shot | CoT VQA | ||||
|---|---|---|---|---|---|---|---|---|
| Caption | VQA | Caption | VQA | |||||
| B-F1 | LLM | Accuracy (%) | B-F1 | LLM | Accuracy (%) | |||
Gemma3-4B | Cult. | 0.67 | 0.47 | 0.60 | 0.68 | 0.45 | 0.45 | 0.58 |
| Food | 0.70 | 0.46 | 0.72 | 0.70 | 0.50 | 0.62 | 0.64 | |
| Hist. | 0.66 | 0.42 | 0.44 | 0.68 | 0.43 | 0.40 | 0.47 | |
| M&M | 0.65 | 0.42 | 0.27 | 0.80 | 0.50 | 0.35 | 0.36 | |
| Nat. Achv. | 0.65 | 0.47 | 0.31 | 0.68 | 0.48 | 0.29 | 0.45 | |
| Nature | 0.69 | 0.58 | 0.59 | 0.69 | 0.56 | 0.63 | 0.52 | |
| Pers. | 0.63 | 0.45 | 0.58 | 0.67 | 0.45 | 0.54 | 0.51 | |
| Pol. | 0.64 | 0.45 | 0.40 | 0.68 | 0.43 | 0.42 | 0.40 | |
| Sports | 0.64 | 0.47 | 0.47 | 0.67 | 0.47 | 0.42 | 0.44 | |
Gemma3-12B | Cult. | 0.68 | 0.40 | 0.62 | 0.68 | 0.43 | 0.65 | 0.78 |
| Food | 0.70 | 0.40 | 0.82 | 0.71 | 0.48 | 0.67 | 0.91 | |
| Hist. | 0.66 | 0.37 | 0.64 | 0.69 | 0.39 | 0.64 | 0.64 | |
| M&M | 0.66 | 0.39 | 0.35 | 0.80 | 0.49 | 0.33 | 0.45 | |
| Nat. Achv. | 0.65 | 0.38 | 0.46 | 0.67 | 0.43 | 0.38 | 0.58 | |
| Nature | 0.68 | 0.41 | 0.47 | 0.68 | 0.45 | 0.54 | 0.42 | |
| Pers. | 0.63 | 0.31 | 0.68 | 0.65 | 0.34 | 0.65 | 0.81 | |
| Pol. | 0.63 | 0.33 | 0.54 | 0.67 | 0.39 | 0.52 | 0.59 | |
| Sports | 0.65 | 0.39 | 0.45 | 0.66 | 0.40 | 0.44 | 0.50 | |
Gemma3-27B | Cult. | 0.69 | 0.48 | 0.70 | 0.69 | 0.48 | 0.68 | 0.78 |
| Food | 0.71 | 0.45 | 0.84 | 0.72 | 0.53 | 0.82 | 0.86 | |
| Hist. | 0.69 | 0.44 | 0.66 | 0.69 | 0.46 | 0.63 | 0.75 | |
| M&M | 0.67 | 0.42 | 0.34 | 0.84 | 0.51 | 0.33 | 0.45 | |
| Nat. Achv. | 0.68 | 0.43 | 0.51 | 0.68 | 0.45 | 0.49 | 0.69 | |
| Nature | 0.69 | 0.46 | 0.59 | 0.70 | 0.50 | 0.60 | 0.63 | |
| Pers. | 0.65 | 0.36 | 0.79 | 0.69 | 0.48 | 0.82 | 0.78 | |
| Pol. | 0.66 | 0.38 | 0.51 | 0.68 | 0.46 | 0.47 | 0.55 | |
| Sports | 0.68 | 0.42 | 0.63 | 0.67 | 0.41 | 0.56 | 0.68 | |
Qwen2.5-VL-7B | Cult. | 0.67 | 0.35 | 0.52 | 0.69 | 0.32 | 0.42 | 0.53 |
| Food | 0.69 | 0.32 | 0.55 | 0.71 | 0.35 | 0.45 | 0.59 | |
| Hist. | 0.68 | 0.42 | 0.58 | 0.70 | 0.43 | 0.55 | 0.52 | |
| M&M | 0.67 | 0.35 | 0.21 | 0.83 | 0.53 | 0.22 | 0.22 | |
| Nat. Achv. | 0.67 | 0.36 | 0.66 | 0.70 | 0.40 | 0.71 | 0.52 | |
| Nature | 0.69 | 0.39 | 0.43 | 0.70 | 0.44 | 0.60 | 0.38 | |
| Pers. | 0.64 | 0.36 | 0.49 | 0.67 | 0.41 | 0.48 | 0.51 | |
| Pol. | 0.66 | 0.38 | 0.39 | 0.69 | 0.44 | 0.34 | 0.36 | |
| Sports | 0.67 | 0.41 | 0.45 | 0.69 | 0.44 | 0.37 | 0.48 | |
Qwen3-VL-8B | Cult. | 0.66 | 0.45 | 0.66 | 0.65 | 0.35 | 0.59 | 0.68 |
| Food | 0.68 | 0.44 | 0.71 | 0.69 | 0.45 | 0.68 | 0.73 | |
| Hist. | 0.67 | 0.44 | 0.72 | 0.66 | 0.35 | 0.67 | 0.72 | |
| M&M | 0.66 | 0.34 | 0.42 | 0.67 | 0.37 | 0.44 | 0.37 | |
| Nat. Achv. | 0.67 | 0.45 | 0.66 | 0.65 | 0.34 | 0.69 | 0.72 | |
| Nature | 0.68 | 0.47 | 0.48 | 0.64 | 0.32 | 0.42 | 0.44 | |
| Pers. | 0.63 | 0.41 | 0.49 | 0.64 | 0.45 | 0.56 | 0.56 | |
| Pol. | 0.64 | 0.44 | 0.42 | 0.66 | 0.42 | 0.41 | 0.42 | |
| Sports | 0.66 | 0.44 | 0.53 | 0.64 | 0.39 | 0.51 | 0.59 | |
GPT-4.1-mini | Cult. | 0.69 | 0.43 | 0.63 | 0.70 | 0.51 | 0.59 | 0.71 |
| Food | 0.72 | 0.46 | 0.84 | 0.72 | 0.50 | 0.66 | 0.86 | |
| Hist. | 0.69 | 0.46 | 0.58 | 0.70 | 0.49 | 0.58 | 0.68 | |
| M&M | 0.69 | 0.40 | 0.47 | 0.90 | 0.59 | 0.43 | 0.50 | |
| Nat. Achv. | 0.69 | 0.48 | 0.64 | 0.70 | 0.55 | 0.62 | 0.76 | |
| Nature | 0.70 | 0.45 | 0.53 | 0.72 | 0.54 | 0.47 | 0.67 | |
| Pers. | 0.65 | 0.36 | 0.63 | 0.70 | 0.51 | 0.51 | 0.71 | |
| Pol. | 0.66 | 0.40 | 0.48 | 0.68 | 0.47 | 0.44 | 0.63 | |
| Sports | 0.68 | 0.46 | 0.47 | 0.70 | 0.50 | 0.47 | 0.73 | |
Full results by model and domain for Sylhet Dialect.
@article{sayeedi2026many,
title={Many Dialects, Many Languages, One Cultural Lens: Evaluating Multilingual VLMs for Bengali Culture Understanding Across Historically Linked Languages and Regional Dialects},
author={Sayeedi, Nurul Labib and Sayeedi, Md. Faiyaz Abdullah and Dipta, Shubhashis Roy and Tabassum, Rubaya and Hridoy, Ariful Ekraj and Mahmood, Mehraj and Sobhani, Mahbub E and Hasan, Md. Tarek and Shatabda, Swakkhar},
journal={arXiv preprint arXiv:2603.21165},
year={2026}
}