16 structured test cases · text · image · audio · video · quality scored by Claude Sonnet 4.6
E2B is 2.4× faster than E4B. Speed gap is hardware-driven: RTX 3090 936 GB/s GDDR6X vs GB10 ~273 GB/s LPDDR5X. LLM inference is memory-bandwidth-bound.
| Modality | Cases | Gemma4-E2B | Gemma4-E4B | Qwen3.5-4B |
|---|---|---|---|---|
| Text | 5 | 7.0 | 7.0 | 8.5 🏆 |
| Image | 5 | 7.5 | 8.0 | 8.5 🏆 |
| Audio | 5 | 8.5 🏆 | 7.5 1 failure | N/A |
| Video | 1 | 7.0 | N/A | N/A |
| Overall | 16 | 7.5 | 7.6 | 8.5 🥇 |
Quality evaluated by Claude Sonnet 4.6 (Anthropic, claude-sonnet-4-6) as independent AI evaluator. Criteria: factual accuracy, reasoning depth, specificity vs ground truth, output structure, hallucination absence.
| Case | Finding | Winner |
|---|---|---|
| text_02 Sheep reasoning |
E4B: "There are 9 sheep left" — no explanation. E2B: "classic riddle." Qwen: step-by-step breakdown of the "all but = all except" idiom. | Qwen |
| text_05 Meditation summary |
Only Qwen mentioned cortisol levels — matching ground truth scientific specificity. E2B and E4B stayed generic. | Qwen |
| img_02 Cyberpunk city |
Only Qwen identified Chinese characters on neon signs, correctly placing the scene as East Asian. E2B missed the cyberpunk framing entirely. | Qwen |
| img_04 Home library |
Qwen hallucinated two fireplaces. E4B described the room accurately — single fireplace, correct layout. | E4B |
| img_05 Neural network diagram |
All 3 models called it a "network graph" or "social network." None identified it as a neural network architecture. Shared blind spot. | None ✗ |
| audio_01 Solar system |
E2B: listed all 8 planets + "Jupiter is the largest planet" — near-perfect. E4B: "discusses the solar system and its planets" — missed all specifics. | E2B |
| audio_03 Medical info |
E4B produced structured bullet list with proper units (mmHg) — more clinically useful. E2B gave accurate prose. | E4B |
| audio_04 Stanford AI news |
E2B: perfect transcription. E4B: HTTP 500 — complete server failure. Only failure in the entire run. | E2B |
RTX 3090 GDDR6X (936 GB/s) vs GB10 LPDDR5X (~273 GB/s). LLM inference is memory-bandwidth-bound at batch size 1. A 2020 GPU outpaces the 2025 DGX Spark for token generation.
8.5 / 10 overall vs 7.6 (E4B) and 7.5 (E2B). More specific language, better reasoning explanations, stronger visual analysis. E4B edges out on hallucination control (library case).
E2B scored 8.5 on audio: near-perfect transcription across all 5 cases. E4B scored 7.5 with 1 hard failure (HTTP 500 on audio_04). For production audio pipelines, E2B is the safe pick.
E4B (4-bit, 4B params) scores higher quality than E2B (bf16, 2B params) on image tasks. More parameters compensate for quantization. Don't assume full precision = better quality.
Need speed? → Gemma4-E2B on RTX 3090 (47.5 tok/s, 100% reliable). Need best response quality for text/image? → Qwen3.5-4B on DGX (8.5/10). Need audio or video? → Gemma4-E2B is the only fully reliable multimodal option across all modalities.