A cross-platform deployment comparison of three open-weights multimodal LLMs across two distinct hardware platforms — NVIDIA RTX 3090 (Rocky Linux) and NVIDIA DGX Spark running NVIDIA DGX OS. Measures throughput, latency, VRAM, temperature, and power consumption.
google/gemma-4-E2B-itmicrosoft/Phi-4-multimodal-instructQwen/Qwen3.5-4BRocky Linux
(NVIDIA RTX 3090 24 GB, CUDA 12.4, Linux). Qwen3.5-4B runs separately on NVIDIA DGX OS
(NVIDIA DGX Spark, ARM aarch64, CUDA 13, BF16 via Docker).
The GB10 (DGX Spark) draws 8× less power than Gemma4 on RTX 3090 during inference. For always-on or edge AI deployments, this translates to dramatically lower electricity costs and no need for active cooling.
All systems remain well within safe operating limits. The GB10 runs 10°C cooler despite its integrated SoC design. RTX 3090 temperatures are comfortable at 58–61°C, leaving significant thermal headroom before throttling (~83°C typical limit).
Gemma 4 E2B uses 1.3 GB less VRAM than Phi-4 despite a larger total parameter count. The Per-Layer Embedding (PLE) architecture keeps embedding tables in a separate lookup space, reducing active computation memory. Qwen3.5-4B on GB10 draws from a shared 128 GB unified pool.
Qwen3.5-4B uses only ~6% of the GB10's 128 GB unified memory pool — leaving massive headroom for very long context (262K tokens) or multi-model serving.
The synthetic audio file (3s, 200–800Hz mixed tones) is not real speech — it generates
ambiguous transcription results from all models. Phi-4 loops "Oh. [no audio]..." (200 tokens),
Gemma4 correctly identifies minimal content ("Oh,"). Both handle real speech well per the v1
benchmark with KOKORO TTS. Qwen3.5-4B does not have audio capability in the current deployment.
Audio verdict: Gemma4 wins on latency (0.44s vs 6.1s for transcription).
For real speech, both models perform comparably. Phi-4 loops on ambiguous input; Gemma4 terminates cleanly.
For text tasks requiring more than 50 tokens, Gemma 4 averages ~49.5 tok/s vs Phi-4's ~38.1 tok/s (+30%) and Qwen3.5-4B's ~15.2 tok/s (+226%). The RTX 3090's 936 GB/s memory bandwidth and Gemma4's efficient PLE architecture combine for highest throughput. Gemma4 also wins on latency: average 2.78s vs 3.76s (Phi-4) and 8.73s (Qwen).
Despite lower throughput, Qwen on the DGX Spark uses only 25.8W average (vs 156–205W on RTX 3090). Its token-per-watt ratio (0.59) is nearly 3× better than Gemma4 (0.20) and Phi-4 (0.22). In this benchmark, DGX Spark plus Qwen establishes a strong low-power operating point for edge-style or always-on deployment. GPU utilization is consistently 90%, showing the platform is heavily used at a fraction of the energy cost of the RTX 3090 runs.
Gemma4 vision throughput (~48 tok/s) dwarfs Phi-4 (~26 tok/s avg) and Qwen (~17 tok/s). All three produce accurate, detailed scene descriptions. Gemma4 uses structured markdown with bold headers; Phi-4 gives concise prose; Qwen produces thorough descriptions with clear section breaks. Quality is comparable across models — Gemma4 wins purely on speed.
The DGX Spark's GB10 SoC peaked at 51°C during 7 consecutive inference calls. The RTX 3090 reached 58–61°C for the same workload. At 27W peak power draw, the GB10 can run fanless-equivalent workloads. This opens up deployment scenarios in noise-sensitive environments (clinical, office, embedded) where the RTX 3090's fan noise and thermal management would be problematic. The 140W total DGX Spark system TDP easily fits on a desk-sized workstation.
Qwen3.5-4B's 262K context window is the largest of the three, suitable for entire codebases, legal documents, or very long conversations. Gemma4 and Phi-4 both expose 128K-token context windows in current official documentation. That still leaves Qwen in the lead for long-document workloads, but Phi-4 should not be treated as a short-context outlier.
Qwen3.5-4B as deployed on NVIDIA DGX OS does not support audio input. Gemma4 and Phi-4 both handle audio, with Gemma4 having dramatically lower end-to-end latency (0.44s vs 6.1s for transcription) and winning audio Q&A throughput (44.0 vs 33.4 tok/s). For full multimodal pipelines requiring audio, Phi-4 or Gemma4 on the RTX 3090 are the options.
This benchmark spans three distinct compute paradigms: (1) RTX 3090 — traditional discrete GPU, high bandwidth, high power, maximum throughput; (2) DGX Spark — Blackwell SoC, unified memory, ultra-low power, edge AI; (3) Three model architectures: Dense+PLE (Gemma4), Dense Transformer (Phi-4), Gated DeltaNet+MoE (Qwen3.5). The 2026 AI ecosystem is no longer "one architecture wins" — deployment target determines the optimal choice.
Gemma 4 E2B is the only model in this comparison with reported video understanding capability. In the benchmark, an 8-second 1280×720 MP4 sampled at 1 fps was analyzed in 13.3s at 42.9 tok/s. Phi-4 has no video support in this benchmark. Qwen3.5-4B as deployed has no video capability. For video AI workflows, Gemma 4 E2B on the RTX 3090 is the only option in this comparison.
| Setup | Avg Power | kWh / month | Cost @ $0.15/kWh | Tokens @ 15.2 avg tok/s |
|---|---|---|---|---|
| Qwen3.5-4B / GB10 | 25.8 W | 18.6 kWh | $2.79 / month | 39.3 B tokens |
| Phi-4 / RTX 3090 | 156.5 W | 112.7 kWh | $16.91 / month | 87.4 B tokens |
| Gemma4 / RTX 3090 | 204.8 W | 147.5 kWh | $22.12 / month | 106.1 B tokens |
Assumes full utilization 24/7. Real workloads are bursty — idle power is much lower (17W for RTX 3090, 11W for GB10). For inference-heavy production workloads, the DGX Spark's 8× lower power draw compounds into significant operational savings, especially when scaled across multiple units.
The corrected picture is a split deployment frontier. Gemma 4 E2B defines the throughput frontier, Qwen3.5-4B on DGX Spark defines the efficiency frontier, and Phi-4 remains relevant for concise text-image-audio pipelines and strong ASR throughput.