Model & Architecture

Category

Parameters

Required VRAM

Fit Status

Est. Speed

Simulate

DeepSeek-R1 (671B Full MoE)MLAMoE

DeepSeek

MoE

671B(37B active)

411.16 GB

OOM (-387.16G)

~0.9 tok/s

Simulate

DeepSeek-V3 (671B Base/Chat)MLAMoE

DeepSeek

MoE

671B(37B active)

411.16 GB

OOM (-387.16G)

~0.9 tok/s

Simulate

DeepSeek-R1-Distill-Llama-70B

DeepSeek

LLM

70.6B

47.6 GB

OOM (-23.6G)

~3.4 tok/s

Simulate

DeepSeek-R1-Distill-Qwen-32B

DeepSeek

LLM

32.8B

23.75 GB

Tight Fit

~35.3 tok/s

Simulate

DeepSeek-R1-Distill-Qwen-14B

DeepSeek

LLM

14.7B

11.94 GB

100% VRAM Fit

~78.8 tok/s

Simulate

DeepSeek-R1-Distill-Llama-8B

DeepSeek

LLM

7.33 GB

100% VRAM Fit

~144.8 tok/s

Simulate

DeepSeek-R1-Distill-Qwen-8B

DeepSeek

LLM

8.03B

7.48 GB

100% VRAM Fit

~144.3 tok/s

Simulate

DeepSeek-R1-Distill-Qwen-7B

DeepSeek

LLM

7.6B

6.49 GB

100% VRAM Fit

~152.5 tok/s

Simulate

DeepSeek-R1-Distill-Qwen-1.5B

DeepSeek

LLM

1.8B

2.61 GB

100% VRAM Fit

~180 tok/s

Simulate

DeepSeek-Coder-V2 Lite (16B / 2.4B MoE)MLAMoE

DeepSeek

MoE

15.7B(2.4B active)

11.21 GB

100% VRAM Fit

~180 tok/s

Simulate

DeepSeek-Coder-V2 (236B / 21B MoE)MLAMoE

DeepSeek

MoE

236B(21B active)

146.23 GB

OOM (-122.23G)

~3.7 tok/s

Simulate

Qwen 2.5 Coder 72B Instruct

Qwen

LLM

72.7B

48.87 GB

OOM (-24.87G)

~3.3 tok/s

Simulate

Qwen 2.5 72B Instruct

Qwen

LLM

72.7B

48.87 GB

OOM (-24.87G)

~3.3 tok/s

Simulate

Qwen 2.5 VL 72B (Vision Flagship)

Qwen

LLM

72.7B

48.87 GB

OOM (-24.87G)

~3.3 tok/s

Simulate

Qwen 2.5 VL 7B (Vision)

Qwen

LLM

7.61B

6.5 GB

100% VRAM Fit

~152.2 tok/s

Simulate

Qwen 2.5 Coder 32B Instruct

Qwen

LLM

32.8B

23.75 GB

Tight Fit

~35.3 tok/s

Simulate

Qwen 2.5 Coder 14B Instruct

Qwen

LLM

14.7B

11.94 GB

100% VRAM Fit

~78.8 tok/s

Simulate

Qwen 2.5 Coder 7B Instruct

Qwen

LLM

7.6B

6.49 GB

100% VRAM Fit

~152.5 tok/s

Simulate

Qwen 2.5 Coder 1.5B Instruct

Qwen

LLM

1.54B

2.45 GB

100% VRAM Fit

~180 tok/s

Simulate

Qwen 2.5 32B Instruct

Qwen

LLM

32.8B

23.75 GB

Tight Fit

~35.3 tok/s

Simulate

Qwen 2.5 14B Instruct

Qwen

LLM

14.7B

11.94 GB

100% VRAM Fit

~78.8 tok/s

Simulate

Qwen 2.5 7B Instruct

Qwen

LLM

7.6B

6.49 GB

100% VRAM Fit

~152.5 tok/s

Simulate

Qwen 2.5 3B Instruct

Qwen

LLM

3.1B

3.59 GB

100% VRAM Fit

~180 tok/s

Simulate

Qwen 2.5 0.5B Instruct

Qwen

LLM

0.49B

1.8 GB

100% VRAM Fit

~180 tok/s

Simulate

Llama 3.3 70B Instruct

Llama

LLM

70.6B

47.6 GB

OOM (-23.6G)

~3.4 tok/s

Simulate

Llama 3.1 70B Instruct

Llama

LLM

70.6B

47.6 GB

OOM (-23.6G)

~3.4 tok/s

Simulate

Llama 3.1 405B Instruct

Llama

LLM

405B

252.77 GB

OOM (-228.77G)

~0.6 tok/s

Simulate

Llama 3.1 8B Instruct

Llama

LLM

7.33 GB

100% VRAM Fit

~144.8 tok/s

Simulate

Llama 3.2 3B Instruct

Llama

LLM

3.2B

4.28 GB

100% VRAM Fit

~180 tok/s

Simulate

Llama 3.2 1B Instruct

Llama

LLM

1.23B

2.41 GB

100% VRAM Fit

~180 tok/s

Simulate

Llama 3.2 11B Vision Instruct

Llama

LLM

10.6B

8.92 GB

100% VRAM Fit

~109.3 tok/s

Simulate

Llama 3.2 90B Vision Instruct

Llama

LLM

89B

59.46 GB

OOM (-35.46G)

~2.1 tok/s

Simulate

Microsoft Phi-4 (14B)

Phi

LLM

14.7B

12.01 GB

100% VRAM Fit

~78.8 tok/s

Simulate

Microsoft Phi-3.5 Mini (3.8B)

Phi

LLM

3.82B

6.91 GB

100% VRAM Fit

~180 tok/s

Simulate

Google Gemma 2 9B Instruct

Gemma

LLM

9.2B

9.79 GB

100% VRAM Fit

~125.9 tok/s

Simulate

Mistral Small 3 (24B Instruct)

Mistral

LLM

24B

18.12 GB

100% VRAM Fit

~48.3 tok/s

Simulate

Mistral NeMo (12B Instruct)

Mistral

LLM

12.2B

10.15 GB

100% VRAM Fit

~95 tok/s

Simulate

Mistral 7B Instruct v0.3

Mistral

LLM

7.25B

6.88 GB

100% VRAM Fit

~159.8 tok/s

Simulate

Codestral 22B (Mistral AI)

Mistral

LLM

22.2B

17.02 GB

100% VRAM Fit

~52.2 tok/s

Simulate

Mistral Large 2 (123B)

Mistral

LLM

123B

79.77 GB

OOM (-55.77G)

~1.1 tok/s

Simulate

Mixtral 8x7B (MoE 47B)MoE

Mistral

MoE

46.7B(12.9B active)

31.15 GB

OOM (-7.15G)

~29.7 tok/s

Simulate

Mixtral 8x22B (MoE 141B)MoE

Mistral

MoE

141B(39B active)

89.67 GB

OOM (-65.67G)

~3.3 tok/s

Simulate

Pixtral 12B (Multimodal)

Mistral

LLM

12.4B

10.28 GB

100% VRAM Fit

~93.4 tok/s

Simulate

Gemma 2 27B Instruct

Gemma

LLM

27.2B

21.26 GB

Tight Fit

~42.6 tok/s

Simulate

Google Gemma 2 2B Instruct

Gemma

LLM

2.6B

3.84 GB

100% VRAM Fit

~180 tok/s

Simulate

GLM-4 9B Chat (Zhipu AI)

GLM

LLM

9.4B

7.79 GB

100% VRAM Fit

~123.3 tok/s

Simulate

FLUX.1 [dev] (12B DiT)

FLUX

ImageGen

12B

15.68 GB

100% VRAM Fit

~96.6 tok/s

Simulate

FLUX.1 [schnell] (4-step)

FLUX

ImageGen

12B

15.68 GB

100% VRAM Fit

~96.6 tok/s

Simulate

Stable Diffusion XL (SDXL 1.0)

SDXL

ImageGen

3.5B

7.51 GB

100% VRAM Fit

~180 tok/s

Simulate

Stable Diffusion 1.5 (Classic)

ImageGen

5.78 GB

100% VRAM Fit

~180 tok/s

Simulate

Whisper Large v3 (1.5B)

Whisper

Audio

1.5B

3.64 GB

100% VRAM Fit

~180 tok/s

Simulate

Kokoro 82M (Text-to-Speech)

Kokoro

Audio

0.082B

1.56 GB

100% VRAM Fit

~180 tok/s

Simulate

Wan2.1 (14B Text-to-Video)

Wan

ImageGen

14.2B

18.72 GB

100% VRAM Fit

~81.6 tok/s

Simulate

Wan2.1 (1.3B Video)

Wan

ImageGen

1.3B

7.28 GB

100% VRAM Fit

~180 tok/s

Simulate

HunyuanVideo (13B Video DiT)

Hunyuan

ImageGen

13B

19.69 GB

100% VRAM Fit

~89.1 tok/s

Simulate

Stable Diffusion 3.5 Large (8B)

ImageGen

8.1B

13.3 GB

100% VRAM Fit

~143 tok/s

Simulate

BAAI BGE-M3 (Multilingual Embedding)

BAAI

LLM

0.56B

2.54 GB

100% VRAM Fit

~180 tok/s

Simulate

Cross-GPU AI Compatibility Index (Static Reference Grid)

Quick cross-reference matrix displaying whether popular AI models fit into dedicated GPU memory at standard Q4_K_M precision with 8,192 tokens context window across 8 landmark architectures:

Model & Params	RTX 5090 (32G)	RTX 4090 (24G)	RTX 3090 (24G)	4070 Ti Super (16G)	RTX 3060 (12G)	RTX 4060 (8G)	M2 Ultra (192G)	Arc B580 (12G)
DeepSeek-R1 (671B Full MoE) 671B • MLA	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
DeepSeek-V3 (671B Base/Chat) 671B • MLA	OOM	OOM	OOM	OOM	OOM	OOM	OOM	OOM
DeepSeek-R1-Distill-Llama-70B 70.6B • LLM	OOM	OOM	OOM	OOM	OOM	OOM	Pass	OOM
DeepSeek-R1-Distill-Qwen-32B 32.8B • LLM	Pass	Pass	Pass	OOM	OOM	OOM	Pass	OOM
DeepSeek-R1-Distill-Qwen-14B 14.7B • LLM	Pass	Pass	Pass	Pass	Pass	OOM	Pass	Pass
DeepSeek-R1-Distill-Llama-8B 8B • LLM	Pass	Pass	Pass	Pass	Pass	Pass	Pass	Pass
DeepSeek-R1-Distill-Qwen-8B 8.03B • LLM	Pass	Pass	Pass	Pass	Pass	Pass	Pass	Pass
DeepSeek-R1-Distill-Qwen-7B 7.6B • LLM	Pass	Pass	Pass	Pass	Pass	Pass	Pass	Pass
DeepSeek-R1-Distill-Qwen-1.5B 1.8B • LLM	Pass	Pass	Pass	Pass	Pass	Pass	Pass	Pass
DeepSeek-Coder-V2 Lite (16B / 2.4B MoE) 15.7B • MLA	Pass	Pass	Pass	Pass	Pass	OOM	Pass	Pass
DeepSeek-Coder-V2 (236B / 21B MoE) 236B • MLA	OOM	OOM	OOM	OOM	OOM	OOM	Pass	OOM
Qwen 2.5 Coder 72B Instruct 72.7B • LLM	OOM	OOM	OOM	OOM	OOM	OOM	Pass	OOM
Qwen 2.5 72B Instruct 72.7B • LLM	OOM	OOM	OOM	OOM	OOM	OOM	Pass	OOM
Qwen 2.5 VL 72B (Vision Flagship) 72.7B • LLM	OOM	OOM	OOM	OOM	OOM	OOM	Pass	OOM
Qwen 2.5 VL 7B (Vision) 7.61B • LLM	Pass	Pass	Pass	Pass	Pass	Pass	Pass	Pass
Qwen 2.5 Coder 32B Instruct 32.8B • LLM	Pass	Pass	Pass	OOM	OOM	OOM	Pass	OOM
Qwen 2.5 Coder 14B Instruct 14.7B • LLM	Pass	Pass	Pass	Pass	Pass	OOM	Pass	Pass
Qwen 2.5 Coder 7B Instruct 7.6B • LLM	Pass	Pass	Pass	Pass	Pass	Pass	Pass	Pass

100% VRAM Fit (Optimal)

The static model weights, dynamic KV cache for the requested context, CUDA runtime buffers, and desktop OS overhead fit completely within physical GPU memory with at least 1.0 GB of safety headroom. No PCIe CPU layer offloading occurs.

Tight Fit (Critical Margin)

Model loads fully in VRAM, but less than 1.0 GB of headroom remains. Running multi-turn conversations or opening browser tabs may trigger background memory reclamation or sudden CUDA Out-of-Memory crashes.

OOM / PCIe Throttled

VRAM capacity is exceeded. The inference engine must offload transformer layers to system DDR RAM across the PCIe bus, reducing generation speed from 30+ tok/s down to ~1.2 tok/s.

Compatibility Matrix: NVIDIA GeForce RTX 4090

Global Compatibility Matrix for NVIDIA GeForce RTX 4090

Cross-GPU AI Compatibility Index (Static Reference Grid)

Model & Architecture	Category	Parameters	Required VRAM	Fit Status	Est. Speed	Simulate
DeepSeek-R1 (671B Full MoE)MLAMoE DeepSeek	MoE	671B(37B active)	411.16 GB	OOM (-387.16G)	~0.9 tok/s	Simulate
DeepSeek-V3 (671B Base/Chat)MLAMoE DeepSeek	MoE	671B(37B active)	411.16 GB	OOM (-387.16G)	~0.9 tok/s	Simulate
DeepSeek-R1-Distill-Llama-70B DeepSeek	LLM	70.6B	47.6 GB	OOM (-23.6G)	~3.4 tok/s	Simulate
DeepSeek-R1-Distill-Qwen-32B DeepSeek	LLM	32.8B	23.75 GB	Tight Fit	~35.3 tok/s	Simulate
DeepSeek-R1-Distill-Qwen-14B DeepSeek	LLM	14.7B	11.94 GB	100% VRAM Fit	~78.8 tok/s	Simulate
DeepSeek-R1-Distill-Llama-8B DeepSeek	LLM	8B	7.33 GB	100% VRAM Fit	~144.8 tok/s	Simulate
DeepSeek-R1-Distill-Qwen-8B DeepSeek	LLM	8.03B	7.48 GB	100% VRAM Fit	~144.3 tok/s	Simulate
DeepSeek-R1-Distill-Qwen-7B DeepSeek	LLM	7.6B	6.49 GB	100% VRAM Fit	~152.5 tok/s	Simulate
DeepSeek-R1-Distill-Qwen-1.5B DeepSeek	LLM	1.8B	2.61 GB	100% VRAM Fit	~180 tok/s	Simulate
DeepSeek-Coder-V2 Lite (16B / 2.4B MoE)MLAMoE DeepSeek	MoE	15.7B(2.4B active)	11.21 GB	100% VRAM Fit	~180 tok/s	Simulate
DeepSeek-Coder-V2 (236B / 21B MoE)MLAMoE DeepSeek	MoE	236B(21B active)	146.23 GB	OOM (-122.23G)	~3.7 tok/s	Simulate
Qwen 2.5 Coder 72B Instruct Qwen	LLM	72.7B	48.87 GB	OOM (-24.87G)	~3.3 tok/s	Simulate
Qwen 2.5 72B Instruct Qwen	LLM	72.7B	48.87 GB	OOM (-24.87G)	~3.3 tok/s	Simulate
Qwen 2.5 VL 72B (Vision Flagship) Qwen	LLM	72.7B	48.87 GB	OOM (-24.87G)	~3.3 tok/s	Simulate
Qwen 2.5 VL 7B (Vision) Qwen	LLM	7.61B	6.5 GB	100% VRAM Fit	~152.2 tok/s	Simulate
Qwen 2.5 Coder 32B Instruct Qwen	LLM	32.8B	23.75 GB	Tight Fit	~35.3 tok/s	Simulate
Qwen 2.5 Coder 14B Instruct Qwen	LLM	14.7B	11.94 GB	100% VRAM Fit	~78.8 tok/s	Simulate
Qwen 2.5 Coder 7B Instruct Qwen	LLM	7.6B	6.49 GB	100% VRAM Fit	~152.5 tok/s	Simulate
Qwen 2.5 Coder 1.5B Instruct Qwen	LLM	1.54B	2.45 GB	100% VRAM Fit	~180 tok/s	Simulate
Qwen 2.5 32B Instruct Qwen	LLM	32.8B	23.75 GB	Tight Fit	~35.3 tok/s	Simulate
Qwen 2.5 14B Instruct Qwen	LLM	14.7B	11.94 GB	100% VRAM Fit	~78.8 tok/s	Simulate
Qwen 2.5 7B Instruct Qwen	LLM	7.6B	6.49 GB	100% VRAM Fit	~152.5 tok/s	Simulate
Qwen 2.5 3B Instruct Qwen	LLM	3.1B	3.59 GB	100% VRAM Fit	~180 tok/s	Simulate
Qwen 2.5 0.5B Instruct Qwen	LLM	0.49B	1.8 GB	100% VRAM Fit	~180 tok/s	Simulate
Llama 3.3 70B Instruct Llama	LLM	70.6B	47.6 GB	OOM (-23.6G)	~3.4 tok/s	Simulate
Llama 3.1 70B Instruct Llama	LLM	70.6B	47.6 GB	OOM (-23.6G)	~3.4 tok/s	Simulate
Llama 3.1 405B Instruct Llama	LLM	405B	252.77 GB	OOM (-228.77G)	~0.6 tok/s	Simulate
Llama 3.1 8B Instruct Llama	LLM	8B	7.33 GB	100% VRAM Fit	~144.8 tok/s	Simulate
Llama 3.2 3B Instruct Llama	LLM	3.2B	4.28 GB	100% VRAM Fit	~180 tok/s	Simulate
Llama 3.2 1B Instruct Llama	LLM	1.23B	2.41 GB	100% VRAM Fit	~180 tok/s	Simulate
Llama 3.2 11B Vision Instruct Llama	LLM	10.6B	8.92 GB	100% VRAM Fit	~109.3 tok/s	Simulate
Llama 3.2 90B Vision Instruct Llama	LLM	89B	59.46 GB	OOM (-35.46G)	~2.1 tok/s	Simulate
Microsoft Phi-4 (14B) Phi	LLM	14.7B	12.01 GB	100% VRAM Fit	~78.8 tok/s	Simulate
Microsoft Phi-3.5 Mini (3.8B) Phi	LLM	3.82B	6.91 GB	100% VRAM Fit	~180 tok/s	Simulate
Google Gemma 2 9B Instruct Gemma	LLM	9.2B	9.79 GB	100% VRAM Fit	~125.9 tok/s	Simulate
Mistral Small 3 (24B Instruct) Mistral	LLM	24B	18.12 GB	100% VRAM Fit	~48.3 tok/s	Simulate
Mistral NeMo (12B Instruct) Mistral	LLM	12.2B	10.15 GB	100% VRAM Fit	~95 tok/s	Simulate
Mistral 7B Instruct v0.3 Mistral	LLM	7.25B	6.88 GB	100% VRAM Fit	~159.8 tok/s	Simulate
Codestral 22B (Mistral AI) Mistral	LLM	22.2B	17.02 GB	100% VRAM Fit	~52.2 tok/s	Simulate
Mistral Large 2 (123B) Mistral	LLM	123B	79.77 GB	OOM (-55.77G)	~1.1 tok/s	Simulate
Mixtral 8x7B (MoE 47B)MoE Mistral	MoE	46.7B(12.9B active)	31.15 GB	OOM (-7.15G)	~29.7 tok/s	Simulate
Mixtral 8x22B (MoE 141B)MoE Mistral	MoE	141B(39B active)	89.67 GB	OOM (-65.67G)	~3.3 tok/s	Simulate
Pixtral 12B (Multimodal) Mistral	LLM	12.4B	10.28 GB	100% VRAM Fit	~93.4 tok/s	Simulate
Gemma 2 27B Instruct Gemma	LLM	27.2B	21.26 GB	Tight Fit	~42.6 tok/s	Simulate
Google Gemma 2 2B Instruct Gemma	LLM	2.6B	3.84 GB	100% VRAM Fit	~180 tok/s	Simulate
GLM-4 9B Chat (Zhipu AI) GLM	LLM	9.4B	7.79 GB	100% VRAM Fit	~123.3 tok/s	Simulate
FLUX.1 [dev] (12B DiT) FLUX	ImageGen	12B	15.68 GB	100% VRAM Fit	~96.6 tok/s	Simulate
FLUX.1 [schnell] (4-step) FLUX	ImageGen	12B	15.68 GB	100% VRAM Fit	~96.6 tok/s	Simulate
Stable Diffusion XL (SDXL 1.0) SDXL	ImageGen	3.5B	7.51 GB	100% VRAM Fit	~180 tok/s	Simulate
Stable Diffusion 1.5 (Classic) SD	ImageGen	1B	5.78 GB	100% VRAM Fit	~180 tok/s	Simulate
Whisper Large v3 (1.5B) Whisper	Audio	1.5B	3.64 GB	100% VRAM Fit	~180 tok/s	Simulate
Kokoro 82M (Text-to-Speech) Kokoro	Audio	0.082B	1.56 GB	100% VRAM Fit	~180 tok/s	Simulate
Wan2.1 (14B Text-to-Video) Wan	ImageGen	14.2B	18.72 GB	100% VRAM Fit	~81.6 tok/s	Simulate
Wan2.1 (1.3B Video) Wan	ImageGen	1.3B	7.28 GB	100% VRAM Fit	~180 tok/s	Simulate
HunyuanVideo (13B Video DiT) Hunyuan	ImageGen	13B	19.69 GB	100% VRAM Fit	~89.1 tok/s	Simulate
Stable Diffusion 3.5 Large (8B) SD	ImageGen	8.1B	13.3 GB	100% VRAM Fit	~143 tok/s	Simulate
BAAI BGE-M3 (Multilingual Embedding) BAAI	LLM	0.56B	2.54 GB	100% VRAM Fit	~180 tok/s	Simulate