Will this model run on your GPUs?

Why the failure comes so late

1. You pull the weights

   A 40 to 400 GB checkpoint lands on your GPU nodes. That part goes fine, because nothing has looked inside it yet.

2. The engine starts loading

   Now vLLM or SGLang finds a tensor it can't split, a kernel your GPU doesn't have, or a token past the end of the embedding table.

3. You find out the hard way

   Sometimes it's a crash. Sometimes it's worse: the server runs and quietly returns nonsense.

NVFP4 · 4-bit · fast on modern NVIDIA GPUs

CRITICAL QPF004: NVFP4 requires Blackwell (sm_100+), target is H100 (sm_90)

Everything it needs is in small JSON files and the first bytes of each weight file. A 38.74 GiB checkpoint took 887 KiB of reads.

What it checks

Each finding names the file or tensor at fault and how to fix it. Pasting a config.json skips the tensor checks. A model ID reads the tensor headers too, and the command line also handles GGUF, tokenizer.json, gated models, and local folders.

Works with

Formats

• AWQ
• GPTQ
• Compressed-Tensors
• FP8
• NVFP4
• MXFP4
• Marlin
• AutoRound
• bitsandbytes
• BitNet
• MLX
• GGUF
• Safetensors

Engines

• vLLM
• SGLang
• TensorRT-LLM
• llama.cpp

GPUs

• T4
• A100
• A100-40GB
• A10G
• L4
• L40S
• RTX4090
• H100
• H200
• B200
• RTX5090
• GB10
• MI250X
• MI300X
• MI325X

Is your model family or GPU missing? Adding one is usually a single TOML file with no Python. Read the contributing guide.

How accurate is it?

Every change runs against 132 small checkpoints labelled with what must fire and what must stay quiet.

That's 96.9% recall and 100% precision on these cases. We wrote them ourselves, so treat this as a regression check and not a promise about your model. The misses are real gaps, listed in the repository.

Real runs changed the tool: they showed which warnings were wrong and fitted the memory estimate. Only T4 GPUs have been tested so far, so rules for other GPUs rest on vendor docs and engine issues. Read the full report, or help test it on your GPU.

Run it on your model

pip install quantpreflight
quantpreflight check Qwen/Qwen2.5-7B-Instruct-AWQ --gpu H100 --tp 4
quantpreflight suggest Qwen/Qwen2.5-7B-Instruct --gpu L4

repos:
  - repo: https://github.com/layer8labs8/quantpreflight
    rev: v0.4.0
    hooks:
      - id: quantpreflight

name: Model preflight
on: [push, pull_request]

permissions:
  contents: read

jobs:
  preflight:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: layer8labs8/quantpreflight@v0.4.0
        with:
          target: ./exported-checkpoint
          gpu: H100
          tp: "4"
          fail-on: high

Each report ends with the GPU count and context length that start, and the command to run. suggest ranks the quantized versions of a model that run on your GPUs. It works on Hugging Face repo IDs, direct URLs, local folders, and single .safetensors or .gguf files. Set HF_TOKEN for gated models. The usage guide covers every option.

What it can't do

• A clean report means no known problem was found. It doesn't prove the model will load or answer well.
• It never reads weight values, so it can't see NaNs or measure quality loss.
• The memory estimate assumes one request unless you pass --concurrency. Real traffic needs more.
• Engines move fast. Its compatibility tables describe known behavior and can fall behind a new release.

See the full list

Support the project

If this project helped you, I'm really glad. If you'd like to support it, you can send a crypto tip below. If not, that's completely okay.

Your wallet or exchange will ask you to choose a network. Pick one listed above and use the address next to it. Crypto payments can't be reversed, so check before you send.