Release / 2026.10

What DeepInfer Ships: DeepSeek's V4.1-Flash Engine

The DeepInfer release is narrower than the launch conversation suggests — and more interesting because of it. Here is the scope, the build, the licenses, and the line between an engine and a service.

Source-based/9 min read/Snapshot 2026-10-20 11:30 UTC

DeepInfer is DeepSeek's open-source inference framework for serving DeepSeek-V4.1-Flash at scale. Released on 20 October 2026 under the Apache-2.0 license, it provides an offline inference engine for NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs — but not the KV cache system, prefill/decode disaggregation, or data-parallel load balancing required for a complete online service. This page covers the scope, build, and licenses.

That paragraph is the whole argument. Everything below is evidence.

The scope, in one table

The README's “Supported features” and “Required for a complete inference service (not included in this release)” lists are unusually clear. We reproduce them side by side, without smoothing over the gap.

Included	Not included
Offline inference for DeepSeek-V4.1-Flash	KV cache system
Optimized kernels and model/sampler runtime	VE / Prefill / Decode disaggregation
Attention DP / balanced CP; MoE EP + EPLB	Load balancing across data-parallel replicas
DSpark and confidence-scheduled verification	A user-facing online API
SWA bounded replay	Support for other DeepSeek models
Asynchronous scheduling	Published Ascend 950 performance data
Batch-invariant inference (NVIDIA)	Third-party benchmark reproduction
Deterministic sampling (NVIDIA)	A production parity guarantee
Structured outputs: regex, JSON Schema, EBNF	—

Read this before the feature list

The left column is what the repository contains. The right column is what a serving team still has to build or operate. DeepSeek's own Q&A describes the missing pieces in detail — which is useful for understanding the design, but does not put the code in the release.

Repository map

The public tree is compact. We list the directories a reader is likely to open; we did not audit every file.

Path	What we found	Why open it
`deepinfer/`	Python package and engine code	The main runtime and scheduling logic
`csrc/`	C++/CUDA native extensions	Kernel and binding work
`doc/`	Deployment Q&A, images	The most detailed official explanation
`examples/`	Offline inference, benchmark, sweeps	Fastest way to see the intended use
`eval/`	AIME 2026, HLE, GPQA Diamond scripts	Reproduce the reference scores
`tests/`	pytest suite, PPL baselines	See what “correct” means to the authors
`scripts/`	Formatting and helper scripts	Contributor tooling
`vendor/`	Vendored third-party code	License and dependency audit
`deep_cluster/`	Cluster-related code shipped in the tree	We did not audit it; no public API claim here

The public history starts with three commits on 20 October 2026: an initial commit at 04:09 UTC, a Q&A documentation update at 09:06, and an English Q&A update at 11:03. That is a code drop, not a long-running public development history. Do not read the commit count as the project's engineering effort.

Build requirements

DeepInfer supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, but DeepSeek's published performance charts cover NVIDIA hardware only. The build itself is data-center software. The README lists:

Linux
GCC ≥ 13.0
Python 3.12
CUDA 13 for NVIDIA; CANN 9.2 for Ascend
NVIDIA SM100 GPUs or Huawei Ascend 950 NPUs

The documented path is a source build:

# NVIDIA; use “ascend” for the Ascend platform
export DEEPINFER_ACCELERATOR=nvidia
./build.sh

# The default checkpoint downloads from Hugging Face on first run.
python examples/offline_inference.py

If you already have the checkpoint locally, set DEEPINFER_HF_MODEL_DIR=/path/to/DeepSeek-V4.1-Flash-1017. The examples README also exposes --ep, --dp, and --cp flags for parallel layouts. None of this changes the hardware requirement: the README lists SM100 and Ascend 950; it does not list H100, H200, H20, or consumer cards.

Not a laptop story. The model is a 552B-backbone MoE with 196B Engram parameters; the engine is built for data-center accelerators. Local experiments with V4.1-Flash exist through separate quantization projects, but they are not DeepInfer.

Licenses, without blending them

Component	License	What it covers
DeepInfer	Apache-2.0	The engine code in `deepseek-ai/DeepInfer`
DeepSeek-V4.1-Flash-1017	MIT	The default model weights
deepseek-recipe	MIT	The separate protocol/tokenization library
Vendored kernels	Check each	FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect and other dependencies

Apache-2.0 permits commercial use and modification, with notice obligations and a patent grant. MIT is similarly permissive. That does not automatically clear the licenses of every vendored dependency or the model's acceptable-use terms. If you are deploying commercially, audit the tree and the model card rather than relying on a summary.

What you can do today

Run offline inference on supported hardware, using the default V4.1-Flash-1017 checkpoint.
Reproduce the benchmark sweeps with examples/sweep/run_sweep.sh and plot with plot_pareto.py.
Run the reference evals for AIME 2026, GPQA Diamond, and HLE via the scripts in eval/.
Study the implementation of Wide-EP, EPLB, DSpark verification, SWA bounded replay, and the async scheduler.
Use deepseek-recipe to align third-party API behavior with DeepSeek's protocol handling.
Build a serving layer on top — but that layer is your work, not the release's.

What you can't do (yet)

Open an API that matches DeepSeek's. The release is offline; the KV cache, PD disaggregation, and load balancing are not included.
Serve other models. Only V4.1-Flash is listed.
Run it on older NVIDIA or Ascend hardware. SM100 and Ascend 950 are the listed targets.
Quote Ascend throughput. DeepSeek published NVIDIA charts only.
Claim production parity. DeepSeek says the framework powers its services, but the public code is a subset; external verification is not possible.

The one-sentence boundary

DeepInfer is an offline inference engine, not a hosted API or a model; the release does not include the KV cache system, prefill/decode disaggregation, or data-parallel load balancing needed for a complete online service.

Version note

There is no GitHub Release or tag at the time of writing. The repository has three public commits, and the Q&A was updated on launch day. Treat this page as a snapshot: if you are making a deployment decision, read the current README and Q&A, and check whether the missing pieces have been added.

Next: how the engine works, or what the missing pieces mean in practice.