DeepSeek open-sources DeepInfer, the inference engine behind its own services

DeepSeek has open-sourced DeepInfer under the Apache License 2.0 [1][2]. DeepSeek says DeepInfer is the inference framework that powers its large-scale production services. The release runs DeepSeek-V4.1-Flash for offline inference, and its GitHub post reached the top of Hacker News on 20 October 2026 [2][3].

What is in the DeepInfer release?

The README lists optimised kernels from DeepSeek's earlier open-source projects, including FlashMLA, DeepGEMM and DeepEP, plus attention data parallelism and mixture-of-experts expert parallelism [2]. It also includes DSpark, a speculative decoding method that drafts tokens and uses confidence estimates to schedule verification [2].

Batch-invariant inference keeps results identical regardless of how requests are batched, and deterministic sampling gives identical outputs for identical inputs and settings, on Nvidia hardware only [2]. Structured outputs can constrain generation to a regex, JSON schema or EBNF grammar [2].

The default checkpoint is DeepSeek-V4.1-Flash-1017, downloaded from Hugging Face on first run [2]. DeepSeek says its benchmarks were measured on 4 SM100 GPUs for prefill and 64 for decode [2].

Why does Huawei Ascend support matter?

DeepInfer supports Nvidia SM100 GPUs and Huawei Ascend 950 NPUs, according to its README. The build instructions include a separate Ascend target that requires Huawei's CANN 9.2 toolkit, alongside CUDA 13 for Nvidia [2].

That makes DeepSeek's own serving stack usable on a Chinese accelerator, not only on Nvidia hardware. The README does not compare performance between the two platforms [2].

What is still missing?

DeepInfer's first release lacks a KV cache system, disaggregation and load balancing for a full service. The README lists these as required for a complete inference service and says most of the engine functionality for disaggregation is already implemented [2].

On 20 October 2026 the Hacker News thread about the release collected more than 2,100 points,, the highest score of any story in our Hacker News scan that day [3]. When we checked the repository on 21 October, GitHub showed about 12,400 stars [1]. We covered DeepSeek's V4.1 Flash update and search API earlier (theaipin.com/trends/deepseek-updates-v4-1-flash-and-opens-search-api).

Our read

Opinion: Releasing the production engine, not a toy version, is a strong open-source move, and Ascend support is the strategic detail. Without the KV cache and disaggregation layers, though, most teams will use it to learn from DeepSeek's tricks rather than to serve traffic tomorrow.

What we could not verify

• The exact release date and time of the repository; we date it by the Hacker News post of 20 October 2026.
• DeepSeek's throughput claims, which are shown only as charts in the repository.
• How DeepInfer performs on Huawei Ascend 950 compared with Nvidia SM100.