DeepInfer explained: how DeepSeek's inference engine works

DeepSeek has open-sourced part of the technology it uses to run its models at scale. It is called DeepInfer. This is a walk through how DeepSeek's inference engine works, which optimisations it ships

DeepSeek has open-sourced part of the technology it uses to run its models at scale. It is called DeepInfer. This is a walk through how DeepSeek's inference engine works, which optimisations it ships and why they matter so much for the performance and the cost of AI. When people talk about DeepSeek, the conversation almost always ends up on the models and its lower price: how many parameters they have, how well they code, how they score against Claude, GPT or Gemini. Another part of its technolo