The affordability of DeepSeek is a myth: The revolutionary AI actually cost $1.6 billion to develop

DeepSeek's new chatbot boasts an impressive introduction: "Hi, I was created so you can ask anything and get an answer that might even surprise you." This AI, a product of the Chinese startup DeepSeek, has quickly become a major market player, even contributing to a significant drop in NVIDIA's stock price.

DeepSeek's success stems from its innovative architecture and training methods. Key technologies include:

• Multi-token Prediction (MTP): Instead of predicting words individually, MTP forecasts multiple words simultaneously, boosting accuracy and efficiency.
• Mixture of Experts (MoE): This architecture uses multiple neural networks, accelerating training and improving performance. DeepSeek V3 utilizes 256 networks, activating eight for each token.
• Multi-head Latent Attention (MLA): MLA focuses on crucial sentence parts, repeatedly extracting key details to minimize information loss and ensure nuanced understanding.

DeepSeek's initial claim of a mere $6 million training cost for DeepSeek V3, using only 2048 GPUs, is misleading. SemiAnalysis revealed a far more extensive infrastructure: approximately 50,000 Nvidia Hopper GPUs (including 10,000 H800s, 10,000 H100s, and additional H20s) distributed across multiple data centers. This translates to a server investment of roughly $1.6 billion and operational expenses near $944 million.

A subsidiary of High-Flyer, a Chinese hedge fund, DeepSeek owns its data centers, fostering control and innovation speed. This self-funded approach enhances flexibility and decision-making. The company attracts top talent, with some researchers earning over $1.3 million annually, primarily recruiting from Chinese universities.

While DeepSeek's $6 million figure only reflects pre-training GPU costs, ignoring research, refinement, data processing, and infrastructure, the company has invested over $500 million in AI development. Its lean structure allows for efficient innovation compared to larger, more bureaucratic competitors.

DeepSeek's success highlights the competitive potential of well-funded independent AI companies. However, its achievements are rooted in substantial investment, technological breakthroughs, and a strong team. Claims of revolutionary budget efficiency are exaggerated. Still, DeepSeek's costs remain significantly lower than competitors; for example, DeepSeek's R1 model cost $5 million to train, while ChatGPT4 cost $100 million.