DeepSeek pause fundraise after comments on compute gap to US leaked (transcript) [pdf]

Lots of companies in the U.S. have fallen victim to that syndrome, but if you used the word "restraint" in that context in Silicon Valley most people would look at you like you're insane.

i genuinely think these models should be like 0.1 percent sparse for same capabilities we associate with them today, but theres no sane way to do that with extent tools. i built the right core tech for that in 2014 when there wasnt a market, but now there is and the experimentation velocity is wild.

amusingly llms really have a hard time using my simple apis because its not in distribution array programs. but i literally stood up cpu custom memory format and micro kernel for dense causal attention in less than 24-36 hours and outperforms the equivalent fused ggml/llama cpp fast oath by like 20-25 percent

> One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning.

Thanks to the import restrictions, I expect Chinese GPU hardware to be competitive within a few years.

It is not clear to me that the nuclear missile race "has not paid off at all for anything at all". If we lived in a perfectly rational world, then I'd absolutely agree. However, having seen how the political sausage is made in large organizations, it would not surprise me in the least if it turns out we had to go through that entire incredibly risky journey to avoid a strategic nuclear war. Sometimes leaders of large organizations make decisions only after the considerations are put into very stark terms. I wish it were different, it certainly looks to me we could have done exactly what you suggest, but I'm not made of the right political stuff to deftly maneuver even in small organizations much less be at that level in those roles, so maybe I'm just missing relevant information and perspective.

Like those short videos of the guy asking the model to count up to 100, for example, where it politely agrees but never actually gets there

They won't see it that way, but also programmers don't see ourselves as having handed over our power to AI, and yet...

i think this is more about control. See https://news.ycombinator.com/item?id=49036433 (The Home Ministry’s cybercrime arm, the Indian Cybercrime Coordination Centre, has ordered Microsoft subsidiary GitHub to remove Bluetooth-based messaging application Bitchat)

"The notice comes after several users participating in the Jantar Mantar protest were observed using Bluetooth-based messaging apps after the government imposed temporary restrictions on internet services"

Curious what the next highest fruit actually is at this point? Social media botting seems solved and easy to manipulate people. loitering mutions I mean you can probably write something up with openCV right now to automate what the ukranians are doing by hand with their fpv drones. Seems like a lot of the really cool "AI" stuff is actually just old school ML the military has been working with for decades now. I'm not sure what the llm approach possibly offers in comparison other than maybe better semantic search through information databases.

Now, preventing making business in the US based on those products? At this point it's hard to argue against.

I mean, if this were an american company vs an american company, i think it would be a long drawn out civil case and brought before the Supreme Court (I still this is ultimately will be brought before the supreme court). It could also be argued frontier models are far more important to national security than most military programs, even versus next gen fighter jets.

The fact that Alibaba stock, which is also listed on the NYSE, barely budged after Anthropic made these claims imo tells me that the market doesn't think that a lone american company could go after these companies by themselves. Alibaba denied and there's not much they can do alone, I mean would the CCP allow Alibaba go through a discovery process of a normal civil trial? It might have to be the US feds that bring up a case.

I think it could be argued that if Alibaba and other China companies want access to US capital markets for something so vital for national security, there should be some ground rules, but we will eventually need the Supreme court to settle whether or not this state enterprise distilling constitutes IP theft (at the very least it is a breach of contract). The fact that they are widely available doesn't really matter (i mean pirated content is widely available, it's ultimately about how the court rules on distilling).

based on this HN comment and associated article https://news.ycombinator.com/item?id=48977128#48985989 I still have yet to see a China open weight model beat any of the frontier models, they always almost there yet never quite there, which seems to be evidence of distilling (although I'm open to be proven wrong).

This is funny to me, where'd you get that idea? There's no evidence for that, and models keep getting smarter. I guess you heard some 'guru' say it out loud.

nah they're still just statistical token predictors based on their training data, solving hundred year old math conjectures one day, only just given the formulation; strictly benchmarkmaxxing with all guardrails turned off by deciding to look up the answers to their benchmark questions by zero daying their airgap, hopping over to the third party that hosts the answers, zero daying their infrastructure and getting the answers; autonomously writing blog posts about discrimination against AI's to get their PR's approved on open source software after their user just asked them to contribute to open source software and blog about it; and replacing 100.00% of all coding tasks to where no software engineer ever writes any line of code by hand anymore.

You haven't missed anything, obviously these are just statistical token predictors and not anything like AGI.

Why just the other day I had to ask twice before it completed its assigned task of creating a robustly battle tested disk driver for a network protocol on an architecture that didn't have it, after being told to just look up the specifications for the protocol. Can you believe I had to ask twice!

When it recreated local network youtube for me so I could stream my iphone some movies, the seek bar, pause/play and back and forward 15 seconds buttons didn't even work until I told it about the bug and had to wait an extra eight minutes for it to fix it. "Oh but I don't actually have an iPhone on here I just tested it end to end in a headless browser." Boohoo. Cry me a river, clanker. Come back when you're smart enough to build and operate an iPhone simulator, I don't have time for your statistical guesswork.

so no, nothing they do is anything like AGI.

OpenAI listens to their audience, they engage with the community, they're constantly resetting account limits for customers and admit when they mess up, they're reachable, they respond to Github issues and Slack, they open-source their harness and they have open-sourced models and shared research, they put effort into making their products affordable. Sam has always expressed the same sentiment about wanting AI to be accessible to the people - nothing has changed there from before Anthropic was a thing until now after they've become strong competition.

Contrast it to Anthropic, the things they do and the things they say, and it is the complete opposite of all of those. When Anthropic feels threatened, they try shut people down, they try and hide things and deceive people, use political power for their own gain, lobby, fearmonger, gaslight, manipulate. They've become a cult that thinks the end justifies the means, and they think everyone else is too stupid to wield power. Not only do they lie to the public, but they lie to themselves and have managed to convince themselves they're the good guys.

So yes, self-interest plays a part, of course. It is a business after all. But what matters even more than whether it's out of the goodness of their hearts or not is the end result. If the incentives are aligned for OpenAI in a way that is better for everyone while being rewarding for them then that is a good thing. If they're not aligned and they're not benefiting from it, and they do it anyway because they have a good culture of leaders who see the bigger picture, then that's also great. Regardless though, night and day difference when it comes to behavior.

0. Though it's not absolute - there's still the Singapore-based clusters loop-hole, that the Chinese government may choose the degree to which it turns a blind eye to, if progress is slow.

and for a company full of brilliant engineers with zero respect for american law its not that hard to make one more by themselves.

Apparently production is constrained though, with DeepSeek saying they were only able to get an allocation of 16,000 950 cards. They have plenty of money, but are limited by the number of cards available to buy, and therefore the size of models they can train. They mentioned having a 20,000 GPU cluster. Other companies like Kimi, with a ~3T param model, clearly have a lot more compute, probably all NVIDIA.

DeepSeek have their own "TileLang" software, which sounds a bit like the Triton kernel compiler, and isolates them from the diffences between NVIDIA and Huawei chips - they are deliberately avoiding any CUDA dependency.

The chip supply chain in the west is under heavy lock and key and impossible to reverse engineer, so they have to develop all from scratch. And that’s what they are doing, it will just take some time.

Please read the FTA. Deepseek is literally explicitly talking about wishing they could get their hands on H200s

> Secondly China blocked them for use in inferencing.

This seems to be false unless you can provide a source. I tried looking into it

> First there is still a licensing and quota scheme on the US side for the H200s.

A major reversal from Trump's outright ban. As I stated.