Zero-Copy GPU Inference from WebAssembly on Apple Silicon

Correct me if I'm wrong, but that reserved memory is for the framebuffer? The iBoot bootloader also reserves some memory for the framebuffer.

dGPUs bring their own VRAM because it's a different type of memory, allowing them to get higher performance than they could with DDR. The M4 Max requires 128GB of LPDDR5X to reach its ~500GB/s bandwidth. The RX Vega 64 had that same bandwidth in 2017 with just 8GB of HBM2.

To the first question: blame Windows I guess. But even on older chips, GPU code could access memory allocated on the CPU side so this didn't cap the amount of data your GPGPU code could crunch.

I'm a little confused what an actor process is. To me a process is inherently local?

This doesn't work with webpages though

I am always a bit baffled why Apple gets credited with this. Unified memory has been a thing for decades. I can still load the biggest models on my 10th gen Intel Core CPU and the integrated GPU can run inference.

The difference being that modern integrated GPU are just that much faster and can run inference at tolerable speeds.

(Plus NPUs being a thing now, but that also started much earlier. Thr 10th gen Intel Core architecture already had instructions to deal with "AI" workloads... just very preliminary)

It's irrelevant because no matter if the system memory is unified or not the point of the article is if WASM adds extra memory copiy.

That's the same no matter the physical memory system architecture.

> The people who have AI write their articles don’t care if it works or if it’s correct.

I'd build on this: The people who have AI write their articles very likely don't know how their thing works or is correct. High chance they'll stumble when they are expected to speak about whatever it is they are presenting with some authority and demonstration of knowledge. Human to human, not being able to do that = obliterates trust. Places it somewhere near the realm of misinformation, which everyone unilaterally has no interest in consuming.

Good luck to people who want to fluff expertise and present as more-capable for job prospects, the world is shit and I know there's more people who need income than there are jobs that provide for our basic human needs, but this level of AI crutching is just going to bode poorly for those who think this is going to get them where they need to go.

I think the days of on-site interviews with whiteboard tests may be drawing to a close faster than you suspect

Calculators have never been the medium in which we communicate our human experience and knowledge transfer. Calculators aren't part of the social fabric or culture. Very 2d extrapolation that somehow resulted in an alleged parallel. Language is woven deeply into civilization and our histories & been a part of our species literal survival against the most unforgiving odds/environments. Using what is effectively a ghost writer nukes trust. You cannot ascertain anything about the person behind the blog if it's clear they used AI to write it. And without that there's no way to infer expertise, rule out hallucinations, falsehoods presented as matter of fact, and the whole broad set of things LLM's get wrong because of their limitations as a technology. I have literally nothing to go off of that would prove this person knows what they are talking about. Why would anyone want to consume that?

Would it kill anyone at all to add a preamble that is forthcoming about using AI to write something? A chance to say these are my ideas and I've used claude to help me state it eloquently because <english is not my first language / i dont write well / claude said it better than i ever could> etc ? Not doing that, presenting as more capable/knowing than one probably is, is what destroys trust immediately the moment it's sniffed out that AI was used to write something.

It's irresponsible, a self-nerf, and it's annoying. Venn diagram there is basically a circle. We're all familiar with how vibe coding appears to weaken your ability to write code, like skipping the gym and expecting good muscle density. All I'm saying is people shouldn't be skipping the gym for literally communicating with each other because there's gonna be a lot of times in life where you're not gonna be able to whip out chat jippity to continue a real conversation with another person. Ceding that turf means you're willingly trading your ability to deal with real life scenarios with other human beings for short term gain. It's funny how the universe tends to find balance. Yeah, being well read and expressing ideas well is a skill, it takes work.

> language itself is one tool of communication invented to supersede body-language and grunting and noises

That's a pretty utilitarian view of language. How would it feel if everyone spoke and wrote like a PR representative? This is what an article written by an LLM is starting to sound like.

I'm even willing to argue that the way in which you convey your ideas is as important as the idea itself. Like we could all be eating soylent for our daily nutritional requirements but we don't. The taste of the food we eat is important. It's the same with writing for me

> the thought and idea is theirs, it was communicated

Are they? I don't know how much they used AI, the entire article could be written from a one sentence prompt and so I'd argue that the thoughts and ideas are not their own.

This isn't like using a spell checker, it's like using a ghost writer.

[dead]

Because its not a feature browser makers support.

it would be hard to share the same memory location with gpu, right ?

Yes, that's the right idea.

It's less about browsers, and more about server/edge/local-agent runtimes.

Wasm lets you have

- sandboxing (untrusted actor code)

- clean snapshot/restore

- portability of actor across machines

If you don’t need those properties, then yes ... native is obviously the better choice