Ask HN: What's the best hands-on path to learn ML inference infrastructure?

Posted by censor5 1 day ago

I'm a backend engineer with 8+ years of experience. Most of my work has been APIs, distributed systems, streaming/real-time systems and cloud-infra.

I'm trying to move towards ML inference infrastructure: model serving, batching, etc. but based on my experience, it has been difficult to get a callback for such jobs.

For people working in this area: what projects or experience would actually convince you that a backend engineer is ready to work on inference infra?

Comments

Comment by vismit2000 18 hours ago

How GPT, Claude, and Gemini are actually trained and served – Reiner Pope: https://www.youtube.com/watch?v=xmkSf5IS-zw

Comment by sigbottle 22 hours ago

Following this. my plan has been to start simple and eventually build out the inference stack piece by piece, however simple. Take some open weights model and from first principles build kernels and a harness for it.

Given that AI is already so good at swe at this point, it feels harder to get a callback because they're probably super AI pilled and I have no idea how good AI is at writing the things that these guys need for performance (I mean, judging from my local work, they seem pretty good at what I'm trying to learn, but I am a beginner in the field). I've received no callbacks either though so I don't know.

Comment by tcp_handshaker 1 day ago

Since Anthropic trains on Sagemaker on AWS, what about some Cloud Certifications?

Comment by lpsatwork 16 hours ago

[flagged]

Comment by araailabs 8 hours ago

[dead]