A Roblox cheat and one AI tool brought down Vercel's platform
Posted by bishwasbh 5 hours ago
Comments
Comment by kstrauser 3 hours ago
So sensitive doesn’t mean encrypted. It means the UI doesn’t show the dev what value’s stored there after they’ve updated it. Not sensitive means it’s still visible. And again, I presume this is only a UI thing, and both kinds are stored encrypted in the backend.
I don’t work for Vercel, but I’ve use them a bit. I’m sure there are valid reasons to dislike them, but this specific bit looks like a strawman.
Comment by nextaccountic 21 minutes ago
Yeah, I'm very confused. It's not possible to encrypt env vars that the program needs; even if it's encrypted at rest, it needs to be decrypted anyway before starting the program. Env vars are injected as plain text. This is just how this works, nothing to do with Vercel.
This situation could some day improve with fully homomorphic encryption (so the server operates with encrypted data without ever decrypting it), but that would have very high overhead for the entire program. It's not realistic (yet)
Comment by rcxdude 1 hour ago
Comment by CodesInChaos 27 minutes ago
Comment by dnnddidiej 19 minutes ago
Comment by lemagedurage 4 minutes ago
Comment by ErroneousBosh 1 hour ago
PoC or GTFO.
I think you'll find it's a bit harder to do than you expect.
Comment by dnnddidiej 22 minutes ago
Comment by nallerooth 1 hour ago
Comment by dkersten 2 hours ago
Comment by darkwater 1 hour ago
Comment by gmerc 1 hour ago
Comment by ErroneousBosh 1 hour ago
One for which the Context.ai employee needs to have their arse booted up and down the car park for.
Comment by sitkack 47 minutes ago
You can blame individuals, but security is a property of the system.
Comment by yoaviram 4 hours ago
Comment by trick-or-treat 3 hours ago
Comment by TheDong 3 hours ago
If you spin up an EC2 instance with an ftp server and check the "Encrypt my EBS volume" checkbox, all those files are 'encrypted at rest', but if your ftp password is 'admin/admin', your files will be exposed in plaintext quite quickly.
Vercel's backend is of course able to decrypt them too (or else it couldn't run your app for you), and so the attacker was able to view them, and presumably some other control on the backend made it so the sensitive ones can end up in your app, but can't be seen in whatever employee-only interface the attacker was viewing.
Comment by trick-or-treat 3 hours ago
Comment by bauruine 3 hours ago
What's best practice to handle env vars? How do poeple handle them "securely" without it just being security theater? What tools and workflows are people using?
Comment by trick-or-treat 3 hours ago
However I do feel now like my sensitive things are better off deployed on a VPS where someone would need a ssh exploit to come at me.
Comment by TheDong 3 hours ago
Notice how their tutorial says "run 'dotenvx run -- yourapp'". If you did 'dotenvx run -- env', all your secrets would be printed right there in plaintext, at runtime, since they're just encrypted at rest.
The equivalent in vercel would be encrypted in the database (the encrypted '.env' file), with a decryption key in the backend (the '.env.keys' file by default in dotenvx) used to show them in the frontend and decrypt them for running apps.
Comment by ErroneousBosh 1 hour ago
Comment by jakewins 3 hours ago
Various certifications require this, I guess because they were written before hyper scalers and the assumed attack vector was that someone would literally steal a hard drive.
A running machine is not “at rest”, just like you can read files on your encrypted Mac HDD, the running program has decrypted access to the hard drive.
Comment by TeMPOraL 1 hour ago
Comment by otabdeveloper4 1 hour ago
(And modern Linux is unusable without root access, thanks to Docker and other fast-and-loose approaches.)
Comment by ErroneousBosh 1 hour ago
Because I never do, unless I'm down in the depths of /var/lib/docker doing stuff I shouldn't.
Comment by burnished 3 hours ago
Comment by otabdeveloper4 1 hour ago
Comment by rcxdude 1 hour ago
Comment by magackame 3 hours ago
Comment by TheDong 3 hours ago
For non-sensitive environment variables, they also show you the value in the dashboard so you can check and edit them later.
Things like 'NODE_ENV=production' vs 'NODE_ENV=development' is probably something the user wants to see, so that's another argument for letting the backend decrypt and display those values even ignoring the "running your app" part.
You're welcome to add an input that goes straight to '/dev/null' if you want, but it's not exactly a useful feature.
Comment by QuantumNomad_ 25 minutes ago
Piping to /dev/null is of course pointless.
What you really want is the /dev/null as a Service Enterprise plan for $500/month with its High Availability devnull Cluster ;)
Comment by ethin 4 hours ago
Comment by post_below 39 minutes ago
The thing that concerns me is that even at a site like HN, where a lot of people are very familiar with LLMs, it seems to be passing.
I hate to think this will become the norm but it's not the first HN linked post that's gotten a lot of earnest engagement despite being AI generated (or partly AI generated).
I'm very comfortable with AI generated code, if the humans involved are doing due diligence, but I really dislike the idea of LLM generated prose taking over more and more of the front page.
Comment by curiousObject 2 hours ago
So I believe the author has exposure to the issue and interest in understanding it, that’s more than AI alone has got.
Comment by progbits 4 hours ago
Comment by mchl-mumo 3 hours ago
Comment by progval 3 hours ago
Comment by croemer 2 hours ago
Comment by paganel 1 hour ago
Comment by parable 2 hours ago
Comment by nilsbunger 3 hours ago
Comment by sitkack 44 minutes ago
Comment by EdwardDiego 4 hours ago
And I thought it was bad when my son got compromised by a Roblox cheat, but they only they grabbed his Gamepass cookies and bought 4 Minecraft licenses, which MS quickly refunded...
Comment by uyzstvqs 1 hour ago
Comment by azalemeth 1 hour ago
Failed to verify your browser Code 11 Vercel Security Checkpoint, arn1::1776759703-rtDgRAtRyXvjD4IoU4RbqvkGmvQQCP7H
Gah.
Comment by jesse_dot_id 4 hours ago
If I don't see asterisks, I'm not hitting save on the field with a secret in it. Maybe they were setting them programmatically? They should definitely still be looking to pass some kind of a secret flag, though. This is a weird problem for a company like Vercel to have.
Comment by apgwoz 4 hours ago
Comment by SOLAR_FIELDS 3 hours ago
Comment by mudkipdev 4 hours ago
Comment by NitpickLawyer 3 hours ago
Comment by voidUpdate 2 hours ago
Comment by rightbyte 1 hour ago
Comment by voidUpdate 1 hour ago
Comment by ashirviskas 54 minutes ago
Comment by varun_ch 4 hours ago
Comment by trick-or-treat 3 hours ago
Comment by aroido-bigcat 2 hours ago
Tools that sit in the middle (like Context.ai) end up becoming a pretty large attack surface without feeling like one.
Comment by jFriedensreich 1 hour ago
(Of course there are tons of other red flags not looked at in the article, eg. how does an employees machine get access to production systems and from there access to customers connected with oauth and how does the attacker get to env vars from a google workspace account)
Comment by trick-or-treat 3 hours ago
Initially, we identified a limited subset of customers whose Vercel credentials were compromised. We reached out to that subset and recommended that they rotate their credentials immediately.
At this time, we do not have reason to believe that your Vercel credentials or personal data have been compromised.
Comment by sitkack 42 minutes ago
Comment by R41 4 hours ago
Comment by ryanisnan 4 hours ago
We'll keep dangerous devices like the SuperBox in our homes, if it helps us get access to free movies and tv.
We'll use single-use plastics, even if we know they're bad for the environment, because they're just so damn easy.
We'll let AI run that thing for us, because it's just too easy.
A whole generation has grown up without knowing what it was like to infect your computer with AIDS trying to download an MP3, and it shows. That caution will come back, just at a terrible cost.
Comment by notpachet 3 hours ago
More generically, our species' Achilles heel is our inability to factor in the long-term cost of negative externalities when evaluating processes that yield short-term positive results.
Comment by kauli 1 hour ago
Comment by trick-or-treat 3 hours ago
Comment by ChrisArchitect 4 hours ago
Vercel April 2026 security incident
Comment by doctorpangloss 3 hours ago
Comment by alanmercer 1 hour ago
Comment by tesders 2 hours ago
Comment by agent-kay 2 hours ago
Comment by souravroyetl 3 hours ago