Trigger.dev customer environment variables exposed

Posted by anurag 10 hours ago

Counter99Comment42OpenOriginal

Comments

Comment by corvad 7 hours ago

I am sorry, this is "Resolved"? How can they even consider marking this resolved in any situation. Yeah we exposed your env vars but we told you about it over email so it's all good... We dont know how or to what extend though.

Comment by janpio 2 hours ago

What would you consider appropriate action for their incident team to be able to mark the incident on their status page as resolved?

Comment by ygouzerh 2 hours ago

Never put it resolved

Comment by Joker_vD 30 minutes ago

Why? What purpose would having an eternally open incident serve?

Comment by theturtletalks 6 hours ago

The biggest case for building bespoke software is that you drop your attack surface area and it’s not as lucrative to hack one person. Don’t even expose it to the internet if you don’t need to and just run Tailscale.

Comment by bitlad 12 minutes ago

Or do onprem or selfhosted. Surface area reduces drastically.

Comment by pelagicAustral 2 hours ago

Tailscale got hacked recently as well, didnt they (?). I think they did, the whole system is poisoned, because I believe one of the vectors had to do with eMudhra certs, which again, is not an issue specific to them, or something they had any say on it... there is literally no escape these days it would seem.

Comment by thewisenerd 1 hour ago

tailscale got hacked recently? sources?

they had a priviledge escalation vulnerability that requires the attacker to already be a part of the tailnet, and more specifically allows a user-to-root escalation devices the attacker can tailscale ssh to.

> Not trying to play down the vuln (its important), but its not a free-for-all RCE that some ppl are acting like it is. [1]

eMudhra/TLS certificates have nothing to do with this (?).

[1] https://news.ycombinator.com/item?id=50266179

Comment by cbluth 5 hours ago

I think you've just explained why people rent infrastructure.

Comment by shepherdjerred 3 hours ago

This is probably correct if you aren’t a target at all, e.g. you aren’t running a business

Comment by iampims 7 hours ago

It’s bold to set status page incident to resolved.

Comment by weird-eye-issue 7 hours ago

If they fixed the exploit then what do you want them to do? Do they need to wait an arbitrary amount of time before setting it to resolved?

Comment by eddythompson80 6 hours ago

I don't disagree with you, but I think generally when the mistake involves compromised user data, the answer is a bit more emotionally driven. People get rightfully mad when things like that happens, and services usually keep the incident in a "Mitigated" status while they "work with their customers to ensure blah blah blah" or whatever. Everywhere I've personally worked the Mitigated->Resolved involves publishing a postmortem, repair items, and general assurances or safe guards based on learnings from the incident that the issue won't happen again. However, the industry as a whole doesn't have a standard for what the line between Mitigation and Resolution is.

Comment by weird-eye-issue 6 hours ago

So the active incident is now fixed and they've already reached out to impacted customers but they can't mark it as resolved yet because they are still yet to publish a more detailed blog post?

Comment by jtchang 6 hours ago

It's not that they can't mark it as resolved. It's just that it is bad business to do so.

How they handle the communication on the incident is crucially as important as remediating the original cause.

Just the fact that they are doing this means I will probably never use this service or recommend this service to anyone else.

Comment by weird-eye-issue 5 hours ago

Okay so let's just say you are one of those affected users

The founder of the company has already emailed you with details and their support team is around for you to ask any follow-up questions. Also the bug/exploit has been fully fixed

At this point are you saying you would actually be upset and leave because the incident on their status page has been marked as resolved?

Comment by tuwtuwtuwtuw 5 hours ago

Yes.

Comment by weird-eye-issue 5 hours ago

Drama queens

Comment by whilenot-dev 4 hours ago

What makes you think it's fixed? They just state that they "blocked the exploit", and that could mean anything:

- did they just take the affected API offline?

- or did they just disable the account of the attacker?

- or is the exploit actually being fixed?

Environment variables are sensible data, and marking a security incident as resolved with that little detail is quite a take. Even changelog entries for harmless bug fixes are more detailed.

Comment by weird-eye-issue 4 hours ago

I assume it's fixed based on it being marked resolved and from what was communicated. Yeah I agree it's a major issue but if it's fixed what they have done so far seems totally fine

Comment by rugma 4 hours ago

they could have explained a bit better with proper postmortem and steps taken to improve, before marking it as resolved

Comment by Traubenfuchs 3 hours ago

They are no longer exposing anything, so they no longer have an incident.

Their customers now have an incident that will be resolved once their secrets are rotated. Maybe they also need to do previous intrusion forensics… In any case, not triggers problem anymore.

Comment by cheeze 7 hours ago

I like the fancy gradient INCIDENT REPORT image as well. At least it looks fancy and AI-ey

Comment by BADCAFE 4 hours ago

This also happened to a trigger.dev competitor a few months ago, inngest.

I was able to patch before it was exploited but that definitely made me question the service.

Comment by reassess_blind 8 hours ago

Have any of these companies disclosed the actual vulnerabilities used to gain access to their systems?

Comment by bbartlomiej 9 minutes ago

Northflank mentioned they patched CVE-2026-98432. This appears to be similar, though not confirmed by trigger.dev yet

Comment by dboreham 7 hours ago

No. Which may mean they don't know.

Comment by tough 4 hours ago

the recent CVE making any non-patched TCP exposed linux server RCE-root-able seems like a good stretch to me.

But could also be any of the recent supply chain attacks hitting a dev too i guess

Comment by fathermarz 6 hours ago

Holy smokes, is security just not a priority at all anymore? Like in what world do we live in, that this is acceptable. And at the volume this has been happening, bad guys are having a FIELD DAY.

Comment by OccamsMirror 6 hours ago

What makes you think security was ever a priority?

Comment by dgellow 1 hour ago

Security has never been a priority in the software world

Comment by asdfsa32 6 hours ago

So at this rate, how long before people start hiding use of AI in development of in their SaaS and services? or people really don't care, do they?

Comment by Kiro 2 hours ago

Hand-made code is riddled with dirty hacks and exploits that LLMs have no problem finding. I would not trust any software that is not using frontier models to fix the messy human code.

Comment by asdfsa32 2 hours ago

Oh that is right, the frontier models yield amazing architecture and secure code in the form of aislop.

There is a difference between using AI to find bugs in written software than using AI to write bug free software, completely different set of skills and outcomes. Most of the high profile stuff you see is AI being driven by people who would have solved the same problem just in slower time going through the code. AI is as good as the operator.

Comment by 7 hours ago

Comment by rvz 6 hours ago

Given the recent Claude Code remote access tool, it would be wise to not install any new software, or use any package manager for a few weeks as they could have already been silently backdoored during the aftermath of that incident.

Comment by tough 4 hours ago

I was paranoid enough to just remove all my Chrome/VSCode etc extensions.

Im now thinking about just running tart VM's on my mac to do my regular stuff from.

Comment by cpursley 6 hours ago

The very fact that Nextjs vibe projects need an external (and paid!) background jobs / async service show how much of an anti pattern that this framework and similar are. Just use a sensible stack from day one that can handle concurrency out of the box like Elixir or something with established open source background jobs library. Then you can worry less about this kind of paid 3rd party nonsense. Ditto on auth. I just can't fathom outsourcing these types or core functionalities, the operational, security and cost risks make very little sense when there's libraries that do this out of the box.

Comment by multiplegeorges 5 hours ago

Yes, outsourcing these kinds of core functionality (your queue, your users table!) is insane to me.

Comment by asdf88990 1 hour ago

Auth0 and friends are useful for social logins but that should be about it, not replacing your user table wholesale like people do.

Comment by cpursley 48 minutes ago

There are open source libraries even for that sort of thing.

Comment by dgellow 1 hour ago

You can run trigger in your own infra. That has very little to do with the libraries used for your web app

Comment by cpursley 45 minutes ago

Trigger and friends exist precisely because Node is not a serious backend runtime so you have to reach for bolt-ons for every little thing. No matter if trigger is open source or managed, it's an anti-pattern. Erlang/Elixir, C#, Go etc all handle this sort of async natively, literally out of the box. Mature frameworks like Rails, Django have pretty easy to use officially libraries as well. This has been a solved problem for literally decades.

Comment by 5 hours ago

Comment by PavelGIL 3 hours ago

[flagged]