Dagger: Define software delivery workflows and dev environments

Hi Kasper, good to see you here! Thank you for the detailed feedback.

1. Yes we got over-excited with the agent runtime use case. We stand by the LLM implementation because we never compromised on the integrity of Dagger's modular design. But our marketing and product priorities were all over the place. We're going to refocus on the original use case: helping you ship software, and more particularly building & testing it.

2. Modules have warts but they are essential. We will continue to improve them, and remain committed to them. Without this feature, you have to write a complete standalone program every time you want to build or test your software. It's too much overhead.

3. Yes you are right. We really thought we could co-exist with CI runners, and get good performance without reinventing the wheel. But for various reasons that turned out to not be the case. So we're going to ship vertically integrated runners, with excellent scalability and performance. DM me if you want early access :)

TLDR: yes we needed to choose and execute. We have, and we are.

Thank you again for the feedback.

Same - we began the migration to Dagger but then switched to just Docker-In-Docker and custom scripts which run vendor-agnostic

Can you explain/link to why you can't really use this without their cloud product? I'm not seeing anything at a glance, and this looks useful for a project of mine, but I don't want to be trapped by limitations that I only find out about after putting in weeks of work

dagger was originally CUE-based, but there was not enough demand so it was dropped. https://dagger.io/blog/ending-cue-support

Same: the promise of defining CI/CD in code is good, but the implementation didn't make sense to me even before the LLM stuff

Do you have examples of your CUE and Github Actions setup?

Do you have any more details on using Cue with GHA? I've also looked at Dagger and been quite disappointed with it (and their terrible documentation).

What I don't get is why would someone code in the terrible GitHub actions dsl which only runs on GitHub actions and nowhere else when there are so many other options that run perfectly fine if you just run it from GitHub actions.

They have SDKs in many languages, not just Go. I use the python one. And they use code, not a DSL.

Hi, I'm the founder of Dagger. I can't speak to the negativity, but if you're looking for a way to make your CI more portable, I recommend joining our Discord and asking our community directly about the pros and cons of using Dagger. Even if you don't end up using it, there are a lot of people there who are passionate about CI and can recommend other alternatives, in a more constructive and pragmatic way than you are getting here.

You can filter input directories before they are loaded, to avoid this. There's no reason you shouldn't be able to get precise cache invalidation in a large monorepo. If you want, DM me on the Dagger Discord and I'll help you out!

I used the old CUE-based version when it came out, and was really excited about it. I liked it, and enjoyed working with CUE, but the API was clunky and incomplete.

Then they completely abandoned not just the CUE frontend, but CUE altogether (while strenuously denying that they were doing so) for a GraphQL-based rewrite that focused on letting people use popular general-purpose languages to construct their workflows. The initial rollout of this was not feature complete and only supported imperative languages (Python and TypeScript, IIRC), which I didn't like.

Instead of porting everything over to all their new interfaces, I hopped off the train and rewrote all of our portable pipeline scripts in Nix, via Devenv. At the time, I'd never used Devenv before, but getting the work done that time still took maybe a tenth of the time or less. More than anything else, this was due to not having to fuck around with the additional overhead Docker entails (fussing with mount points, passing files from one stage to another, rebuilding images, setting up VMs... all of it). I got the reproducibility without the extra bullshit, and got to work with interfaces that have proven much more stable.

I still think there's a place for something like Dagger, focused just on CI, perhaps even still using Docker as a deployment/distribution strategy. But I no longer trust Dagger to execute on that. I think a proper external DSL (probably special-purposw but still Turing-complete, e.g., Nickel) is the right fit for this domain, and that it should support multiple means of achieving repeatability rather than just Docker (e.g., Nix on bare metal and Podman, to start). An option to work on bare metal via reproducible environment management tools like Nix, Guix, or Spack is a valuable alternative to burdensome approaches like containers.

I haven't looked at Dagger in several months, but the other big piece that is missing for portable CI workflows is a library that abstracts over popular CI platforms so you can easily configure pull/merge request pipelines without worrying about the implementation details like what environment variables each platform exposes to indicate source and target branch.

Idk anything about all the AI horseshit; I was off the Dagger bandwagon before they took that turn. I don't know if it's serious or a nominal play to court investors. But that kind of pivot is another reason not to build core infra on top of the work of startups imo. If the product is 70% of what you want, you have no way of knowing whether filling that 30% gap is something the maintainers will suddenly pivot away from, even if their current direction looks aligned with yours.

I'd recommend considering tools in this space only if (a) they're already close to 100% of what you need and (b) they're open-source. Maybe you can relax (a) if it's really easy to extend the codebase (I find this to be true for Devenv's Nix modules, for example.)

It's also a type of blade, been around for hundreds of years!

/s I've never heard of Dagger the DI framework but I have heard of this Dagger. Names will overlap sometimes and it's not a big deal.

Yeah I initially thought of this coming from android

have used it, and i do like it, but the licensing situation is not great. It open source but its not free software by any means.

I've tried it, but there's too much "sorry not in the open-source edition, please buy the entreprise edition" stuff all around, which makes it quite unusable

Same, the marketing was even worse a few months ago.

Works fine for us for glueing a bunch of CI steps together which would've been a pile of bash otherwise. Works well with depot.dev caches. We don't use it for anything AI either.

No, it's code based CI/CD not a build system.

I use it to do builds in our monorepo. We got onboard before the LLM trash features. The base design is ok but there's things I'd do different today if I knew the build stuff would fade away for the LLM push.

It’s actually a CI/CD as code tool, where some pieces can be LLM agents.

But, the marketing heavily focuses on LLM stuff to the point of making everyone confused.

I've never tried it. My first impression based purely on reading the homepage is it adds complexity to something I can already do with a Dockerfile and bash. What can it do that I can't already do more simply?

It does a pretty good job of caching and that does help speed up builds. I also run all of my end to end tests from it because I can coordinate secrets and clusters of containers through it.

Instead of YAML workflows you write code, and there is an analog of the Github Actions Marketplace (the Daggerverse).