The future of Terraform CDK
Posted by mfornasa 5 days ago
Comments
Comment by vbernat 5 days ago
I did try Pulumi a while back, but the compatibility with Terraform modules was not great, so I've switched to CDKTF, which can handle unmodified modules. Dunno if I'll switch back to Pulumi or just use OpenTofu directly.
Comment by jjice 5 days ago
All their branding does this now, including the HashiCorp logo on their website [0]. There's gotta be a name for this specific branding pattern, but I don't know it.
Comment by stingraycharles 5 days ago
Comment by huddo121 5 days ago
Comment by pretext-1 5 days ago
Comment by dandellion 5 days ago
Comment by miki123211 5 days ago
Granted, that was in the 1930s or something, but still.
Comment by abhiyerra 5 days ago
Comment by dandellion 5 days ago
Comment by packetlost 5 days ago
Comment by willio58 5 days ago
Comment by here2learnstuff 5 days ago
Comment by Garlef 4 days ago
One example:
You can't really build custom TS providers for AWS resources.
Why?
Because this feature is built using the compilation magic that makes inline lambdas work.
But the compilation step omits the AWS SDKs since these are present in a lambda anyways. So you can't use the AWS SDK in custom providers.
Comment by lokar 5 days ago
Comment by weakfish 5 days ago
Comment by jen20 5 days ago
Probably some specifics might be more useful there...
Comment by katdork 5 days ago
Comment by purpleidea 5 days ago
Comment by lighthazard 5 days ago
Comment by mfornasa 5 days ago
Comment by smithcoin 5 days ago
Comment by benatkin 5 days ago
Comment by Hamuko 5 days ago
Comment by cube2222 5 days ago
That also includes a new “enabled” meta argument, so you don’t have to hack around conditional resources with count = 0.
[0]: https://opentofu.org/blog/opentofu-1-11-0/
Disclaimer: affiliated with the project
Comment by lijok 5 days ago
Comment by cube2222 5 days ago
It's pretty seamless.
Comment by joombaga 5 days ago
Comment by cube2222 5 days ago
There’s some samples in the docs[0] on safe access patterns!
[0]: https://opentofu.org/docs/language/meta-arguments/enabled/
Comment by darkwater 5 days ago
EDIT: Oh just realized the sibling asked the same, but the doc doesn't state that clearly, although it seems to me that the doc implies that yeah, it doesn't use the array form anymore.
Comment by lijok 5 days ago
Comment by Hamuko 5 days ago
Comment by atonse 5 days ago
Comment by Pet_Ant 5 days ago
Comment by cr125rider 5 days ago
Comment by selkin 5 days ago
Or it's legal trying to preempt a risk.
If it was the author just wanting to point at IBM, they'd mention it just once or twice, but using that awkward phrase throughout the text makes me think it was an edit mandated by a careful lawyer.
Comment by nsonha 5 days ago
Common sense would be IBM mandating that branding, as opposed to Hashicorp.
Comment by roboben 5 days ago
Comment by firesteelrain 5 days ago
Comment by viraptor 5 days ago
Comment by richardfontana 5 days ago
Comment by firesteelrain 5 days ago
1. https://business.adobe.com/customer-success-stories/red-hat....
2. https://www.openpr.com/news/4100338/linux-operating-system-m...
Comment by richardfontana 5 days ago
Comment by crimsonnoodle58 5 days ago
HCL just does not have the modularity and expressiveness that Python, or other languages CDKTF supports.
I guess I'll spend another year migrating to Pulumi now..
Comment by lijok 5 days ago
Comment by crimsonnoodle58 5 days ago
Comment by lijok 5 days ago
Comment by everforward 5 days ago
SQL is also declarative, but incredibly expressive. A thousand character query contains enough complexity that it's hard to reason about. A thousand characters of Terraform will barely stand up a CRUD app on AWS.
Designing a language from first principles for this was a mistake. HCL is awful; they should have gone the Starlark route and made a stripped-down version of an existing language instead of making their own language from scratch. This feels like the worst of both worlds. The language is practically imperative, but it has its own syntax that isn't useful outside of this one single domain.
Comment by solatic 5 days ago
Anyway you shouldn't have too many resources in a single Terraform workspace, for performance reasons. The real issues with Terraform come when you start to want to orchestrate different workspaces triggering each other, and trying to write that orchestration language, which itself would be declarative.
Terraform built a Stacks feature, but support is Terraform Cloud-only. OpenTofu has issues in the area that have been open for years: https://github.com/opentofu/opentofu/issues/931 https://github.com/opentofu/opentofu/issues/2860 and progress is slow, in part (IMO) because a genuine solution requires server-side evaluation (i.e. triggering applies as Kubernetes Jobs) and the open-source implementation of Terraform Enterprise/Cloud is a completely separate project with a completely different group of maintainers, Terrakube.
Comment by Uvix 5 days ago
Comment by solatic 4 days ago
You split the workspace into smaller workspaces precisely to tell Terraform that you haven't made any changes to the networking layer, so don't bother trying to refresh the status of the networking layer to see if any changes are needed, it's not relevant when you're trying to scale up your Kubernetes cluster or whatever.
Comment by pxc 5 days ago
I worry that comments like this lead the average newbie to overlook (or worse, avoid) declarative languages (both among DSLs and among general-purpose languages) because they will associate the term with hacky, confining, gotcha-ridden messes like Terraform's HCL, Azure DevOps' standards-breaking "YAML" DSL, etc.
Incidentally I agree that a language like Python is a terrible fit for this domain, but it's also plain to see that HCL is a shitty tarpit. It's not hard to understand why people want to get away from HCL.
And concretely, you can use Pulumi in a pure functional style with F# or Scala.
Comment by dastbe 5 days ago
It also makes it easier to reason about that output as you can avoid awkward iteration in your declarative spec.
Comment by crimsonnoodle58 5 days ago
The end result is still declarative, your just using an imperative language to keep your IaC DRY.
Comment by lijok 5 days ago
Comment by Spivak 5 days ago
At the point where we are templating Terraform files we've already lost the plot. You might as well get to use a real programming language.
Comment by theevilsharpie 5 days ago
Not only does grafting this functionality onto a language after-the-fact inevitably result in a usability nightmare, it also gets in the way of enabling developer self-service for these tools.
When a developer used to the features and functionality of full-featured language sees something ridiculous like Terraform's `count` parameter being overloaded as a conditional (because Terraform's HCL wasn't designed with conditional logic support, even though every tool in this class has always needed it), they go JoePesciWhatTheFuckIsThisPieceOfShit.mp4 at it, and just kick it over to Ops (or whoever gets saddled with grunt work) to deal with.
I'm seeing the team I'm working with going down that same road with Helm right now. It's just layers of templating YAML, and in addition to looking completely ugly and having no real support for introspection (so in order to see what the Helm chart actually does, you essentially have to compile it first), it has such a steep learning curve that no one other than the person that come up with this approach wants to even touch it, even though enabling developer self-service was an explicit goal of our Kubernetes efforts. It's absolutely maddening.
Comment by bigstrat2003 5 days ago
Comment by JojoFatsani 5 days ago
Comment by pizza234 5 days ago
I also find refactorings considerably harder in a declarative language, since configurations have a rigid structure.
Comment by vanschelven 5 days ago
That seems like rather short notice.
Comment by HashiCorps 5 days ago
[0] - https://news.ycombinator.com/item?id=46192130#46198058
Comment by yearolinuxdsktp 5 days ago
Terraform CDK had promise as a blessed infrastructure-as-actual-code solution from the official maintainer of Terraform, so easier to sell internally rather than something from a new vendor like Pulumi. I feel sorry for those teams who have migrated to TF CDK.
Internal vs external DSLs explained in the middle of this page: https://martinfowler.com/bliki/DslQandA.html
Comment by chuckadams 5 days ago
Comment by jeroensoeters 5 days ago
Comment by mfornasa 5 days ago
Comment by lillecarl 5 days ago
Comment by mfornasa 5 days ago
Comment by preisschild 5 days ago
Comment by an-engineer 19 hours ago
There is currently a live proof of concept fork of the main cdktf repository that one developer made: https://github.com/TerraConstructs/terraform-cdk
And one Open Tofu developer said he and some other Open Tofu developers would be happy to collaborate with that community-driven effort to keep CDKTF alive: "The OpenTofu maintainers are happy to collaborate with that project once it's up and running, but we will not be directly involved." https://github.com/opentofu/opentofu/issues/3585
Comment by kennu 5 days ago
Comment by GardenLetter27 5 days ago
Comment by tapoxi 5 days ago
If I really need to migrate off of AWS at some point I'll throw an LLM at it.
Comment by manquer 5 days ago
In my experience claude/codex to wrangle CDK constructs be complicated, it frequently hallucinates constructs that simply do not exist, options that are not supported etc.
While they can generate IaaC component mostly okay and these problems can be managed, Iterations can take a lot of time, each checkpoint, goes the deploy/ rollback cycles in CF. CloudFormation is also not particularly fast, other IaaC frameworks are not that different.
Running an agent to iterate until it gets it right is just more difficult with IaaC refactor projects. Hallucinations, stuck loops and other issues, can quickly run the infra bill up not to mention security.
Comment by manishsharan 5 days ago
I got a nice and comprehensive infrastructure requirement document out of this.
Now I am using it to create Terraform repo , deploying it via OpenTofu and comparing it to my existing AWS cloud formation . This part is still a WIP .
Comment by raw_anon_1111 5 days ago
I am very detailed about all of the security group requirements, tell it that I don’t need Internet access and tell it which VPC endpoints. I don’t do “agentic coding”.
Comment by ryandvm 5 days ago
Besides, actual full-scale Cloud migrations are exceedingly rare.
Comment by emoII 5 days ago
Comment by raw_anon_1111 5 days ago
Comment by thayne 5 days ago
Comment by GuinansEyebrows 5 days ago
Comment by exidy 4 days ago
Not picking on you personally but having had this conversation many times over many years with many clients I find it confounding. Oftentimes TF itself was heavily promoted as a way of "avoiding lock in".
Well guess what? Now you're locked-in to IBM, whose motivations may not be perfectly aligned to you goals of simply and efficiently using your cloud provider of choice to deliver your business outcomes.
What we refer to as lock-in is simply an expression of risk, with one axis being the cost of getting off $solution and the other being the likelihood of needing to do so. Having stepped through this exercise a few times, the cost of rewriting your e.g. AWS API Gateway + Lambda + SQS + RDS + CloudWatch etc architecture invariably vastly dwarfed the cost of changing the IaC language it is expressed in.
Anytime you feel the urge to overbuild on a cloud provider's services, stop, and do a really rigorous cost/benefit analysis. If you truly have unique drivers the data should tell the story.
Comment by rendaw 5 days ago
- More accurate types/type safety than the CDK (for static feedback on required parameters, etc)
- No CLI required - just plain Rust (provider definitions can be published as normal rust packages so you don't have to generate them yourselves, and I've published a bunch of common ones - docker, aws, etc)
- Simpler: Terraform CDK had this crazy flow where it (go code) generated typescript code then used some transpiler to generate target language code. The output wasn't pretty, and there were bugs. Your project directory would get filled with boilerplate generated files.
It generates tf json files and has a fairly safe way for handling variable interpolation and escapes - I haven't hit any weird bugs with it.
Comment by tetha 5 days ago
It's a bit of an "Caveman solve problem with rock" approach, but for very regular projects it's great. A new cluster is some group vars, larger changes to the structures can be easily reviewed - and if you really really have to, you can also just modify the generated code by hand to fix something your generation code can't deal with right now.
Comment by scruff3y 5 days ago
Comment by rendaw 5 days ago
When writing stacks you need normal language features: loops, yes, but also if statements, reuse (functions), being able to do stuff like complex string parsing and re-formatting, etc etc.
HCL supports loops, modules can be kind-of used as functions with lots of footguns, there are awful hacks for some other things, and some stuff just couldn't be done.
Comment by cholantesh 5 days ago
Comment by stackskipton 5 days ago
Downsides are doing infrastructure in a programming language was always problematic unless developer was skilled at Ops which most who used TFCDK were not.
Comment by cholantesh 5 days ago
Comment by coenhyde 5 days ago
The real value of cdktf was more dynamic infrastructure provisioning while still having the plan / apply pattern.
Comment by mfornasa 5 days ago
Comment by resonious 5 days ago
Terraform is ugly but it works well enough for me and seems ingrained enough to be durable to this kind of thing (i.e. I bet for sure the community would pick it up (I wish I could say that I'm part of that community but I can't say I use it quite that often))
Comment by re-thc 5 days ago
This is clearly a business decision rather than technical.
Pulumi is meant to be semi-automated (in generating the bridges) so perhaps is slightly better off in maintenance.
Comment by srmatto 5 days ago
Comment by sshine 5 days ago
Comment by lillecarl 5 days ago
Comment by madjam002 5 days ago
Comment by deadfece 5 days ago
Comment by NeckBeardPrince 5 days ago
Comment by tonnydourado 5 days ago
Comment by chickensong 5 days ago
Comment by egorfine 5 days ago
Comment by Havoc 5 days ago
Comment by borisbanjo 5 days ago
CDKTF stacks are great and the construct pattern gives you modularization without all the baggage.
Comment by nevon 5 days ago
Comment by nijave 5 days ago
That also means you end up with things like the language's native JSON not doing what you expect and having to use a special Terraform function call.
Comment by zer0-c00l 5 days ago
Comment by leetrout 5 days ago
You can combine it with tools like Dhall or my personal preference Jsonnet instead of imperative languages for an interesting experience for reusable pieces outside of module concepts.
Comment by Duologic 5 days ago
I wrote a generator a little while ago that can create jsonnet libraries from the TF schemas: https://github.com/Duologic/soysonnet
Example lib here: https://github.com/Duologic/soysonnet-aws
I only needed it for AWS so I didn't spend more time on it.
Comment by joeduffy 5 days ago
Comment by here2learnstuff 5 days ago
Comment by moltar 5 days ago
Comment by rubenvanwyk 5 days ago
Comment by thayne 5 days ago
Although, I would hope a fork of cdktf would target opentofu instead of terraform.
Comment by callumgare 5 days ago
Comment by moltar 5 days ago
Comment by ptdorf 4 days ago
transpiled to JSON
Comment by dev_l1x_be 5 days ago
Comment by jpitz 5 days ago
Also, for large providers like AWS, GCP, Azure, etc - these are often largely authored by the hyperscaler themselves, for better or worse.
Comment by lijok 5 days ago
Comment by dev_l1x_be 3 days ago
Comment by kbar13 5 days ago
Comment by DangitBobby 5 days ago
Comment by lijok 5 days ago
Comment by lloydatkinson 5 days ago
Comment by so0k3311 4 days ago