Show HN: Ctrlb-decompose: Strip the noise from logs before sending to LLMs

Posted by ruhani_grover 3 hours ago

Counter45Comment6OpenOriginal

Comments

Comment by hugopuybareau 1 hour ago

Have you benchmarked it in real condition to test if it truly reduces costs ? I feel like many of those type of tools often end up making the llm more lost about the logs and make it more spend than just feeding the raw logs or letting it code to discover the content itself

Comment by pranav_rastogi 2 hours ago

Finally, a tool that tackles this problem without forcing you to adopt an entire ecosystem. I appreciate how this leans into the Unix philosophy by just focusing on the decomposition step and doing it well. Really appreciate the modular architecture here.

Comment by ruhani_grover 1 hour ago

Thank you and we do really appreciate it. The Unix philosophy was certainly one of the influences upon us, we wanted CtrlB Decompose to carry out one task well, converting noisy logs into structured and compact representations which are easier for both humans and LLMs to handle.

Comment by firasd 1 hour ago

Very interesting.

A few days ago I made something to use in addition to head/tail: Venetianblinds.js shows equally spaced chunks of a file https://github.com/firasd/venetianblinds

So I can kinda see these as being part of the same workflow like VBlinds for 'what does this file even look like' before calling the decompose

Comment by nextblock 2 hours ago

[flagged]

Comment by CuriousBuilder 2 hours ago

[flagged]

Comment by gokohl 2 hours ago

[flagged]

Comment by ruhani_grover 1 hour ago

Thanks! Happy to hear it solved a real problem for you.

Comment by syron 2 hours ago

[flagged]

Comment by ruhani_grover 1 hour ago

Thanks a lot, I truly do appreciate the thoughtful feedback.

We agree that there are a great many interesting downstream possibilities concerning OpenTelemetry, correlation, derived metrics, and more sophisticated telemetry pipelines once you are able to reliably extract structure from noisy logs.

To us, CtrlB Decompose represents the initial step. Although we launched it as a standalone tool for producing logs that are friendly to LLMs, it also feeds into part of CtrlB's special log compression pipeline. The identical structured representation aids in reducing both storage space and token usage without losing the information required for search, investigation, and AI-assisted debugging.

The idea you've advanced regarding enriching the logs with trace context and generating higher-level signals before sending them on downstream is indeed a very interesting one, and we are currently considering it as the project develops. Thank you for bringing it to our attention!