Ask HN: How are you using LLMs in production?

Posted by Anon84 4 days ago

I’m curious how people are leveraging LLMs in production. We’re all familiar with using them to write code through your IDE, or agent, etc… but what applications have you built that use LLMs as part of the functionality?

    - What applications/industries have you found them to be the most useful in?
    - What tools do you use for orchestration? LangGraph/CrewAI/built your own/etc?
    - What models? Centralized or locally deployed?

Comments

Comment by morisil 3 days ago

Regarding applications/industries - after software development as an industry I see scientific research benefiting the most from agentic AI and autonomous machine reasoning. Another domain which is still not saturated would be something I call "personal guardian angel", which usually extends to whole families. Quite often people use OpenClaw for these use cases. Personally I build my own harnesses. One instance is set up as a CEO of my organization, taking care of operations, and another one, completely separate, operating over private knowledge about my family matters, schedules, medical history, etc. I predict context graph and metacognitive use cases to get rapid adoption this year.

- I've built 2 harnesses, one called Claudine - an older sister of Claude Code which now I use for teaching harness engineering, and Golem XIV, with context/knowledge graph management and self-directed metacognitive research.

- In practice rather Anthropic models if the quality of the metacognitive reasoning process and self-improvement loops are considered.

Comment by Anon84 1 day ago

Thank you

Comment by nicbou 4 days ago

I have a little pipeline that monitors specific parts of specific webpages for a value. If it changes, it makes a pull request to update a constant in a file. Basically, "if the minimum wage changes, update MINIMUM_WAGE in constants.json".

I use a tiny model (currently OpenAI, soon self-hosted Qwen) to extract values from raw text.

This helps me maintain a growing collection of guides about German bureaucracy. I monitor about a hundred values. I aim to watch as many facts as possible that way.

Comment by Anon84 3 days ago

That’s interesting, thank you.

Comment by sminchev 4 days ago

I have done integration it two applications: (1) I use it to analyze behavior, sleep and still patterns, and detect frequently visited locations, based on recorded and provided data. It is an monitoring app for elderly people; (2) suggestions, parsing files, as database. The first two are clear. The third, instead of supporting and maintaining a database, I call the AI to give me the information;

Used LLMs: Gemini 2.5 Flash. ChatGPT 5

Comment by Anon84 4 days ago

Thank you, that’s helpful.

Comment by myui 3 days ago

For financial and stock investment domains, heard lots of product stories around LLM like one in https://www.google.com/finance/beta

Comment by laptop-man 3 days ago

currently working on a sales presentation generator.

it is provided curated research and its goal is to make a compelling sales deck someone can present. current feedback is its better than our old version.

there's a master agent that have 8 sub agents it can run as tools. it can run them up to 65 times. its been really fun playing with the agents as tools.

I was extremely excited when I seen the critic agent provided a critique->main agent called -> story written for slide x -> creative writter for slide -> then critic again to verify.

Comment by hdogrukan 2 days ago

I am using Azure for LLM products. I've built four rag app in-house.

Comment by Leomuck 4 days ago

I'm a full-stack software dev, proficient in AI but also sceptical. I've found that staying away from the hype is key. Stop thinking about "WHAT COULD THIS DO", but rather try to find cases where LLMs actually benefit. I've seen so many projects trying to throw LLMs at things that could have been solved deterministically.

My personal opinion is: LLMs give you the power of language. So far we could define rules, based on structured data, we couldn't process unstructed data that well. Now we can use LLMs to take any kind of input and either create responses to it or transform it to structured data. That is a huge leap of advance. But also, there are a million cases where it's not necessary.

On the side, I'm working for a NGO caring about sustainable finance. They have a manually gathered database, they have lots of resources, but most users couldn't care enough to actually click through everything. So offering a chatbot to make that data available seemed reasonable. It works, quite well, and still most requests are so trivial you could have just blocked them.

On my paid job, I'm working for a german radio/tv broadcast station and they're trying to involve AI in solving simple internal user issues. It seems to work quite well. We've built a RAG system based on Qdrant and LlamaIndex and it provides all available information in a format users couldn't find before - because the systems were chaotic and complciated. So in my book, that's a good use case. Users in a very complicated environment with lots of information.

I've worked with OpenAI API, Anthropic API, Azure Foundry, local models, IONOS Model Hub, etc. One thing that keeps coming up is privacy and (in Europe) GDPR-compliance. Use the capabilities of LLMs without sacrificing data that should not go into the next training round.

Anyway, I think LLMs offer a lot of possibilities, but many people tackle them from the wrong side - "what could we do with this?" instead of "what problems do we need to solve?".

Comment by Anon84 4 days ago

Thank you for the thoughtful answer

Comment by sdevonoes 4 days ago

We are not. We are still making money and providing payslips for real humans. We are doing fine