Making My Blog Readable by AI Agents

One trap when you name AI bots specifically is that is that robots.txt groups don’t merge. Crawlers will only follow the most specific group so if GPT bot has its own group named nothing under User-agent: * applies to it anymore so you need to repeat every disallowed line under each group.

Are llms.txt actually read by bots now? I found one experiment that showed that no bot has accessed the file in 3 months.

https://www.rebootonline.com/geo/llms-txt-experiment/