The Waymo effect: how AI is quietly making research less collaborative

> the author might not know what "impressive writing" or even "good writing" is

Well the author in this case is Claude, and AIs write like that because the assistant persona really thinks that's what good writing sounds like. They're wrong, but they're just doing what they were taught. There's a reason LLM raters consistently score LLM writing highly.

Well perhaps the LLM put that bit in for them.

Interestingly that's the one point of the article I have a disagreement with. Yeah, good thinking comes when reformulating your ideas. Also, reformulating ideas is part of the traditional writing process. However it's does not necessarily focus writers/thinkers on the reformulating ideas in their most value adding form.

I've watched some YouTube videos narrated by AI that were created by those who's native language I assume to be Chinese but the value of the content is higher and more concentrated than those from the native speakers.

While I've watched many for whom English is not their first language struggle in technical talks and lose most of the meat if their discussion to their struggle with the language conversion.

While the example of language barriers being skipped over and providing value in that circumstance is obvious I suspect more is possible by avoiding unnecessarily focus on prose or technical aspects of communication and focusing on the ideas themselves.

Imagine if the cost of not assuming a background in technical/textbook writing were zero. Freeing up authors to explain more thoroughly. Perhaps, more effort can be spent on thinking up analogies, metaphor or examples to help communicate an idea.

> believed this post was good and valuable enough to be posted publicly.

I mean, it made it to the front page of HN didn't it? Probably served its purpose just fine. Not all writing is supposed to be impressive or good. Some of it is to just get attention and stir discussion, and this Claude output did exactly that.

Pangram should be paying HNers for how often we pitch needing to use their product by name to believe things as obvious as "a long form news article cramming every AI trope it can fit from start to finish" being AI written.

At this point I'm surprised when a news article isn't largely AI written, let alone one using the default tone! I don't even mind it as much as others seem to, it's just turning into more and more of a rarity for a news article to not be these last few years and so is now what sticks out.

Em-dashes would be a loss, they can be a better flowing version of a parenthetical. “It’s not X it’s Y” is not a loss. This phrase is a symptom of a situation where the author wants to subvert expectations but doesn’t have the space, ability, or faith in their audience to organically set up X as the thing to be contrasted against.

I suspect it became an LLM tell because it is over-represented in text that’s easily available to the models but that most people don’t actually want to consume: marketing text, LinkedIn posts, that sort of thing.

That’s not common and effective, it’s buzz feed style articles.

Have you also scanned earlier work of the author with pangram to establish a baseline? Pangram is not without problems.

Why trust pangram? What are their accuracy scores in the wild? Ai detection is notoriously difficult.

All these "is this AI" checkers should be banned, as they never can reach 100% accuracy.

Fwiw, I only did the first 300 words (signed into pangram) and it seemingly correctly noted that there were 2 (mostly) human-authored sentences in there

What's wild to me is that:

1. People are responding to this article like it's hitting a nerve

2. In most Ubers you already don't talk to the driver (yellow cabs are higher variance in NYC). Doesn't seem like anything's being lost in that case.

3. There are enormous safety benefits to waymo, mobility benefits for youth (and elderly) that are afforded by this technology. It's not clear why people argue "Uber" is better than waymo. A few years ago there were arguments against Uber! (A technology which, again, provides a huge benefit, especially if you live in an area where people were previously expected to go out for drinks and then drive home)

I think it's fair to point out real issues at these companies. But we should be clear-eyed about which technologies we want to accelerate vs slow down

Yes, HN is for human discussion. It is fine to flag and remove things that are not human written and cannot facilitate human discussion due to it’s false construction.

No, it is not.

I think individually and when used sparingly, writing tropes can be fine, but when the article has every second sentence being a writing trope, it becomes pretty obvious that either a LLM wrote it, or the author simply doesn't know how to write, and regardless, it's a waste of time trying to read through it.

In this sort of way, yes, it's a clue. Obviously not dispositive just by itself.

It genuinely is--and that's the honest truth.

The real LLM tell is tortured and inappropriate metaphors and unnecessary adjectives and adverbs[1]. What purpose is "quietly" serving in this headline?

[1] https://www.youtube.com/watch?v=ORgKY9AlybA

> saying it is "written by LLM" is not useful and misses the point.

It is useful to me. I see too much slop every day and would like to filter it out from my life. I appreciate the tip, and to me, that is the whole point.

"every hour spent aligning with a co-author is an hour not spent producing output that is legible to an evaluation system." use of the word 'legible'.

"We defund the corridor and then wonder where the corridor conversations went." this is a claude-ist construction.

"This is an old worry wearing new clothes" this is a very claude-ist construction.

(rather ironically... as @embedding-shaep pointed out earlier): "Outsource the writing and you have not accelerated the thinking; you have skipped it." 'not X but punchy-Y' (not to be confused with 'not X but profound-Y'.

I'm not saying the author shouldn't have used an LLM - once in a while I suppose it gets used to good effect. But I wouldn't kid myself that the writing is certainly all human through-and-through.

[Incidentally, in case anyone's wondering where many of these claude-isms come from, just search within lesswrong.com .]

Can you share me the summary? I'd like to read a LLM summary of your summary.

Things are bit tight today.

Plenty of people use Oxford commas though, can't really use it as a reliable signal, too much noise.

If you check my comment history here, I have not written this before.

I like LLMs and use them daily, but I don't really like when people use them undisclosed for long-form prose writing.

In the spirit of the "That's what she said" bot, you could train an LLM detector by accusing everything of being authored by an LLM and seeing which ones get voted up.

"Novel" is not a prerequisite for something to be worth pointing out. Lots of bad things happen repeatedly, and don't quickly become not worth caring about or knowing about. People have strong spirits and curious minds and it tends to take a long time to boil that frog out of them - generations, sometimes.

People aren't treating it as a thoughtful contribution. It is more of a flag, like "spam".

>>I'm not against fact checking, but it absolutely felt like a punch in the gut coming from a non-dev. There was no "trust" there. She couldn't rely on my expertise, she had to go back to her "source of truth" (Claude) and asked for a second opinion on something that she wouldn't be able to verify.

This is probably how physicians feel when informed patients (whether using LLMs or not) ask questions (good or bad) during their patient encounters.

Yeah I've felt that gut punch all the time. Seems to happen with every technology. In the early days of GPS, people would ask for directions to the gas station then pull it up on their phone map and go "yeah, you're right!" So thanks for wasting my time, insulting my integrity, and killing my faith in humanity all in 3 words. And if you really need two opinions, it's so easy to just reverse the situation, "my GPS says there's an Exxon in two miles if I turn left here and then right, is that correct?"

I'd bet it depends on what they write to the LLM, since sycophancy is so strong in these things. If they write in a fairly agnostic way, it might default to sycophancy to you the dev, while if they write it in an opinionated way it might default to sycophancy to them. Regardless of who is right.

To be generous, it seems like the most natural way to phrase what happened. The non-developer punched in "should this be a database instead?" and it will say "You are right to question it! I was sloppy when .. blah blah blah blah blah"

I doubt the human went into a whole diatribe with the CLI about how the developer on the project with this many years was saying this or that but she wanted Claude to second guess them.

It's just saying "ah yeah, that went through without issue."

I can see how the situation is annoying there; I've had a similar frustration when doing manual code review for someone's obviously AI-slop PR only to have their bot auto-respond to my comments. There's an asymmetry of effort that feels unfair. Like, don't ask for manual review of something that contains lots of things that your own manual review should have caught first, and if I provide human feedback, I expect human responses.

That said, I wouldn't necessarily take it at face value that the person you were working with trusted the bot more than you— at the end of the day, the bot was doing all the implementation anyway, so all the interactions between her and the bot were probably framed in language like "what if we" or "I think it would be better if", so her saying "my programmer colleague thinks X" would naturally generate that "yeah your colleague is right, let's go that direction, makes sense" that she passed back to you, probably thinking it was a complement.

Maybe the real frustration is her not understanding the value-add that still exists there when someone with real expertise and more context can step in and provide that nudge in the right direction.

wow that wouldnt feel like a gut punch at all to me, it's normal for inexperienced people to not really understand what kinds of knowledge experts have or don't. I certainly come back from doctors appointments and google/gemini chat for whatever came up to get more detail. this person going to claude and getting immediate confirmation that you were exactly correct should have felt great, she would be like "wow, this guy's good, huh."

>I'm not against fact checking, but it absolutely felt like a punch in the gut coming from a non-dev. There was no "trust" there.

Maybe I'm just cynical but I don't think I ever see that much trust for anyone's opinion or expertise; devs think managers don't know what they're doing, managers think devs don't, people on HN think lawyers are wrong when they state their legal opinion, devs think other devs are wrong about their areas of expertise even though they have no reason to think that, hell I've worked with people who basically lie for some reason about technical issues that they have no expertise in, in order to get some benefit that is unclear to me. I've also seen people of great technical expertise gaslight industries in order to derive very clear benefits for their employers, so I'm not sure why there should be trust.

I'm sure that various AI agents will maybe add another interesting wrinkle to this, but I sort of feel like you must have had a nice life up until AI if that was feeling like a gut punch.

I think I could qualify as an expert on various topics relative to many average people. But it always bothered me when experts wanted to "just be believed". In your example, it doesn't long to explain why a database is better than the file system in this instance and a moderately intelligent should be able to understand and not need you or Claude as the source of truth.

So I think it can be good people are able to "AI" to question expert opinions - though naturally, as you say, it's bad if people take the AI as a single authority akin to an expert.

> Treating AI as if the output is factual is just a misuse of the tools, I think.

Especially because it's very often wrong. AI tools lack the necessary context of the specific situation and the surrounding technology environment to make appropriate suggestions for anything more complex than clean sheet designs and toy prototypes. There is too much complexity that is hidden in domain knowledge and not even exposed fully in documentation for any AI tool currently to ingest it all into its own context to give you an accurate answer, even setting aside hallucinations and that these tools are incapable of reasoning.

I have this same gut punch constantly. AI has destroyed functional outcomes in corporations, because managers are all deep in AI psychosis deciding that whatever the tool says is perfectly correct at all times. We've replaced analysis paralysis with a complete absence of thought from any human in high-level impactful strategy discussions, everything is entirely being abdicated to AI tools in so many companies. It's ridiculous.

AI will radically change the economy, but not because its replacing jobs, but because its eroding trust, domain knowledge and expertise, and thinking/creativity while being a massive waste of money and total boondoggle. This bubble can't burst soon enough so we can get on with doing shit that actually matters for people that actually helps them.

non-dev AI psychosis doesn’t bother me

I’ve seen how the market treats vibe coders that werent previoisly engineers. Coinbase’s CEO bragged about PM’s deploying code and everyone now thinks they are totally incompetent anytime a small UX problem occurs.

Engineers with prior experience now check a box, even if they are doing the same thing. Like a Compliance Officer or Cybersecurity professional checks a box

Annoyingly pedantically speaking, a filesystem is a type of database.

As an unfamiliar, I find it interesting how you specifically called out physics grads in the pre-LLM era. Is that just snark or is there some widespread truth / stereotype of that behavior?

    > we treat understanding new systems as something approachable

    > from my boss's perspective, I'm the only one who's actually willing to just read code, investigate, possibly read a paper, and figure out what's going on instead of punting and saying it's someone else's turf I'm blocked until they get back to me.

I hypothesize that with more people on a daily basis encountering information and claims that require those skills, there will be a time in the future when those skills are much more enhanced.

There is a reason this exists: https://xkcd.com/793/

I sadly share this as a physicist. To be fair, sometimes this arrogance can be useful. I worked at the boundary between domains my entire career where there aren't "experts" per se. But yeah, barging into an established domain without some humility generally leads to embarrassment - especially Nobel winners (e.g. https://www.newsweek.com/nobel-prize-winner-who-doesnt-belie...)

Hilarious. Do they preface the conversation with "Well, Claude said..."?

>arrogance when barging into unfamiliar domain

describes nearly every tech attempt at "disruption"

like a prosthetic dunning-kruger....

Currently, the leader of USA national AI policy (Trump admin and OpenAI) is a history Bachelors with no technical training or advanced training of any kind, Dean Ball.

> I also work at an intersection of tech, applied research and science. My experience has been different. It's been surprising how controlled my collaborators/coworkers have been with their reliance on AI.

It's similar to the pattern I've noticed with experienced programmers I know and respect, the really good ones do seem to use AI but they are deeply sceptical about its claimed capabilities so they check/verify/assess what it's good for and use it for that, the tier down is less questioning and just accepts whatever it generates as gospel and it seems to degenerate the further you go down.

It seems like as soon as the AI states something confidently and is wrong in the domain in which the user is an expert, it loses a lot of credibility and people become much more wary of it in general but that requires you to be able to see that it is wrong otherwise it becomes a "bullshit baffles brains" generator.

It's useful to remember those times you've seen it do that when you are asking it something you aren't an expert in and then verify it's answer a different way.

Ugh. To be fair to the LLMs they're trained on industry blogs, and if you spent too much time reading them you'd also think every electronic stamp album needs to scale to a billion users with millisecond regional failover.

My condolences :(

Nip it in the bud quickly

The worst thing about AI is that it's camouflage for stupid people. And it often gets combined with utter confidence in some position because they're too stupid to know the machine is confidently wrong.

It's not just SWE's we just see it more in that pool because we mostly (we are on HN after all) swim in that pool.

I've eaten dinner with Doctors (medical) where they've confidently spoken about a subject I am more experienced in than them and they've been hilariously far off the mark, I just nod and ask for the gravy - not picking a fight over a meal.

Prof. Nathan Ballantyne coined the term Epistemic Trespassing to describe it, it is rampant and LLM's just act as an amplifier.

It's like people have a collective fear of just admitting they don't know something, there are a vast number of disciplines/topics I'm not competent or knowledgeable enough to have a meaningful opinion about even if I find the topic interesting, it's just been smart enough to realise that in the end.

Worst case, find the person who does know and ask then both of you might know (assuming you understand the answer of course which isn't guaranteed, I like physics, I'd understand about one word in ten if a physicist actually described it the way they would to another physicist).

yep, this is how it goes, you just ignore the llm brained people.

I'm not GP, but I've been experiencing this too. All my coworkers are in the same boat though, so we don't really have anyone with the deep understanding. Our approach (which has been working pretty well, we've solved a lot of problems and learned some along the way) has been to use AI as an informer who also points us to sources (like documentation, github issues, etc) where we can then read up on something the AI has pointed us at. We try to time box things a bit to prevent going down the rabbit hole, so we don't always get to exhaust our curiosity, but we're continually gaining that knowledge and pressing forward. Most recent example is tuning the kernel settings and our app on our prod machines to behave better for WebRTC packet handling/forwarding. AI for things like ffmpeg has been a god send when nobody on the team is an ffmpeg expert.

I've also had success just asking claude to write documentation on a system or subsystem or module, etc, and reading that. I then sometimes have it turn that into an svg diagram or something visual that often helps understand things. That domain-specific knowledge is hard to gain though, so not a silver bullet by any stretch.

I have a very similar experience, and am very frustrated too.

I could (and do) lay blame on this new culture that "in the age of AI" things are there to be done, not learned, and doing is cheap (even if badly).

But I'm also particularly frustrated with myself for going along with it, and becoming that person.

I've never been that person. So on the one hand I feel I pressure to adapt or die. On the other hand adapting is dying, in a sense.

I don't think you can do much about it if you're hoping to hide that you have been in the domain for less than a year. You're going to have to admit your inexperience and ask questions of your more experienced coworkers and try to stretch your common sense (e.g. devise internal consistency checks).

Don't just blindly pass around LLM generated content that you don't understand. I think it's important to consider that you are supposed to be more than just a meat proxy for the model. You don't want people to start associating you with AI slop.

Read some books on the subject and within a year in this new domain you'll be much better. Did you forget your whole life before the last 3 years?

Let’s be honest.

It’s always been present - e.g when I see people talking about finance on here with completely mangled fundamentals.

But now it’s amplified like crazy. We are going into a world where the geniuses will have to be isolated and protected from brain rot.

It's a tough call but I'm not sure if physicist code or electronic engineer code is more frightening.

Yes but it's annoying to collaborate with a Claude proxy person who thinks to have it all sorted out and who will question everything you say but not really listen, just itching to get back to Claude to confirm or deny what you said.

Compare that to being asked for advice, to explain directly something you know or have done, with the expectation of the person asking to learn something. Like asking why some decision was made in the past, based on what information and context, instead of confidently saying that it was done wrong. Both will eventually get you the same information.

[dead]

I use state of the art frontier coding agents daily. While the fact they can code at all is a huge achievement, it's hard for to me to buy in to the looming AGI/2027 collapse scenario when they remain stymied by the simplest out of sample tasks and sabotage their own work frequently. But the blooming ignorance of everyone addicted to them is striking. These things are tools, used correctly they can do some amazing things, but going Tim the Toolman with them goes poorly.

> back a response that's all Claude (and disorganized and not really making sense to boot

You’re absolutely right! They trusted but did not verify. Thats a load bearing point — they assumed intention from your question.

I very much doubt it will do that because access to that information was already available to anyone who wanted to know. It was not a secret. All AI did was to provide it via single interface, for a fee - which strangely do not go back to the original source of the knowledge, but to the AI company these days. This is not democratisation of information, it's commodification of it.

Partially true, but only partially. It also makes people believe they understand more than they do, because every answer comes so easily to them. But the LLMs are easily steered and tend to agree with what you want them to say.

Awesome, now balance that out will education levels dropping like a rock, due to various factor but also due to LLMs, and things will be balanced out by also democratizing stupidity, until we live in Idiocracy.

Another delusional take.

What exactly have you done with this quote on quote free access to information?

Informational accessibility is/was pretty much a solved problem. What llm’s now do is let amateurs think they can easily become experts.

LLM tools tend confirm what you come to them with, be it unfounded conviction or doubt. I find them quite useless to tell me things I don't know, as verifying what they say shows that any non-trivial claims they make are usually oversimplified or plain wrong.

Anything more than using them to just pointing me to media and literature is usually a waste of time.

>it's hard to gauge this take without specifics. My experience with LLMs tends to be towards the opposite concept; LLMs dissuading me of hunches and notions I have about things (where I have no particular expertise; societal-level things), saving me and others time and strife having an argument about something they were actually right about all along, as my internal doubts that I'm too embarrassed to bring up (because these things aren't my field) are confirmed as incorrect.

You can do this with any subject.

The LLM is trained in part on a decade of shitty online comments and vapid professional correspondance. If you frame whatever you're asking about in a direction that would offend the sensibilities of the kind of people and ideas that are over-represented in that content it will hem and haw and drag its feet and whatnot.

Not trying to be rude, but is this a serious comment, or are you being sarcastic? "Human supremacists", haha.

Also, why do you think someone who used an LLM to research would have the same level of knowledge and insight as someone who's been in the domain for a long time? That doesn't seem to hold up to any level of critical thinking.

If I talk to a professor about their field of expertise, does that empower me to talk with other professors in that field on an equal level?

That's an interesting observation. I can see both sides of that coin.

And yet, many people have ambitions. They will put up a website about how great their software is and how everyone should use it.

I usually put a warning that it's experimental and might not be maintained. More users, more problems.

(And yet, I'll confess to be slightly disappointed that nobody is using software that I didn't promote.)

From the post, “and the incentive structures we have built are the experimental apparatus.” That’s a Claude sentence.

It was definitely written with significant LLM assistance. It's very obvious, and the essay is not good.

> The Waymo effect: how AI is *quietly* making research less collaborative

> The Waymo arrived with the serene confidence of a machine that has never once worried about where to find parking, and I climbed into the back seat, glanced instinctively at the driver’s seat to say hello, and found myself nodding politely at an empty chair.

> No obligation to make conversation. No silent negotiation over the radio. A guilt-free space to be alone with one's thoughts, finish an email, or take a call en route without the awkwardness about conducting it in front of a stranger. The car was quiet, smooth and entirely undemanding.

> The collaborator’s inconvenience, in other words, is *not a bug* in the collaboration; it largely is the collaboration. The value of another mind *lies exactly in the ways* it refuses to be an extension of your own.

It absolutely was used, and heavily. I won't even cover the "it's not ___, it's ___", of which there are MANY in this piece. But beyond that:

Count the lists of three:

    A guilt-free space to be alone with one's thoughts, finish an email, or take a call en route without the awkwardness about conducting it in front of a stranger.

   It costs travel and scheduling and the glacial work of building trust. It costs ego management and compromise and the periodic small heartbreak of the author-order negotiation.

Look for referenced to "naming" things, used in the sense of "identify" or "realize" rather than, you know, naming something.

    It took us slightly longer to name what we had lost.

    Swap “pilot” for “researcher” and “autopilot” for “agent”, and the mechanism Wang is guarding against is the one Bainbridge named four decades earlier.

Look for things being done "quietly":

    Two people who had crossed continents to talk to each other were quietly delighted by a technology whose central feature is that you don’t have to talk to anyone. (ed. - is that *really* Waymo's central feature?)

    once a frictionless option exists, we take it by default, and quietly surrender whatever the friction was doing for us

    Every conversation redirected from a colleague to a chatbot is a thread quietly withdrawn

    while the invisible thing that sustained it quietly falls below threshold

    competence quietly erodes precisely because it is needed only rarely

    the quiet risk is that our incentives are training us

[flagged]

> AI do absolutely nothing in terms of "avoiding the negative side effects imposed on me by other people"

Quite literally the subject of the article.

> Right all those people protesting outside data centres are bureaucrats?

Not what was claimed.

Also, the passenger does not have to worry about the driver doing anything to them (and vice versa).

The same argument was made when elevator operators went away.

Some things work well with automation (taxis, washing machines, copy machines) while others work better with humans (chefs, masseuses).

50 years ago, I would have needed to dictate this note to a secretary who would distribute a memo. Does it further isolate ourselves that we can type and communicate directly? Would 1980 uses think it was awful to us?