Ideas aren't getting harder to find

Here's a possible scenario that I think address at least a decent chunk of the idea production part:

- Research is not embarrassingly parallel. Adding researchers doesn't lead to more different things being researched (see also, Amdahl's Law), but to multiple groups racing to research the same things.For the most part, useful research directions can be identified with far less effort that it would take to actually explore them. (It's not that ideas are harder to find, it's that researchers can't be allocated efficiently.)

- The "publish or perish" constraint that's famous from academia applies to patents. Researchers prefer to split up their results into as many separate patents as possible. (patents count is not a consistent measure)

- Research is not embarrassingly parallel. These split-up ideas are not independent, but form a chain where each builds on the last. Each small patent still gets referenced in all the following split-up small patents. (the "breakthrough patent" measure doesn't work)

---

Unrelated to the above... haven't there been articles in recent years complaining that it's harder for young people to get jobs because old people are staying working longer? Assuming that's actually true, could the constant-ish growth rate and recent decline in that rate be similar to the "science advances one funeral at a time" effect?

My apologies for being off-topic, but it's been a long time since a website made my jaw drop with its design. So simple, nothing extra, beautiful. The images and illustrations carry it so well. Also, there are multiple cover images per issue and a random one is shown on each page visit. And the typography-- I love everything about it. Makes me miss web dev.

edit: the typography combo is different for every article whaaat

I thought this article was engaging. I think it taps into something many of us can feel at times, that collective sense that truly breakthrough ideas are significantly more difficult to come by. It piqued that sense of curiosity, even if I have some questions about the methodology.

The comments section here is also quite a study. Many of the responses are so quick to dismiss the article with basically a "Well of course, this is all obvious, it's clearly due to [insert dubious reasons]." Thankfully not all of them: Some of the top comments show some genuine curiosity and deeper reflection, rather than just a pithy shoulder-shrugging dismissal. But it's definitely a pattern.

It's a little funny to read this and then immediately see comments in other posts lamenting how LLMs are always "confidently incorrect." I suppose LLMs really are just mimicking what they've been trained on.

Is it possible we have too much capital in too few hands? Ideas become harder to sell because the ratio of investors to ideas is lower.

I don't know what the solution would be. I tend to favor letting the market figure it out but dunno if that can happen here.

> This is a puzzle! Why would the market fail to reward innovative firms, or, conversely, why does it continue rewarding less innovative firms? Unfortunately, here we don’t have clear answers.

Puzzle no more, the answers are obvious! There are two interlinked mechanisms leading to this phenomenon. The rise of inequality (centralisation of power and wealth) and the rise in private debt. Both require coordinated governmental intervention to address, which won’t happen until the next economic crisis and dramatic drop in standards of living. Wish it was different, but economic theory (mainstream anyway) doesn’t account for our present situation and the control system is cycling into instability.

The upside is that we might learn the lessons this time around.

> Through two world wars, the Great Depression, the global financial crisis, and the Cold War, US real GDP per capita has grown steadily at 2% per year.

I can't help but note that while the graph is adjusted for inflation, it is not adjusted per capita.

>Bloom et al. looked across many sectors, from agriculture to medicine to computing. In each field, productivity measures have grown at the same rate as before. This sounds like good news, except that the number of researchers in each of these fields has exploded. In other words, each researcher produces much less than they used to — something you might expect if ideas really are getting harder to find.

Each researcher produces less on average, but that is just restating the statistic in different terms.

I suspect the answer is just that increasing the number of people in a research field does not mean it produces more innovations. Almost all the big innovations are produced by a tiny number of people. Let's call them the geniuses. The geniuses of a field adore the field, were never going to study anything else, and would contribute to innovation no matter what. Everyone else just fiddles around the edges. That's why making PhD-level research much more accessible hasn't increased the amount of innovation even close to commensurately.

We now have tidier, cleaner theories. They cover more edge cases. They're neater. All the little side branches are investigated and filled in. But we aren't getting more big leaps.

It's an OK article, but it's basically an economist doing a survey article of what other economists have written. No reliable metric for "ideas" is proposed.

This may be related to Baumol's cost disease, which was on HN yesterday. Most of the areas where innovation is effective involve manufacturing or other technologies that are not labor intensive. So, while they can push costs down in some areas, they do so in areas that are already highly efficient. If you could cut the price of basic steel by 30%, few would notice, because the raw steel cost of most products is tiny.

There's a fad problem in US finance. Right now, so much investment is going into AI data centers that anything else is hard to fund. US electric cars, US copper mining, and US rare earth separation, and solar farms are all under-funded, despite known good ROI. They're all boring.

What passes for capitalism in the US isn't really that good at capital allocation any more. It's too detached from physical reality.

Maybe one way to think about the efficiency of markets is that an efficient market is honest. So then the analysis here is essentially saying that the markets are getting more inefficient and corrupt, or at least not getting better. And this is borne out by things like the surveys of Americans' trust in government. But this corruption is more systematic than individualized. It is reflected in things like the lack of options if your Google account is banned. It is essentially the lack of human intervention and instead the use of AI systems. The failure modes of these systems are not obvious, and these "corruptions" essentially do not show up in typical statistics, only in super macro statistics like these productivity things.

The puzzle amounts to asking ourselves what is so special about 2%? And given the inevitable large swings in market theory fads, policy fads, generational changes, etc. why is the 2% so robust?

thinking in terms of half-lives or mean lifetimes may give a better hint than the annual percentage change.

log(1/2)/log(0.98)=> ~34 years half-life

Observe that calculating in the other direction, a half-life of 30 or 40 years results in very similar 98%:

exp(ln(1/2)/30) = 0.977

exp(ln(1/2)/40) = 0.983

the constancy is just the relative insensitivity to the exact half-life, suppose the half-life models how long a horse will run after a carrot dangling from a stick mounted to its head. Some will give up earlier, some will give up later and it can easily fluctuate by 33% (30 years or 40 years), yet the annual drop-off percentage would be a similar 2%

Perhaps we should think of things that could be universal, like approximate age a person starts working, approximate age a person simply can no longer work, or how many times one can fundamentally fool a person before totally demotivating them.

suppose we assume 15 till 65 years, a ~50 year career maximum (and some effective career length in between, ended either by age related problems or disillusionment):

lets take one of many "scams" or "white lies" or whatever one wishes to call them, like pension funds, when you start working and you place money in the fund, but that money devalues, and by the time you are on your pension, the money has halved in value so to speak, suppose a rough inflation rate of 2% (whenever the nation as a whole was more productive, that productivity was printed away by the central bank issuing more money, devaluating everyone's savings). After 50 years 0.98^50 = ~0.36

At some point (and it turns out this has been going on since time immemorial) people just chug along satisfying themselves with minimum wage, because the effort for a marginal increase is not commensurate to the gains, regardless of how fast you invent new trinkets for life that do not fundamentally change our (un)happiness with the status quo.

At some point its just a constant measuring how long you can fool the population before they are too old to revolt.

How much effect has the regulatory environment had on overall productivity? How much of a company's (or individual's) resources are wasted trying to jump through hoops that do nothing to increase output?

Tax compliance. Defending against frivolous lawsuits. Chasing permits. Settling labor disputes. Sensitivity training. Wading through government red tape.

Each of these and dozens of others drain resources (time and money), but contribute little to productivity.

> The decline in allocative efficiency should be more of a main focus — we need to throw more of our intellectual capital at understanding how to increase competitiveness and the market potential for innovative firms and technologies, in the same way that we've focused on understanding how to make better technologies.

In layman terms, it’s the age of the startup.

Shouldn't the chart that shows effective number of researchers show them as a factor of the population? Not a factor based on how many there were when we had a much smaller population?

Why doesn't anyone pay attention to the elephant in the room: median age? It has increased by 10 years in those 50 years. And, money is now overwhelmingly concentrated in the older cohorts just simply because there were no significant "reset" events for a long time such as massive wealth expropriations, wars, destruction - last time government robbed people of their property on a massive scale, was in 1865. It means that people who consume things, are now risk averse and unlikely to break with their habits.

See how different is the trajectory of electric cars in the US, EU, and China. In the US, average new car buyer is 50, in EU, 53, in China, it is 36. Plus, in China old people are dirt poor - peak income age is 35, in EU too, crushing taxes prevent wealth accumulation so old people are poorer than young who work more and are nimble enough to avoid taxes.

This cannot be fixed, because it requires either a Lebensborn-style forced reproduction program, or mass confiscations/redistributions, or both. We better accept situation the way it is.

Go to the army for five minutes and you’ll find 10 different ideas of products to create. Your failure is based on your limited and narrow perception, not the lack of ideas of things to create

I’m working on it.

TLDR; monopolism or quasi monopolism is holding back innovation at a country scale notable enough to begin changing the two century long GDP trend of 2%?

[flagged]