Measuring developer productivity with the DX Core 4

Posted by saikatsg 3 days ago

Counter41Comment31OpenOriginal

Comments

Comment by stiray 7 hours ago

I would like to present DX team another metric.

It is called "not wanting to work in a company that is insulting me with measures like metrics, spying on screens, squeezing every drop of sweat and exploiting its workers for already huge profit.

My last job change was exactly due to company acquisition by USA company that started to exploit everything work related, removing wfh, ruining life-job ballance,... And who leaves first? Those who can.

So you are not really measuring developer productivity but helping company get rid of most productive people, while all others stay.

Comment by Cthulhu_ 6 hours ago

My only experience with GetDX has been filling in a questionaire once a quarter that asked stuff like "how often do you go to production"; I don't know if you work in a more hostile environment, but in my case it was always used to identify points to improve, not necessarily targets.

Comment by Anonyneko 5 hours ago

That's exactly what they're introducing all of that crap for, to make people quit voluntarily, layoffs are more expensive...

Comment by 3 hours ago

Comment by brabel 6 hours ago

I think metrics can be used for good. For example we are constantly try to improve our dev experience, but how do you know if you are improving anything? Developers are notably inaccurate in reporting efficiency. Unfortunately flying blind is not really an option when you need to justify paying for tools that claim to improve productivity.

Comment by BoxOfRain 6 hours ago

Surely no data is better than bad data? What's the point of a metric that doesn't measure anything tied to the objective reality of what it is that you're measuring?

All I see tools like this doing is getting people to manipulate the metrics, while at the same time producing enormous amounts of worker alienation. I'd even speculate the demoralising effects of workplace surveillance schemes undermine the alleged productivity benefits.

Comment by tripledry 6 hours ago

I see more of people using data and bad metrics to justify whatever they want to push rather than genuinely trying to improve conditions.

I don't have any data to prove this :)

Comment by nh23423fefe 6 hours ago

This response is low on originality metric.

Comment by aevv 3 hours ago

my current place uses DX and it's well received, if done transparently and for the right reasons. there's a lot of very interesting research in these areas (e.g. EngThrive) where the focus is on measuring things that if gamed result in better outcomes

there's strong correlation when onboarding an engineer between the time to first/tenth/fiftieith PR and their PR throughput two years later, so the answer is make it really easy to do PRs when onboarding, and they saw the same outcome even if those initial PRs were trivial

through DORA/SPACE/DevEx they also found that asking your engineers if they feel productive is generally as useful and reliable than trying to measure every possible dimension - which is why DX is based largely around the survey with subjective responses. if you actually use those responses to address friction and frustration, from my experience you end up with more, better, easier work getting done

it's possible to work in a team that cares about measuring their effectiveness, and using those measurements to understand how to be more effective. most/all engineers will have an idea of how they could improve their work, but the reality when you actually spend time to measure often shows a lot of different things you might not have been aware of

Comment by wokwokwok 9 hours ago

It’s remarkable (but not reassuring) how carefully they avoid saying “part of the Atlassian family” in any promotional material for some reason.

Comment by igleria 9 hours ago

> Whether you’re a C-level leader or a frontline manager, there is no denying that measuring developer productivity—whether to understand performance or guide improvement—is a daunting challenge.

good

Focus on the stuff that matters, not on tormenting the people that make you money.

Comment by BearOso 3 hours ago

What stuck out to me with that sentence was that a competent manager can determine productivity by looking at what the developer produced, easily and quickly. If the person has no clue what they're looking at and needs this software then they're not qualified.

Comment by lbriner 7 hours ago

Developers don't like being measured for good and bad reasons. The good reason is that there is lots of effort that we expend that isn't always captured in metrics like long pieces of design or training others etc. The bad is that we don't being told that we aren't as productive as other people even if we are!

My experience is that these are only useful over a long enough period and across enough people that we can spot genuine outliers. For example, your average across the last 6 months is "15" and the average of everyone else is "25", can you help me see whether you are being given too much off-target work or are there any other issues that are blocking you getting stuff done?

Comment by pipes 6 hours ago

Also trying to reduce productivity to some thing as silly as pull requests per hour etc ignores valuable contributions else where. Design, push back, quality, mentoring, leadership. I thought that is why meta had the "coding machine archetype" and other archetypes, recognising that just PRs aren't the only measurements.

Comment by tripledry 5 hours ago

And if you start measuring amount of commits or PR's I'd bet money on there suddenly being smaller PR's and more commits. As soon as the metric is tied to peoples livelihood, it will be gamed.

Funny anecdote: I had a large company selling their LLM tools with a slideshow starting with a mention of Goodhart's Law, and then proudly present number of PR's as a metric in the next slide.

Comment by Cthulhu_ 6 hours ago

From my limited experience with this tool / DORA metrics, it's not so much about performance (PRs per hour) but more about whether you can act fast if you need to. But I'm not sure what the other metrics are about.

Comment by zeroc8 6 hours ago

The problem is that there are so many people involved in software development who have never developed anything but still somehow are in the position to manage others who do.

That's why there are so many stupid ideas floating around.

Comment by greenchair 3 hours ago

Probably also related to the fact that many devs don't want to switch to the mgmt track.

Comment by jcarrano 9 hours ago

Did anyone here ever worked under these sorts of "metrics"? I can't even imagine what a torture it would be.

Comment by anon7000 2 hours ago

Yes, I’ve worked in companies that use DX. Generally you get a survey once a quarter, and you can see some basic stats like how many PRs/code reviews are happening.

My company leadership does not use these as a one size fits all metric. In fact, the code review & PR stats never even show up in performance reviews. Mainly, they are used to improve systems around dev productivity. Like, we see that generally, we’re shipping less PRs and the survey shows people think it’s getting slow to merge changes. That is an opportunity to look at where our systems are holding us back.

I think broad stats are also probably still useful too. For example, there have been times that I’m reviewing more code than everyone on the team combined. I WANT my manager to know that’s happening and figure out how to even the workload. Or, why is my team shipping like 2-3 times as many PRs as a similar team with similar headcount? It’s not immediately a PIP or something, but it’s highly likely something odd is happening. Could leadership improve direction & focus, make priorities more obvious?

Comment by data-ottawa 5 hours ago

Basically every company I’ve worked for has had some sort of rubric like this. I don’t think that’s inherently bad, it can be quite good for consistency.

I have quit companies that made their performance review systems too onerous/frequent/intense. It’s exhausting to constantly be justifying your existence and having to sell yourself instead of doing the real work.

At one point I was at a 50/50 ratio of time spent documenting/justifying my work vs doing my work, and that was miserable. My level of anxiety reduced my job performance, which created a very unhealthy cycle. Pressure creates diamonds but it also suffocates and crushes most things.

That’s why seeing things like in-context sampling in the article make me suspicious. I don’t think I would be happy at all being interrupted as I’m in flow state to be asked if I could be more productive. I also feel some of this like speed (“time to 10th PR”) or working on new stuff is creating bad incentives.

Fundamentally the most important metric is how hard your boss will fight for you. The second most important metric is whether you can meaningfully discuss areas of improvement honestly and safely with your boss.

If either of those two are out of whack you should question whether to make a change. If I’m missing those I’m not going to be able to do my best work, so it cuts both ways.

Comment by majorbugger 8 hours ago

Could be worse like crossover

Comment by cultofmetatron 9 hours ago

the whippings will continue until Dx core 4 metrics improve

Comment by duzer65657 8 hours ago

you forgot the (tm) on the metrics system. That's a huge smell.

Comment by unglaublich 6 hours ago

Lol, number of diffs and prs? Number of prs that focus on new features?

Those are horrible incentives.

Comment by imeron 9 hours ago

Developers will optimize for what is measured.

Comment by amelius 4 hours ago

> DX Core 4

Sounds like the name of a new CPU/GPU.

Comment by adornKey 3 hours ago

486 DX4

Comment by williamcotton 4 hours ago

What about measuring management productivity?

Comment by kamil55555 10 hours ago

>key metric >diffs per engineer

Comment by ghthor 6 hours ago

I don’t think the product is all bad; my co is using it and I feel like it is giving us some useful signals about our internal dev UX that my team can action on; we own the base level dev tooling.

Comment by cynicalsecurity 7 hours ago

How to ruin your company in 3 simple steps. Step 1: DX Core 4.

Comment by zaydmulani 3 hours ago

[dead]