"We have data on the performance of >50k engineers from 100s of companies. ~9.5% of software engineers do virtually nothing: Ghost Engineers.”

Last week, a tweet by Stanford researcher Yegor Denisov-Blanch went viral within Silicon Valley. “We have data on the performance of >50k engineers from 100s of companies,” he tweeted. “~9.5% of software engineers do virtually nothing: Ghost Engineers.”

Denisov-Blanch said that tech companies have given his research team access to their internal code repositories (their internal, private Githubs, for example) and, for the last two years, he and his team have been running an algorithm against individual employees’ code. He said that this automated code review shows that nearly 10 percent of employees at the companies analyzed do essentially nothing, and are handsomely compensated for it. There are not many details about how his team’s review algorithm works in a paper about it, but it says that it attempts to answer the same questions a human reviewer might have about any specific segment of code, such as:

  • “How difficult is the problem that this commit solves?
  • How many hours would it take you to just write the code in this commit assuming you could fully focus on this task?
  • How well structured is this source code relative to the previous commits? Quartile within this list
  • How maintainable is this commit?”

Ghost Engineers, as determined by his algorithm, perform at less than 10 percent of the median software engineer (as in, they are measured as being 10 times worse/less productive than the median worker).

Denisov-Blanch wrote that tens of thousands of software engineers could be laid off and that companies could save billions of dollars by doing so. “It is insane that ~9.5 percent of software engineers do almost nothing while collecting paychecks,” Denisov-Blanch tweeted. “This unfairly burdens teams, wastes company resources, blocks jobs for others, and limits humanity’s progress. It has to stop.”

The Stanford research has not yet been published in any form outside of a few graphs Denisov-Blanch shared on Twitter. It has not been peer reviewed. But the fact that this sort of analysis is being done at all shows how much tech companies have become focused on the idea of “overemployment,” where people work multiple full-time jobs without the knowledge of their employers and its focus on getting workers to return to the office. Alongside Denisov-Blanch’s project, there has been an incredible amount of investment in worker surveillance tools. (Whether a ~9.5 percent rate of workers not being effective is high is hard to say; it’s unclear what percentage of workers overall are ineffective, or what other industry’s numbers look like).

Over the weekend, a post on the r/sysadmin subreddit went viral both there and on the r/overemployed subreddit. In that post, a worker said they had just sat through a sales pitch from an unnamed workplace surveillance AI company that purports to give employees “red flags” if their desktop sits idle for “more than 30-60 seconds,” which means “no ‘meaningful’ mouse and keyboard movement,” attempts to create “productivity graph” based on computer behavior, and pits workers against each other based on the time it takes to complete specific tasks.

What is becoming clear is that companies are becoming obsessed with catching employees who are underperforming or who are functionally doing nothing at all, and, in a job market that has become much tougher for software engineers, are feeling emboldened to deploy new surveillance tactics.

“In the past, engineers wielded a lot of power at companies. If you lost your engineers or their trust or demotivated the team—companies were scared shitless by this possibility,” Denisov-Blanch told 404 Media in a phone interview. “Companies looked at having 10-15 percent of engineers being unproductive as the cost of doing business.”

Denisov-Blanch and his colleagues published a paper in September outlining an “algorithmic model” for doing code reviews that essentially assess software engineer worker productivity. The paper claims that their algorithmic code assessment model “can estimate coding and implementation time with a high degree of accuracy,” essentially suggesting that it can judge worker performance as well as a human code reviewer can, but much more quickly and cheaply.

I asked Denisov-Blanch if he thought his algorithm was scooping up people whose work contributions might not be able to be judged by code commits and code analysis alone. He said that he believes the algorithm has controlled for that, and that companies have told him specific workers who should be excluded from analysis because their job responsibilities extend beyond just pushing code.

“Companies are very interested when we find these people [the ghost engineers] and we run it by them and say ‘it looks like this person is not doing a lot, how does that fit in with their job responsibilities?’” Denisov-Blanch said. “They have to launch a low-key investigation and sometimes they tell us ‘they’re fine,’ and we can exclude them. Other times, they’re very surprised.”

He said that the algorithm they have developed attempts to analyze code quality in addition to simply analyzing the number of commits (or code pushes) an engineer has made, because number of commits is already a well-known performance metric that can easily be gamed by pushing meaningless updates or pushing then reverting updates over and over. “Some people write empty lines of code and do commits that are meaningless,” he said. “You would think this would be caught during the annual review process, but apparently it isn’t. We started this research because there was no good way to use data in a scalable way that’s transparent and objective around your software engineering team.”

Much has been written about the rise of “overemployment” during the pandemic, where workers take on multiple full-time remote jobs and manage to juggle them. Some people have realized that they can do a passable enough job at work in just a few hours a day or less.

“I have friends who do this. There’s a lot of anecdotal evidence of people doing this for years and getting away with it. Working two, three, four hours a day and now there’s return-to-office mandates and they have to have their butt in a seat in an office for eight hours a day or so,” he said. “That may be where a lot of the friction with the return-to-office movement comes from, this notion that ‘I can’t work two jobs.’ I have friends, I call them at 11 am on a Wednesday and they’re sleeping, literally. I’m like, ‘Whoa, don’t you work in big tech?’ But nobody checks, and they’ve been doing that for years.”

Denisov-Blanch said that, with massive tech layoffs over the last few years and a more difficult job market, it is no longer the case that software engineers can quit or get laid off and get a new job making the same or more money almost immediately. Meta and X have famously done huge rounds of layoffs to its staff, and Elon Musk famously claimed that X didn’t need those employees to keep the company running. When I asked Denisov-Blanch if his algorithm was being used by any companies in Silicon Valley to help inform layoffs, he said: “I can’t specifically comment on whether we were or were not involved in layoffs [at any company] because we’re under strict privacy agreements.”

The company signup page for the research project, however, tells companies that the “benefits of participation” in the project are “Use the results to support decision-making in your organization. Potentially reduce costs. Gain granular visibility into the output of your engineering processes.”

Denisov-Blanch said that he believes “very tactile workplace surveillance, things like looking at keystrokes—people are going to game them, and it creates a low trust environment and a toxic culture.” He said with his research he is “trying to not do surveillance,” but said that he imagines a future where engineers are judged more like salespeople, who get commission or laid off based on performance.

“Software engineering could be more like this, as long as the thing you’re building is not just counting lines or keystrokes,” he said. “With LLMs and AI, you can make it more meritocratic.”

Denisov-Blanch said he could not name any companies that are part of the study but said that since he posted his thread, “it has really resonated with people,” and that many more companies have reached out to him to sign up within the last few days.

  • mochisuki@lemmy.world
    link
    fedilink
    English
    arrow-up
    3
    ·
    1 hour ago

    The old adage of the engineer paid to know where to tap an X comes to mind: https://quoteinvestigator.com/2017/03/06/tap/?amp=1

    Frankly anyone telling you they can measure the value of a line of code without any background knowledge is selling BS.

    But I welcome this new BS system as the previous system of managers not so secretly counting total commits and lines added was comically stupid.

  • 2ugly2live@lemmy.world
    link
    fedilink
    English
    arrow-up
    6
    ·
    2 hours ago

    What a fucking snitch. 9.5% of engineers gotta go, but the CEO getting paid buckets and buckets of money isn’t draining the company? Fire 9.5% of engineers that actually have knowledge and are skilled enough to demand a high price for their skills, or CEO fuck-all who comes in via zoom once a quarter and couldn’t open a pdf if they’re life depended on it. Hmm, what a hard choice 🤔

  • Viri4thus@feddit.org
    link
    fedilink
    English
    arrow-up
    15
    ·
    edit-2
    5 hours ago

    How to scare managers into hiring me with my PoS black box software.

    Step 1: make wild claims about wasted resources.

    <- you are here

    • a4ng3l@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      ·
      2 hours ago

      Or architects, infrastructure engineer… plenty of peripheral functions are hired as « IT engineers » and not pushing code in a repo. What a weird article.

    • stoly@lemmy.world
      link
      fedilink
      English
      arrow-up
      35
      ·
      12 hours ago

      Alternatively they are on an engineering team and providing their expertise via other means beyond code submission. This entire thing sounds like a sledgehammer trying to do the work of a scalpel.

  • BassTurd@lemmy.world
    link
    fedilink
    English
    arrow-up
    49
    ·
    14 hours ago

    It’s a long article that I admittedly didn’t read all of. I got to the part where it said the details of his algorithm are basically unknown, which means his data means nothing. If someone can’t provide the proof to their claims, they have no merit.

    An LLM that’s built entirely on code repo data, and is somehow claiming workers “do virtually nothing” without any sort of outside data, is insane.

    • JollyG@lemmy.world
      link
      fedilink
      English
      arrow-up
      10
      ·
      10 hours ago

      One of my big beefs with ML/AL is that these tools can be used to wrap bad ideas in what I will call “Machine legitimacy”. Which is another way of saying that there are many cases where these models are built up around a bunch of unrealistic assumptions, or trained on data that is not actually generalizable to the applied situation but will still spit out a value. That value becomes the truth because it came from some automated process. People cant critically interrogate it because the bad assumptions are hidden behind automation.

  • socsa@piefed.social
    link
    fedilink
    English
    arrow-up
    40
    ·
    edit-2
    16 hours ago

    I don’t doubt the thesis, but reviewing commit history is next to useless. I’m probably not top 50% of activity within our organization but I’ve literally invented most of our tech and my name is on the patents.

    If anything, it’s the people who spend all day making pedantic code review comments just to boost git actions who have nothing better to do.

  • yildolw@lemmy.world
    link
    fedilink
    English
    arrow-up
    31
    arrow-down
    1
    ·
    16 hours ago

    The thing about being a big organization is that you need to have slack capacity most of the time in order to be able to go quickly in a different direction at certain times. If you don’t have excess capacity sitting idle, an unforeseen event can paralyze you

    • ddh@lemmy.sdf.org
      link
      fedilink
      English
      arrow-up
      4
      ·
      8 hours ago

      And slack capacity can be used effectively e.g., spend some time on process improvement. There’s always some saw to sharpen or some technical debt to repay.

  • fluxion@lemmy.world
    link
    fedilink
    English
    arrow-up
    45
    ·
    edit-2
    17 hours ago

    Some easy tasks involve pumping out mounds of code/commits, some tasks involve monumental amounts of inter-department cooperation or design discussions with open source communities online or at yearly conferences and result in relatively small amounts of code especially in terms of LOC/day.

    This study purports to take this into account to some degree but i call bullshit. I can barely explain this level of nuance to anyone above my first-line manager everyone else is just like what’s taking so long can we throw an intern on it to speed things up? and its like sure… after you hire them full-time and spend the next couple years training them. Oh you want me to do it that too?

    The whole tone of this “researcher” makes the bias so clear but I’m sure we’ll have all kinds of fancy new monitoring and lay-offs of good people thanks to these sorts of bullshit metrics.

    If you want to know whether employees are a waste of space or not, hire good fucking managers that know what they are doing. If they farm that out to tools like this, it’s a good sign they don’t.

    • Wotan@lemmy.world
      link
      fedilink
      English
      arrow-up
      18
      ·
      edit-2
      16 hours ago

      Well, this is so typical of people who just see employees as numbers. How is it possible that a company thinks they have an overemployment problem? Doesn’t it mean that the whole company’s management is a pile of crap?

      Just build your team on people who care, who have time and will to do 1on1s and who can build a culture of trust in your company. Then you will not need to waste money on an algorithmic whip.

      Now, if someone comes up with an AI budy who helps with 1on1s and helps to build a culture of trust in a company, let me know, anything else is just waste of time and money solving problems created by crap management.

  • phoneymouse@lemmy.world
    link
    fedilink
    English
    arrow-up
    9
    arrow-down
    1
    ·
    12 hours ago

    They’re actually better known as Program and Product Managers. When engineers get tired of actual work, they try to convert to these jobs.

  • MTK@lemmy.world
    link
    fedilink
    English
    arrow-up
    118
    arrow-down
    2
    ·
    20 hours ago

    “We have to let you go as from our analysis you do mostly nothing, mr senior engineer”

    1 week later everything is crashing and no one knows why

    Ah yes, the classic evaluation of stupid shit that ends up shooting the company in the foot.

    • state_electrician@discuss.tchncs.de
      link
      fedilink
      English
      arrow-up
      1
      ·
      5 hours ago

      Yes, but there’s also people actually not doing anything. I am dev lead and after building a team, which was a lot of work, I am at a point where I am doing fuck all on most days. Maybe join a few meetings, make some decisions and work on my own stuff otherwise.

      • MTK@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        3 hours ago

        Yeah, there are plenty of truly pointless workers, I’m not denying that. But doing stupid metrics like commit counting or lines of code per day is stupid and counter productive, and it emphasizes the out of touch and inhuman methods of corporate idiots

    • BearOfaTime@lemm.ee
      link
      fedilink
      English
      arrow-up
      58
      arrow-down
      2
      ·
      19 hours ago

      Yep.

      This question doesn’t address what else these engineers do besides write code.

      Who knows how many meetings they’re involved in to constrain the crazy from senior management?

      • Saik0@lemmy.saik0.com
        link
        fedilink
        English
        arrow-up
        28
        ·
        18 hours ago

        Who knows how many meetings they’re involved in to constrain the crazy from senior management?

        This is more than half of my job. Telling the company owners/other departments “No”. Or changing their request to something actually reasonable and selling them that they want that instead.

    • enkers@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      32
      ·
      edit-2
      18 hours ago

      Makes me think of a trend in FTP gaming, where there was a correlation between play time and $ spent, so gaming companies would try and optimise for time played. They’d psychologically manipulate their players to spend more time in game with daily quests, battle passes, etc, all in an effort to raise revenues.

      What they didn’t realise was that players spent time in game because it was fun, and they bought mtx because they enjoyed the game and wanted it to succeed. Optimising for play time had the opposite effect, and made the game a chore. Instead of raising revenues, they actually dropped.

      This is why you always have to be careful when chasing metrics. If you pick wrong, it can have the opposite effect that you want.

      • MTK@lemmy.world
        link
        fedilink
        English
        arrow-up
        9
        ·
        12 hours ago

        When your data “scientists” don’t understand the difference between causation and correlation

        • pdxfed@lemmy.world
          link
          fedilink
          English
          arrow-up
          6
          ·
          10 hours ago

          And why economists and sociologists are important to have in the room when marketing and sales heads throw stupid fucking ideas on the table.

  • sudo42@lemmy.world
    link
    fedilink
    English
    arrow-up
    35
    ·
    17 hours ago

    How much hubris/ignorance this guy has to believe his algorithm is accurate enough to detect “10%” of employees were deadbeats? What precision! If it found 50% deadbeats, that would mean the algorithm might be working.
    The worst companies have only 10% deadbeats? Any company with only 10% deadbeats means their management team is doing a great job hiring/managing. Any company that only 50% deadbeat managers would be outstanding.

  • gedaliyah@lemmy.world
    link
    fedilink
    English
    arrow-up
    81
    arrow-down
    1
    ·
    20 hours ago

    I am not a coder, nor do I work in or have much knowledge of the industry. But I can tell immediately that this looks like some extra fancy BS. Designing a program to detect the quality and quantity of a person’s code commits sounds like AI mumbo-jumbo from the start. Even if it were technically possible, it would not tell you whether someone is an effective communicator, coordinates with other team members, shares productive ideas, etc.

    The headline should have been:

    “Consulting Firm Desperately Tries to Justify its Existence.”

    • gedaliyah@lemmy.world
      link
      fedilink
      English
      arrow-up
      48
      arrow-down
      1
      ·
      19 hours ago

      Just googled the paper’s author. Yep, sure enough he seems to contract with “FounderPartners,” which describes itself as, " a team of serial entrepreneurs and M&A advisors."

    • Nighed@feddit.uk
      link
      fedilink
      English
      arrow-up
      9
      ·
      19 hours ago

      It does mention that they send some of these in, and sometimes they get responses back that they are fine.

      That covers all of your senior engineers that end up spending more time speccing/investigating things than code.

      This kind of tool is probably very useful in ‘fiefdom’ companies where middle managers refuse to fire people because then they lose a headcount, or just protect their cronies. Having a central team that cuts across the company investigating that would be a good idea.

      Unfortunately in a lot of cases, I can see people being fired off that even though they are doing other work, just because management don’t understand what they do. Or worse because someone sells the tool as being flawless and they just fire anyone it picks up.