Curious how it would track more ide things like a refactor to method command rather than cut and paste. I can understand not tracking paste as you don't really know where it came from, but rtm is using the same code just moving it for you.
This does make more sense to some of the big claims "95% of our code is ai generated" though. By the logic here a lot of my code would be considered generated for years. ide auto complete on variables, functions, closing brackets, class names etc. Lines changes by linters and formatters to add so much as a comma are not mine anymore. Add in refactoring and my use of ide templates and framework generators to make common files and increasingly small amounts would be considered my written code.
Then it should at _least_ be maintaining a third “unknown” category, but then they'd have to acknowledge (to the management, no less!) that they don't know as much as they're claiming.
Kudos on the analysis, the conclusion is right. I would go further and say even if the metric was completely fair and unbiased it would still not tell you anything useful, and any manager or executive that used it as part of any kind of headcount decision (firings, layoffs, which team to grow, etc) is a moron who probably should be facing tough questions from their own management chain.
AI is a tool, everything it does is attributable to the person who prompted it. Anything else is no different from the long-understood fallacy of counting lines of code.
PCW clustering around ~85-95% regardless of usage is a measurement bias, not a real signal. In manufacturing, this would fail measurement system analysis by having a larger variation than you're trying to detect. Companies trying to make headcount and copyright decisions on that are doing the AI version of measuring with a broken ruler.
Curious how it would track more ide things like a refactor to method command rather than cut and paste. I can understand not tracking paste as you don't really know where it came from, but rtm is using the same code just moving it for you.
This does make more sense to some of the big claims "95% of our code is ai generated" though. By the logic here a lot of my code would be considered generated for years. ide auto complete on variables, functions, closing brackets, class names etc. Lines changes by linters and formatters to add so much as a comma are not mine anymore. Add in refactoring and my use of ide templates and framework generators to make common files and increasingly small amounts would be considered my written code.
> as you don't really know where it came from
Then it should at _least_ be maintaining a third “unknown” category, but then they'd have to acknowledge (to the management, no less!) that they don't know as much as they're claiming.
Kudos on the analysis, the conclusion is right. I would go further and say even if the metric was completely fair and unbiased it would still not tell you anything useful, and any manager or executive that used it as part of any kind of headcount decision (firings, layoffs, which team to grow, etc) is a moron who probably should be facing tough questions from their own management chain.
AI is a tool, everything it does is attributable to the person who prompted it. Anything else is no different from the long-understood fallacy of counting lines of code.
so, what you're saying is that we can definitely expect this metric to be used to make hiring / firing decisions then
PCW clustering around ~85-95% regardless of usage is a measurement bias, not a real signal. In manufacturing, this would fail measurement system analysis by having a larger variation than you're trying to detect. Companies trying to make headcount and copyright decisions on that are doing the AI version of measuring with a broken ruler.
That's re-assuring. It'll be much easier to fake any AI usage metric my management sets.
You mean productivity? Sure — but speed isn’t the same as replacing humans.
Such lying statistics are not at all unexpected, but it is very good to have actual evidence about this practice.