Engineering Impact Of Generative AI, Give Outstanding Code Reviews
Issue #14 Bytes
🙏 Points of inspiration for a perfect Sunday ⤵️
🎙️DORA Implementation & Tech Debt
Watch the exclusive podcast dedicated to all things DORA and beyond! In this episode, Peter Peret Lupo, Head of Software Engineering at Sibli, speaks in detail with our host, Kovid Batra, about the vital role engineering metrics play in shaping organisational culture. They delve into topics such as code collaboration, the relationship between metrics and technical debt, and the challenges and timelines involved in implementation and adoption.
Don't miss this insightful conversation! (Kovid, has a special message inside)
Article of the Week ⭐
“Despite analysing the activity of over 218,000 developers in the GenAI-based comparison group, fewer than 1% (2062) developers used AI-generated code without making significant changes.”
The Impact of Generative AI on Software Developer Performance
Editor note—Hey, this is Denis. We prefer not to headline articles of the week that are behind soft-walls, but the topic of GenAI and Copilot have been highly sought after.
BlueOptima launched a massive survey on GenAI usage, focusing on Copilot and its adjacent ecosystem in Software Engineering teams. The analysis encompasses a mind boggling 🤯 218,354 professional developers and about 880 million commits from mid-2022 to mid-2024.
Despite the large sample size, only a small fraction of developers consistently used GenAI-authored code without significant human rework.
Key Findings
Productivity Increase:
GenAI enhanced developer productivity by ~4%.Wow, just four percent!?
Developers with moderate usage emerged as the highest performers, suggesting an optimal balance between AI assistance and human expertise.
Code Quality:
Overall code quality was maintained, with no significant reduction due to GenAI usage.
The low rate of unaltered AI-authored commits highlights the need for further refinement in integrating these tools.
Adoption Patterns:
Unexpectedly, 64% of licensed GenAI users employed GenAI before officially being granted licenses, indicating a strong grassroots adoption trend.
High Friction, Low ROI
The study identified three groups for its evaluation:
High levels of AI usage
Low levels of AI usage
No AI usage (control group, the largest of the three)
The study showed a clear, albeit small advantage in productivity for the AI-using groups of engineering. Measurement details at the end of the paper, linked below.
However, the highest performing groups were those who used GenAI tools moderately, finding a balance between AI assistance and their own expertise. The study hints at a correlation between an increase in time spent correcting AI-generated code as usage increases on the High-GenAI group.
Despite this, both groups showed similar levels of code quality, keeping it firmly in the domain of human engineers. For now.
⚙ Join the Webinar in 3 days | Implementing Engineering Metrics ft. Google DORA Lead, Nathen Harvey
End all your DORA questions and drive engineering success!
Join our 45-minute live webinar on July 31, with DORA expert Nathen Harvey and special guest Ido Shveki, VP R&D at BeeHero. Learn to decode successful DORA implementation for your dev teams.
Interested? Register here 👇
Other highlights 👇
Leadership Lessons from CrowdStrike Outage
CrowdStrike’s outage has been the talking point among tech experts for the past week. A prime example of software practices causing harm and disruption in the real world. The world has jumped to speculation, but we have waited for the official postmortem to hopefully shed some learning insights from the situation.
We suggest you read the preliminary report or executive summary (linked below) for full context. To keep things informative and brief, here is what CrowdStrike leaders propose as changes to prevent it from reoccurring (paraphrased).
1. More unit tests (local to the developers)
It seems the incident would have been easily preventable by exhaustive unit, snapshot or even mutation testing. This includes regression-testing this particular incident
2. Graceful Error Handling
In short, crash gracefully—but let the customers continue to operate their businesses, like operating planes itineraries, handling patients in hospitals, etc.
3. Staggered Deployment Strategy + Monitoring
One would wonder why such an expensive and complex engineering product isn’t already taking advantage of canary deployments for this particular set of features.
Unexpectedly, the incident itself did not involve essentially complex machinations or black swan circumstances. Merely the basics of Minimum CD adoption would capture such incidents when adopted to an adequate standard that they seemed to have skipped or reduced.
How is your organisation’s continuous delivery adoption?
How to give outstanding code reviews?
Tobias walks the fine line between engineering excellence and developer well-being. He brings 9 tips on how to be more empathic and practical in asynchronous, written code reviews to create more spontaneous and welcome environment for your team.
Below are the top 3.
1. Give reviews as timely as possible
Unfinished work being handed off for review or additional re-work is one of the most pervasive forms of waste in Lean theory.
Small changes often wait hours or days for someone to review and merge them. Much of the lead time is spent "waiting for review," which is inefficient and lowers team performance. It is also frustrating.
2. Have clear expectations
It’s easy to get lost in the immediate solution presented by the to-be-reviewed code snippet without full understanding of the problem being solved. Before jumping on the first few problems, zoom out and have a clear sense what outcomes the code change aimed to solve and whether it fulfilled on that promise.
Looking for a mentor to connect with & get some actionable personalized advice on solving your leadership challenges or your next career move? -Let’s Connect
3. Tests first
If [tests] are missing, this is a red flag or at least justifies extra caution while proceeding with the review.
A single bundle of change within a review or pull request should focus on one solution. If the tests are missing or communicate about a completely different problem area, this is a sign of hidden complexity or risk that is still waiting to be managed.
The rest of the tips include guidance on curiosity, learning, root cause analysis and collaboration.
😆Just for laughs
When we heard the news of the #Microsoft Outage, we immediately thought of this!
That’s it for Today!
Whether you're hustling with your side projects, catching up with the latest technologies, or simply relaxing & recharging, wish you all a lovely day ahead.
See you next week, Ciao 👋
Credits 🙏
Curators - Diligently curated by our community members Denis & Kovid
Writers of the week - Tobias Mende
Sponsors - This newsletter is sponsored by Typo AI - Ship reliable software faster.
1) Subscribe — If you aren’t already, consider becoming a groCTO subscriber.
2) Share — Spread the word amongst fellow Engineering Leaders and CTOs! Your referral empowers & builds our groCTO community.
Thanks for mentioning my newsletter and highlighting the top 3 again. 😊🙏