Here’s an uncomfortable scenario Chief Technology Officers (CTOs) in healthcare software companies are all too familiar with.
The Senior team has gathered together for their monthly progress report outs. Going first, the Chief Revenue Officer (CRO) notes her team has not hit their sales numbers for the month, details where they fell short and lays out a five-point plan for getting the team back on track to include placing one of her sales executives on probation.
Pleased with the update and proposed action plan, the CEO turns to the CTO for his report.
Launching into a quick review of ongoing projects, the CTO ends by claiming the engineering team has more work than they can handle (noting this a common issue for tech companies) and pleas for more resources or a reprioritization of projects.
When pressed by the CEO to explain the productivity of his team members, the CTO awkwardly concedes he has limited insight into individual team member performance relying completely on the number of features getting out on time (“feature velocity”) as his sole measure of performance. Clearly frustrated by the CTO’s inability to hold individual team members accountable, especially in light of the responsibilities assumed by other members of the Senior team, the CEO offers a less than committed response to the CTO’s resource request before moving on to the next report out.
THE CTOs CHALLENGE
The scenario described above is not uncommon because the reality is, reporting on engineer productivity is challenging.
Defining the productivity of software developers is difficult. Writing software code is an inherently creative and collaborative process with efforts to establish a clear link between the various inputs and outputs (rather than business outcomes) an inherently challenging task. A task that has grown more challenging with the explosive use of AI for coding. While tools like GitHub Copilot or ChatGPT can help write code faster and make junior coders look like “rockstars”, they also come with “hallucinations” and errors, making the use of generated code in production risky, which ultimately hobbles productivity.
There is a “language” barrier CTOs need to navigate when dealing with the CEO and other non-technical personnel. The technical work engineers do is difficult to translate for non-technical stakeholders such as managers and investors. The brilliance of an engineering team can easily get lost in translation.
Managing remote (especially offshore) engineering teams has its own unique set of challenges. With limited to no continuous insight into the work offshore teams produce, managers are not only operationally “blind” but must work through time zone and communication differences.
Yet, if other groups can measure team and individual performance, then is it not absurd to think engineering cannot? How else can health tech leaders assess / compare investment opportunities, manage team and individual engineer performance, know which type of engineers to hire / fire, etc.?
At least that was McKinsey’s thinking when they set off a heated debate in 2023 on the possibility of measuring engineer productivity.
A CONTINUED ROLE FOR TRADITIONAL METRICS
To be clear, the debate around how to define, measure and improve both individual engineer and software team productivity is far from settled. While nuanced measurement approaches such as DORA and SPACE have emerged and clearly have a role in the CTO’s tool belt, there seems to be a growing sentiment questioning the value of traditional engineer product activity metrics (e.g., lines of code written; hours logged; number of commits). Understandable, as traditional engineering metrics can be misleading, even damaging, if used incorrectly. CTOs heavily focusing on the number of code lines an engineer writes for example, may inappropriately incentivize developers to prioritize quantity over quality, or even engage in unproductive practices to meet some arbitrary target.
Yet, in a world where feature velocity is highly significant, traditional productivity measures still have a role. This is especially true for CTOs in identifying their top talent (those “heavy-lifters” generating the most impactful code). There is no need to shy away from leveraging these types of metrics or the performance tracking software tools which allow CTOs to automatically track an individual engineer’s software development performance. Instead of throwing the proverbial “baby out with the bath water”, the real need is to focus on a balanced mix of quantitative and qualitative metrics that reflect quality, impact, and actionable insights. Not just vanity numbers.
In doing so, CTOs are better positioned to:
- Ensure alignment with business goals: Productivity measures that mirror the company’s product roadmap ensure engineers are spending their time on the highest-impact work.
- Improve team efficiency and morale: When engineers see that their meaningful contributions are recognized and measured, they’re more engaged and motivated.
- Identify and remove bottlenecks: Product- centric code metrics help uncover friction in the delivery process, be it from unclear requirements, inefficient tooling, engineer competency, or unnecessary meetings.
NO MORE AWKWARD MEETINGS
Measuring productivity through the lens of value creation, not just motion, allows CTOs to lead with clarity, invest where it matters most, and scale their teams effectively. And while emerging qualitative measures surrounding productivity are important to weave into performance measurement processes, they should not come at the expense of eliminating traditional code performance metrics. Senior Leaders crave accountability and traditional productivity metrics provide the glimpse into engineering team performance CTOs can build on when discussing their team’s performance.
Without them, the Senior Team meeting will continue to be awkward for CTOs.
Paul Mathews Co-founder and CTO of TripleKey
Paul is a West Point graduate with a BS in Engineering with over two decades of software engineering experience. Paul has spearheaded teams at renowned firms such as Raytheon, Aristocrat Leisure, Xcite, and YouScience. Known for scaling teams and technology in high-growth startups, some with over a million active users, Paul excels at building teams that deliver solutions in highly regulated industries that prioritize trust, security, and compliance.
