Communications of the ACM
Measuring Up: How to Properly Measure Your Programmers
To measure or to not measure, that is the question. Excuse me for waxing poetically, but measurement is one of the touchiest subjects among programmers and managers. Should programmers be measured? Or is doing so intrusive and ineffective? Often, the answer depends on where you stand and the job title on your business card.
Ask programmers and many will tell you that measurement is a fool’s folly. Measurement undermines the team spirit, or it reduces dynamic solutions into simplified numbers that can never reflect the true complexity of the project. Measurements can even set off a hero complex among the most ambition programmers, setting them up for failure while reducing collaboration.
Those aren’t my words, but hang out with programmers enough and you’ll likely hear similar critiques. And you know what? There is some truth to such statements. However, I’d argue that there are also a lot of misconceptions. The wrong measurements and poorly designed performance reviews can indeed undermine teams, and in the long run, also productivity.
At the same time, however, without measurement, managers must cede most, if not all, control of a project to their programmers. In an extreme case, managers will simply have to take their programmer’s word for it that the project is advancing successfully and that resources are being effectively put to work.
In a worse case scenario, the project can devolve into anarchy. Managers are reduced to de facto administrative assistants, and programmers run the show with little oversight. Surprise, surprise, the project starts to go off the rails, with development slowing and bugs proliferating through the code like so many swarms of locusts.
Yet should we measure measurements by the worst-case scenarios? In my opinion, absolutely not. Poorly designed measurements in incompetent hands can destroy projects, break apart teams, and ultimately overwhelm companies. Yet the right metrics and review processes can bring order to chaos, streamlining work and producing excellent results.
Okay, so what does good measurement look like? Let’s take a look at some promising metrics.
Mentee results: Some people believe that measurement can break apart teams. New team members, including mentees, are perhaps the most at risk, producing measurably poor results early on. Why not flip the script? Measure mentees by reasonable standards AND give props to those mentors who are helping them along. Now, senior team members have an vested interest in seeing new talent succeed.
Features implemented: Yes, measuring software and programming performance is tricky. One reasonable approach is to measure the features delivered or implemented. Still, every time a programmer on GitHub or an internal ticket system completes a ticket and adds a feature, you can check it off and give them credit. Just make sure that the product manager, not the programmer, closes the ticket.
Uptime: Uptime is a great, high-level way to measure success. High uptime suggests that the software was well-programmed and not suffering from critical bugs or other issues. Consider a variety of measures, including the mean time between failures (MTBF), mean time to failure (MTTF), failure rates, and other relevant DevOps metrics.
Bugs measured and fixed: Measuring bugs fixed is a great way to improve software. Some companies punish programmers when bugs crop up. Guess what? Many programmers simply try to hide or paper over the bugs, resulting in software that even upon launch is unreliable.
The solution is simple: reward programmers both for uncovering bugs and fixing them. Once you do this, your programmers won’t let bugs slip by, and will be properly motivated to fix them. Just make sure that either the product manager or another programmer checks off both bugs that have been found and fixed.
Cost of Pull Requests: Great programmers will quickly merge submitted pull requests (PRs). The faster pull requests are peer-reviewed and sent to quality control, the better. Reward your team members at every step, including jumping in to peer review pull requests.
Admittedly, junior programmers may submit massive or complex PRs, which can wreak havoc during peer review. But getting junior programmers up to speed with pull requests is a great thing to reward mentors for.
Releases Published: Ideally, new versions of a project are released on a daily basis. Amid chaos? New versions might not come for weeks. Releases are stressful. However, by rewarding programmers for regular, timely updates, you encourage frequent releases.
Published Documentation: Some programmers don’t want to bother with documentation; after all, they already know the code. However, proper and extensive documentation ensures stability both now and in the future. Reward your programmers for how much documentation they produce; just make sure it’s validated before publishing.
Start measuring the above and you’re off to a good start. However, I need to emphasize something: peer review and quality control are essential. Without it, programmers will submit pages of terrible documentation or will report non-existent bugs. Everything must be double-checked.
Further, don’t enslave your team to metrics. Ultimately, metrics shouldn’t be ends in-and-of themselves; the goal is to keep the team and project on track. That requires some flexibility. Take bugs, for example; some are minor and not particularly urgent, while others are complex and fixing them is vital. No two bugs are the same, so take note of urgency and severity. You should adjust your metrics to account for this.
Ideally, programmers will be rewarded for performance as measured by metrics. In an ideal world, we’d toss out salaries and instead pay people for the features added, new versions launched, and documentation pages published, among other things. This way rewards commensurate with contributions.
For now, salaries may be here to stay. Ultimately, poor managers don’t bother with measurement and wages bloat up as they try to keep everyone happy. Great managers, on the other hand, implement a sound measurement system and reward those who put in the effort.
Yegor Bugayenko is founder and CEO of software engineering and management platform Zerocracy.
No entries found