I really enjoyed Will Larson’s recent post on measuring engineering. Excellently written as always, and Will provides a great framework for thinking about measurement for different audiences. He provides a taxonomy for thinking about measuring for different stakeholders. This is a recent shift that we have made at Tatari, from measuring for ourselves to doing it for stakeholders.
Reporting your “measure to plan” measurements are fine – everyone wants to know if you are hitting your own goals. Similarly “hard” measurements like headcount-to-plan and costs are straightforward to report. But in my experience the company wants more features, faster, with fewer people, at lower cost, and with zero bugs! They want to know how your metrics are helping you to achieve all that.
The importance of trade-offs
Unfortunately most things in engineering-life do not come for free. Thus one of my primary goals is to help our external stakeholders understand the critical trade-offs that we are making in engineering.
So we have created a basic framework that will be familiar to anyone in engineering:
- Velocity vs.
- Quality vs.
- Cost
We can and do make regular trade-offs to increase velocity, accepting less up-front design and testing time that may result in bugs found later. Similarly we can lower engineering investment by leveraging cloud spending instead, such as letting Databricks operate Spark for us rather than running it ourselves.
So we anchor our measurement for stakeholders in this framework of trade-offs. Generally, we try to define quality and cost as constraints, and get as much velocity as we can within those constraints. The Product team are key partners in balancing the trade-offs. But we solicit other stakeholder feedback to tune our constraint levels, or to understand when it may make strong business sense to surge on velocity at the cost of other constraints.
The engineering trade-offs reflect the company-level trade-offs that the business has to make: how much to invest in marketing, sales, and engineering, and customer success. When to surge in one area, or shift resourcing to support a strategic goal.
Effectively sharing good engineering metrics with your stakeholders allows you to better align your engineering trade-offs with the strategic goals and trade-offs that the company is making. Good numbers, well communicated, are a critical tool to drive alignment across business functions.
scott persinger 