A key part of your DevOps programme, and instrumental in its success, is how DevOps and your applications are measured. If this measurement is done well, it will provide accurate ROI, help you to manage progress towards your DevOps goals, and be a great help in evangelising DevOps success.
Metrics supporting the business
Well-defined and relevant metrics also help to focus the organisation on delivering the right business outcomes. They make sure that every change implemented is a step towards achieving a defined business goal. However, metrics used badly can stall or even reverse the cultural changes in progress. This will delay your DevOps ambitions and disrupt your long term goals. Bad metrics can for example be ones that put different teams in direct conflict – which is the very state we are trying to move away from.
For the most accurate information on your progress, start measuring from day 1 of your DevOps journey – or earlier, if possible. This allows you to compare your current state with your starting point throughout the entire process.
Measuring issue resolution
A good metric to start measuring might be MTTR – Mean Time To Recovery. This measures how long it takes for an issue to be resolved in production. High MTTR generally translates to long downtime for the customer or business users, which results in loss of revenue and direct business impact. Low MTTR corresponds with high availability, fast responsiveness and minimal customer impact.
With low automation, high MTTR is generally down to the manual nature of a lot of the work required for releases. When a manual release goes wrong, a manual rollback or hot fix is often rushed in. This change can bring increased risk, as the rollback process is not likely to be thoroughly tested under the added time pressure. If a quick hot fix is needed, the developer will write the code quickly and try to get as much testing as possible done before the hot fix deadline. But without continuous integration and continuous delivery, the full set of tests (including regression tests) will inevitably not get run.
Reducing MTTR directly translates to better reliability and better customer service. It creates cohesion within the entire application team, as there is no conflicting goal between any groups.
Encouraging innovation
Something which goes hand in hand with measuring MTTR is not being afraid to innovate and make mistakes. With a low MTTR we have greater confidence that any problem can be resolved quickly, so the impact of issues in production is not so high. This means that each issue becomes less critical. This attitude relies heavily on underlying cultural change to encourage innovation and not focus the blame on incidents.
When creating goals and metrics for tracking progress, the final key characteristic I want to highlight is the creation of goals that are applicable to each stakeholder. This means that everyone can take ownership for achieving them. Having individual goals for each team member should encourage success, as everyone has a vested interest.
What works for your business?
In this blog, I have only scratched the surface of the vast topic of metrics and measurement in DevOps success, but I will be discussing this more in future blogs. Please let me know if you agree or disagree, and if you have any other useful metrics that you have seen used successfully – or badly, for that matter!
I have touched on one particular metric here, which is useful for some organisations adopting DevOps. Our recent white paper goes into much more detail about KPIs for DevOps success from a broader perspective, including tips and tricks to build a KPI framework suited to your organisation.
Bronwyn Davies
Latest posts by Bronwyn Davies (see all)
- Winner! Unicom Most Innovative Project 2023 - 27th February 2023
- Sandhata Gini – Our internal chatbot - 1st February 2023
- Goodbye 2022 - 19th December 2022