Plenty of organizations have commissioned a DevOps assessment, received a thick report with a maturity score and a list of recommendations, and then watched very little change. The report was not wrong. It simply was not built to be acted on.
An assessment is worth doing only if it changes what people do next week. That goal shapes everything: who you talk to, what evidence you collect, how you write the findings and what you hand over at the end. Here is how we approach it.
Start with a decision, not a maturity model
Before any interviews, agree on the decision the assessment should inform. “Where should we invest in delivery over the next two quarters?” is a good one. “How mature are we?” is not — a score gives people something to defend or dispute, but rarely tells them what to do.
A clear decision also keeps the scope honest. If the question is about delivery speed for customer-facing services, you probably don’t need to inventory every internal batch job.
Write down, in a few sentences:
- the decision the assessment informs and who will make it,
- which teams, systems and value streams are in scope,
- what “done” looks like — usually a short findings document, a prioritized roadmap and a readout session.
Follow real changes, not process diagrams
Documented processes describe how work is supposed to flow. To see how it really flows, pick one or two recent, representative changes and trace them from idea to production.
For each step, ask: who did the work, how long did it take, and how long did it wait before someone picked it up? In most organizations the waiting dominates — reviews queued behind other work, environments shared between teams, approvals from people who are in meetings all day. This is classic value-stream mapping, and it works because it replaces opinions with a timeline everyone recognizes.
Combine the walkthrough with data you already have. Your version control and CI systems record when changes were committed, built, deployed and rolled back. Even a rough calculation of the four DORA metrics — deployment frequency, change lead time, change failure rate and failed deployment recovery time (formerly “time to restore service”) — gives the conversation a factual anchor. Be explicit about how you calculated them; definitions matter more than precision at this stage.
Talk to the people who feel the pain
Interview across roles, not just up the hierarchy: developers, testers, operations and on-call engineers, security, product owners and the managers who balance all of them. Keep the sessions small and confidential so people can be candid.
Useful questions are concrete:
- “Walk me through the last time you deployed. What happened?”
- “What do you wait for most often?”
- “What would you fix first if you had a free week?”
- “When something breaks in production, how do you find out?”
Patterns appear quickly. When three different teams independently mention the same flaky integration environment, you have found something worth fixing.
Write findings people can act on
A good finding has three parts: what we observed, why it matters, and what we recommend. Tie each one to evidence — an interview theme, a step in the value-stream map, a data point — so it can be discussed on its merits.
Then prioritize ruthlessly. A useful structure is:
- Quick wins — changes a team can make within a few weeks without new budget, such as removing a redundant approval or fixing the flakiest tests.
- Foundations — larger investments that unblock several improvements, such as reusable pipeline templates or self-service environments.
- Bets — bigger changes worth exploring with a pilot before committing, such as a new platform or operating model.
Fewer, sharper recommendations beat a long list. If everything is a priority, the organization will choose by politics instead of evidence.
Hand over a roadmap, not a report
The most important output is a roadmap that names owners, first steps and the measures you will use to judge progress. Review it with the people who will do the work before it goes to leadership — if the teams don’t believe in it, it will stall.
Finally, schedule a follow-up. Revisiting the roadmap after a couple of months, with the same measures, shows what has actually changed and keeps the assessment from becoming another document on a shared drive.
A short checklist
- The assessment answers a specific decision, with a named decision-maker.
- At least one real change was traced end to end.
- Delivery data was gathered, with its definitions written down.
- Interviews covered every role involved in delivery, not just managers.
- Every finding links to evidence and to a recommendation.
- The roadmap has owners, first steps and a review date.
If you are planning an assessment and want a second pair of eyes on the scope, we are happy to compare notes.