On the 10th anniversary of its first survey, the multi-vendor State of Devops report concludes that DevOps is still “rarely done well at scale,” and that “almost everyone is using the cloud, but most people are using it poorly.”
The report is opinionated and there is no vendor pitch, perhaps because it is sponsored by a variety of DevOps companies including CircleCI, Puppet, bmc, New Relic, Snyk and Splunk. The first survey was in 2011 so this the eleventh report but a decade after the first.
“We don’t want to produce something lightweight, there’s a lot of crappy surveys out there, that I don’t think are particularly robust in their methodology or insightful in their findings,” Puppet’s Field CTO Nigel Kersten, who has been involved with the State of DevOps Report from the beginning, told us.
The survey was completed by 2,657 individual DevOps professionals across the world, with around 75 per cent from the USA, Canada or Europe, and covering a wide range of organisation sizes.
The general approach is to divide organizations into those that are highly evolved, mid-level performers, or low performers. “Highly evolved” means frequent deployments, very short lead time for application changes (“less than an hour”), quick response to failures (“less than an hour”), and a change failure rate of less than 5 per cent. “Low” is the other extreme, with a lead time for changes of “between a week and six months.” According to the report, “the vast majority of organizations are stuck in the middle.”
This division informs the rest of the analysis, which compares the responses of high, mid, and low performers on questions like “Members of my team have clear roles, plans and goals for their work.” 89 per cent of high performers agreed with this, compared to just 46 per cent of low performers.
Have some standards!
Another striking example: 57 per cent of high performers felt their IT systems conformed with all five of the US NIST (National Institute of Standards and Technology) essential characteristics of cloud computing, these being “on-demand self-service, broad network access, resource pooling, rapid elasticity or expansion, and measured service,” whereas just 5 per cent of low performers could tick these boxes.
“Almost everyone is using the cloud, but most people are using it poorly,” the report says. Asked to expand on why, Kersten told us that “most people are not taking advantage of those things, and essentially lifted and shifted their existing processes and practices into the cloud.”