Author and researcher into DevOps Gene Kim explains how the methodology overcomes technical debt and structural problems.
DevOps is a cultural approach to running a technology organization successfully - an organization that is able to change is cyber-secure and, most importantly, pleases its customers. For CXOs, whether they have a technology remit or not, the principles of DevOps are a powerful way to enact change, retain talent, and meet the expectations of your vertical market.
"DevOps is the architectural, technical practices and cultural norms that allow us to increase our ability to deliver applications and services quickly," says author and DevOps expert Gene Kim. "This allows us to deliver value to our customers in the fastest possible way…we can win in the marketplace," he adds. Kim also cites the DevOps definition coined by Jon Smart: "better value sooner, safer and happier." Whether in the development of software applications or other business outcomes, Kim says DevOps, "drives genuine business value," and this is why organizations should adopt it as a methodology.
In IT teams, Kim's research found: "High-performing teams are massively out performing their peers on deployment frequency." The author of The Unicorn Project says that DevOps is as transformational to technology and knowledge sector organizations as the Lean methodology was to manufacturing and supply chains in the 1980s when the Japanese automotive and industrial sectors pioneered the approach. "DevOps takes the principals of Lean and applies them to how we work every day," he says of how both Lean and DevOps create high engagement patterns amongst employees and customers and continuous improvement to products, services, and processes.
Lean and DevOps-focused organizations are more engaged because they have an internal structure that promotes engagement, learning, and empowerment. All of these enable continuous improvement and business agility. This same structural approach to the organization also tackles technical debt head-on, which Kim says is central. He says in many organizations, there are invisible structures that hinder the development of new products; these can include an inability to get data from decision-makers, data that would enable innovation. Or opposition to new ways of working. These invisible structures become the architecture of the business, and it is then unable to change.
The very digital businesses that have reshaped the last two decades had invisible structures that led to significant challenges, businesses such as Amazon, eBay, Etsy, Google, LinkedIn, Microsoft, and Twitter. At Etsy, an online retailer, the organization was struggling with releasing new features to its online store, the reason - Sprouter. Store Procedure Router (Sprouter) was introduced to act as a central place where developers and back-end database administrators could come together to complete new technology services. Each team worked independently on a new business service and then would meet on Sprouter to connect their work and release the new service. "This required a degree of coordination that was rarely achieved," Kim says. "Almost every deployment became a mini outage."
Kim's work in the DevOps field has led him to believe, write about and advise that making teams work together is risky and leads to the mini outages that he observed at Etsy. "When you have two teams that have to work together, then they have to coordinate, communicate, schedule, priorities, and marshal conflict,"
he says. DevOps succeeds, Kim says, because teams are empowered to carry out the entire process, taking on the work of back-end and front-end teams. "At Etsy, they fully enabled developers to work by themselves and not be dependent on DBAs. If there are multiple teams, it doesn't take a lot to go wrong before you are looking at delays that are measured in weeks, months, or quarters."
In a DevOps environment, team members are responsible for the entire process. "These teams are far more likely to succeed without causing issues as, if there is a problem, they can fix it within hours. They integrate security, and they are twice as likely to achieve business mission goals and be a great place to work," Kim says.
"It is not enough to move boxes around an organizational chart. You must have an architecture that allows teams to work independently as architecture is one of the top predictors of performance," Kim says of developing a completely new cultural approach. In Kim's research for his books and lectures, he has found a myriad of ways that organizations define the size of teams and their responsibilities, ranging from online retailer Amazon defining teams by the number of people that two pizzas feed.
The second invisible structure that Kim says hampers technology organizations is technical debt. "Small changes affect the whole system, so like financial debt, it gets worse over time without an active intervention,'' he says. Risto Siilasma, a former executive of mobile telecommunications firm Nokia, had just this experience: "The build time for the Symbian operating system meant it was two days before any engineer could determine if a change had worked." Referring to the issues that technology giants Amazon, eBay, Etsy, Google, LinkedIn, Microsoft, and Twitter experienced, Kim, says these organizations realized that technical debt was central to their problems, alongside the structure of the business. "Bill Gates published a memo that said if developers ever have to choose between working on a feature or a security issue, you must always pick the security issue as the survival of the company depends on it," Kim says of how not dealing with a security issue becomes technical debt. "These firms took feature development down to zero, paid down their technical debt, and increased quality."
Ideas Flow
With the right culture in place and technical debt in hand, Kim says organizations can then improve the service to customers and get into a flow that means change is normal and not disruptive. "Flow is that amazing feeling of enjoying our work, so we lose track of time," says Dr. Mihaly Csikszentmihalyi.
Kim says businesses fail to achieve flow because their organizational architecture often has two opposing value streams. Product or service design and development requires an organization to be uncertain and to experiment, which rarely fits with the business's operational demands and culture.
One way to tackle this is for organizations to improve the way they work on a daily basis, with a focus on daily improvement. To get to this position, Kim says feedback is vital and one of the key principles of DevOps. "We can validate and learn more. This is at the core of a learning organization," he says. Once again, DevOps has learned from the Lean movement in manufacturing, and the Fremont car factory in northern California is a prime example of how Lean and DevOps culture change can take root and make a significant difference. When the Fremont plant was a General Motors (GM) factory, it had a terrible reputation for poor-quality products. In later years Fremont would become the US home of Toyota, the Japanese car maker that would become one of the leading proponents of Lean. On a Toyota factory line, every worker has an Andon cord above them; if they have a problem with a faulty part or an item takes longer to install than expected, it is the cultural norm for that employee to pull the cord, which stops the entire line. Although this disruption adds time to the manufacturing process, Kim says it is a great example of a working culture that deals with technical debt. If there is a faulty component or an issue with the business processes, by pulling the cord, the organization stops what it is doing and deals with that issue. "If you don't fix it then you will have the same problem 55 seconds later," he says.
Together these processes develop the right culture across the organization, which leads to transparency. Kim says DevOps thrives on a transparent culture where there is a general sense of inquiry - the opposite of a blame culture and work arounds. This, in turn, leads to an organization that has a strong customer focus. An example of customer focus Kim shares is Compuware, a provider of data center services. At the main Compare data center, the organization has a blank space with the area where server racks once stood marked out on the floor. Within each of these marked-out areas is a sign indicating the applications that once resided on these servers and the cost of operating them. What this scene depicts is a saving of $80 million in back office expenditure that Compuware has taken out of its business costs. That money has been reinvested back into the Compare services. Kim cites Compuware as an example of an organization that has understood that although it is essential for a business to have payroll and the software to run payroll, this is not what the customer pays for. So a customer-centric organization removes as much of this cost base as possible. The savings are invested into products and services for the customer.
DevOps Leadership
As an organization adopts a DevOps culture, the role of leaders and the CXO team is to ensure that the dynamic of DevOps remains in place. Kim says that as a dominant culture and its methods grow within an organization, it can become staid. "Teams become incapable of solving problems that they were not designed for," he says. "Organizations get into a groove, but that then becomes a rut," writes Clayton Christensen in the Innovator's Dilemma. A groove is a good thing, it is the flow that Kim describes above, but a rut is negative, as it prevents creative thinking.
To combat this in large enterprises, Kim says independent teams, which use DevOps methods, should be created to tackle changes in the way the organization operates. Kim cites the example of German automobile maker BMW taking a team from its battery department and tasking them with creating a electric car. Away from the mainstay of the organization, they were able to create an entirely new vehicle, the BMW i3, which has been a success for the organization. Kim says this is an example of a business unit being created away from the "performance engine" of the business. In other words, the core of the business and its revenue targets.
A key part of the success was to ensure the i3 team had a leader who could act as what Kim describes as the interface. Although the unit was creating something new, it would have needed to work with other business units along the way, the team that makes brakes, for example. Having a senior leader that can ensure these relationships form and deliver outcomes is vital, Kim says.
"Leaders have to be a maestro with high energy, high standards, an ability to see the system as a whole, but they can also ask good questions, and they love walking the floor," he says of the CXO role in a DevOps culture.
Kim was the founder and Chief Technology Officer (CTO) of Tripwire for 13 years, and since 1999 he has been an author, researcher, and speaker focusing on high-performance technology organizations. His books include The Unicorn Project, The Phoenix Project, and the DevOps Handbook.
