The “Spotify Model” Demystified and Made Applicable through Essence
This article is a collaboration between Joakim Sundén and Stefan Malich. We do not work at Spotify and the views and opinions expressed here are our own, and do not express the views or opinions of Spotify. Thanks to Simon Girvan for review and creation of Mural gameboards.
Initial Chaos and Scaling Challenges at Spotify
According to old Spotify lore, as the company grew, the ad hoc collaboration that had been good enough for a small startup no longer worked when scaling up. It was no longer clear who was doing what, who to talk to when help was needed, how things were depending on each other, and so on. It was a bit chaotic at times and some people got really stressed out, crying out for help: “If we don’t do something about this, I’ll quit!” To address these challenges, Spotify brought in a Scrum trainer and introduced a pretty rigid Scrum prescription for all teams. Too rigid and bureaucratic for some apparently, who now said: “If we don’t do something about this, I’ll quit!”. It was clear that Spotify required a solution to grapple with the challenges of scaling and introducing Scrum by the book wasn’t the right way.
Balancing Chaos and Bureaucracy
This would become the constant balancing act as Spotify grew in the coming years; how can Spotify have some kind of process that helps teams and people to avoid chaos but at the same time doesn’t constrain them too much? The expression “Minimum Viable Bureaucracy” was introduced and already in the first Squad Health Check back in 2011 one of the six categories for the teams’ self-assessment was “a process that fits the team”. The aspiration (indicated by a green light in the health check) was described as “the team has actively optimized the process for their use and are continuously improving their way of working” and the failure (indicated by a red light) as “the team doesn’t follow any process”. If a team felt they were somewhere in between (indicated by an amber light), the option was described as “the team follows our process but haven’t put much effort into optimizing it for their use”.
Describing the Way of Working
It was clear within the Spotify organization that teams need to take ownership of their way of working as it needs to be adapted to fit their specific context and needs. It also has to be continuously adapted and improved as the team learns and their context changes. Values, principles, culture, training, coaching and management support would guide the teams on this never-ending journey.
One thing that was missing however, was a more structured way of describing which practices the teams were using. It is difficult to agree on what is working or not and what to do differently, if you don’t have a shared understanding of what you are currently doing. A good description and visualization of your way of working makes it possible to have constructive conversations about what to change to continuously improve. Capturing improvements, experiments, and lessons learned also facilitates sharing of knowledge and ideas so teams can learn and take inspiration from their colleagues. Unfortunately many of the teams at Spotify did not capture their way of working and those who were doing it often did it in an ad hoc way that wasn’t easy to share.
Enter Essence
When Joakim was introduced to Essence by Ivar Jacobson, it was immediately apparent that Essence could be a way to help address some of these challenges. Essence isn’t a new practice or method. It is an OMG standard that defines a common ground for describing, adopting and improving software engineering practices and methods.
The descriptions based on Essence establish a common vocabulary and body of knowledge which among other things helps practitioners compare practices/methods and make better decisions about their way of working. Thus, Essence empowers teams to learn and continuously adapt their way of working. In addition, it supports continuous performance improvements on various levels (e.g., organization, project, product, team). Furthermore, the Essence language allows organizations to create new or bespoke practices that can live alongside established practices in a Practice Library or ecosystem.
These key features of Essence match very well with Spotify’s approach to design and evolve organizational structures and way of working to fit the context of specific teams. And Spotify is of course not alone to have encountered these challenges. According to industry surveys there are many enterprises that are using some kind of customized version of agile frameworks, some even claim to use the “Spotify model”, even though it doesn’t really have a formal description. This got us thinking, could Essence actually be used to describe the “Spotify model” and support those who are adopting it or looking to it for inspiration?
Sidebar: Using Essence to Describe a Way of Working
The common ground defined by Essence includes the essential elements that are always prevalent in every software engineering endeavor (so-called Alphas, like requirements, software system, team, and way of working). These elements have states representing progress and health, so as the software endeavor moves forward the states associated with these elements progress. Moreover, the elements can be applied on different levels to assess the progress and health on each level. Essence supports scalability including from one product to many, from one team to many, and from one practice/method to many. On the level of teams the elements can be used by an individual team to guide their work and way forward based on the element’s states and practice descriptions.
Furthermore, Essence supports agility and autonomy on the level of teams because practices and methods can be refined and modified by an individual team to reflect its experiences, lessons learned, and changing needs.
Essence provides a common vocabulary for talking about solution development and a framework on which practices and methods are described. It tears down the language barrier that exists in many teams and software endeavors by enabling practices to be described using a single vocabulary—the so-called Essence kernel. In addition, Essence provides a visual form for each of its elements. Particularly, it defines a visual view based on cards which contain a mix of symbols and text that are related to an element. These cards can be used as physical, printed cards or as digital images on slides, virtual whiteboards, etc. They represent hands-on guidance for project teams supporting the definition, composition and communication of their way of working.
Is There Really a “Spotify Model” Though?
The Spotify model is a very loosely defined “framework” for agile at scale. The canonical “documentation” consists mainly of an outdated white paper from 2012 – that clearly states it’s “only a snapshot of our current way of working” – and two “engineering culture videos” from 2014 captured by a consultant who, in his own words, was “just the messenger”. If you want to learn more than that you’ll have to wade through a vast amount of presentations and articles, many of them published by Spotify employees saying things like “there is no Spotify model”, “don’t copy the Spotify model” and “you can do better than the Spotify model”.
In spite of the lack of a formal definition, the scarcity of semi-official descriptions and many cautionary tales, the “Spotify model” continues to appeal to many organizations. We believe it’s a conservative estimate to say it has been adopted in one way or another by hundreds of organizations by now. It is even being used by the big consulting firms as an agile transformation approach. So, no matter what people tell you, there is de facto a Spotify Model out there. Let’s stop kidding ourselves and instead actually help people better understand what the Spotify Model is and to evolve our collective understanding of how it can be of value. We believe Essence can be an enabler to better understand, communicate and adopt the Spotify Model.
Towards a Spotify Model Essentials
We have in collaboration put together a first draft of what we believe are the essential building blocks of the Spotify Model described with Essence. It is based not only on the aforementioned white paper and videos, but draws on Joakim’s experience from co-developing the model during his employment at Spotify from 2011-2017. Joakim has since also co-developed (with Jimmy Janlén, who consulted at Spotify for several years) the course “Agile at Scale, Inspired by Spotify”, pouring hundreds of hours of research into understanding it even better; incorporating perspectives and feedback from other current and former Spotify employees, as well as some of the experience from the hundreds of people who have attended the course over the last five years. He has also worked with several organizations that have used many of the Spotify Model essentials.
The Spotify Model Essentials fundamentally consists of organizational patterns, activities, and core beliefs.
The organizational patterns are the ones most people familiar with the Spotify model probably will recognize. Organizational structures consist of teams, such as Squad, Chapter, Guild, and Tribe; and roles, such as Chapter Lead, Agile Coach, and Product Manager. However, there are also some that you won’t find in most common Spotify Model material, for example POTLAC, the Tribe Lead Trio, and different types of Tribe Leads (Technology, Product, and Design).
For activities, a lot of practices used both at Spotify and in other organizations adopting the Spotify Model are already described in Essence as they are commonly used agile practices. For example retrospectives, team stand-up, iteration planning, product backlog, blameless post-mortems, and user stories. In our Spotify Model Essentials we chose to describe a few specific practices that we believe are useful for many contexts: Squad and Tribe Kick-Starts, Squad Health Check, and Tribe Gathering.
A really important part of the Spotify Model, that too often is overlooked, is the core beliefs. This is partly because people tend to focus on the more tangible structural elements and partly because what these beliefs are has not been as clearly communicated. Spotify Model Essentials is trying to change that by describing what we mean are the guiding principles for the entire model. For now, we have divided them into the categories of supporting the belief of “autonomy” and of supporting the belief “alignment” – both parts of the overarching belief in “Aligned Autonomy”.
“Empower and Trust”, “Developing products is about developing people”, and “Strong teams always beat rock stars”, are mainly about supporting empowered autonomous people and teams.
“We’re all in this together” and “Be autonomous, but don’t suboptimize” emphasize the need for alignment in service of the collective outcome.
These core beliefs are the real litmus test for whether you are just copying Spotify Model structures or if you are capturing the essence of the Spotify Model: deliberately experimenting with different patterns and practices to achieve aligned autonomy. If your implementation of the Chapter Lead role is not helping you develop people who are empowered and trusted, that might work for you, but it’s not the Spotify Model. If you call your component delivery teams “Squads” and your backlog administrator “Product Manager”, go for it, but it’s not the Spotify Model.
Benefits of Describing the Spotify Model Essentials
In addition to being an enabler, Essence provides specific benefits to the Spotify model. The model defines a foundation to design and implement organizational structures and the way of working. Particularly related to the latter, the Spotify model leverages many other practices like product management, DevOps, and daily stand-ups. However, the Spotify model doesn’t define how a team like a Squad can combine those practices in everyday’s work to compose a distinct way of working. The Spotify Model Essentials supports composition with other practices that are also described using Essence.
In addition to the definition of a common ground, Essence also defines the essential elements that are always prevalent in every software engineering endeavor (like the so-called Alphas). Those essential elements also enabled us to identify the essential building blocks of the Spotify Model by mapping the Spotify Model to the essential elements. Moreover, the mapping supported us putting the Spotify Model into perspective and describing its scope. This resulted in the fundamental structure of the Spotify model practice into core beliefs, organizational patterns, and activities.
Moreover, the Spotify Model is centered around the principle of autonomy and the organizational pattern of the Squad. A Squad is like a mini-startup that is autonomous and self-organizing. Particularly, this means that a Squad defines and evolves its own way of working. In practice each Squad could have its own way of working at any point in time. Essence supports this autonomy by providing hands-on guidance for Squads to define, compose and communicate their way of working. A Squad can for example leverage the games we’ve provided as part of the Spotify Essentials Gameboard based on Mural template.
Using the Spotify Model Essentials
There are many use cases for the Spotify Model Essentials. Altogether they help to understand, adopt, optimize, and improve the individual use of the model. Accompanying the release of the Spotify Model Essentials, we’ve made available additional materials for leveraging this new practice.
First, we’ve developed a classification that guides the adoption of the Spotify model.
This classification distinguishes between the elements which should be embraced when adopting the Spotify model and elements which could be considered to complement and support it, depending on context and needs. Particularly, the classification lists and groups other practices which are self-contained and typically used when the Spotify model is adopted.
Second, in collaboration with Simon Girvan we have released a virtual whiteboard based on Mural that demonstrates how the cards of the Spotify Essentials practice can be leveraged to play so-called serious games. These games do not have entertainment, enjoyment, or fun as their primary purpose. They are played in person and address business problems in creative, engaging and, most importantly, effective ways (see the book and website on gamestorming). The Spotify Essentials Gameboard contains five games that will help to understand, adopt, optimize, and improve the individual use of the new Spotify Model Essentials practice. Furthermore, Simon developed another game which we’d like to play openly with you: How well do you Spotify? We’d like to invite you to analyze the elements of the Spotify Model Essentials practice and assess them by using simple sticky notes. This gameboard is shared by all players and thus we’ll get an idea of how the Spotify Model is used in practice.
We Want Your Feedback And Contribution!
The Spotify Model Essentials and additional materials demystify and make the model more easily applicable. However, we still feel that there is a lot we don’t know about how people out there are using the Spotify Model, in actual practice, and what they believe is core to the model. That is why we want to release this early draft, even though it’s missing a lot of pieces of the full Spotify Model puzzle, because we are eager to involve a wider community of Spotify Model practitioners to help capture a shared understanding of what it is to all of you. We’d love to get your feedback, comments, ideas, and of course also criticism. We suggest leveraging the LinkedIn group Essence for Agility for following and participating in the discussion about Essence in general and also the Spotify Model Essentials practice.
– Joakim & Stefan
About the article authors
Joakim Sundén
Joakim Sundén is a consultant with Crisp, a small Swedish consultancy with world-renowned Agile experts who help develop sustainable organizations. From 2011 to 2017 he worked as an Agile Coach at Spotify where he was part of a group of people working together with the CTO to develop a new approach to Agile at scale, aka “The Spotify Model” with Tribes, Squads, Chapters and Guilds.
Stefan Malich
Stefan Malich empowers people and organizations to reduce the complexity and risk inherent in the development of software-intensive systems by improving the way they design, evaluate and implement software architectures. This encompasses the enhancement of architecture-related methods and processes, the adoption of technologies and application of innovative architectural patterns and designs.
Simon Girvan
Simon is a principal consultant and agile coach at Ivar Jacobson International where he has a particular focus on developing Essence products and practices and in helping organizations use Essence to help power up their development practices. He is an experienced agile coach and co-author of the book ‘Agile From First Principles’.
1 Comment
Comments are closed.
I have mostly seen organizations renaming existing teams with Spotify names ( squads, tribes, etc) They are not considering the rest of the elements you are presenting here. It is an “easy way” to say they are now an agile organization. And most of the time these organizations just do it in one area, not the whole company.