In this guest blog post Alan O’Callaghan explains how governance when applied traditionally can severely damage an agile transformation and negatively impact business results. His conclusion: we need a new model of governance for agile.
Governance seems to be one of those frightening words that threatens to stop an Agile transformation effort dead in its tracks. I’ve been hearing it whispered, and even screamed once or twice, quite a lot recently. There’s no big surprise here. As the big corporations and Government agencies get increasingly fascinated by frameworks like Scrum, they are mandating their IT departments to “Go agile” and then, sooner or later…governance!
Types of Governance
There is operational governance represented in the defined processes that organizations and teams are expected to follow when software is in production. There is project management governance, perhaps dictated by PRINCE2 or similar, while the product is in development. PRINCE, by the way, is an acronym standing for ‘PRojects In a Controlled Environment’.
Project management governance is often a subset of a wider IT governance. According to the TOGAF version 9.1, IT governance supposedly provides the framework and structure that links IT resources and information to enterprise goals and strategies. “IT governance institutionalizes best practices for planning, acquiring, implementing and monitoring IT performance…” Standards like COBIT, which stands for Control OBjectives for Information and related Technology, might be in place. And then, of course, there is a range of issues to do with compliance to the requirements of external regulators in all public bodies, as well as commercial institutions in sectors such as insurance and banking. So, is this a case of an irresistible force (Agile) meeting an unmovable object (governance)?
What is Governance?
Let’s go back to first principles. What is governance? Here’s a definition I came across recently in the TOGAF 9.1: “The discipline of monitoring, managing and steering a business to deliver the business outcomes required.” As an Agilist, I have no problem with governance described in this way. An obstinate focus on the delivery of business outcomes is the very stuff of Agile. It is the way governance is applied that is the problem. Typically specialist governance bodies are set up in a hierarchy at each level of which procedures are mandated, documents are required for sign-off, and auditing procedures abound. The Agile principle of trusting professionals to get the job done is about as welcome as a trap door in a canoe. Fear of non-compliance drives very different behaviors from those we are trying to grow with Agile.
When these kinds of bodies and procedures get imposed on software development teams there is one guaranteed result: a delay in the delivery of value to the customer (often a protracted delay at that). Phase-gates block the development path. Waiting for sign-offs builds queues of work items. Sometimes the whole development ‘track’ is idle, like a train sitting at a signal waiting for it to turn green.
Software Development is Designful
The scenarios I have just described are in no way conducive to the achievement of business outcomes. In fact, very often, it is the governance procedures which are the major obstacle to their delivery? Why is this?
Business value is much more likely to be delivered in the Agile Model because of its fast feedback cycles. These are necessary because software development is inherently unpredictable. Its practical processes are more like what goes on in the design rooms of product development than it is like the assembly lines of mass manufacture.
“Design is not passive. It is wise for designers to harmonize with the ways of nature. But the ways of nature follow context and change.”(1) Mass manufacture is predictable. The uncertainties have been ironed out and removed in the design “phase”. But software development is not.
While not everything in software development is design (problem analysis should shape design, after all), its ‘implementation’ is creative. Even when we are crafting code we are only designing the instructions that the computer will run. Design is dominated by uncertainty. New information emerges during the process itself and is incorporated as quickly as possible. Feedback cycles surface that information, allowing the development team to converge on a solution that delivers business value to the customer.
A New Model of Governance?
Traditional governance procedures follow the motto “Plan the work; work the plan”. They assume predictability. There are many corporate procedures where governance for predictability is appropriate. Software development is not one of them. Governance for feedback; governance for responsiveness is what is required. In many Agile-adopting organizations, a “two-speed” solution is evident. Where predictability reigns, traditional governance procedures are maintained. Where feedback drives success, different approaches operate.
In Scrum, the Product Owner is responsible for achieving the business outcomes inherent in the product development. She owns the product for the business, and is accountable for, amongst other things, its alignment with the strategic goals of the business. She is also a peer member of the Scrum team. One of the reasons you rarely see formal, outline business cases, followed by detailed business cases and post-project audits against them in Scrum is because it is unnecessary. The Product Owner role and the Sprints’ inspect-and-adapt cycle takes care of that stuff in a more ‘light touch’ way.
Similarly, the quality of the product is best ensured by embedding testers as developers in the Scrum team. The goal of testing then itself becomes feedback. The ‘Whack-The-Mole’ pattern means defects can be removed by the Development team before the increment gets to the Sprint Review, let alone Release. The ping-pong between programmers and testers that so characterizes (and delays) waterfall development is eliminated by making the Scrum team responsible for quality.
While any overall solution to the governance issue will be situational, and may yet require some external oversight, the general line of approach seems obvious to me. Business stewardship and quality assurance are, in my opinion, stronger in Scrum than they are in waterfall precisely because the specialist skills needed have been brought into the Scrum team, and the team is accountable for them. The Scrum Development team is supposedly fully cross-functional: it should contain all the skills necessary to deliver the product. If that requires specialists in governance, then include them in the team.
Let the team figure out, through conversation and collaboration with whoever it needs – external regulators included – the best way to ensure the proper delivery of business outcomes.
(1) J.O. Coplien and L.Zhao Towards a General Formal Foundation of Design: Symmetry and Broken Symmetry. Monograph
This blog post is republished from the Learning Tree blog. The author Alan O’Callaghan is principal product owner at Emerald Hill Limited as well as the Instructor and Course Author for some of Learning Tree’s most popular Agile training courses.
The looks of Agile Governance
Thank you Alan for raising the topic of governance and explaining how the way that it’s applied often conflicts with agile. I fully agree with you that we need a new model of governance for agile. Actually, the new model that you describe might also be useful for organizations not planning to go for agile. When organizations implement effective feedback loops, they usually become more effective and efficient, and that also increases predictability.
One way to establish governance through feedback in organizations is to do retrospectives beyond the team level. Agile retrospectives play an important role when it comes to getting value out of agile. Only when teams and their surrounding organization truly dare to reflect and learn from what is happening, and take action based on that, then improvement is possible.
A practice that I highly recommend when it comes to governance is visualization. Making things visible, things that people don’t see or might have different views on, makes it possible for people to discuss them, share their views, and align their thinking. It also improves collaboration: Once things are visible it becomes a lot easier to work together on something.
How does your organization apply governance? Does that hurt or help your agile transformation?
The other day I came across a job description for Scrum Master role that included familiarity with PRINCE and responsibility of “documentation as per PRINCE”.
IMO, this is just wrong. There is nothing wrong with documentation as per PRINCE or anything other governance model but deciding this must be done by the Scrum Master beforehand is a sign of the organization following top-down model of management. The SM may end up doing this, but the how, when and who should be determined by the team collectively.
It would be great to get your opinion on this.
Isn’t it interesting how often job ads for ScrumMasters require knowledge or skills that the SM doesn’t need at all – but the team does. I agree with you Mohammed. The SM should be coaching the Scrum team to make their own decisions about documentation, not doing it for them
Visualising the work is a great way of establishing some form of governance. Kanban board is the essence of this.
But when talking about governance, you need to give it a purpose so you know what you are governing towards. If governance is still to deliver a project on time / on scope, you are likely to drop into the same trap.
So you need to establish what other goal / what next goal should we have and start making yourself accountable to this. Delivering better quality, delivering more often, avoiding wasted effort, reducing tech debt, they are all great goals to start with that will provide an additional consideration for governance that will progress in the right direction with things that should matter to the business.
Too often agilists are quite dogmatic and forget to connect the effort of agility to the enterprise goals.
A great point. If the purpose of governance is business alignment, then everyone involved needs clarity on the business goals
I think this is a great topic to explore, but I found the post to be 90% about describing the problem and maybe 10% of uncovering how to change the traditional governance model to be more aligned with agile development. I understand the role of Product Owner, but it would be helpful to read about some examples of where this was put into practice in a large organization that already had a mature IT governance model and how they adapted it to support agile. Any suggested reading here?
Good article and quite appropriate for heavily governed entities such as banks, Financial Institutions, etc.
From personal experience at a bank, I thought about governance for those who are ‘Scaling Scrum at Large’, across multiple teams, programs, or portfolios. We found that injecting governance (auditing, infrastructure, security, etc.) at the beginning, at each synchronized program increment (we utilized a modified SAFe approach), and eventually just before release, eventually worked well after training those personnel. It slowed the flow of work down initially, but the true issue in our initial release was that we failed to consider external governmental auditors, who had little to no Agile/Scrum Training and hence caused us to fail their auditing. We didn’t consider ‘seeing the whole’ process, from end to end, which included them.
If you work in a similar regulated environment, it would be wise to consider their point of view in your work and try to embrace their feedback, as much as possible (early and often, if they can).
Thanks, Brian, for this example. My intention was to open the door to discussion of experiences, and you can probably see from Antonio’s comment there is a thirst for that. There isn’t going to be a “one size fits all” solution for everybody, but the more information is made available about how these challenges have been met in practice, the better
Hey, Great Information.
A board governance software is a solution that these private sector companies are reaching out for in order to fulfil their board governance compliance requirements. In the wake of the constantly increasing number of board meetings, annual shareholders meeting, special committee or sub-committee meetings and other meetings, there is increased pressure on the top officials of the company. To know more about the board governance software read here : http://www.axardigital.com/