Reflections on an Agile BI Program
Agile is not a software feature — it’s a process.
by Chris Sorensen
Does this sound familiar: “I told IT what I wanted and all they ever do is hide for 6 months, never to be heard from and then ultimately they do not deliver what I wanted! Their process doesn’t work!”
Agile has certainly become the buzzword of the Business Intelligence industry over the past few months. With the craze comes abuse of the term and a slew of misconceptions about what it is, is not and what the benefits are. Suddenly, all vendor products enable Agile! [Editor's note: "Creating an Agile BI Environment -- Delivering Data at the Speed of Thought" is the theme of the 2010 TDWI World Conference in San Diego.]
Agile is not a feature in software. It is a mindset and a process. Like any process, it depends heavily for success on people and the organizational culture in which it is being implemented (see Being Agile versus Doing Agile).
Our business intelligence team has been working under the methodology since 2006 and I believe we still have not hit full stride. What does that mean for your program? Prepare for a long journey. It takes time, dedication, and the desire to improve to achieve the highest state of refinement and that is what we keep working on. In this article I present a collection of facts (in no particular order) that I often touch on when talking to others about our program.
As luck would have it, we were backed into using the methodology via an organizational change. In early 2006 we were moved under the direction of the software development team who had already been living and breathing agile for years. The fortunate part for us was that the culture, success stories, methodology, and expertise already existed. In other words, the conditions for success were in our favor out of the gate.
The roots of agile began in early 2000 with the agile manifesto, but its true roots go back even further in that it borrows from the core principles of lean manufacturing made famous by companies such as Toyota. Just in time, bottlenecks, continuous flow, total quality management, elimination of mudda (waste), and continuous improvements have their roots in the lean movement and these concepts have worked their way into agile.
Development of BI applications is often compared to a restaurant or manufacturing process — taking raw materials and transforming them into an asset that customers will demand and that can be sold at a profit. BI mirrors these concepts except with data. If we expect our programs to have a long, useful life, they should be proven to generate a positive ROI. The processes build something and the process used to do this can be analyzed and improved using lean techniques.
More than Just Speed
One of the biggest misconceptions about agile is that it is about getting more done faster. This is simply false. It is about delivering the right things of value, with a high degree of quality and in small iterations. The word “more” should be dropped. It is about avoiding waste or “mudda” when creating value. Have you ever delivered something that took a long time to build, only to have it never be used? Ask yourself why that was the case. This is the waste that we seek to avoid.
One of the biggest difficulties is to get the heads of business users and technical teams wrapped around thinking iteratively. Remember that delivering in small bits with communication built into the process is a foreign concept to many. Most people are equipped to deal with big bang and are unsure how to engage with a process that requires constant communication and participation. Others are simply afraid that once you deliver something you will never be seen again, so they ask for everything at requirements-gathering sessions.
