In August of 2012, I began a series of blog postings about the experience that The DTCC Learning Group had while implementing Agile as a development methodology for training programs (click here for a link to the first post). For those who are not familiar with the term, Agile is a software development technique that supports frequent releases of product features and functionality to customers. Agile accomplishes this by reducing the amount of administrative overhead typically associated with product development, focusing on human interactions more that on tools and processes,  delivering working software as opposed to product documentation, collaborating with customers as opposed to relying on legal documentation, and responding to change as opposed to being restricted by a plan.

At DTCC, we felt that the changing nature of customer demands and expectations that were causing businesses to increase the speed to market and, quickly respond to changing customer requirements was also applicable to the training organizations that supported the business.  Thus, like many software developers, we abandoned the waterfall approach to (learning) product development and adopted Agile.

The initial results of results of the Agile conversion were promising. In the first two months (post Agile implementation) we saw a 75% increase in the number of learning assets that the teams were able to deliver to customers on a monthly basis.  In addition the internal customers that the team supported seemed to feel that the learning organization was more responsive to their needs.  One business partner stated the following: “My experience with dealing with your team over the past month has been extremely positive….your team was extremely helpful with their professionalism, quick turnaround on developing the SIMS and the overall quality of work.” Training employees also appeared to be more engaged.  Several members of the training team stated that they really felt empowered by the process.

Monthly Comparison

While we were quite happy with these results, we wanted to ensure that the team was not experiencing the Hawthorne effect.  After operating in the Agile mode for over 9 months, we looked at the data again, this time comparing the organizational performance from Q1 2012 when we were using the waterfall approach to Q1 2013 when we were operating totally in Agile.  The results were astounding.

2012 vs 2013 deliverable counts

The number of learning assets delivered to customers in Q1 2013 as compared to the same period of the previous year increase by 91 deliverables.

We realized that some learning assets were naturally larger than others and as a result require more effort.  We wanted to ensure that we were looking at an apples to apples comparison, so we normalized the deliverable data by looking at what we called work units (Agile would call these story points). We identified the smallest learning asset that we could possibly deliver as a single work unit, and sized the larger assets appropriately.  An article was identified as requiring a single work unit of effort while developing a webinar might be 50 work units of effort and a product simulation 30.

When we compared the number of work units delivered from year to year, the results were even more stark.

2012 vs 2013 work units

The work units of learning assets delivered increased almost ten times, from 394 work units of learning assets delivered in Q1 2012 to 3,604 during Q1 2013.  This meant that we not only delivered more learning assets, but we were also able to deliver larger learning assets.

Perhaps the most telling comparison of year over year performance came when we looked at the types of learning assets that the team was able to deliver.

2013 deliverables breakdown | 2012 deliverables breakdown

In Q1 2012, the teams almost exclusively delivered articles or procedural documentation as learning assets to customers.  During the same period in 2013 the teams delivered a much more diverse set of learning solutions including webinars, webcasts, product simulations, and instructor led training.

The quantitative results from the year to year comparisons certainly indicate that the Agile approach for learning product cannot just work, but can work well.

I’ll be discussing our experience with Agile and learning product development at Training Industry, Inc.’s Partnering for Performance conference in Raleigh NC, this week. I invite anyone who is interested to attend the conference.