Introduction
This is the first article in the Agile Thinking series. The series has been inspired by real life experiences, working with teams who are trying to work in challenging environments that for many reasons are not friendly to Agile adoption. Still the teams battle on trying to change and do as much good as they can within the environmental and contractual limitations.
We believe it is a myth to say that Agile cannot be applied in all software development environments, sometimes it just takes a little work and some negotiation and compromise. This range of articles aims to make no judgments but to present experiences and techniques for working in these Wild West style environments.
We give an example of what one of these environments might indeed look and feel like in this article written some months ago: http://www.cloud-coder.co.uk/blog/index.php/2012/11/agile-or-wagile-discussion/
We hope you find this series of articles challenging and engaging if a little non-conformist.
Article Approach
We believe the best approach we can take in this article is to look at a scenario. We will aim to present situations we have encountered in the past and generalize the context to cover wider audiences. While there is no magic bullet for any of these situations we will attempt to present some approaches to each, which hopefully will become talking points for teams in similar situations.
While the scenarios here are based on our experiences they do not reflect any particular engagement.
Scenario [Pressured Redevelopment]
Context
The company and product
The company has a number of products that are unbranded and considered white label. The product is then sold to 3rd parties as part of licensing / SAAS style arrangement. The 3rd parties then use the products for direct sale to the general public via a number of interfaces:
· Electronic Point of Sale [EPOS] – vb6 application
· Web [Ecommerce] – .net framework V1.1 web forms
· API [3rd party systems connect] .Object C
The company had offices internationally in Australia, USA and Europe. Customization for a given client was performed by the local office. Development of the core products was centralized in the UK, utilizing a development team of 30.
Deployment, administration and support was performed by the local office and local technical teams as well as design agencies working on the clients behalf.
The product code base had been patched and iterated on top of with no concern for maintainability. Basic patterns and practices such as the SOLID set and any form of Test Driven or post code Test development had not been implemented. Reuse was not a priority and code duplication had made the code pathways chaotic and difficult and in some cases impossible to trace execution. No form of documentation had been kept and the code was littered with outdated and confusing comments. Source control was undisciplined and most of the product set couldn’t be built from source without post compilation hacking. The POS products used controls which were no longer in production and were unsupported.
The development team had taken the decision 18 months earlier to redevelop the whole product suite from the ground up in .net 2.0 using Winforms and a Webforms. They decided to reverse engineer the product suite into a set of requirement documents, prior to having the full set of requirements the business were told a number of statements.
· The business would have new maintainable products which had the same feature set as the current product set, plus a delta from the current set to the full backlog that had mounted up over a several years.
· The development could be done with the current head count.
· There would not be any releases of any of the products for 18 months, while the work was undertaken.
Katie’s Note – Although this situation might seem extreme, it is in fact common place in older product teams. It is normally motivated by a need to replace a product set which is still evolving and generally failing to perform under load; we quite often see huge support footprints and budgets associated with this type of product.
The business, the support team, the sales team and the development team are all in agreement that it must be replaced and in a tight time line. The combined product team is generally under huge commercial pressure with no willingness to undergo a staged release cycle. A classic statement you might hear from business in this sort of situation is “Just make it do what it does now, and all of the X feature set as well. We need it by the end of the year, you can’t fail”, whenever we hear statements like this we push back for a full discovery process prior to initial development of the first iteration and we make sure expectations are set and written down between business, sales, deployment and the development team.
It should also be noted that in situations like this it is common for either an ineffective Quality Assurance team to be present or for no QA team at all to be employed and for developers to be performing their own testing. We also see practices such as hot hacking on live sites and ad hock patching of live servers.
It is important to remember that all elements of the invested product team will be under severe pressure and have generally been driven to this point by desperations and rational thinking or reasoning might not be a governing factor in the decisions they make or the conversations being held internally.
What happened over the next 2 years?
The business green lighted the redevelopment, the development team ploughed forward diving into code with the CTO directing by pair programing with the developers. No plan or documentation was produced, the Project Manager continued to gather requirements and fed them into the JIRA back log.
The business and the sales team, with no guidance as to what the new product would contain, sold every feature they could dream of when speaking to potential client’s. Clients were on boarded on to the old system with promises of the new super system within 2 years.
Factors that came into play most of which were not considered at the project inception:
· The products turned out to be very hard to deconstruct and the code was very unreadable and much of the product had not been worked on for a number of years.
· The code modules had no separation of concerns very little code could be isolated or replaced in situ.
· The maintenance of the existing product, plus an increasing user base triggered by the new sales activity and promises being made to new clients, ate 70% of the development team’s capacity with no increase in head count or resetting of expectations.
· At some point during the life of the product set, someone had decided to drop all the foreign / primary key relationships with in the huge database that supported all the products. They had also randomized the table names to obfuscate them. The net effect of this was that the stored procedure which contained a great deal of the business logic were the only real statement of data truth, this caused months to be spent trying to map out a data schema and understand the data interactions.
· The UI, Data and Business logic code was spread and intermixed across layers, this prevented opportunities for replacing code horizontally layer by layer.
· The development team did not have the knowledge or skill bank to move to the technology set purposed by the CTO. As such the work which was undertaken was slow and the end result was still of the same quality stamp as the code it was supposed to replace.
The end result of this combination of factors was that the project got behind schedule. Initially this was not made visible to the business who continued to set expectations with clients, thus the pressure on the development team increased for early delivery. Eventually it became obvious to all parties that the project was nowhere near complete. The impact of non-delivery was to send shock waves through the organization, staff members left taking embedded knowledge with them and the sales team increased the pressure on the business to deliver the new features even if on the old code base.
The CTO resigned, this caused a focus point for the business and they decided to take action.
Katie was called in
Day 1 {Business}
He spent day 1 understanding the business position; resources available, time frames and working with sales to try to produce a holding statement for the client base to try to remove some of the immediate pressure. This included communication coaching and assistance with public statements.
Day 2 {Moral, Development Team , Team Building}
Working with the development team to understand the moral and other issues in the team. Producing a management report, including action points for increasing team moral. Prepared a brief for the CEO to deliver to the development team which was designed to base line the situation and reduce the heat in the interdepartmental relationships.
Day 3 {External Interfaces, Expectations}
Communicated with the international teams and issued reassurances. Sketched out the high level steps of an action plan to perform a delivery within the next quarter.
Katie’s Note – Quite often the first job in a failing project is to create a holding position and to reopen communication channels. This generally involves telling people things they do not want to hear. However, as long as you have a plan to move the situation forward and fresh credibility, it is perfectly possible to have sensible conversations even with upset people.
It is important to create space within which the team can breathe and the project can be given a new shape, this generally involves easing the pressure across multiple verticals and reopening departmental communication channels.
Day 4 {Technical, High level}
Full Architectural review of the existing systems, the end product of this was a power point presentation outlining the main problem points within the current architecture and an architectural map document.
Day 5 {Business}
Positioning conversations with the business, presenting the dynamics of the current situation and talking about potential ways forward. The end result of this conversation was to abandon the rebuild project and develop features on the current code base. Where possible it was decided to use in place refactoring to replace some of the existing code base. It was also agreed that a visible project management approach was needed and that the business would need to see some fast wins. With all of the above in mind Katie pitched an agile approach to the development team, after some deep conversations it was decided that a hybrid agile approach would be used and that the existing Project Manager would work with Katie to produce story cards for the new requirements.
Day 6 and 7 {Planning}
The next 2 days were spent isolating features from the backlog and removing duplicates the end product of the exercise was the production of a word document with a title to represent each feature detailed as follows:
|
Feature Title |
|
|
Feature Description |
|
|
Feature Relative Size |
|
|
Risk |
|
|
Dependencies |
There was a set of physical cards created using this template to describe the proposed features.
Day 8 {Business, Planning}
Katie organized a group of stake holders representing each areas of the business and the interrelated departments. The stake holders over a number of meetings facilitated by Katie and led by the Project Manager were encouraged to play a high level planning game.
Katie’s Note- To reach day 7 involved a lot of empowerment of the existing team. It is important in these situations to try to restore the rightful place of the team elements, and to support and facilitate the existing team back into a position where they can communicate between each other, and with the business elements. One of the tools to achieve this is to hold an initial retrospective so the past can be discussed, actions identified and then the team can move on.
What is a Retrospective – http://www.amazon.co.uk/Agile-Retrospectives-Making-Pragmatic-Programmers/dp/0977616649
As a consultant my job is not just to get the project back on track, but to leave in place a competent and empowered team. It’s very important the team depends on itself and does not become dependent on the consultant. The consultant role is rather as an ‘agent of change’ and should always stand slightly apart from the production team and keep objectivity.
High Level Planning Game
Katie and the Project Manager distributed a word version of the features to each stake holder along with donuts and plenty of coffee, the meeting was introduced and the objected set, the meeting was time boxed to 1hr. The master set of feature cards were then spread out on the table each card was then spoken about in brief detail and any questions answered, the key question was then asked “Does anyone want to champion this feature?” if no one was prepared to it was torn up and thrown in the bin, if someone was prepared to champion the card then it was given to the stake holder.
After the initial sort of the feature cards had been performed, the group were left with 30 cards, Katie then told the group they needed to work out between themselves in what order the features were played, Katie and the Project manager then left the room setting a time limit of 30 minutes for the group to discuss and decide the order of work.
Katie and the Project Manager returned to the room to find an ordered deck and a happy set of steak holders who main question was then when will we see the first featured completed. Katie promised to break down the first 2 features and run an iteration planning meeting, in the next 2 days and feed back with a progress report. The current plan is for the first iteration to start by the end of the week, everyone broke up in good spirits.
Katie’s Note- It’s possible that a high level planning game doesn’t fit all situations , but what is important here is getting the business stake holders involved in the planning and making decisions about prioritization, each from their own perspective. The point of note here is do not give deadlines for feedback you think you might break, its important if you say I will come back to you with XYZ on this date , that you keep your promises.
First Eight Day’s Recap
What has been achieved in the first seven business days?
· Architectural assessment
· Alignment of the business view as to the real state of the project
· Communication channels opened up between the different areas of the business
· Air cleared retrospective held
· First set of requirements broken out into actionable items and prioritization responsibility distributed to a group of stake holders.
· Project Manager empowered
This is pretty typical of the first week of an engagement of this type, so far this engagement is following a very healthy pattern. The next stage is to plan an Agile cycle with the team and implement correct tooling and support for the team. Once this support is in place we can then look at Patterns and Practice’s and the technical elements of coding inside an older code base.
Katie’s Note- It’s important at this stage not to overload a team but also to establish and keep a momentum of change happening, the worst thing that can happen is you hit a blocker and stagnate. For this reason do not use linear planning; always have a backup plan that you can switch the team to. As an Agent for Change in these initial phases it’s very easy to become personally over loaded; I find List’s help. It is also important to take down time. For instance take your lunch hour go for a walk, do fun stuff after work and under no circumstances should you take your work home with you. It’s important to remember the team is dependent on you, it is key that you do not get to a point where you cannot process or think objectively. On a side note it also helps to have a support system in your life. It doesn’t matter if this is a group of close friends or a loving partner or perhaps an understanding cat, being an Agent for change can be a lonely and challenging place at times.
One final note of personal Qualities , you cannot afford in this initial period to have a personal bad day. It is better you stay at home than appear to the team that you do not have everything 100% under control. It’s very important you walk with 100% confidence and appear at all times to be filled with energy and enthusiasm. Remember the team is likely to be in a beaten up state from what has happened in the past they need to look to you and know that everything is sparkling bright and there are no hurdles they cannot now get past.
Individual Maintenance – It is important to find time for each member of the team, take them to lunch find out how they feel it is going. Give them an opportunity to express their fears and thoughts in a free environment. Take actions from these meetings and personally fight their corner where needed to try to provide the working environment they need to function as part of the team.
Finally don’t forget the donuts! Celebrate every small success as a team, do provide team lunches, donuts, team drinks etc. at every excuse. Team cohesion is the most important goal to continued success it is important the team feels like an extended family, one unit.
Day 9 {Team, Agile, Planning}
The day starts with the first team stand up meeting. We follow a very simple format members speak about:
· What they worked on yesterday
· What they will be working on today
· Any blockers
After the stand-up meeting Katie and the Project Manager did a full team huddle and fed back on the planning meeting, discussed in brief the features that had been selected as the first 3 to be attacked and announced their plan to hold a Team Iteration Planning Meeting the following day.
Katie then worked with the lead developer the project manager to break out stories from the features selected in the High Level Planning Meeting.
Katie’s Note- Normally this would be the job of the Business Analyst , but quite often you find in these situations that one is not employed although this is a role experience tells me you should push to be filled ASAP.
What is a Story?
A story is a breakdown of a feature into more granular requirements, the language used to define the story varies from team to team as does the costed complexity; to put this another way the size of the story divisions within the feature. The following recipe is based on what I have found most effective when working with teams but is by no means exclusive of other methods, there are after all many ways to cook a fine chili.
Katie wrote up the first feature on the white board. Katie then started interviewing the team, driving them to talk about the key parts of the feature. He grouped them on the white board in to word groups. He then picked on the first common pattern of words and started to write stories around the word groups in the following format.
As a ….
Given that ……
When I…..
Then I Expect….
He used “And” where it made sense for concatenation and where stories became complex he wrote a header story and broke out scenario statements below.
|
As a Registered UserGiven that I am not logged in And I am at the home page When I browse to the product page Then I expect to be challenged for my credentials Scenario [Valid Credentials entered] ….. Scenario [Invalid Credentials entered] ….. Scenario [No credentials entered] ….. |
When the group had finished breaking apart the stories the Lead developer guessed at the size of each, where a story was over 3 days it was split into smaller stories.
· If the story contained scenarios it was relatively straight forward to extract the scenarios in to separate stories
· If the story contained highly technical elements , the team were able to extract a number of technical stories
· If the story was simply too large and had to many moving elements this was more difficult to split up, but the 3 day rule was stuck to.
The stories were then put in priority order and any interdependencies noted on the story , when the team was happy with the stories the Katie and the Project Manager then wrote out the stories onto A5 index cards and entered them into Jira as back log items, noting the Jira number on the physical card.
The planning team was now ready for their first iteration planning meeting with the larger team.
Katie’s Note- I have found it to be very important to perform the initial sizing exercise with a developer element present. This really does help you to get the size of the stories correct, it is also important to set a limit in terms of story size and to stick to that limit no matter what other pressures there might be. It is worth remembering that a larger a story is the more Risk is carried in the story. If it is too big you will end up with the developer/s who pick up the story feeling like they have to climb a mountain. It is important however to remember this is guide sizing only and when you enter the planning meeting the team may resize the stories again (hopefully this doesn’t happen too often). If this does happen then the Planning team need to be flexible and be prepared to either withdraw a story from this iteration or to split it on the fly.
In a lot of cases we used Jira as this was is the in place requirement management system , however Jira in my opinion struggles to model an agile process. This can be improved by bolting on Green Hopper [
http://www.atlassian.com/software/greenhopper ] to your Jira installation. However this is still a compromise compared to more Agile requirement management systems such as Mingle from Thought Works [ http://www.thoughtworks-studios.com/mingle-agile-project-management ]. An additional note on Team Foundation Server from Microsoft, we have found that the default templates actually introduce complexity to a team’s simple Agile cycle and planning. It’s important to remember that a tool should work the way you want to work. You should not have your teams working habits dictated by a tool. In short if you are to adopt TFS please spend time and budget making it fit your process. It is my opinion that thoughts works Mingle is the ultimate in flexible requirement management solutions, but it does require time and effort to fit it to your process. Unlike TFS it does not dictate rigid process upon the team, but due to its flexibility it is not usable straight out of the box so to speak, excluding for the most basic of projects. Normally I would advise a team to budget 2-4 days to configure a Mingle instance pared with Thoughtworks pre and post-sales support, which has always been excellent.
Day 10 {Technical, Patterns and Practises}
The Project Manager is taking a break to go over the requirements and make sure he understands them in depth, ready for tomorrow’s requirement management meeting. Katie is now turning his mind to the technical challenges of implementing new functionality inside VB6 while consuming best practice. Katie has prepared a couple of brown bag sessions for the team he will present today, both will include a live coed example.
Sessions
· T/BDD covered in this post – http://www.cloud-coder.co.uk/blog/index.php/2012/11/step-by-step-guides-getting-started-with-white-box-behaviour-driven-development/
· Making C# classes COM visible and consuming them from within VB6
http://juststuffreally.blogspot.co.uk/2008/02/steps-to-make-your-net-dll-useable-from.html
http://blogs.artinsoft.net/mrojas/archive/2010/06/23/exposing-c-classes-thru-interop.aspx
Katie then showed how both techniques could be used to produce test driven assemblies in C#, which could be used to replace huge swathes of functionality in the existing VB6 application, whilst keeping the forms and interface consistent with the user’s existing expectations. He explained that this form of black box refactoring is highly effective for replacing existing applications from the inside out, when the application must be kept in a deployable state.
Katie drew the following diagram on the white board to illustrate.
He then explained that they could use the new functionality required to replace the sections of the existing logic with C# test driven functionality. They could then make them COM visible and consume from them the existing VB6 form’s and take the opportunity to migrate the business logic that exists in the current forms plus the new logic changes to the test driven C# assembly’s. He also explained and showed an example of using C# and XAML Controls to provide new views which could be wrapped in the existing VB6 application.
The development team spent the rest of the day learning Pairing skills and how to write code from a Test First Perspective using the following sample exercise [https://www.box.com/shared/3hbnko03d7 ] Katie tripled with each pair helping to teach technique and approach.
Katie’s Note- It is really important to teach solid foundations. A good place to start is with TDD but over time you really want to promote SOLID principals as a minimum [http://www.blackwasp.co.uk/SOLIDPrinciples.aspx ].
Day 11
The iteration planning meeting started with the project manager presenting each story in sequence , he also presented Balsamiq Mocks [ http://www.balsamiq.com/ ]images where they added clarity, and other artifacts such as excel spread sheets where this added important information or context.
The team then voted on the size of the requirements the numbers which were recorded on to the card:
· Quality Assurance Size [ How long it will take to test the requirement ]
· Development Size [How long it is estimated to develop ]
Katie had agreed an initial maximum size that they would play in their first 2 week iteration with the team. When this was reached the Project Manager, also acting as the customer representative in this case, had to decide which story’s to push to later iterations.
Final Comments
This article has attempted to show how within in 11 days you can turn a project around. Obviously this is not the end of the story so to speak, but the rest of the engagement is covered elsewhere under the many good posts on Agility and how to run an agile process. What we have tried to show in this article is how to get a failed project back into a state were the Agile life cycle can be applied. We hope you have found this article of use and engaging.
Willow @Technical Thoughts 