Agile Documentation

In a previous post - Waterfall or Agile? Neither, there was a discussion (see the comments) on various points. One of the more interesting points was documentation. I argued against what I thought was the attitude to documentation by the agile community. From my personal experience, people generally interpreted the principle of face-to-face-discussion is preferable to documentation as a license to abandon documentation altogether. Today I still experience this attitude from many professionals, who regard documentation with contempt and rely on agile theory to argue that documentation is not required (even bad) for software development. And that I could not accept. I have seen enough how crazy things are without documentation.

So, I decided to do some further googling to see what people say about documentation and about agile documentation in particular. And I had a pleasant surprise. I came across an excellent white paper by Scott Ambler, titled: Agile Documentation: Strategies for Agile Software Development. It was really a delight to read it. Finally, I saw a very well balanced analysis of the issue of documentation in software development. With very few exceptions, I liked and agreed with everything Scott writes.

In one of the comments in my post above, I said that my goal in the post was to move the agile pendulum a bit back from its extreme swing. From Scott's paper I can see I did not have to do that. The pendulum has already moved back from its extreme position. At least for those professionals who learn agile principles carefully and thoroughly. And now, I think, the pendulum is right where it should be. Not tough, rigid documentation regiment, nor no-documentation at all. Just Agile Documentation. I don't need to repeat hear the principles of agile documentation. Just read Scott's paper above. great read.

If what Scott writes represents the agile community position about proper SW documentation strategy than I am certainly more supportive of agile methods than I was a week ago. The only sad thing is that I find that too many people have not yet realized the correct position on agile on documentation and still think: Agile documentation = no documentation.

And final note to Dave Nicolette - thanks for your very good comment in the post above. I now have a better and more supportive perspective of Agile methodology. It is definitely maturing.

In my next post, I will describe some of my "techniques" for generating SW documentation and show that, at least to some degree, I have been doing agile documentation without realizing it.

----------------------------------------------------------------------------

Technorati Tags: Agile, Waterfall, Project Management, Project Manager, Software Engineering, Software Project Management

Innovation and Project Management - Mutually Exclusive?

One of the most common complaints we hear against project management is that it kills innovation. "Oh, here come those project cops. They will put rules on everything, force us to do all kinds of unimportant things. This is the end of innovation. This project is dead because the project manager won't not let us work freely so we can innovate. Because only innovation will bring the really great products".

Sounds familiar? So many PMs have to struggle with teams (mostly engineers) who don't believe in project management. Who see project management as a damaging force that will stop any real production and will eventually kill the project. Over time, everybody begins to accept this mindset. Where there is project management, there cannot be innovation. And, where there is innovation, project management will suffocate it.

Powerful argument. Not because it has any merit, because it doesn't. But still, it's powerful because everybody seems to accept it as some kind of tautology. Too often, executives are scared to bring project management into the company (or group) because they are afraid to kill this wonderful tender baby - innovation. PMs are also at a loss as to how to make projects and innovation live in piece together and so, they end up unintentionally fulfilling exactly the "curse" they want to avoid.

This whole concept of "project management kills innovation" comes from a misunderstanding about what innovation involves. Everybody knows intuitively what innovation is and what it is supposed to give us. The problem is that most people usually interpret "innovation" (often subconsciously) as "give us a free hand to do whatever we want, however we want, whenever we want. We are artists and therefore, must have full freedom from schedules, scope, planning, cost constraints, etc. But don't worry, if you give us this essential freedom, we will do great things for you. Not quite sure what, not sure when. Not even sure we will deliver. But trust us, if and when we are done, it will be great. Because it's innovation." So, move aside project management, here comes innovation.

Not so. I hope, everybody realizes that complete freedom does not work. Yes, if you remove all the annoying restrictions (schedule, scope, cost, etc.), the engineers, given enough time, will eventually do something very nice and innovation will have proven itself. What I mean is that in our industry, we usually do not have this luxury. The customer wants its product with the agreed scope, on time and within budget. To the executives and the customer, innovation is not the goal. At best, it's the means. The goal is the exchange of added value (deliverables for money), and on time. If the PM is not going to manage the project and try to ensure the team hits the above constraints, he/she will be considered to have failed and even the project itself may fail. A failed project means a great deal of money that was invested for nothing and non-materialized revenue.

Now, in order to prevent misunderstanding, let me make it very clear:

Innovation is good! Innovation is very important. Every company should have some.

The trick is, how do we have innovation and project management?

The points that I'm trying to make are:

• Innovation must not exclude project management (and vice verse).
• Innovation and project management can live together.

Yes, innovation and project management don't have to be mutually exclusive. The trick here is:

Innovation should not be prevented. It should be controlled.

Many people will think that this statement is an oxymoron. Controlled innovation? I say yes. You can do great innovation even with some control mechanisms. This is done all the time in R&D organizations. Even innovators should (and can) do some planning and execute to the plan. The difference is that with innovation, the plan should be less restrictive and more flexible. But there must be some plan and tracking of execution to the plan. So, say "No" to something, someway, sometime. It's OK to ask even innovators What? How? and when?

Let's look at two general cases:

• Innovation within a regular project.
• Innovation as a project.

Innovation Within a Regular Project

With the right planning, innovation can live within a project. Project Managers must be very careful not to completely block innovation within their project. Instead, they need to plan for some innovation and be prepared to accommodate it.

Do not allow ambiguity in the innovation work in the project. Instead:

• Innovation work must be officially identified, requested and approved. This is the most important part. Innovation must not be in "Skunk works mode". It should be presented, and justified. Not only why this is a good idea but also why now, in this project?
  
• Innovation work must be incorporated into the plan and tracked as a risk.
• Mechanisms should be in place to allow termination of the innovation work, if it exceeds its planned constraints.
• If a project no longer can accommodate the innovation activity, have in place plan to:
• Complete the project without the innovation part.
• Spin the innovation work off to a stand-alone effort or into another project.

Innovation as a Project

This, IMHO, is the best way to go. As I note above, innovation is very important for a company and any company should cultivate it. However, within a regular project, this is possible only on a limited basis. To really cultivate it, the company should recognize that innovation work should be given its own, separate space. When a company prepares its high-level strategy, it should include R&D project work. By including it at the strategic-level planning, it makes the commitment to allow and support innovation. This means dedicated R&D projects that go in parallel with the "regular" (revenue generating) projects. This means the company will put aside the resources and money to cultivate innovation. This way, innovation will have its own life to prosper and ultimately benefit the company.

Note that I don't say "innovation work". I say "innovation projects". This is not accidental. Even in its own "innovation track" innovation work still cannot be allowed to do something, sometime, somehow. Any work in a commercial enterprise, even innovation work, must be subject to (at least) the famous 3 constraints - scope, time, cost.

Innovation and project management are not mutually exclusive.
---------------------------------------------------------

Technorati Tags: Innovation, Project Management, Project Manager, Software Engineering, Software Project Management
---------------------------------------------------------

Waterfall or Agile? Neither

This is the debate of the century. Waterfall or agile? Everybody has an opinion, emotions run high on both sides of the line and those who don't have an opinion are confused. It looks like everyone must have an opinion to be respectable. OK, I want to be respectable too so here is my opinion.

The first and IMHO, most important point is that the debate makes it look like there are only two alternatives for the "right" software development methodology. If it's not waterfall, then it must be agile. If it's not agile, then it must be waterfall. I say "not so". There are more alternatives. For me it's not waterfall, nor agile. Or should I say "both"? The closest methodology to what I like (and try to do) is Iterative Development. Yes, many of you will say that iterative development is a fundamental component of agile development and they will be right. However, agile development also has several things that I don't like. So, I cannot honestly say that I support agile principles.

Let me say up-front that, in my mind, classical waterfall is completely out of the question. I think that this methodology is so defunct that there is no need for me to explain and justify this position. So, in regards the question "waterfall or agile?", one half of the answer is straight forward: surely not waterfall. But if not waterfall, then what? Well, for me it's not agile either.

Let's cover briefly the major principles of agile and see what I think about them. I will take the principles directly from the original Agile Manifesto:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

This I like. At the end of the day, customer satisfaction is the ultimate goal. I also like the important principle of useful software. With regard to rapid and continuous, I like that too, but remember that it takes two to tango. Meaning, the customer needs to agree and work with us to accept our deliverable and rapidly test and provide feedback. You might be surprised, not all customers will want that.

Which brings me to the an important point. Agile experts agree that full customer cooperation with the project team is essential for the success of the agile methodology. This means that the customer must:

• Thoroughly understand agile techniques.
• Agree to work with the project team by the agile principles.
• Do their part in the "deal".

The above requires significantly more involvement, commitment and resource allocation from the customer. So, not only you need all of the above from your own company, you need the customer to deliver too. Double challenge. My experience:

• Rarely will the customer deliver on the above requirements.
• Most companies that try to implement agile methodology fail to secure the customer's cooperation but still go ahead with the methodology. In this tango you dance alone and the results are accordingly.

Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

I have a problem with this. Change is usually unavoidable in software projects, but welcome? No, I don't think so. This is engineer talk. Project managers call it Scope Creep. I am not saying that change must be blocked "at any price". I am saying that change has to be controlled carefully and allowed only under the right circumstances. Both the project team and the customer (again the customer) have to answer the following questions:

• Is the change really necessary?
• Is it necessary now?
• What will be the benefit?
• What will be the cost?

If the answers show that the change must happen, then so be it. But welcome? Changes are a necessary evil. Sorry, I am the old fashioned type.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

I like that. It's a good technique and it works of you are disciplined enough. Agile is even more strict. It requires timeboxing - even better.

Business people and developers must work together daily throughout the project.

Absolutely. But I think this is true and critical in any methodology. Show me a methodology that will disagree. So, even though this is true, it's not unique to Agile.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

There are two parts here:

• Support the team members. Again, true but universal. Not unique to agile.
• Trust them. Well, where I come from, we had a saying (crude translation): "Respect him and suspect him". As an experienced project manager, I like that better. Not that I think that team members are evil or anything. Just that they are human beings. And that includes me - the project manager - too, of course.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

This one probably irks me most. Not because of what it says but because of what it does not say. What it does not say is: "we hate documentation". Yes, I know. We all hate documentation. More precisely, we hate to write the documents. I have never met a project team member that did not like (and actually appreciated) documentation if:

• It was good quality
• Other people wrote it

It's much easier and more fun to communicate verbally. But verbal communication has serious drawbacks. This is a significant issue that deserves its own post, so I will not go into detail. I will just note here: face-to-face conversation has no records (other then in people's minds) and, therefore, no memory. Again and again, experience shows us that there is no escaping documentation. The big trick is to cleverly decide what documentation, how much? how? etc. But there is no alternative to good documentation. Don't get me wrong, face-to-face conversation is a great and vital means of communication but, sorry, it cannot make documentation unnecessary.

Working software is the primary measure of progress.

I agree assuming the working software does what the customer wants.

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

OK, this is not really a guideline. It's more of a promise. I do not have enough statistics to make a determination if this has proven itself. I will be glad if someone can enlighten me on this.

Continuous attention to technical excellence and good design enhances agility.

Again, this is not unique to agile. All methodologies support this, even those that do not call it out specifically.

Simplicity--the art of maximizing the amount of work not done--is essential.

I like this. The KISS principle is a good principle.

The best architectures, requirements, and designs emerge from self-organizing teams.

Here again, I do not have enough insight to make a judgement. Let me know if you have any concrete info about this.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

This is what the PMBOK calls Lessons Learned and is basic project management. It's good and again not unique to agile techniques.

So this is a brief summary on how I feel about agile development. I will sometime write about my own technique for which there is no formal name but I will characterize it as a combination of iterative and flexible waterfall.

Let me know what you think.

-----------------------------------------------------------------

Technorati Tags: agile, waterfall, Project Management, Project Manager, Software Engineering, Software Project Management

-----------------------------------------------------------------

Vendor Management - a Project within a Project

In previous posts (Cost Management is not Optional and Risk Management is not Hard - do it), I covered two very important project management Knowledge Areas (PMI terminology) which are not practiced enough in the software industry. I noted that they can be practiced relatively easily (at least at the basic level). Vendor management, on the other hand, is the opposite. It is something most SW projects have to deal with and it's not easy. The reason: if you look at it closely, it actually comprises a whole project unto itself. A project within a project. You need to plan and execute a special version of all Knowledge Areas for the vendor management, separately (yet interdependently) from your own project. And then some.

• The vendor represents its own stakeholders that require stakeholder management.
• You need to develop the vendor schedule..
• You need to define the scope of the vendor work.
• You need to work out the financial part.
• And you need to plan and execute all the other areas: Human Resources, Quality, Risk, Communication, etc.

And, as I note above, there is more that is unique to the vendor management:

• The vendor's interests are usually contradictory to yours. You want maximum scope with minimum cost. The vendor wants minimum scope with maximum pay (that's cost for you). This forces you to
• Negotiate.
• Be extra vigilant.

• There is the whole legal part. There must be a contract between your company and the vendor. Although putting the contract together is not your job, you, as the project manager, have a responsibility to ensure it happens. Don't dare to work without one. it may work out a couple of times but then there will be that one time when things go wrong, there is no contract in place and you'll regret it for the rest of your life. Besides, agreeing to work without a contract demonstrates that you are not professional.

All of the above introduce a full range of activities of a typical project into your project. As I said: a project within a project. Make sure to manage your vendor as a project.
---------------------------------------------------------------------------------

Technorati Tags: Vendor management, Procurement Management, Project Management, Project Manager, Software Engineering, Software Project Management
---------------------------------------------------------------------------------