Waterfall or Agile? Neither

This is the debate of the century. Waterfall or agile? Everybody has an opinion, emotions run high on both sides of the line and those who don't have an opinion are confused. It looks like everyone must have an opinion to be respectable. OK, I want to be respectable too so here is my opinion.

The first and IMHO, most important point is that the debate makes it look like there are only two alternatives for the "right" software development methodology. If it's not waterfall, then it must be agile. If it's not agile, then it must be waterfall. I say "not so". There are more alternatives. For me it's not waterfall, nor agile. Or should I say "both"? The closest methodology to what I like (and try to do) is Iterative Development. Yes, many of you will say that iterative development is a fundamental component of agile development and they will be right. However, agile development also has several things that I don't like. So, I cannot honestly say that I support agile principles.

Let me say up-front that, in my mind, classical waterfall is completely out of the question. I think that this methodology is so defunct that there is no need for me to explain and justify this position. So, in regards the question "waterfall or agile?", one half of the answer is straight forward: surely not waterfall. But if not waterfall, then what? Well, for me it's not agile either.

Let's cover briefly the major principles of agile and see what I think about them. I will take the principles directly from the original Agile Manifesto:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

This I like. At the end of the day, customer satisfaction is the ultimate goal. I also like the important principle of useful software. With regard to rapid and continuous, I like that too, but remember that it takes two to tango. Meaning, the customer needs to agree and work with us to accept our deliverable and rapidly test and provide feedback. You might be surprised, not all customers will want that.

Which brings me to the an important point. Agile experts agree that full customer cooperation with the project team is essential for the success of the agile methodology. This means that the customer must:

• Thoroughly understand agile techniques.
• Agree to work with the project team by the agile principles.
• Do their part in the "deal".

The above requires significantly more involvement, commitment and resource allocation from the customer. So, not only you need all of the above from your own company, you need the customer to deliver too. Double challenge. My experience:

• Rarely will the customer deliver on the above requirements.
• Most companies that try to implement agile methodology fail to secure the customer's cooperation but still go ahead with the methodology. In this tango you dance alone and the results are accordingly.

Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

I have a problem with this. Change is usually unavoidable in software projects, but welcome? No, I don't think so. This is engineer talk. Project managers call it Scope Creep. I am not saying that change must be blocked "at any price". I am saying that change has to be controlled carefully and allowed only under the right circumstances. Both the project team and the customer (again the customer) have to answer the following questions:

• Is the change really necessary?
• Is it necessary now?
• What will be the benefit?
• What will be the cost?

If the answers show that the change must happen, then so be it. But welcome? Changes are a necessary evil. Sorry, I am the old fashioned type.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

I like that. It's a good technique and it works of you are disciplined enough. Agile is even more strict. It requires timeboxing - even better.

Business people and developers must work together daily throughout the project.

Absolutely. But I think this is true and critical in any methodology. Show me a methodology that will disagree. So, even though this is true, it's not unique to Agile.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

There are two parts here:

• Support the team members. Again, true but universal. Not unique to agile.
• Trust them. Well, where I come from, we had a saying (crude translation): "Respect him and suspect him". As an experienced project manager, I like that better. Not that I think that team members are evil or anything. Just that they are human beings. And that includes me - the project manager - too, of course.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

This one probably irks me most. Not because of what it says but because of what it does not say. What it does not say is: "we hate documentation". Yes, I know. We all hate documentation. More precisely, we hate to write the documents. I have never met a project team member that did not like (and actually appreciated) documentation if:

• It was good quality
• Other people wrote it

It's much easier and more fun to communicate verbally. But verbal communication has serious drawbacks. This is a significant issue that deserves its own post, so I will not go into detail. I will just note here: face-to-face conversation has no records (other then in people's minds) and, therefore, no memory. Again and again, experience shows us that there is no escaping documentation. The big trick is to cleverly decide what documentation, how much? how? etc. But there is no alternative to good documentation. Don't get me wrong, face-to-face conversation is a great and vital means of communication but, sorry, it cannot make documentation unnecessary.

Working software is the primary measure of progress.

I agree assuming the working software does what the customer wants.

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

OK, this is not really a guideline. It's more of a promise. I do not have enough statistics to make a determination if this has proven itself. I will be glad if someone can enlighten me on this.

Continuous attention to technical excellence and good design enhances agility.

Again, this is not unique to agile. All methodologies support this, even those that do not call it out specifically.

Simplicity--the art of maximizing the amount of work not done--is essential.

I like this. The KISS principle is a good principle.

The best architectures, requirements, and designs emerge from self-organizing teams.

Here again, I do not have enough insight to make a judgement. Let me know if you have any concrete info about this.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

This is what the PMBOK calls Lessons Learned and is basic project management. It's good and again not unique to agile techniques.

So this is a brief summary on how I feel about agile development. I will sometime write about my own technique for which there is no formal name but I will characterize it as a combination of iterative and flexible waterfall.

Let me know what you think.

-----------------------------------------------------------------

Technorati Tags: agile, waterfall, Project Management, Project Manager, Software Engineering, Software Project Management

-----------------------------------------------------------------

Vendor Management - a Project within a Project

In previous posts (Cost Management is not Optional and Risk Management is not Hard - do it), I covered two very important project management Knowledge Areas (PMI terminology) which are not practiced enough in the software industry. I noted that they can be practiced relatively easily (at least at the basic level). Vendor management, on the other hand, is the opposite. It is something most SW projects have to deal with and it's not easy. The reason: if you look at it closely, it actually comprises a whole project unto itself. A project within a project. You need to plan and execute a special version of all Knowledge Areas for the vendor management, separately (yet interdependently) from your own project. And then some.

• The vendor represents its own stakeholders that require stakeholder management.
• You need to develop the vendor schedule..
• You need to define the scope of the vendor work.
• You need to work out the financial part.
• And you need to plan and execute all the other areas: Human Resources, Quality, Risk, Communication, etc.

And, as I note above, there is more that is unique to the vendor management:

• The vendor's interests are usually contradictory to yours. You want maximum scope with minimum cost. The vendor wants minimum scope with maximum pay (that's cost for you). This forces you to
• Negotiate.
• Be extra vigilant.

• There is the whole legal part. There must be a contract between your company and the vendor. Although putting the contract together is not your job, you, as the project manager, have a responsibility to ensure it happens. Don't dare to work without one. it may work out a couple of times but then there will be that one time when things go wrong, there is no contract in place and you'll regret it for the rest of your life. Besides, agreeing to work without a contract demonstrates that you are not professional.

All of the above introduce a full range of activities of a typical project into your project. As I said: a project within a project. Make sure to manage your vendor as a project.
---------------------------------------------------------------------------------

Technorati Tags: Vendor management, Procurement Management, Project Management, Project Manager, Software Engineering, Software Project Management
---------------------------------------------------------------------------------

Risk Management is not hard, do it

This is another area where Software Project Managers are not very good at: Risk Management. Why not? I found 2 main things and both are typical of the SW industry:

• There is no pressure on PMs and no expectation that they perform risk management. Most executives know about risk management but they don't realize that it's part of a PM's responsibilities.
• Too many PMs in the SW industry just don't have hard skills, so even if they understand the importance of risk management, they don't know how to do it. So they just don't.

Interesting anecdote: I occasionally teach SW Project Management at the local community college. One of my ex-students asked me to come to his workplace and make a presentation to the PMs in the company. In order to decide the topic, he ran a poll among the PMs, asking them for the topic they would most be interested in. Interestingly, Risk Management came in first with 22 votes (second place had a much lower rank - Stakeholder Management with 16 votes). So, this was clearly at the top of their mind. Not big statistics but I was not surprised. I have not yet given the presentation (scheduled for late January) but I will try to understand why they were so concerned about Risk Management.

Anyway, just as I noted in Cost Management is not Optional, I think that Risk Management shouldn't be optional either. It's important for the project and for the PM as a skill. And, it's really pretty easy. I don't want to "teach" Risk Management here. As I noted before, text books can do it much better than a blog. But I do want to give some tips here:

• It's not hard. Any basic book on project management gives the basics of Risk Management. Take a little time to learn it. It has a lot of overlap with Issue Management, which most PMs do regularly.
• Use mitigation to avoid contingency. A lot of PMs forget that it's much better to get rid of the risk before the risk event happens, rather than execute the contingency plan after the risk event has happened. Why be the smart guy (or gal) if you can be the wise one?
  
• Assign risk owners. You don't have to resolve all the risks all by yourself. You have a team. Let them help out. In fact, there will be risks which you are not the right person to own anyway.
• Track regularly. Risks are like issues. You need to track them regularly to make sure they are being taken care of.
• Find a risk log template . Templates make life so much easier. Don't waste time re-inventing the wheel. There are zillions of templates out there.

Once you start to do regular risk management on your projects, you will see that it's not hard and you'll enjoy the benefit of better project control and the appreciation of all of your stakeholders.

Let me know what you think.
---------------------------------------------------------------

Technorati Tags: Risk Management, Project Management, Project Manager, Software Engineering, Software Project Management
------------------------------------------------------

The Engineer as a Communication Tool for Remote Teams

We all have experience working with remote teams. And it's most likely not a very good experience. Remote is hard. Many experienced professionals will tell you that the shortcomings outweigh the benefits and that, therefore, companies should not allow dispersed teams. But we all know better - ain't gonna happen. The lure of cheap labor is too strong for the executives. So, remote teams are here to stay - at least for a while.

One of the most painful problems that I have experienced with remote teams is the communications issue. Mostly, the members of the remote team are very good professionals. They know their job. They have to. They have to prove themselves even more than the local team members. The problem is the difficulties in the communications. Distance also means time. They are not only far away, they are also usually asleep when we are awake and visa versa. They have a different culture (including work culture). But the most important factor (IMHO) is: Being remote means not being involved. Not that they don't want to. The time-space difference just makes it so hard. And it goes both ways. Not only are they not able to be involved with our work, we can't be involved with theirs. So we are all familiar with frustration around this disconnect. If we don't understand these dynamics we just conclude that they are just lousy professionals. Those of us who are more experienced know better: it's the communications, idiot!

Now, you are the PM and you have a local team plus a remote one (or more than one). How do you handle this communications problem? Well, the first thing you should do is probably something you haven't considered: try to see if you can avoid the remote team altogether. Yes! Remote teams are not something from God. Just from the executives. Your first reaction should be: "Do I really have to work with them?" Find the answer to the following:

• Do I need their skills in my project?
• Are there available local professionals who can do the same job?

Go to your manager/executive and try to convince them to allow you to replace the remote team with locals. I know that this attitude does not sound so nice. Where is the team spirit? Where is globalization embracing? Where is your tolerance for openness to other cultures? You know... My answer is: if the project fails, nobody will give you credit for having team spirit, embracing globalization or being open to other cultures. Your job and commitment are to complete the project successfully and if you think remote teams are a risk, it's your responsibility and right to try to eliminate this problematic element. Now, we all know that you don't have a lot of chance to succeed. This is usually something that is imposed on you and it's not optional. But it does not heart to try and who knows, maybe this is going to be your lucky day...

Assuming you must work with the remote team, how do you handle the communications problem? Textbooks offer a variety of techniques (email, con calls, Video conferencing, status reports, etc.), so I'm not going to repeat that here. Textbooks can do it much better than a blog. What I'm going to recommend is one technique which is slightly more radical but has proven extremely beneficial. Not theory, real experience.

The idea is: have an engineer from the remote team come and stay with the local team and act as a liaison between the local and remote teams. The engineers from the remote team will rotate in the liaison role. Each engineer should stay for an extended period (typically one to three months) so they can get really knowledgeable about how things work locally. This person will spend a significant period of his/her time on the following:

• Talk with team members to get to know them well.
• Thoroughly learn the product (code, docs, etc.)
• Learn and understand the local team's development "culture" (tools, processes, etc.)
• Learn and understand the business aspects (time pressures, clients, executive perspectives, etc.)
  
• Participate in most meetings, to understand exactly what is going on with the team, product and project.
• Be the main communication "tool" with his/her "home" team. Typically, have a regular, daily (probably more like nightly) phone call with the remote team to update them on what has been going on locally and ensure that the remote team is well updated.
• Represent the remote team locally.

I think that the benefits of this technique are veryu clear to see. I have experienced this twice in my career and was very happy with the results.

Needless to say that this also has disadvantages:

• Costly. You have to pay for the liaison's transportation, food, accommodations, etc.
• Complicated logistics.
• A large chunck of the liaison's time is spent on the communications part, so they contribute less to the generation of the product artifacts.
• Their time schedule is typically somewhat shifted - they work late into the night and report to work late in the morning.

Yet, my experience is that the benefits of this arrangement far exceed the disadvantages. I highly recommend this. Talk to your manager tomorrow.

Let me know what you think.
---------------------------------------------------------------

Technorati Tags: Communications Management, Project Management, Project Manager, Software Engineering, Software Project Management
---------------------------------------------------------------

Excellent Blog: David's Software Development Survival Guide

Hi everybody.

I just came across this blog: David's Software Development Survival Guide. I like it a lot and if you like my blog (more or less), you'll like that one too.

David writes about similar topics as I write. His material is very informative, very practical. It clearly shows that he's been around and learned a thing or two (and more). He provides a good mix of theory and practice, formal and informal. Highly recommended. I have also blogrolled him (on the right). Take a look.

Keep up the good work David and thanks.
---------------------------------------------------------------------------------------

Technorati Tags: Project Management, Project Manager, Software Engineering, Software Project Management
---------------------------------------------------------------------------

Story Time - Stakeholder Management

This really happened to me. Naturally, I hide identifying details.

Several years ago I got a contract position as Program Manager for a startup company. The company had just signed their first contract with a very large client. This was a do-or-die thing, so they figured it was time to bring someone in to make sure the project gets done properly.

The company had four executives:

• CEO, who was also the founder
• CTO who was also a co-founder
• VP1 who was the son of the chair of the board (smell trouble? No kidding)
  
• VP2

It took me two weeks to realize that I had landed right in the middle of a major war within the executive layer. The war was over the future strategy of the company. Actually, over the future of the company.

Camp 1: CEO and CTO: think the company is viable, has a great future and needs to continue with the current strategy.
Camp 2: Chair (daddy) and VP1 (son): think that the company is OK right now but will not succeed long term. Therefore, it should be sold ASAP to recover the investment.

I was the only manager in the company (other than the executives), so I had to somehow keep the boat afloat while the gods were fighting. I focused on the success of the project and did my best to stay out of the line of fire. But, as these things go, the line of fire chased me. I soon became their secret ally. One day one camp would sit down with me, tell me their secret war plans and urge me not to tell the other camp. The next day, the other camp would do the same. I listened politely but stubbornly avoided becoming a part of that in any way. From my perspective, each one of the executives could fire me in a blink of the eye if he thought I was siding with the other camp, so it was clear to me that the smartest thing was to stay out of that war as much as I could. So I continued to "stay the course" (ouch, not the smartest choice of words these days...) and concentrate on the project execution.

But things turned very problematic very quickly. It turns out that as the war intensified, the camps started with more personal and direct criticism and accusations. And so, an argument (one of many) broke-out: was our product ready for prime time or not? VP1 and VP2 claimed that the product was in a prototype state - not ready for prime time, which proved that CEO and CTO were incompetent. CEO and CTO claimed that the product was mature and ready for prime time, hence, they were doing a good job. Daddy-chair secretly sided with his son (VP1) but could not do so openly. So, the BOD was looking for somebody neutral but knowledgeable to "tell them the truth". And guess who they found. Right, me. My manager (VP1... Son) notified me that I was invited to the BOD's next meeting to "testify".

And then, to make matters worse, the CEO fired VP1 (my boss) ! So, I get a phone call from (ex) VP1 telling me that he had just been fired. He explained to me that he was sure he would be reinstated because his daddy would take care of that. Oh, and also, now, it was more important than ever that I deliver the right message to the BOD. Get it?

So what do I do? Here I was, just doing my job and suddenly, I decide the future of the world. And what's worse, the way I looked at it from my perspective, this was a lose-lose case. If I support camp 1, camp 2 will get rid of me. If I support camp 2, camp 1 will get rid of me. Damned if I do, damned if I don't. Either way, once I "testify", my employment with the company would be very short. This was still the dot com bust time and losing a job was even less fun than usual. I will admit that I had a few not easy days. I had a family to support and all that.

So, I thought and thought and strategised and strategised: which side should I support and yet survive?. I saw no way out. And then I had this eureka: to choose a side is the wrong consideration altogether! I must not choose sides in a political-power war. I must not think what's good for me, I must do what's right! I am first and foremost a professional and I must act professionally. And if I loose my job (I was sure of that) so be it. I just refuse to play along with this political power struggle.

So, the next day, I met with both camps (separately) and officially notified them that they should not expect me to support any camp and that what I would tell the BOD will be my true and honest professional judgment of the status of the project and the product. And after making that statement, I closed my eyes and waited for the ax to come down.

How big was my surprise. First I should say how big was their surprise. That, obviously was the one answer they had not expected. But when they heard it, they liked it! They apologized for dragging me into that war and said that all they wanted from me from now on was just to continue with my work and make sure that the project was successful. Which was what I wanted to do all along :)

I was so happy and proud that I had made the decision not to play the wrong game and promised myself that I would continue to conduct my job by this simple principle:

Be professional.

These are not empty words, It's important, it's the right way and sometimes it even works :).

So, happy ending to this story. Talk about stakeholder management...
------------------------------------------------------------------------------------

Technorati Tags: stakeholder management, Project Management, Project Manager, Software Engineering, Software Project Management
------------------------------------------------------------------------

The Test Engineer is the Project Manager's Best Friend

Or should be...

Over the years, the status of the QA profession has evolved (or, more correctly, devolved). For some reason, it has become the norm to view the QA engineer as some kind of a "second rate citizen" in the project team. Most other professionals, in particular the development engineers, look down on them. The test engineer is considered some sort of a failing development engineer. If you try to be a developer and And, as it often times happens in life, this misconception becomes sort of a self-fulfilling prophecy. Because of the poor image, nobody really wants to be a QA engineer, so only the no-so-good engineers end up "falling" down into this "lower caste" profession.

QA engineers are usually treated with resentment and viewed as a necessary evil and an obstacle on the way to the product release. Engineers treat them dismissively,try to order them around and pressure them to not delay the release. Usually, the QA group is officially subordinated to the engineering group, reporting to the Director or VP of Engineering. This is a clear conflict of interest! The QA people are managed by the development manager whom they are supposed to audit and whose product code they are required to test.

But for you - the project manager - the test engineer is the best friend! The QA engineer is your last "line of defense" in the release process before the product goes to the customer. He/she gives you the last chance to find the problems, catch the unacceptable bugs and generally, give you a professional, impartial assessment of the readiness of the product for release. And they are, by their profession, the most qualified to do that job. As a project manager you know (I hope) that it is so much better if the annoying QA engineer finds all these embarrassing issues, rather than the annoyed customer. So, the QA engineer can save you a lot of embarrassment and headache.

And it is actually not at all true that QA engineering is less sophisticated and technically challenging than the job of the developer. A good QA engineer must have the same technical skill level as the developers. A major reason for the misunderstanding is because too many people think: "QA Engineer = black-box testing". Indeed,black-box testing is not rocket science. However, any good QA engineer must be skilled in white-box testing and this is a whole different game altogether. I can easily find development engineers who will be challenged by QA engineers who implement white-box testing. And what about test automation? How many developers are skilled in building test automation tools? And what about testing of real-time, embedded software?

So, in your project, be sure to get the best QA engineers you can; protect them and support them and take their test findings seriously. If you don't the customer will. And do not save effort to change the team's attitude towards them Everybody must realize once and for all that testing is one of the best things in the release process.

Let me know what you think.
-----------------------------------------------------------------------------------------

Technorati Tags: test, testing, QA, test engineer, QA engineer, Project Management, Project Manager, Software Engineering, Software Project Management,