mosgot's World of Software Project Management

"My" Agile Documentation

2007-02-04T01:14:00.001-08:00

In the last post I discussed agile programming and noted that I was managing documentation development in my projects the "agile way", even though I did not realize I was doing agile. Here I will summarize briefly some of "my" agile documentation principles.

At least do the Software Requirements Specification

I have written about this before - see here and here, so I will not repeat here. Just to recap. I can accept not writing any product documentation, except the SRS. But this one is not optional.

Start with something

I never thought that writing a full, complete, "perfect" document up-front was feasible. It almost never is. Trying to enforce that is futile. It will stress the team members, take an inordinate amount of time and effort and will become increasingly out of date as the project progresses. Soon, large parts of its content will be obsolete, which means (among others) that we have wasted a great deal of time and money.

Instead, start the document with a minimum and flash it out as you go. In the beginning, most everything is fuzzy and changing, hence, write only more general, more high-level things. As the project progresses, more and more things stabilize. update the docs with these things.

Update on the fly

As the project progresses, no need to go into heavyweight documentation cycles. Just update the documents as soon as new information is identified. Any team members can do that at any time. This makes the project dynamic and makes it easier to develop the documents quickly.

Review, approve, disseminate

Documents are useless if nobody uses them regularly. To make them useful you must make sure that what they say is

Acceptable - make sure the right people review the documentation and ultimately approve it.
Known - the best document in the world is useless if people don't use it. Most team members will not if you do not distribute the document formally and make sure people use it.

Keep changes at bay

As I noted in previous posts, unlike the agile principle, I do not "welcome" change. I treat change as unavoidable (don't want to say "evil") in the project. So, I know very well that it will come. However, nobody says you must accept it unquestionably. On the contrary, do question the need for any change. If you can avoid the change, all the better. If not, make sure it's reflected in the documentation, especially, if the documents already cover this topic.

Keep the documentation in sync with the code

Ideally, the documentation should drive the code. In reality a lot of changes are decided and implemented in the code without updating the document. This renders the documents obsolete very fast. Get your team into the habit of updating the documentation at the same time they update the code. It's actually easy because if they keep it up to date all the time, the updates are done in many small steps, rather than major update efforts.

Formatting is not critical

I had cases where we did cut-and-paste directly from emails or from comments in code into the document. This creates a rather sloppy-looking document but it's much better than no document at all and often better than spending all that time/money on high-quality tech-writing. The high quality can follow later. Just make sure that the document is understandable, updated and correct.

Solidify as the release approaches

Changes and updates to the documentation should be harder as the release gets closer. If the project is managed properly, this will be natural. If changes continue to arrive in large quantities even as the release is near, something is wrong. To me, it will indicate that the product is immature and unstable. this will prompt me to consider pushing the release date out. on the other hand, if things are done right, the pressure for changes will go down as the release approaches.

But this is not enough. You (project manager) should implement some process that will intentionally make it harder to get new changes in as the release approaches. People should have stronger and stronger justification to incorporate changes as the release approaches.

Instead, concentrate more on changes to the documentation that do not imply changes to the product (code). Corrections and more details are always good. They improve the documentation without requiring changes in the code.

I can probably think of some more but this is a good start and it's getting late...

-----------------------------------------------------------------------------

Agile Documentation

2007-01-28T22:22:00.001-08:00

In a previous post - Waterfall or Agile? Neither, there was a discussion (see the comments) on various points. One of the more interesting points was documentation. I argued against what I thought was the attitude to documentation by the agile community. From my personal experience, people generally interpreted the principle of face-to-face-discussion is preferable to documentation as a license to abandon documentation altogether. Today I still experience this attitude from many professionals, who regard documentation with contempt and rely on agile theory to argue that documentation is not required (even bad) for software development. And that I could not accept. I have seen enough how crazy things are without documentation.

So, I decided to do some further googling to see what people say about documentation and about agile documentation in particular. And I had a pleasant surprise. I came across an excellent white paper by Scott Ambler, titled: Agile Documentation: Strategies for Agile Software Development. It was really a delight to read it. Finally, I saw a very well balanced analysis of the issue of documentation in software development. With very few exceptions, I liked and agreed with everything Scott writes.

In one of the comments in my post above, I said that my goal in the post was to move the agile pendulum a bit back from its extreme swing. From Scott's paper I can see I did not have to do that. The pendulum has already moved back from its extreme position. At least for those professionals who learn agile principles carefully and thoroughly. And now, I think, the pendulum is right where it should be. Not tough, rigid documentation regiment, nor no-documentation at all. Just Agile Documentation. I don't need to repeat hear the principles of agile documentation. Just read Scott's paper above. great read.

If what Scott writes represents the agile community position about proper SW documentation strategy than I am certainly more supportive of agile methods than I was a week ago. The only sad thing is that I find that too many people have not yet realized the correct position on agile on documentation and still think: Agile documentation = no documentation.

And final note to Dave Nicolette - thanks for your very good comment in the post above. I now have a better and more supportive perspective of Agile methodology. It is definitely maturing.

In my next post, I will describe some of my "techniques" for generating SW documentation and show that, at least to some degree, I have been doing agile documentation without realizing it.

----------------------------------------------------------------------------

Innovation and Project Management - Mutually Exclusive?

2007-01-20T21:28:00.000-08:00

One of the most common complaints we hear against project management is that it kills innovation. "Oh, here come those project cops. They will put rules on everything, force us to do all kinds of unimportant things. This is the end of innovation. This project is dead because the project manager won't not let us work freely so we can innovate. Because only innovation will bring the really great products".

Sounds familiar? So many PMs have to struggle with teams (mostly engineers) who don't believe in project management. Who see project management as a damaging force that will stop any real production and will eventually kill the project. Over time, everybody begins to accept this mindset. Where there is project management, there cannot be innovation. And, where there is innovation, project management will suffocate it.

Powerful argument. Not because it has any merit, because it doesn't. But still, it's powerful because everybody seems to accept it as some kind of tautology. Too often, executives are scared to bring project management into the company (or group) because they are afraid to kill this wonderful tender baby - innovation. PMs are also at a loss as to how to make projects and innovation live in piece together and so, they end up unintentionally fulfilling exactly the "curse" they want to avoid.

This whole concept of "project management kills innovation" comes from a misunderstanding about what innovation involves. Everybody knows intuitively what innovation is and what it is supposed to give us. The problem is that most people usually interpret "innovation" (often subconsciously) as "give us a free hand to do whatever we want, however we want, whenever we want. We are artists and therefore, must have full freedom from schedules, scope, planning, cost constraints, etc. But don't worry, if you give us this essential freedom, we will do great things for you. Not quite sure what, not sure when. Not even sure we will deliver. But trust us, if and when we are done, it will be great. Because it's innovation." So, move aside project management, here comes innovation.

Not so. I hope, everybody realizes that complete freedom does not work. Yes, if you remove all the annoying restrictions (schedule, scope, cost, etc.), the engineers, given enough time, will eventually do something very nice and innovation will have proven itself. What I mean is that in our industry, we usually do not have this luxury. The customer wants its product with the agreed scope, on time and within budget. To the executives and the customer, innovation is not the goal. At best, it's the means. The goal is the exchange of added value (deliverables for money), and on time. If the PM is not going to manage the project and try to ensure the team hits the above constraints, he/she will be considered to have failed and even the project itself may fail. A failed project means a great deal of money that was invested for nothing and non-materialized revenue.

Now, in order to prevent misunderstanding, let me make it very clear:

Innovation is good! Innovation is very important. Every company should have some.

The trick is, how do we have innovation and project management?

The points that I'm trying to make are:

Innovation must not exclude project management (and vice verse).
Innovation and project management can live together.

Yes, innovation and project management don't have to be mutually exclusive. The trick here is:

Innovation should not be prevented. It should be controlled.

Many people will think that this statement is an oxymoron. Controlled innovation? I say yes. You can do great innovation even with some control mechanisms. This is done all the time in R&D organizations. Even innovators should (and can) do some planning and execute to the plan. The difference is that with innovation, the plan should be less restrictive and more flexible. But there must be some plan and tracking of execution to the plan. So, say "No" to something, someway, sometime. It's OK to ask even innovators What? How? and when?

Let's look at two general cases:

Innovation within a regular project.
Innovation as a project.

Innovation Within a Regular Project

With the right planning, innovation can live within a project. Project Managers must be very careful not to completely block innovation within their project. Instead, they need to plan for some innovation and be prepared to accommodate it.

Do not allow ambiguity in the innovation work in the project. Instead:

Innovation work must be officially identified, requested and approved. This is the most important part. Innovation must not be in "Skunk works mode". It should be presented, and justified. Not only why this is a good idea but also why now, in this project?
Innovation work must be incorporated into the plan and tracked as a risk.
Mechanisms should be in place to allow termination of the innovation work, if it exceeds its planned constraints.
If a project no longer can accommodate the innovation activity, have in place plan to:

Complete the project without the innovation part.
Spin the innovation work off to a stand-alone effort or into another project.

Innovation as a Project

This, IMHO, is the best way to go. As I note above, innovation is very important for a company and any company should cultivate it. However, within a regular project, this is possible only on a limited basis. To really cultivate it, the company should recognize that innovation work should be given its own, separate space. When a company prepares its high-level strategy, it should include R&D project work. By including it at the strategic-level planning, it makes the commitment to allow and support innovation. This means dedicated R&D projects that go in parallel with the "regular" (revenue generating) projects. This means the company will put aside the resources and money to cultivate innovation. This way, innovation will have its own life to prosper and ultimately benefit the company.

Note that I don't say "innovation work". I say "innovation projects". This is not accidental. Even in its own "innovation track" innovation work still cannot be allowed to do something, sometime, somehow. Any work in a commercial enterprise, even innovation work, must be subject to (at least) the famous 3 constraints - scope, time, cost.

Innovation and project management are not mutually exclusive.
---------------------------------------------------------

Waterfall or Agile? Neither

2007-01-15T01:59:00.001-08:00

This is the debate of the century. Waterfall or agile? Everybody has an opinion, emotions run high on both sides of the line and those who don't have an opinion are confused. It looks like everyone must have an opinion to be respectable. OK, I want to be respectable too so here is my opinion.

The first and IMHO, most important point is that the debate makes it look like there are only two alternatives for the "right" software development methodology. If it's not waterfall, then it must be agile. If it's not agile, then it must be waterfall. I say "not so". There are more alternatives. For me it's not waterfall, nor agile. Or should I say "both"? The closest methodology to what I like (and try to do) is Iterative Development. Yes, many of you will say that iterative development is a fundamental component of agile development and they will be right. However, agile development also has several things that I don't like. So, I cannot honestly say that I support agile principles.

Let me say up-front that, in my mind, classical waterfall is completely out of the question. I think that this methodology is so defunct that there is no need for me to explain and justify this position. So, in regards the question "waterfall or agile?", one half of the answer is straight forward: surely not waterfall. But if not waterfall, then what? Well, for me it's not agile either.

Let's cover briefly the major principles of agile and see what I think about them. I will take the principles directly from the original Agile Manifesto:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

This I like. At the end of the day, customer satisfaction is the ultimate goal. I also like the important principle of useful software. With regard to rapid and continuous, I like that too, but remember that it takes two to tango. Meaning, the customer needs to agree and work with us to accept our deliverable and rapidly test and provide feedback. You might be surprised, not all customers will want that.

Which brings me to the an important point. Agile experts agree that full customer cooperation with the project team is essential for the success of the agile methodology. This means that the customer must:

Thoroughly understand agile techniques.
Agree to work with the project team by the agile principles.
Do their part in the "deal".

The above requires significantly more involvement, commitment and resource allocation from the customer. So, not only you need all of the above from your own company, you need the customer to deliver too. Double challenge. My experience:

Rarely will the customer deliver on the above requirements.
Most companies that try to implement agile methodology fail to secure the customer's cooperation but still go ahead with the methodology. In this tango you dance alone and the results are accordingly.

Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

I have a problem with this. Change is usually unavoidable in software projects, but welcome? No, I don't think so. This is engineer talk. Project managers call it Scope Creep. I am not saying that change must be blocked "at any price". I am saying that change has to be controlled carefully and allowed only under the right circumstances. Both the project team and the customer (again the customer) have to answer the following questions:

Is the change really necessary?
Is it necessary now?
What will be the benefit?
What will be the cost?

If the answers show that the change must happen, then so be it. But welcome? Changes are a necessary evil. Sorry, I am the old fashioned type.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

I like that. It's a good technique and it works of you are disciplined enough. Agile is even more strict. It requires timeboxing - even better.

Business people and developers must work together daily throughout the project.

Absolutely. But I think this is true and critical in any methodology. Show me a methodology that will disagree. So, even though this is true, it's not unique to Agile.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

There are two parts here:

Support the team members. Again, true but universal. Not unique to agile.
Trust them. Well, where I come from, we had a saying (crude translation): "Respect him and suspect him". As an experienced project manager, I like that better. Not that I think that team members are evil or anything. Just that they are human beings. And that includes me - the project manager - too, of course.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

This one probably irks me most. Not because of what it says but because of what it does not say. What it does not say is: "we hate documentation". Yes, I know. We all hate documentation. More precisely, we hate to write the documents. I have never met a project team member that did not like (and actually appreciated) documentation if:

It was good quality
Other people wrote it

It's much easier and more fun to communicate verbally. But verbal communication has serious drawbacks. This is a significant issue that deserves its own post, so I will not go into detail. I will just note here: face-to-face conversation has no records (other then in people's minds) and, therefore, no memory. Again and again, experience shows us that there is no escaping documentation. The big trick is to cleverly decide what documentation, how much? how? etc. But there is no alternative to good documentation. Don't get me wrong, face-to-face conversation is a great and vital means of communication but, sorry, it cannot make documentation unnecessary.

Working software is the primary measure of progress.

I agree assuming the working software does what the customer wants.

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

OK, this is not really a guideline. It's more of a promise. I do not have enough statistics to make a determination if this has proven itself. I will be glad if someone can enlighten me on this.

Continuous attention to technical excellence and good design enhances agility.

Again, this is not unique to agile. All methodologies support this, even those that do not call it out specifically.

Simplicity--the art of maximizing the amount of work not done--is essential.

I like this. The KISS principle is a good principle.

The best architectures, requirements, and designs emerge from self-organizing teams.

Here again, I do not have enough insight to make a judgement. Let me know if you have any concrete info about this.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

This is what the PMBOK calls Lessons Learned and is basic project management. It's good and again not unique to agile techniques.

So this is a brief summary on how I feel about agile development. I will sometime write about my own technique for which there is no formal name but I will characterize it as a combination of iterative and flexible waterfall.

Let me know what you think.

-----------------------------------------------------------------

Technorati Tags: agile, waterfall, Project Management, Project Manager, Software Engineering, Software Project Management

-----------------------------------------------------------------

Vendor Management - a Project within a Project

2007-01-06T22:31:00.000-08:00

In previous posts (Cost Management is not Optional and Risk Management is not Hard - do it), I covered two very important project management Knowledge Areas (PMI terminology) which are not practiced enough in the software industry. I noted that they can be practiced relatively easily (at least at the basic level). Vendor management, on the other hand, is the opposite. It is something most SW projects have to deal with and it's not easy. The reason: if you look at it closely, it actually comprises a whole project unto itself. A project within a project. You need to plan and execute a special version of all Knowledge Areas for the vendor management, separately (yet interdependently) from your own project. And then some.

The vendor represents its own stakeholders that require stakeholder management.
You need to develop the vendor schedule..
You need to define the scope of the vendor work.
You need to work out the financial part.
And you need to plan and execute all the other areas: Human Resources, Quality, Risk, Communication, etc.

And, as I note above, there is more that is unique to the vendor management:

The vendor's interests are usually contradictory to yours. You want maximum scope with minimum cost. The vendor wants minimum scope with maximum pay (that's cost for you). This forces you to

Negotiate.
Be extra vigilant.

There is the whole legal part. There must be a contract between your company and the vendor. Although putting the contract together is not your job, you, as the project manager, have a responsibility to ensure it happens. Don't dare to work without one. it may work out a couple of times but then there will be that one time when things go wrong, there is no contract in place and you'll regret it for the rest of your life. Besides, agreeing to work without a contract demonstrates that you are not professional.

All of the above introduce a full range of activities of a typical project into your project. As I said: a project within a project. Make sure to manage your vendor as a project.
---------------------------------------------------------------------------------

Risk Management is not hard, do it

2006-12-30T22:32:00.000-08:00

This is another area where Software Project Managers are not very good at: Risk Management. Why not? I found 2 main things and both are typical of the SW industry:

There is no pressure on PMs and no expectation that they perform risk management. Most executives know about risk management but they don't realize that it's part of a PM's responsibilities.
Too many PMs in the SW industry just don't have hard skills, so even if they understand the importance of risk management, they don't know how to do it. So they just don't.

Interesting anecdote: I occasionally teach SW Project Management at the local community college. One of my ex-students asked me to come to his workplace and make a presentation to the PMs in the company. In order to decide the topic, he ran a poll among the PMs, asking them for the topic they would most be interested in. Interestingly, Risk Management came in first with 22 votes (second place had a much lower rank - Stakeholder Management with 16 votes). So, this was clearly at the top of their mind. Not big statistics but I was not surprised. I have not yet given the presentation (scheduled for late January) but I will try to understand why they were so concerned about Risk Management.

Anyway, just as I noted in Cost Management is not Optional, I think that Risk Management shouldn't be optional either. It's important for the project and for the PM as a skill. And, it's really pretty easy. I don't want to "teach" Risk Management here. As I noted before, text books can do it much better than a blog. But I do want to give some tips here:

It's not hard. Any basic book on project management gives the basics of Risk Management. Take a little time to learn it. It has a lot of overlap with Issue Management, which most PMs do regularly.
Use mitigation to avoid contingency. A lot of PMs forget that it's much better to get rid of the risk before the risk event happens, rather than execute the contingency plan after the risk event has happened. Why be the smart guy (or gal) if you can be the wise one?
Assign risk owners. You don't have to resolve all the risks all by yourself. You have a team. Let them help out. In fact, there will be risks which you are not the right person to own anyway.
Track regularly. Risks are like issues. You need to track them regularly to make sure they are being taken care of.
Find a risk log template . Templates make life so much easier. Don't waste time re-inventing the wheel. There are zillions of templates out there.

Once you start to do regular risk management on your projects, you will see that it's not hard and you'll enjoy the benefit of better project control and the appreciation of all of your stakeholders.

Let me know what you think.
---------------------------------------------------------------

The Engineer as a Communication Tool for Remote Teams

2006-12-23T23:00:00.000-08:00

We all have experience working with remote teams. And it's most likely not a very good experience. Remote is hard. Many experienced professionals will tell you that the shortcomings outweigh the benefits and that, therefore, companies should not allow dispersed teams. But we all know better - ain't gonna happen. The lure of cheap labor is too strong for the executives. So, remote teams are here to stay - at least for a while.

One of the most painful problems that I have experienced with remote teams is the communications issue. Mostly, the members of the remote team are very good professionals. They know their job. They have to. They have to prove themselves even more than the local team members. The problem is the difficulties in the communications. Distance also means time. They are not only far away, they are also usually asleep when we are awake and visa versa. They have a different culture (including work culture). But the most important factor (IMHO) is: Being remote means not being involved. Not that they don't want to. The time-space difference just makes it so hard. And it goes both ways. Not only are they not able to be involved with our work, we can't be involved with theirs. So we are all familiar with frustration around this disconnect. If we don't understand these dynamics we just conclude that they are just lousy professionals. Those of us who are more experienced know better: it's the communications, idiot!

Now, you are the PM and you have a local team plus a remote one (or more than one). How do you handle this communications problem? Well, the first thing you should do is probably something you haven't considered: try to see if you can avoid the remote team altogether. Yes! Remote teams are not something from God. Just from the executives. Your first reaction should be: "Do I really have to work with them?" Find the answer to the following:

Do I need their skills in my project?
Are there available local professionals who can do the same job?

Go to your manager/executive and try to convince them to allow you to replace the remote team with locals. I know that this attitude does not sound so nice. Where is the team spirit? Where is globalization embracing? Where is your tolerance for openness to other cultures? You know... My answer is: if the project fails, nobody will give you credit for having team spirit, embracing globalization or being open to other cultures. Your job and commitment are to complete the project successfully and if you think remote teams are a risk, it's your responsibility and right to try to eliminate this problematic element. Now, we all know that you don't have a lot of chance to succeed. This is usually something that is imposed on you and it's not optional. But it does not heart to try and who knows, maybe this is going to be your lucky day...

Assuming you must work with the remote team, how do you handle the communications problem? Textbooks offer a variety of techniques (email, con calls, Video conferencing, status reports, etc.), so I'm not going to repeat that here. Textbooks can do it much better than a blog. What I'm going to recommend is one technique which is slightly more radical but has proven extremely beneficial. Not theory, real experience.

The idea is: have an engineer from the remote team come and stay with the local team and act as a liaison between the local and remote teams. The engineers from the remote team will rotate in the liaison role. Each engineer should stay for an extended period (typically one to three months) so they can get really knowledgeable about how things work locally. This person will spend a significant period of his/her time on the following:

Talk with team members to get to know them well.
Thoroughly learn the product (code, docs, etc.)
Learn and understand the local team's development "culture" (tools, processes, etc.)
Learn and understand the business aspects (time pressures, clients, executive perspectives, etc.)
Participate in most meetings, to understand exactly what is going on with the team, product and project.
Be the main communication "tool" with his/her "home" team. Typically, have a regular, daily (probably more like nightly) phone call with the remote team to update them on what has been going on locally and ensure that the remote team is well updated.
Represent the remote team locally.

I think that the benefits of this technique are veryu clear to see. I have experienced this twice in my career and was very happy with the results.

Needless to say that this also has disadvantages:

Costly. You have to pay for the liaison's transportation, food, accommodations, etc.
Complicated logistics.
A large chunck of the liaison's time is spent on the communications part, so they contribute less to the generation of the product artifacts.
Their time schedule is typically somewhat shifted - they work late into the night and report to work late in the morning.

Yet, my experience is that the benefits of this arrangement far exceed the disadvantages. I highly recommend this. Talk to your manager tomorrow.

Let me know what you think.
---------------------------------------------------------------

Excellent Blog: David's Software Development Survival Guide

2006-12-17T22:34:00.000-08:00

Hi everybody.

I just came across this blog: David's Software Development Survival Guide. I like it a lot and if you like my blog (more or less), you'll like that one too.

David writes about similar topics as I write. His material is very informative, very practical. It clearly shows that he's been around and learned a thing or two (and more). He provides a good mix of theory and practice, formal and informal. Highly recommended. I have also blogrolled him (on the right). Take a look.

Keep up the good work David and thanks.
---------------------------------------------------------------------------------------

Story Time - Stakeholder Management

2006-12-15T19:42:00.000-08:00

This really happened to me. Naturally, I hide identifying details.

Several years ago I got a contract position as Program Manager for a startup company. The company had just signed their first contract with a very large client. This was a do-or-die thing, so they figured it was time to bring someone in to make sure the project gets done properly.

The company had four executives:

CEO, who was also the founder
CTO who was also a co-founder
VP1 who was the son of the chair of the board (smell trouble? No kidding)
VP2

It took me two weeks to realize that I had landed right in the middle of a major war within the executive layer. The war was over the future strategy of the company. Actually, over the future of the company.

Camp 1: CEO and CTO: think the company is viable, has a great future and needs to continue with the current strategy.
Camp 2: Chair (daddy) and VP1 (son): think that the company is OK right now but will not succeed long term. Therefore, it should be sold ASAP to recover the investment.

I was the only manager in the company (other than the executives), so I had to somehow keep the boat afloat while the gods were fighting. I focused on the success of the project and did my best to stay out of the line of fire. But, as these things go, the line of fire chased me. I soon became their secret ally. One day one camp would sit down with me, tell me their secret war plans and urge me not to tell the other camp. The next day, the other camp would do the same. I listened politely but stubbornly avoided becoming a part of that in any way. From my perspective, each one of the executives could fire me in a blink of the eye if he thought I was siding with the other camp, so it was clear to me that the smartest thing was to stay out of that war as much as I could. So I continued to "stay the course" (ouch, not the smartest choice of words these days...) and concentrate on the project execution.

But things turned very problematic very quickly. It turns out that as the war intensified, the camps started with more personal and direct criticism and accusations. And so, an argument (one of many) broke-out: was our product ready for prime time or not? VP1 and VP2 claimed that the product was in a prototype state - not ready for prime time, which proved that CEO and CTO were incompetent. CEO and CTO claimed that the product was mature and ready for prime time, hence, they were doing a good job. Daddy-chair secretly sided with his son (VP1) but could not do so openly. So, the BOD was looking for somebody neutral but knowledgeable to "tell them the truth". And guess who they found. Right, me. My manager (VP1... Son) notified me that I was invited to the BOD's next meeting to "testify".

And then, to make matters worse, the CEO fired VP1 (my boss) ! So, I get a phone call from (ex) VP1 telling me that he had just been fired. He explained to me that he was sure he would be reinstated because his daddy would take care of that. Oh, and also, now, it was more important than ever that I deliver the right message to the BOD. Get it?

So what do I do? Here I was, just doing my job and suddenly, I decide the future of the world. And what's worse, the way I looked at it from my perspective, this was a lose-lose case. If I support camp 1, camp 2 will get rid of me. If I support camp 2, camp 1 will get rid of me. Damned if I do, damned if I don't. Either way, once I "testify", my employment with the company would be very short. This was still the dot com bust time and losing a job was even less fun than usual. I will admit that I had a few not easy days. I had a family to support and all that.

So, I thought and thought and strategised and strategised: which side should I support and yet survive?. I saw no way out. And then I had this eureka: to choose a side is the wrong consideration altogether! I must not choose sides in a political-power war. I must not think what's good for me, I must do what's right! I am first and foremost a professional and I must act professionally. And if I loose my job (I was sure of that) so be it. I just refuse to play along with this political power struggle.

So, the next day, I met with both camps (separately) and officially notified them that they should not expect me to support any camp and that what I would tell the BOD will be my true and honest professional judgment of the status of the project and the product. And after making that statement, I closed my eyes and waited for the ax to come down.

How big was my surprise. First I should say how big was their surprise. That, obviously was the one answer they had not expected. But when they heard it, they liked it! They apologized for dragging me into that war and said that all they wanted from me from now on was just to continue with my work and make sure that the project was successful. Which was what I wanted to do all along :)

I was so happy and proud that I had made the decision not to play the wrong game and promised myself that I would continue to conduct my job by this simple principle:

Be professional.

These are not empty words, It's important, it's the right way and sometimes it even works :).

So, happy ending to this story. Talk about stakeholder management...
------------------------------------------------------------------------------------

The Test Engineer is the Project Manager's Best Friend

2006-12-08T23:38:00.000-08:00

Or should be...

Over the years, the status of the QA profession has evolved (or, more correctly, devolved). For some reason, it has become the norm to view the QA engineer as some kind of a "second rate citizen" in the project team. Most other professionals, in particular the development engineers, look down on them. The test engineer is considered some sort of a failing development engineer. If you try to be a developer and And, as it often times happens in life, this misconception becomes sort of a self-fulfilling prophecy. Because of the poor image, nobody really wants to be a QA engineer, so only the no-so-good engineers end up "falling" down into this "lower caste" profession.

QA engineers are usually treated with resentment and viewed as a necessary evil and an obstacle on the way to the product release. Engineers treat them dismissively,try to order them around and pressure them to not delay the release. Usually, the QA group is officially subordinated to the engineering group, reporting to the Director or VP of Engineering. This is a clear conflict of interest! The QA people are managed by the development manager whom they are supposed to audit and whose product code they are required to test.

But for you - the project manager - the test engineer is the best friend! The QA engineer is your last "line of defense" in the release process before the product goes to the customer. He/she gives you the last chance to find the problems, catch the unacceptable bugs and generally, give you a professional, impartial assessment of the readiness of the product for release. And they are, by their profession, the most qualified to do that job. As a project manager you know (I hope) that it is so much better if the annoying QA engineer finds all these embarrassing issues, rather than the annoyed customer. So, the QA engineer can save you a lot of embarrassment and headache.

And it is actually not at all true that QA engineering is less sophisticated and technically challenging than the job of the developer. A good QA engineer must have the same technical skill level as the developers. A major reason for the misunderstanding is because too many people think: "QA Engineer = black-box testing". Indeed,black-box testing is not rocket science. However, any good QA engineer must be skilled in white-box testing and this is a whole different game altogether. I can easily find development engineers who will be challenged by QA engineers who implement white-box testing. And what about test automation? How many developers are skilled in building test automation tools? And what about testing of real-time, embedded software?

So, in your project, be sure to get the best QA engineers you can; protect them and support them and take their test findings seriously. If you don't the customer will. And do not save effort to change the team's attitude towards them Everybody must realize once and for all that testing is one of the best things in the release process.

Let me know what you think.
-----------------------------------------------------------------------------------------

Cost Management is Not Optional

2006-11-24T20:31:00.000-08:00

All too often, I see companies and/or projects where project cost management is not done at all. Project Managers just don't deal with the money aspect. In many companies, the PM is not expected to manage the cost of the project. In some, the PM is even discouraged from doing this.
I saw many organizations where the executives somehow did not even know that PMs can do this for their projects, let alone should do it. When the (skilled) PM brings the topic up and demonstrates how this can be done and how beneficial this is, they are surprised. And happy.
But it is a fact that in many places, PMs are not required to manage the project cost.

The point I want to make in this post is:

Even if you (the PM) are not expected to manage your project cost, you should still do it!

Considering the fact that cost is one of the 3 famous project constraints, this is one of the most fundamental skills that PMs must have and must apply for their projects. And, it is one the most important activities that the PM must engage in. Even if the PM is not required to manage cost, they must feel compelled to do it anyway. At least two reasons:

For your own professional development. As a PM you must become skilled in cost management. Without some basic skill and experience in this area, IMHO, you cannot be regarded as a senior professional. If you work in a company where cost management is not expected of you, what better environment can you have to practice and learn? And educate the management?
Cost management is a critical tool for better managing and executing projects and programs. Without cost management, it is impossible to calculate such things as ROI, NPV and Earned Value. If a company wants to have a solid project/program strategy, it must use these (and many other) tools. Without project cost management, the company gropes in the dark.

Company financials are not a simple matter at all. In order to perform really good project cost management, the PM must be familiar with general business financial management and the specifics of his organization. This requires from the PM a significant investment in time and effort to achieve a good level of expertise. This is undoubtedly a discouraging factor and may explain why cost management is a 'weak link" in software project management. However, I want to claim that this should not stop the PM from practicing some basic-level of cost management. Two important points:

Even minimal cost management can provide significant benefit
Forget all the company financial policies and general finance management theory. All you need to do is estimate costs and track expenses for your project. That's all. You'll be surprised how many good things can come out of this. And...

Minimal cost management is actually pretty easy to do
Current project management software tools make it relatively easy to estimate costs and track expenses. Here are some guidelines, using MS Project (although any project management software tool can do that).

In MS Project, go to the Resource Sheet. All your resources are listed here. For each resource it is possible to define various parameters. One of them is Std. Rate (= Standard Rate). This is measured in $/hour. All you need to do is enter the rate of each resource in the table. True, usually, you don't know how much a person's salary is but I am sure you can make a reasonable guess. As soon as you entered rates for all the resources, MS Project automatically calculates the cost of each task in your schedule, plus roll-up of totals to summary tasks, all the way up to the full project cost. Plus totals and breakdowns per resource. That's it. So simple, so powerful.

You can also easily add fixed costs (such as purchases, travel expenses, fixed-price consultaning, etc.). Add a task for each of the fixed cost items. Add the Fixed Cost pre-defined column in the table (see MS Project help on how to add columns). For each fixed cost entry in the schedule, enter its cost in the Fixed Cost column. That's it. So simple, so powerful.

Don't forget to baseline.

As the project progresses, be sure to update the schedule to correctly reflect the project status. You update the tasks (start date, end date, percent complete) and MS Project will calculate updated current costs. And with the baseline that you remembered to save, MS Project will also calculate all the Earned Value info for you, full break down and roll up.

Once you start doing this minimal work you will be surprised how easy and useful it is. And when you show it to your manager or executive (who did not even think you could do something like that), they will be delighted and never let you off the hook again.

This is why I say: cost management is not optional.

------------------------------------------------------------------------

To Pad or Not to Pad

2006-11-18T01:20:00.000-08:00

To many, this is still the question. We all know this practice. Make some time estimate, then pad it to compensate for... Actually for what not? Overloading, distractions, interruptions, pressure to speed up work, Murphy's Low and what not. Actually, one more: the inability of the programmer to do good estimates in the first place. Even the programmers admit that they are over-optimistic and that their time estimates are always too short.

This typical predicament is nicely expressed in Rita Mulcahy's book, PMP Exam Prep:

"I have no Idea how long it will take. I do not even know what I am being asked to do. So, what do I say? I will make my best guess and double it!"

In general, padding is bad. Again, as Rita explains it, "Padding is a sign of poor project management!". if you are not sure about your team's time estimates, you should treat is as a risk and not camouflage it as hidden padding.

Actually, why is it bad to pad? Here are several reasons:

Padding means that there is an inherent deficiency in the estimation (and likely in the estimator's skill). By padding, we disguise the problem instead of facing it and dealing with it.
When the resource sees their padded tasks, they naturally assume that their task completion date is the one that includes the pad. You are giving the resource "extra time" before they even started the task.
Once padded, the original estimation disappears. There is no way to hold the estimator accountable for their original estimates.

So, what do we, PMs, do? We know that if we take the estimators' input "at face value" we are most likely in trouble right from the first moment of the project. There are all kinds of things that a Project Manager can (and should) do to improve the situation. None of them is a magic bullet but in combination, they can improve things significantly. Here are some suggestions:

Give them more time. One of the most compromising factors is the time that we (don't) give to the estimator. If we ask the estimator to estimate a task and want an answer right then, we may assume that the estimator is just throwing a number in the air to get us off their back. Give them time and encourage them to use it to think more carefully and wisely.
Review the estimates with a critical eye. If you have experience in time estimation and in SW development, you can identify cases where the estimation doesn't seem right.
Let an expert review. An expert will be able to provide a lot of insight.
Train the estimator in time-estimation techniques. There are techniques for estimating time. verify that the estimators use them.
Remind them of the implied sub-tasks. Remind the estimator that there are typically several activities included in a single programming task. Once the programmer realizes that, he/she will benefit in 2 ways: It will be easier to make a more accurate estimation and it will help them remember to do all of these things

Some thinking, planning, designing.
Prototyping
Implementing
Developing unit tests
Unit-testing
Fixing bugs.

All the above are things that the estimator can do to improve their estimates. But there are also a couple of things that you, the Project Manager, can do.

Estimate Effort, not Duration

Estimators easily fall in this trap. They think in terms of duration rather than effort. This automatically introduces padding into the estimation process. You need to make sure they provide their estimates as effort. You usually will have to coach them on this and continuously verify that the numbers they give you are indeed effort and not duration. Estimates in duration muddy the waters.

Nobody Ever Works at Full Load

No human being executes their tasks at 100% of their time. People are human beings. They take breaks, rest, take care of administrative things, go to the doctor, etc. When an estimator provides an estimation of the effort, the number should reflect continuous, non-stop work. In real life, this never happens. Project Management SW tools allow you to specify the load level of the resource. I usually set the load of Individual Contributors to 80% and of Managers to 50%. Granted, this is probably a form of padding but it does not come from uncertainty in the estimate but rather from sound experience. It's not an uncertainty, it's a certainty that people do not perform at 100%. The only uncertainty is the actual percentage. This is up to you to estimate.

Add Project-Level Buffers

This is a mechanism that I invented years ago, only to learn that it was a standard element in Critical Chain Management. I found it extremely powerful even if I don't use Critical Chain Management. Here is how it works:

The Gantt chart shows a project with a series of tasks (in Blue). At the end of the project, I added 3 Milestones:

Optimistic. This is when all the tasks are complete. No buffers, no padding. In the chart it is on 5/18.
Realistic. This is a date I got by adding 20% to the total duration of the project (from day 1 to the Optimistic milestone). In the chart it is on 6/15.
Pessimistic. This is a date I got by adding 40% to the total duration of the project (from day 1 to the Optimistic milestone). In the chart it is on 7/12.

As the project progresses a delay may develop in the project end-date. This will cause the first (Optimistic) milestone to move to the right. The other 2 milestones never move. This way we can see how the current projected end-date moves relative to the buffers. In other words, how much of the buffers is being consumed by the delay. This is an excellent indicator that allows the PM to see "how bad things are" and decide if it's time to take some corrective actions.

These are my safety buffers. Generally, I prefer not to show the 3 buffers to my team members. I only show the Optimistic milestone and do not even call it Optimistic. As far as they are concerned, that is the target date that they need to hit. The 3 milestones are for the people that I am accountable to: My manager, the project sponsor, the executives and the customer. I explain these milestones to them as follows:

Optimistic. We will finish the project on this date only if everything goes perfectly. No delays, no surprises. This is the ideal end date. I have a low level of confidence in this date but I am going to drive the project to this date and try to hit it.
Realistic. I think this is the date that I can realistically hit and I recommend that you assume and make your plans to this date. I have a good level of confidence in this date.
Pessimistic. This is the day we will be done if many things go really wrong. I have a high level of confidence in this date.
If the project runs later than the Pessimistic milestone, something is really bad in the project and we need to reevaluate the entire project.

I used this approach in the past with great success. Managers and customers were very impressed and pleased with this approach. They stopped being fixated on a single delivery date. Now the project is not held by its throat to a single date. There is flexibility but it's OK with management and customers because it is not open-ended. It is bounded. They accept the fact that I do not commit to a single date because they know it's not realistic but they are not worried that I open the door to a runaway project because the end date has a reasonable range.

Let me know what you think...

---------------------------------------------------------------------------------

Technorati tags: project manager, software project management, project management, software engineering, schedule, project schedule

---------------------------------------------------------------------------------

Why Hasn’t Software Development Gotten Any Better?

2006-11-17T21:16:00.001-08:00

This is the title of a very good post that Ed Yourdon published in his blog. Highly recommended. Read it and also my comment.

-------------------------------------------------------------------------------

Technorati tags: Software Engineering, software project, software project management

-------------------------------------------------------------------------------

Software Requirements Specification Is King - Part 2

2006-11-12T14:47:00.000-08:00

In Part 1, I explained why the Software Requirements Specification (SRS) is the most critical and valuable product document. In this part - Part 2, I will provide some guidelines, suggestions, tips, etc. that Project Managers need to know in order to make the SRS experience good and beneficial.

We will start where I left off in Part 1:

If you don't intend to use it seriously, don't bother to write it.

This is actually applicable to all project and product related documents. So many project teams write so many documents (sometimes even reviewing and approving them) only to put them in the drawer and forget them forever. What's that good for? A frustrating and boring task, expanding resources with no benefit. So, don't bother.

Those of you who have seen a well written SRS know that it's not a trivial undertaking at all. Writing a high-quality SRS is very demanding, both in quality and quantity. This is one of the main reasons why it's often hard to get the team to write it. I have found in my experience that you - the PM - can make things easier if you and the team follow some simple rules.

Substance is more important than form. In the beginning, don't worry too much about the appearance and format. Just make sure that the info is there. Worry about looks later. In fact it will probably be best to get a tech writer for that.
Something is better than nothing. Trying to write a full, complete, comprehensive spec in the first shot is discouraging. It's a very large task. Encourage your team to start with something. Start small and add as you go.

These 2 principles will allow the team to start with a relatively small effort and get something started. The team will routinely add and improve and over time the document will be high quality.

Now we want to cover some important principles for the contents of the SRS. I will not cover this material here. There are tons of books out there and I don't need to repeat that. No added value. But I do want to recommend that you take a look at Scott Sehlhorst's article: Big Ten Rules - Writing Correct Requirements. This is a very good review of the characteristics of good requirements. It is probably more than you want to know or do for starters. But as you improve the quality of your SRS, this is certainly something that you should look at. The only thing that I want to add here is: if you start the SRS small, as I recommend, then, IMHO, the most important principle in Scott's blog above is actually the last one - correct. The important point here is: errors are unacceptable. They cause more damage than if there was no SRS in the first place. So scrutinize the SRS very carefully for correctness. Worry about the other rules later.

This is it for now. I covered some principles for generating the SRS. Sometime in the future (sooner than later, if there is interest), I will cover the principles for using the SRS.
Let me know what you think.

--------------------------------------------------------------------------------

Technorati tags: Software Project, Software Project Management, Software Engineering, Software Requirements Specification

-------------------------------------------------------------------------------

Software Requirements Specification Is King - Part 1

2006-11-09T22:28:00.000-08:00

One of the problems that Software Project Managers encounter is the struggle with the team over product development documentation. Documents, documents, documents. Theory says that the development of the SW product must be accompanied by a series of documents: Market Requirements Specification, Software Requirements Specification (SRS, SW Architecture, SW Design, SW Detailed Design, test documentation, Release... On and on.

People usually hate to write documents. They like to do their job, be it market research, SW design, SE architecture, etc. But they hate to sit down and put all their knowledge on paper. And on top of that, there are the theorists who claim that documenting is a bad idea anyway (especially if they have to do it :). And to make matters even worse, even if they finally agree (or are forced) to write one of these documents, it's usually bad quality because they are not skilled in writing such documents. This surely does not add to the popularity of generating and maintaining documentation.

In this post, I don't intend to go into this debate about: does a SW product need documentation? This will require a rather elaborate discussion. To be clear, I do have an opinion - I think documentation is very important as long as it's done wisely, but this is for another discussion. What I want to cover here is the following rule that I strongly believe in:

If nothing else, at least maintain and use a decent Requirements Specification.

The Software Requirements Specification (SRS) is the most important and useful of the various SW development documents. It tells everybody, and I mean everybody, what this piece of SW should be doing. If you think for a moment, I think you will be surprised when you realize the long list of project stakeholders that will find the SRS useful and beneficial (and that's why I say everybody). Here is a list (let me know if I forgot anybody):

Product Manager - needs to be able to tell potential customers what the product does
Sales Manager - needs to be able to tell potential customers what the product does
SW Architect - uses it to develop the architecture.
SW Designer - uses it to design the product.
Programmer - uses it to make sure his implementation provides the correct features/behavior
QA will test against the SRS
Customer will know what to expect and what to test (acceptance testing)
Technical Writer will use it as the reference for the user's documentation (User's Guide, User's Manual, etc.).
Executives will use it for various purposes (such as in case of dispute, especially with the customer).

Without a SRS, every stakeholder has their own picture in their mind of what this product should do and how it should behave. How many times have you seen the bewildered Product Manager asking why is this feature different form what we agreed? Or the angry customer asking what is this new thing that was never mentioned to us? What does it do? How do we use it? Why do we need it? Did you ask us if we needed it? And where is this important feature that we did want to have and you agreed to provide? Or the angry engineer who says to the QA tester "Why does this test fail? The implementation is correct. You don't know how to write tests". And the tester answering "according to my information, this should behave differently. My test is correct". And who is right? If it's not written down, who knows?

No other document has this power of putting all the stakeholders on the same page like the SRS. So, please, if you don't write any other document, please, at least, write a Requirements Specification.

I will continue in the next post and provide some useful suggestions to make the chore of writing and maintaining the SRS easier. I will also suggest some rules for maintaining the SRS and how to effectively use it. I will just give you a little heads-up here as a teaser:

If you don't intend to use it seriously, don't bother to write it.

--------------------------------------------------------------------------------------------------------

Technorati tags: Software Project Management, Software Engineering, Software Project, Requirements Specification

--------------------------------------------------------------------------------------------------------

Why do SW PMs need to know Software Engineering?

2006-11-03T23:20:00.000-08:00

This sounds pretty trivial. It sure makes sense that SW PMs need to know SW Engineering. After all, SW Engineering is a very important discipline in the world of SW development. No brainer. Construction PMs need to understand construction technology, right? Ship building PMs need to know ship construction technology, right? Aerospace PMs need to know Aerospace technology, and so on. Same for SW PMs.
All the above is true and right and we all know it. Nothing new here. However, the point that I'm trying to make is that in the SW development world, it is so much more important that PMs know SW engineering. And why?

Because Software Engineers don't know Software Engineering !

OK, at this point all SW engineers are ready to jump on me and finish me off. So, let me explain. First, what actually is SW Engineering? Here is where there is a lot of confusion. Wikipedia notes that there are many different definitions for this term. The one that I like most is from IEEE Standard 610.12:

Software engineering is "(1) the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software, that is, the application of engineering to software," and "(2) the study of approaches as in (1)."

"the application of engineering to software" - this is the essence of this. It's not the coding. It's not the testing. It's the approach to SW development as an engineering endeavor. This means the classical activities of requirements definition, analysis, design, implementation, etc. And it doesn't matter right now if this is Waterfall, RUP, Agile or whatever. And when I talk about the above activities, I don't mean only the actual execution of the activities (such as writing the requirements, coding, running the tests, etc..) I mean the Methodology. The practices, processes, policies, etc. All of those things that everybody loves to hate but we (PMs) know that the project can't succeed without them. And yes, I know and agree that we need to be careful not to overkill with these things but we must have some.
Now, everybody basically understands the basic concept of the SDLC. We all know how to recite the waterfall phases or the Agile principles. But how many people really know how to do them? In detail?
Consider this: a person goes to college and graduates with a degree in Computer science. What is his profession? software Engineer? No - Computer Scientist. It's not the same. But when that same person gets a typical job at a SW shop, what will his title be? Computer Scientist? No, it will almost always be... Yes, you guessed - Software Engineer. But does he really have the skill set (not to say experience) required for a Software Engineer? No. Colleges do not teach software engineering almost at all. Interestingly, this person will not work as a Computer Scientist either. In most of the cases, the computer scientist by education (and some skills) and Software Engineer by title is neither. He is a programmer. All this stuff above with terms and titles is not just playing with words. These different titles do really mean different things, require different skills and are all important in the SDLC world. But the vast majority of professionals in the SW development world are programmers. Nothing else.
Now, if you are a single person who sits at home and uses his programming skills to write a little nice program, you do not need Software Engineering skills. But if we talk about a large company in which multiple programmers (and other professionals) have to work together to produce multiple products - possibly under multiple programs with multiple versions and multiple configurations for multiple customers... You can see where this is going if there is a lack in Software Engineering skills. This is a guaranteed mess.
I want to emphasize at this point that all of the above is not something theoretical. I have been around and I have seen time and again how companies struggle and they don't even know why. And when you put a SW Engineering methodology in place, things become so much better. This is a real problem in the SW technology world and it is everywhere.
So if you are the PM, you operate in this kind of demanding SW production environment and the SW engineers don't know SW Engineering, good luck. What do you do? Well, you need to know at least the basics of SW engineering so that you can:

Educate the engineers (and others).
Intelligently observe the project execution by the various professionals and make your own judgment about the quality of the operation.

This is why it's so important for SW PMs to know SW Engineering. Sadly, in the current state of affairs, these who should know don't know so you'd better know.
In the coming posts, I will go into more details on what I think PMs need to know about SW Engineering. In the meantime, let me know what you think. All comments welcome.

Technorati tags: Software Project Management, Project Management, Software Engineering

So, what is this all about?

2006-10-31T22:32:00.000-08:00

Yes, this is my first post. It's going to be short.

I am a software project manager and have a passion for this profession. I have a pretty good idea about what works, what doesn't, how to fix things, improve things, etc.
In particular, I like improving PM in small to medium size companies. These are the classical places where PM is just coming in. There is a lot of confusion, a lot of uncertainty about PM, a lot of suspicion, a lot of resistance. Sometimes also some support. Usually, most everybody feels that things are not going very good but none is sure what needs to be done to fix it. Actually, everyone has their own solution. Someone comes up with the idea that "maybe we need project management here" and stress goes up to the sky.
So, do you feel this pain in your company too? Let's discuss this and all sorts of other topics too. Here are some topics:

To Agile or not to Agile? What is Agile anyway?
Why do SW PMs need to know Software Engineering? (most don't)
Are Software Engineers really software engineers? (I say, no)
Have you ever seen Risk Management in your world? Do you know how to do it?
Can you convince the executives that PM is good? What are the benefits?

This is just a small sample. I have lots of issues and have opinions too. In the coming days I will start to covers these topics and more. I hope you will all join and we'll have a lively discussion.

Technorati tags: Software Project Management, Project Management