Monday, August 25, 2008

Practical Estimation Lesson

In wring this blog as response to the comment from Raj in one of my earlier post. He's asked me about the estimation techniques I use for testing efforts.
I've never used any standard estimation process such as Function point analysis, COCOMO etc. Testing estimation always is very much tied to the development estimates. So an independent estimation without taking into consideration development schedules always fails. At a very high level, I ask team members and test leads to use work break down structure (WBS) to estimate how much it'd take to complete the task on an optimistic and pessimistic count. I've found WBS method to be effective when testers have performed similar task previously. These estimates are subjective, and that is also fine. Behavioural and emotional factors also comes into picture when it comes to estimation in WBS. It always helps if you meet as a group and ask 'why'/'what-if'/'how' questions to further refine the estimates. Probing questions can be 'why is time taken for testing testing the feature more on Unix than on windows?', 'did you consider the factor in your estimates that feature XXX may be unstable because its based on a libraries our developers had inherited from a team that no longer exist?', 'how did you come up with this estimate?'. Many of these questions are intrusive in nature, some testers may be uncomfortable with this style. However, the team lead/manager should remember the need to perform this within a team meeting context. It requires to be stated very clearly that its the process that is being questioned and not the person .
What I typically do, is to have a 'long-term' estimate for the management when the project is initiated. This is generally a one-time estimate and as a test manager, you'd need to convince the stakeholders that the estimates comes with 'risk'. Identification of risks is important at this stage, and the most important risk is the 'risk the feature not delivered for testing in the schedule time', 'no clarity on what the requirement is' etc.
What I also do is to have internal estimates of individual tasks. These are 'near-term' estimates of individual tasks. These tasks are more granular and gives a foresight of tasks for typically next 3 weeks. Tasks performed by individual team member tasks should be stated to a level not exceeding 2 to 3 days. These estimates are more realistic and have dependencies. For example, these tasks can be very specific something like 'verify defect fixed in last 2 weeks', 'test the fund transfer module for inoperative accounts', 'get the latest build installed on the UNIX box', 'review the latest modifications for the user doc'. For each and every task taken-up by team members, a standard question to be asked is 'how long will it take?'. In my experience, typically for a mid-sized project, on a typically day, you may tend to ask this question over 3 to 4 times. Often so, every one in the team gets used to this question that they have this answer ready. It helps to track the near-term estimates if you are using a spread sheet or similar project tracking tools.
From my experience and the context in which we are working on, the stakeholders are more interested progress of work rather than the estimates. Enough advanced communication with the stakeholders about the progress, issues and risks are more important than the accuracy of the estimates.
With enough experience in the project, you can derive your next long term extimate from your previous near-term estimates. I've heard of "Wideband Delphi" estimation and seems to be a promising technique for estimating testing efforts.

3 comments:

Inder P Singh said...

Thanks for sharing your estimation technique!

I think that estimation is one of the more difficult responsibilities of a test manager. You have mentioned useful suggestions such as using a work breakdown structure (this encourages breaking down the testing tasks to an appropriate level of granularity and the estimating the individual tasks at the lowest level individually) and asking probing questions to better understand the rationale behind the individual estimates (this encourages the clarification of assumptions made by the test manager and the team member who provided an individual estimate). The discussion part of the latter suggestion is a key step in the Wideband delphi estimation method.

Anonymous said...

Hi,

Thanks for your article on ESTIMATION.

Though the defacto method of estimation is "WBS" which i too use for test estimation, FPA's are more devlepment centric.

TPM also mentions few estimation techniques.

Rgds,
Raj

Anonymous said...

Thanks for your help and understanding on the estimation process.

Also could you through some light on the below as it would be helpful for aspiring "Test Leads" and "Test Managers":

1. What is "Test Methodology"?
2. What is "Test Estimation"? What are the different types?
3. What is "Effort Estimation"? What are the different types?
4. How do you arrive at the figure of this effort estimation? Is there an industry standard?
5. As based on your effort estimation, we need to say how many 'testers' are required for the project? (or is there any other way of determinig this?)
7. Is there a standard value (cost) in 'Billing' a resource for testing to a client?
8. What to know more information on 'Writing Test Proposals', different project/test models?
9. What factors need to be considered when it comes to 'Billing a clinet'? Are there any standard values and methodology? How is it calculated?

Rgds,
Raj

 
Creative Commons License
The Elusive Bug by Rajesh Kazhankodath is licensed under a Creative Commons Attribution-Share Alike 2.5 India License.