Estimating Test Effort

Posted: July 13, 2010 | Author: | Filed under: Planning, Software Metrics, Tips and Tricks | Tags: | Leave a comment »
The best way for a ball park estimate for the testing effort is relatively simple- just take one third of the effort required for the complete development of an application. However, sometimes this total effort may not be available, for example, if the development and test vendors are different or the development and testing practices are not integrated. In that situation, the testing team needs do an independent sizing of the application, using a technique like Function Points and then computing the number of test cases.

One of the best known techniques are the ones provided by Caper Jones and David Longstreet. According to Caper Jones,

Total number of test cases =  (Count of Function Points) raised to the power of 1.2.

David Longstreet provides a similar formula for computing the number of UAT test cases:

Total number of UAT test cases, which is = 1.2 x (Count of Function Points).

As with every such empirical model, one has to use them as a guide and evaluate them with real data. In my own experiences with web based (Microsoft dot net platform), the numbers are slightly different, and approximate better as follows:

Total Number of test cases = (Function Point Count) raised to the power 1.05
Total number of UAT test cases = 1.35 x (Function Point Count)

Once the count of the test cases has been found, apply the productivity factors for test case authoring and test case execution to arrive at the total testing effort.

The following is the data for a 3000 function point project on which the above formula have been tweaked.

Unit Test Cases                 : 13,000
Integration Test Cases       : 5,000
System Test Cases            : 13,000
UAT:                                : 4,100
Total                                : 35,100

References: Estimating Test Cases and Defects by David Longstreet
Software Estimation Rules of Thumb by Caper Jones (pdf)

Reverse Engineering for Software Systems

Posted: June 30, 2010 | Author: | Filed under: Software Engineering, Tips and Tricks | Tags: | 2 Comments »
In its most rudimentary form, reverse engineering can be defined as “going backwards through the development cycle”. It has  its origins in espionage when rival countries would break apart a product from the opposite country and figure out its mechanism in order to build a similar product. Wikipedia defines reverse engineering as:

Reverse engineering (RE) is the process of discovering the technological principles of a device, object or system through analysis of its structure, function and operation. It often involves taking something (e.g., a mechanical device, electronic component, or software program) apart and analyzing its workings in detail to be used in maintenance, or to try to make a new device or program that does the same thing without utilizing any physical part of the original.

When I started my career at an Indian public sector telecommunication company, one of the projects that I worked on was a versatile multiplexer. At that time the Indian market was still closed and technological know how from the more developed countries was limited. We did not even have a sample of the multiplexer, just a technical brochure with a feature list of the product. I am not sure if that can be called reverse engineering, but our team did manage to build a working product. By the time it hit the market, I had left the company so I do not know how well it fared. However, I had other brushes with reverse engineering, both successful and not-so-successful.

Read the rest of this entry »

Agile or Waterfall?

Posted: July 7, 2009 | Author: | Filed under: Tips and Tricks | Tags: , , , | Leave a comment »
A popular buzzword in software development is Agile methodology. Cynics call it the programmer’s approach to software development while the agile manifesto is full of praise for it. My own experience is that both agile and waterfall are just models and real life does not follow either of these to the dot. Agile projects need the discipline that the waterfall approach imposes and the waterfall approach also needs to demonstrate sufficient flexibility. Change is a perennial issue in software development and software engineering can very much be called as nothing more that ‘managing chaos’.

There is a very insightful passage that Roger Pressman1 quotes from a paper by Nogueira and colleagues that needs to be quoted in full:

The edge of chaos is defined “a natural state between order and chaos, a grand compromise between structure and surprise”. The edge of chaos can be visualized as an unstable, partially structured state… It is unstable because it is constantly attracted to chaos or to absolute order.

We have the tendency to think that order is the ideal state of nature. This could be a mistake. Research…. supports the theory that operation away from equilibrium generates creativity, self- organized processes, and increasing returns. Absolute order means the absence of variability, which could be an advantage under unpredictable environments. Change occurs when there is some structure so that the change can be organized, but not so rigid that it cannot occur. Too much chaos, on the other hand, can make coordination and coherence impossible. Lack of structure does not always mean disorder.

In a SharePoint based project, for example, I found that the management of user requirements can best be done by managing them in an agile based manner, even though the overall project was done in waterfall model. The diagram below (from the Ambysoft site) proved to be very useful.

I created a requirements management sheet with a number of columns with the following key ones:
  • Use Case
  • Description
  • Priority (High/ Medium/Low)
  • Development Complexity (High/ Medium/Low)
  • OOTB/RUC/Configuration/ Customization
  • Iteration in which the feature will be built

OOTB: out of the box feature available in SharePoint to be used without any changes
RUC: Reusable component already available
Configuration: an OOTB feature but needs significant amount of effort in configuring it
Customization: Feature that needs custom code to be written, for example, in C#

By keeping the above parameters visible to the customer and end users, I found that it was very easy to negotiate as each iteration of the system was built. Each time this re- stacking was done, a number of features that were initially high priority go down the stack as users became aware of the implementation complexity and  its impact on timelines and cost. This negotiation is required because there are always the constraints of time and funds. This excel sheet based “requirements stack” is pretty much all one needs for projects based on platforms like MOSS 2007.

Notes:
(1) Roger S. Pressman, Software Engineering: A Practioner’s Approach, 6th Edition (2005), page 77

Using the 80/20 rule

Posted: July 1, 2009 | Author: | Filed under: Tips and Tricks | Tags: , | 2 Comments »

The 80/20 rule is so called after the Italian economist Vilfredo Pareto, who observed that 80% of the land in Italy was owned by 20% of the population. In software, the principle is particularly applied in defect fixing as it is often observed that 80% of the defects can be addressed by fixing 20% of the causes.1

Besides defect analysis and fixing I find the rule to be particularly useful when making transitions from one phase to the next in the software development cycle. For example, once 80% of the requirements analysis has been done, it is time to get into the design phase, once 80% of the design phase is completed, move into the development phase and so on. Trying to complete the remaining 20% of the work usually takes not so much effort but time. This remaining work usually ends during the middle of the next phase. There is also a tendency, especially where there are a lot of users and it is difficult to get a consensus on a feature, for issues to settle down as the issues posed by the next phase start coming up.

There is at least one situation where the reverse rule can be used. When it is time to roll out the application into production, I usually prefer to address only the top 20% of the defects found during the acceptance testing. Usually the critical and high priority defects are just around 20% of the total defects and cause 80% of the unease for the user.

Image Source

What Function Point is not

Posted: May 2, 2009 | Author: | Filed under: Tips and Tricks | Tags: | Leave a comment »
Function Points not an effort estimation method. It is a method to estimate the size of a software application. Using productivity data, it can be used to estimate effort.

The implication is that while different people and organizations can be pretty close to the same figure for the size of the application, the effort estimate can vary substantially across different organizations.

Sizing done using Function Points is verifiable since it uses a generally accepted method. In contrast, the other popular method of computing effort- the Work Breakdown Structure (WBS) is almost entirely based on implicit productivity of an organization or team. The productivity factor that uses the FP count also makes it easier to compare the productivity across teams and organizations.

Another aspect of Function Points is that it is based on statistical techniques, and hence small variations in the  computation tend to cancel out each other. Despite this, it is very reasonable to see a variation of 15% or so in the calculations done by different people or teams even when using identical inputs. For larger projects, I prefer to have the estimation done independently by two teams and then have it cross checked with an estimation done using WBS. Then use a mean of the three.

How to Estimate in 5 minutes

Posted: May 1, 2009 | Author: | Filed under: Tips and Tricks | Tags: | Leave a comment »
This is a thumb rule that I use based on some past projects. It works pretty well to compute the Rough Order of Magnitude for mid to large sized business applications. Here is the rule:

  1. Estimate the number of screens in the application that need to be developed
  2. The RoM of the project in person months = Count of Screens to be developed
  3. E.g. if there are 100 screens to be developed, the RoM of the project is 100 person months.

Note that this is only an RoM method, or estimation with a large degree of tolerance, perhaps 25% or so.

I have used this thumb rule with a large degree of success for dot net based projects. My guess is that it should work pretty much the same with other platforms like Java and even SAP and other ERP systems as well. I do not have any historical data, however, to substantiate this.

Since this is an effort estimation rule and not a sizing rule, the figure will differ from one team to another. You will need historical data to arrive at a figure that better reflects the productivity of your team.

The point is that there is a correlation between the number of screens and the effort. Obviously, it is rather simplistic and hence suitable only for a quick computation of the RoM. For business applications, I have used function point for dog’s years, and trust it very much. That is what I would recommend for a more accurate estimation.

Follow

Get every new post delivered to your Inbox.