One of the best known techniques are the ones provided by Caper Jones and David Longstreet. According to Caper Jones,

Total number of test cases =  (Count of Function Points) raised to the power of 1.2.

David Longstreet provides a similar formula for computing the number of UAT test cases:

Total number of UAT test cases, which is = 1.2 x (Count of Function Points).

As with every such empirical model, one has to use them as a guide and evaluate them with real data. In my own experiences with web based (Microsoft dot net platform), the numbers are slightly different, and approximate better as follows:

Total Number of test cases = (Function Point Count) raised to the power 1.05
Total number of UAT test cases = 1.35 x (Function Point Count)

Once the count of the test cases has been found, apply the productivity factors for test case authoring and test case execution to arrive at the total testing effort.

The following is the data for a 3000 function point project on which the above formula have been tweaked.

Unit Test Cases                 : 13,000
Integration Test Cases       : 5,000
System Test Cases            : 13,000
UAT:                                : 4,100
Total                                : 35,100

References: Estimating Test Cases and Defects by David Longstreet
Software Estimation Rules of Thumb by Caper Jones (pdf)

Moving from project lead to project manager was relatively seamless. There were a few changes though. The most significant was the focus I had to bring in laying down project specific processes and ensuring compliance. I also became less hands on as far as programming and architecting was concerned. My reliance on metrics increased. Unknowingly, I also carried forward the habit of micro managing the project, keeping an eye on each and every team member and their activities. This included spending time with individual team members at their desks while they worked, sometimes helping them with debugging their programs. An interesting aside was that I realized while one may grow out of the habit of programming, one almost never loses the ability to debug programs. 

This observation came to my help much later. In my role as a program manager, I had to manage multiple projects- often in multiple technologies and across different types of projects.

In one such role I was managing support for about 100 business applications, an identity management application, SAP implementations in ABAP and BW as well as smaller independent projects in JDE Edwards, C++ and so on. It became almost impossible for me to micro manage all these projects. Intuitively, I started devoting more time to project start up, ensuring that the fundamental processes were laid out and the team was committed to follow those, and then getting into project details only when something did not go right.

This was akin to debugging- that is it is sometimes easier to manage what is going wrong rather than keeping a tab on each and every a line of code. In more sophisticanted parlance, it is known as management by exception. Wikipedia defines this as:

a “policy by which management devotes its time to investigating only those situations in which actual results differ significantly from planned results. The idea is that management should spend its valuable time concentrating on the more important items (such as shaping the company’s future strategic course). Attention is given only to material deviations requiring investigation.”

Like any other concept, an over reliance on management by exception can be risky. Handling too many exceptions is not a good sign of the health of the projects either. By concentrating only on exception management, the manager may not be able to invest much time and attention to building processes and grooming talent. The trick is to kill two birds with one stone- that is, while attending to exceptions, the manager has to ensure that he or she identifies the root cause and then endeavors to put checks and balances in the processes to avoid future recurrences.

Reverse engineering (RE) is the process of discovering the technological principles of a device, object or system through analysis of its structure, function and operation. It often involves taking something (e.g., a mechanical device, electronic component, or software program) apart and analyzing its workings in detail to be used in maintenance, or to try to make a new device or program that does the same thing without utilizing any physical part of the original.

When I started my career at an Indian public sector telecommunication company, one of the projects that I worked on was a versatile multiplexer. At that time the Indian market was still closed and technological know how from the more developed countries was limited. We did not even have a sample of the multiplexer, just a technical brochure with a feature list of the product. I am not sure if that can be called reverse engineering, but our team did manage to build a working product. By the time it hit the market, I had left the company so I do not know how well it fared. However, I had other brushes with reverse engineering, both successful and not-so-successful.

Trivia: The original article was typed on a PC- XT machine using Word Star 7.

I had used email for just over a month then using a corporate account and the browser I was then using were Mosaic and Gopher !

Anyone remember using these?

Read the full article here.

1. Rely on his or her instinct and not measurable parameters
2. Manage people and not activities/ deliverables
3. Not set expectations with all stakeholders

The set of what one perceives to be the most common mistakes is likely to vary from person to person. Nevertheless, the second point came in for a sharp response at the forum:

I have a contention on point # 2 in your response. In any people centric organization, managing people is an integral part of Project Management. I am not sure why you consider it as a mistake. Can you explain?

This is not the first time that I have encountered this criticism- that software is “people centric” and hence the focus should be on managing people and not the deliverable. The root cause of this observation goes much deeper- that somehow, software development is something very unique, that it is not similar to other engineering disciplines, that it should not emulate manufacturing.

I aver that while, like any other discipline, software has its unique aspects, it is still part of the long tradition in engineering. Indeed, over the years, consistent attempts have been made to align it with engineering- establishment of standards, the ISO and CMM models all intend to achieve the same objective.

By stating that software is somehow “people centric”, the real point is lost- where software differs fundamentally from manufacturing is that its work products are intangible- they do not have a tangible physical inventory of items that can be measured discreetly, in that software is closer to process engineering where the basic ingredient undergoes a number of changes throughout its lifecycle. In the absence of such tangible artifacts, it seems that somehow it is people centric because that is the nearest tangible association one can make.

Over the years, there has been an increasing recognition of the engineering or manufacturing aspect within  the software industry- where there is the increasing preponderance of “components” (like from Microsoft), frameworks, standards, metrics and processes.

I am sure the the debate will, nevertheless continue, and provide better insights into this still- young discipline.

The feedback received on the project’s artifacts was related not only to the defects in the work products but a major part was non-defect comments (see diagram below). We were able to demonstrate to the customer through an analysis that all review comments were not defects. A detailed classification was also provided to substantiate the data. This enabled the customer’s project manager to go back to his team and guide them in controlling the non-defect review comments. This resulted in reduction of re-work by the team and enhanced the customer’s perception about the quality of the deliverables.

Key Steps:

• Consolidate all review comments in a single document (a tool like MS Excel can also be used).
• Categorize each review comment by the phase, artifact and type.
• Prepare a visual representation (example pie chart) of review comments and include as part of the weekly status

Outcome: Help the customer analyze the project feedback by categorizing and visually representing the review comments for a better perspective. This will help both the team and the customer by reducing re-work and enhancing perceptions about deliverables.

The x axis in the graph below shows the phase of the project, and the y axis shows the relative cost of finding a defect in the software.

Source: Roger Pressman, Software Engineering, ed 5

The next post will identify some of the metrics that can be derived from the above diagram. Meanwhile, can you take a shot at identifying what kind of testing metrics can be useful in reducing the cost of defects?

When you can measure what you are speaking about, and can express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind: It may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the stage of science. -Lord Kelvin

Software metrics used to measure attributes associated with a software application or a project. The purpose of measuring is self evident- one measures to improve. In a software development project, one measures attributes that help to improve quality, speed up execution and reduce costs, in other words, they should be useful to make software development faster, better and cheaper. This series of posts discusses some of the software metrics that are particularly important from a testing perspective and the benefits and challenges associated with gathering, analyzing and using these metrics.

What is to be measured?

“The risk with any metric is that people will come to see it as a description of reality, rather than a tool for a conversation about reality… one metric or another can function well only when managers know why they are measuring and for whom… In the world of social value-creation, context is king.” (The Economist survey of wealth and philanthropy from the February 25th, 2006 issue). Source

While the talk of software metrics abounds- indeed a number of companies do have a metrics plan- few implement it, or implement it in a systematic form. One reason could be that significant costs are associated with such measurements, one estimates puts the cost as 4-8% of the total development budget¹. Another is that while metrics are useful primarily from a management perspective, the source of the data are the technical people on the project. More often than not, the latter are so much in the thick of meeting deadlines that data is either not collected sufficiently or not on time, when it is too late to collect it. Metrics, particularly related to defects, are certainly collected when the applications or product does not exhibit stability or cause too many issues for the end user. However, more often than not this is as part of a damage control action, rather than as part of a systematic practice of gathering data. On the other end of the spectrum, data that might be gathered as part of practice may not relate or even obfuscate the intended objective. It is, therefore, important that while data gathering and metrics analysis is done as part of the routine, care is taken to gather only a few but key metrics.

1. NE Fenton and SL Pfleeger, Software Metrics: A Rigorous and Practical Approach, 2ndedition, Boston: PWS Publishing, page 28