How Unique is Software Development?

In a recent short discussion on “3 common mistakes a project manager makes’ over at Linkedin, I mentioned that the three most common mistakes a manager makes are:

1. Rely on his or her instinct and not measurable parameters
2. Manage people and not activities/ deliverables
3. Not set expectations with all stakeholders

The set of what one perceives to be the most common mistakes is likely to vary from person to person. Nevertheless, the second point came in for a sharp response at the forum:

I have a contention on point # 2 in your response. In any people centric organization, managing people is an integral part of Project Management. I am not sure why you consider it as a mistake. Can you explain?

This is not the first time that I have encountered this criticism- that software is “people centric” and hence the focus should be on managing people and not the deliverable. The root cause of this observation goes much deeper- that somehow, software development is something very unique, that it is not similar to other engineering disciplines, that it should not emulate manufacturing.

I aver that while, like any other discipline, software has its unique aspects, it is still part of the long tradition in engineering. Indeed, over the years, consistent attempts have been made to align it with engineering- establishment of standards, the ISO and CMM models all intend to achieve the same objective.

By stating that software is somehow “people centric”, the real point is lost- where software differs fundamentally from manufacturing is that its work products are intangible- they do not have a tangible physical inventory of items that can be measured discreetly, in that software is closer to process engineering where the basic ingredient undergoes a number of changes throughout its lifecycle. In the absence of such tangible artifacts, it seems that somehow it is people centric because that is the nearest tangible association one can make.

Over the years, there has been an increasing recognition of the engineering or manufacturing aspect within  the software industry- where there is the increasing preponderance of “components” (like from Microsoft), frameworks, standards, metrics and processes.

I am sure the the debate will, nevertheless continue, and provide better insights into this still- young discipline.

Posted in Software Engineering | 4 Comments

Software Metrics-III: An unorthodox metric

Here is an example of a non- orthodox metric that I happened to use for a large project.

The feedback received on the project’s artifacts was related not only to the defects in the work products but a major part was non-defect comments (see diagram below). We were able to demonstrate to the customer through an analysis that all review comments were not defects. A detailed classification was also provided to substantiate the data. This enabled the customer’s project manager to go back to his team and guide them in controlling the non-defect review comments. This resulted in reduction of re-work by the team and enhanced the customer’s perception about the quality of the deliverables.

Key Steps:

  • Consolidate all review comments in a single document (a tool like MS Excel can also be used).
  • Categorize each review comment by the phase, artifact and type.
  • Prepare a visual representation (example pie chart) of review comments and include as part of the weekly status

Outcome: Help the customer analyze the project feedback by categorizing and visually representing the review comments for a better perspective. This will help both the team and the customer by reducing re-work and enhancing perceptions about deliverables.

Posted in Software Metrics | Tagged , , , | Leave a comment

Software Metrics-II

The figure below offers a good starting point to determine what the most important metrics from a testing standpoint could be. The cost of detecting and fixing a defect escalates as the defect is detected later in the project. The obvious implication is that testing should aim to find defects earlier in the lifecycle. Another is that defects should be predictable- this can often be achieved based on historical data and a few heuristics.

The x axis in the graph below shows the phase of the project, and the y axis shows the relative cost of finding a defect in the software.

Source: Roger Pressman, Software Engineering, ed 5

The next post will identify some of the metrics that can be derived from the above diagram. Meanwhile, can you take a shot at identifying what kind of testing metrics can be useful in reducing the cost of defects?

Posted in Software Metrics | Leave a comment

Software Metrics- I

When you can measure what you are speaking about, and can express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind: It may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the stage of science. -Lord Kelvin

Software metrics used to measure attributes associated with a software application or a project. The purpose of measuring is self evident- one measures to improve. In a software development project, one measures attributes that help to improve quality, speed up execution and reduce costs, in other words, they should be useful to make software development faster, better and cheaper. This series of posts discusses some of the software metrics that are particularly important from a testing perspective and the benefits and challenges associated with gathering, analyzing and using these metrics.

What is to be measured?

“The risk with any metric is that people will come to see it as a description of reality, rather than a tool for a conversation about reality… one metric or another can function well only when managers know why they are measuring and for whom… In the world of social value-creation, context is king.” (The Economist survey of wealth and philanthropy from the February 25th, 2006 issue). Source

While the talk of software metrics abounds- indeed a number of companies do have a metrics plan- few implement it, or implement it in a systematic form. One reason could be that significant costs are associated with such measurements, one estimates puts the cost as 4-8% of the total development budget1. Another is that while metrics are useful primarily from a management perspective, the source of the data are the technical people on the project. More often than not, the latter are so much in the thick of meeting deadlines that data is either not collected sufficiently or not on time, when it is too late to collect it. Metrics, particularly related to defects, are certainly collected when the applications or product does not exhibit stability or cause too many issues for the end user. However, more often than not this is as part of a damage control action, rather than as part of a systematic practice of gathering data. On the other end of the spectrum, data that might be gathered as part of practice may not relate or even obfuscate the intended objective. It is, therefore, important that while data gathering and metrics analysis is done as part of the routine, care is taken to gather only a few but key metrics.

1. NE Fenton and SL Pfleeger, Software Metrics: A Rigorous and Practical Approach, 2ndedition, Boston: PWS Publishing, page 28

Posted in Software Metrics | Tagged | Leave a comment

Agile or Waterfall?

A popular buzzword in software development is Agile methodology. Cynics call it the programmer’s approach to software development while the agile manifesto is full of praise for it. My own experience is that both agile and waterfall are just models and real life does not follow either of these to the dot. Agile projects need the discipline that the waterfall approach imposes and the waterfall approach also needs to demonstrate sufficient flexibility. Change is a perennial issue in software development and software engineering can very much be called as nothing more that ‘managing chaos’.

There is a very insightful passage that Roger Pressman1 quotes from a paper by Nogueira and colleagues that needs to be quoted in full:

The edge of chaos is defined “a natural state between order and chaos, a grand compromise between structure and surprise”. The edge of chaos can be visualized as an unstable, partially structured state… It is unstable because it is constantly attracted to chaos or to absolute order.

We have the tendency to think that order is the ideal state of nature. This could be a mistake. Research…. supports the theory that operation away from equilibrium generates creativity, self- organized processes, and increasing returns. Absolute order means the absence of variability, which could be an advantage under unpredictable environments. Change occurs when there is some structure so that the change can be organized, but not so rigid that it cannot occur. Too much chaos, on the other hand, can make coordination and coherence impossible. Lack of structure does not always mean disorder.

In a SharePoint based project, for example, I found that the management of user requirements can best be done by managing them in an agile based manner, even though the overall project was done in waterfall model. The diagram below (from the Ambysoft site) proved to be very useful.

I created a requirements management sheet with a number of columns with the following key ones:

  • Use Case
  • Description
  • Priority (High/ Medium/Low)
  • Development Complexity (High/ Medium/Low)
  • OOTB/RUC/Configuration/ Customization
  • Iteration in which the feature will be built

OOTB: out of the box feature available in SharePoint to be used without any changes
RUC: Reusable component already available
Configuration: an OOTB feature but needs significant amount of effort in configuring it
Customization: Feature that needs custom code to be written, for example, in C#

By keeping the above parameters visible to the customer and end users, I found that it was very easy to negotiate as each iteration of the system was built. Each time this re- stacking was done, a number of features that were initially high priority go down the stack as users became aware of the implementation complexity and  its impact on timelines and cost. This negotiation is required because there are always the constraints of time and funds. This excel sheet based “requirements stack” is pretty much all one needs for projects based on platforms like MOSS 2007.

(1) Roger S. Pressman, Software Engineering: A Practioner’s Approach, 6th Edition (2005), page 77

Posted in Tips and Tricks | Tagged , , , | 1 Comment

Using the 80/20 rule

The 80/20 rule is so called after the Italian economist Vilfredo Pareto, who observed that 80% of the land in Italy was owned by 20% of the population. In software, the principle is particularly applied in defect fixing as it is often observed that 80% of the defects can be addressed by fixing 20% of the causes.1

Besides defect analysis and fixing I find the rule to be particularly useful when making transitions from one phase to the next in the software development cycle. For example, once 80% of the requirements analysis has been done, it is time to get into the design phase, once 80% of the design phase is completed, move into the development phase and so on. Trying to complete the remaining 20% of the work usually takes not so much effort but time. This remaining work usually ends during the middle of the next phase. There is also a tendency, especially where there are a lot of users and it is difficult to get a consensus on a feature, for issues to settle down as the issues posed by the next phase start coming up.

There is at least one situation where the reverse rule can be used. When it is time to roll out the application into production, I usually prefer to address only the top 20% of the defects found during the acceptance testing. Usually the critical and high priority defects are just around 20% of the total defects and cause 80% of the unease for the user.

Image Source

Posted in Tips and Tricks | Tagged , | 2 Comments

Team Organization: Star or Diamond?

Short term and long term projects need different types of team organization. Short term projects, say those up to 6 months, tend to have a “star” configuration while longer term projects are better served with a “diamond” shaped configuration.

In a “star” based organization, the team hinges around one strong lead resource while the others are like spokes around it. The word ‘star’ also fits in the sense that such a project succeeds because of a star performer. This organization is efficient and the line of communication is very clearly established. The major risk with this type of organization is that attrition or burn out of the key resource may happen. This makes it unsuitable for longer term projects.

What works better for a long term project is a diamond shaped structure i.e. one that has a few highly skilled resources, a large mix of medium skilled and a few low skilled resources.  Each of these serve a function while making the project both low risk as well as low cost. Fred Brooks in his book The Mythical Man Month states that the difference in productivity between the most skilled and an average programmer is about 10 times with the difference in cost being just twice 1. The problem is that these highly skilled resources are scarce and volatile. A number of times, I have found that it is not so much salary or financial consideration that drives such people, but purely intellectual challenges (it does not mean that they can be treated unfairly!). To sustain their interest the manager needs to to apply their soft skills and ensure that they continue to be challenged through out the project. At an organization level, they need to be cared for across projects.

What I call the diamond shape is what has been termed as The Surgical Team by Brooks2. Brooks uses the following metaphors to define the various roles in the project team:

  • Surgeon
  • Copilot
  • Administrator
  • Editor
  • Secretaries
  • Program Clerk
  • Toolsmith
  • Tester
  • Language Lawyer

My own classification of the various skills required in a project team are slightly different than those mentioned above, but the idea is very similar.

1: The Mythical Man Month page 30
2: The Mythical Man Month page 32

Posted in Team Organization | Leave a comment