What the CMMi and PMBoK don’t tell you
Software project management as a field has matured significantly over the past couple of decades and the industry standard frameworks such as the CMMi and PMBoK have played their role in this progress. This raises an interesting question as to what extent these frameworks can be relied upon to ensure successful delivery. Are they complete by themselves or constitute only a part of a complete solution? And if partial what are the additional practices needed to complete ?
Primary questions surrounding software delivery
To elaborate, let us understand how this question matters to different stakeholders in software delivery:
- Can a BU (Business Unit) head go for CMMi assessment for organization, PMP certificates for the project managers (PMs) and be assured that delivery results improve automatically? If not, what are the other practices to be implemented?
- Can a customer rely solely on CMMi assessment and PMP certifications of the vendors and outsource software development to them? If not, what other capabilities should a customer look for among the vendors?
- Can a PM attend CMMi and PMP classes and be confident that he or she can deliver end-to-end, fixed price projects? If not what additional skills should s/he acquire?
When we speak to a number of software professionals across the spectrum and refer to research material (such as ” Scientific Research and Essays Vol. 6(10), pp. 2174-2186, 18 May, 2011 “), we find that these frameworks are good and increases maturity level in software delivery. But they are by no means sufficient and there are problems that persist despite the framework. These problems call for advanced practices beyond CMMi and PMBoK and a closer look at these problems and available solutions is needed.
Capabilities and limitations of CMMi and PMBoK
The CMMi and PMBoK have many similarities and differences as well. It would be out of scope for this article to examine these differences, and the article will be treating the superset of practices of both the frameworks as a whole.
Capabilities: The practices prescribed in the frameworks provide complete solution to a number of issues in software delivery:
- Configuration management: Many non-CMMi delivery units lack this practice and face consequences such as poor quality, rework and frustration due to absence of version control. Configuration management provides a complete solution.
- Reviews: This is one area that is the backbone of software quality and CMMi systematizes reviews. Huge benefits such as rework avoidance, better quality can be reaped if reviews are implemented properly.
- Project tracking and communication: The frameworks introduce discipline in the way projects are tracked and status communicated.
- Release procedure: Releases are made smoother and flip-flops are avoided by implementing CMMi.
Limitations: The limitations are many and cut across multiple areas such as soft skills of project managers, experience, software engineering, technological aspects such as architecture and so on. However, the fact that technology related skills, soft skills and software engineering skills are out of scope for CMMi and PMBoK is well understood. Therefore, this article will be focusing on 3 areas, which are considered very critical to project success, that are within the scope of CMMi and PMBoK, but don’t address adequately and requires software organizations to stretch much beyond implementing CMMi and getting their staff trained on PMBoK.
- Requirement management – Requirements related practices are well within the scope of both CMMi and PMBoK, but are not adequately addressed as will be analyzed below.
- Estimation accuracy – Estimation and cost are well within the scope of CMMi and PMBoK, but organizations need estimation methodologies suited to their context and these estimation methodologies don’t come packaged with PMBoK / CMMi.
- Schedule control – Project schedules are well within the scope of both CMMi and PMBoK, but how to draw effective schedule and use it to deliver project within schedule is an insight that is critical to project success and IT organizations need to go beyond PMBoK and CMMi to get these insights.
Analysis of limitations and solutions
- Poor requirement elicitation: None of the frameworks including agile frameworks have implicit methods for ensuring clarity and completeness of requirement elicitation. While, it may not be feasible to produce at the end of requirements phase, a water-tight set of requirements that will never change, significant improvement is possible. As Leffingwell (Dean Leffingwell, “Managing Software Requirements, A use case approach”, Pearson, 2011) puts it, requirements change because of what are called internal factors and external factors. Internal factors are not eliciting requirements from right stakeholders and non-usage of appropriate elicitation techniques. External reasons are market condition changes, regulatory changes and changes to competitive landscape. While one has to be flexible to incorporate changes due to external factors, changes arising out of internal factors can and must be avoided.
Solution: Striving to achieve clarity and completeness in requirements elicitation by taking care of internal factors is a practice that IT teams need over and above the industry frameworks. Requirement changes due to internal factors can be eliminated by eliciting requirements from the right stakeholders and usage of right elicitation techniques.
- High degree of requirement changes: CMMi prescribes that the requirements must be frozen after the requirements phase and subsequent changes must be subject to requirement change control procedure. Enforcing the requirement change control procedure imposes an overhead and when the number of changes becomes too many, the overheads become unmanageable and change control procedure breaks down.
Solution: First part of the solution is to reduce the changes by improving requirements elicitation. Second part of the solution is to eliminate requirement changes arising out of communication gap – this needs a good requirements document to the right level of detail. Third part of the solution is to have shorter release cycles and when requirement change does occur, it must be subject to a filter – whether the change asked for adds business value? If the change is important for business, can it be incorporated in the next release? After passing the change through these questions, if it turns out that the change is a must for the current release, then the change control procedure should be applied. In summary, how the change control procedure must be leveraged, is an insight that IT teams need beyond CMMi/PMBoK.
- Estimation accuracy: While the CMMi and PMBoK stress on cost estimation, they don’t offer estimation methodologies that can accurately estimate the effort. This is another area where delivery teams have to look for practices beyond the frameworks.
Solution: There are many technology specific estimation methodologies that are in practice today and some of them can be leveraged.
- Unusable project schedules: Many times the schedules drawn up using MS project are not used for driving the project. The task sequence and task durations in the schedule do not reflect ground reality, resource loading would not be optimal and as a result, the project schedule is only updated based on ground situation that takes shape independent of the schedule. Ideally, the ground situation should be driven based on the schedule. This laxity allows project schedules to slip out of control.
Solution: Well researched, step-by-step procedures can be used to draw usable schedules. How to draw up schedule is an area not addressed by the CMMi / PMBoK frameworks.
There are many other software delivery areas beyond CMMi and PMBoK such as software engineering excellence, test case optimization etc that demand attention and this article has highlighted only the most important ones. Implementing advanced practices in these areas can be highly beneficial. In one case study that I know of, a software service organization has implemented these practices across multiple projects and this has resulted in measurable improvements. Across a sample of 36 projects, the effort overrun was reduced by more than 10% and defect rate was reduced by more than 10 times.
In conclusion, I would say that software organizations can improve their delivery by investing on advanced practices beyond the CMMi and PMBoK frameworks. Which area of advanced practice do you think should be targeted first? Estimation or Requirements or Scheduling? Let us know why.