Assess-IT will once again be providing Independent Process Improvement Consulting Services

It is with great pleasure that I can announce, on November 2, 2010 Assess-IT will once again be providing Independent Process Improvement Consulting Services.  Our consulting practice focuses on Project Management and Systems and Software Engineering Process Improvement.  Building on a number of different models, methods and tools, we predominantly apply CMMI, Lean, Six Sigma, TOC, Agile and ITIL to our consulting solutions.

I look forward to continuing to blog about topics of general interest in the areas mentioned above.

You DONT have to rearchitect your processes to be CMMI compliant

I'm still surprised at how many clients have the misperception that they have to rearchitect their processes to be CMMI compliant.  There is nothing in the model that says you have to contort yourselves into some CMMI shaped box to comply with the guidelines in the model.  In fact, it's nearly impossible to do so and still carry on with your business model.  The CMMI is written as functional stovepipes (in alphabetical order), and your company processes, in all likelihood, are written to accomodate your business pursuit and delivery lifecycles.

The real trick to becoming CMMI compliant is to capture all of your business and technical processes, arrange them in some reasonable lifecycle order, then map that order to the CMMI to see where you have adequate model coverage and where you don't, develop a set of plans to address the gaps, and finally implement the plans to close the gaps.

If you have one monolithic Project Management process that addresses all the practices in the PP, PMC, IPM, and RSKM PAs, so be it.  Don't go rearranging everything to match the model.  Put together a mapping matrix that allows you to keep everything where it is, while at the same time allowing you to check yourselves against the model's guidelines.

CMMI SCAMPI Appraisals are CHEAP, Relatively Speaking

Quite a few clients are surprised by the cost of a SCAMPI Appraisal. While a SCAMPI Appraisal may seem expensive, when considered within the context of what it costs to implement a process improvement program within an organization, well, a SCAMPI Appraisal is CHEAP, relatively speaking.

Industry data show that a Process Improvement (PI) effort takes on the order of a 5-7% investment of resources: planning, improving, training, measuring, estimating, quality assurance, configuration management, status, etc. Essentially 5 FTEs of your 100 FTE organization need to be dedicated to your PI effort. Similarly, PI will take 5 weeks of effort on your 100 week project and $5K will be spent on your $100K project. Industry data also show that you can expect to receive on the order of a 4:1 Return On Investment (ROI).

If you create a process group early on, and allocate the 5-7% that we expect it to take, when it comes time for the appraisal, you’ve already got the appraisal resources you need, without taking away from billable work at the always critical stage. The other people in the organization will also realize that senior management takes this effort seriously by their demonstrated and not just verbalized commitment to the effort.

PI, if it’s really important to your organization, needs to be treated like any other important project. To not give it the benefit of “Real Project” status is to all but doom it to failure. “Real Projects” get real project managers, they have Work Breakdown Structures (WBSs)/Task Lists/Deliverables, they have identified staff who are committed to the effort at a negotiated effort level at a specific frequency of interaction over a particular duration, real projects get budget, schedule, resources, training, standard processes, and senior management face time.

So . . .How much does an appraisal really cost; IT DEPENDS :-)

In a typical 300 person development organization it takes roughly 3 to 4 people to generate $1M in revenue. So our hypothetical organization is generating $75-$100M. A 5% commitment to PI would be $7.5-$10M. Please feel free to substitute your own numbers.

A 4:1 ROI benefit to the organization would be $30-$40M or 4 times the $7.5M investment (almost 50% equivalent improvement organizationally) in reduced waste, greater throughput capability, more profitability, increased customer satisfaction, fewer defects, faster turn-around time, etc.

A fully featured appraisal with all the bells and whistles, planning, training, consulting, paperwork, materials, travel, etc., should cost on the order of $100K. A $100K appraisal investment against a $10M PI Project investment is 1%. If your current project estimating methodology is not producing estimates that are accurate to within 1% of the actual outcomes, I can legitimately (tongue in cheek) make the argument, using your data, that the appraisal is free, because the margin of error exceeds the appraisal cost.

Get Started Now, things will never really change until you change them.

Before you hire a CMMI consultant or SEPG Lead, at least check them out.

I have several clients who are in the process of selecting a CMMI Consultant or SEPG Lead. 

I offered to do a quick check just to get a feel for who was claiming to be a CMMI Expert.  I didn't do much:

I tried to add one of the candidates to an open appraisal, and I couldn't; it seems that they had some internal CMMI training but had never taken the formal class. 

I looked to see if another was an authorized/certified provider of anything and no luck there either.

Then I did the simplest thing of all, I googled them, and believe it or not, when I googled "firstname lastname CMMI", no results came up.  I thought you could google any three words and something would come up, or you win some type of prize.

There are a number of other simple credential checks that can also be done. Although no guarantee of finding a good consultant, at least you have a better chance.

Before you go through all the trouble of seeing if there is a fit for your business context, culture, project domain, etc. there are some very simple things you can do to thin the herd.  I highly recommend it.

Can CMMI, Lean, TOC, 6 Sigma, and Agile Play Together – They Better

There’s been a lot of chatter in the blogosphere lately about various different models, methods, and tools; and, not unsurprisingly about which one is best. It’s not a zero sum game; there can be more than one winner. In fact, if you adopt the best attributes of all of them, you’ll certainly be better off for it.

CMMI presents a set of industry agreed to best practices. If you develop your own company’s best practices for the management and development of projects, and then map them to the CMMI with an eye toward addressing the CMMI best practices, you can’t really go wrong.

Lean is based foundationally on the Toyota Production System, and is a combination of methodology and tools that seeks to improve the flow of work products through the system, while at the same time eliminating the many causes of waste inherent in any system. By mapping the as executed processes, you can determine if there are any wasted steps, motions, meetings, etc. The improvement of flow and the elimination of waste can not help but improve the efficiency of your business.

Theory of Constraints is based on the premise that there are constraints or bottlenecks in any system. To the extent that you can successfully mitigate the effect of the constraint on the system, your organization’s throughput will increase.

6 Sigma is a statistically based tool that seeks to identify and analyze the causes of process variation so they can be reduced or eliminated. Variation causes risk to the success of your project, because variation means that your predictions are only accurate to within the limits of the variation. To mitigate the risk you must allocate contingency buffers to ensure that your degree of uncertainty is covered by the additional set asides, which then can not be used for other purposes.

Agile is a project management methodology that builds foundationally on Lean and 6 Sigma; with its most obvious characteristics being frequent face to face meetings of the stakeholders to ensure the latest plan/iteration pair is on track; and, decomposing large projects into small timeboxes, iterations, sprints depending on your terminological preferences, so you can’t go too far astray before the next meeting to sync up on progress.

Nowhere in any of them does it say if you use one you may NOT use the other.

Institutionalization - It’s Not Just Another 20 Letter Word

Institutionalization – It’s just the way we do it here, Maturity Level by Maturity Level.

• Initial/Performed (ML1/CL1) – The work is done, the work products are produced, the Goals are satisfied. You have smart people doing good stuff.
  
• Managed (ML2) – The work is done according to a plan, the people are available and skilled/trained, adequate resources are assigned/acquired, identified work products are created, stored, evaluated and ultimately delivered, the work is monitored and controlled, and senior management takes an interest in the project throughout its lifecycle. ML2 is focused at the project level – Similar projects may do similar things dis-similarly.
  
• Defined (ML3) – All of the above, and there are documented characteristics and attributes that process descriptions possess: Purpose, Roles, Inputs and Entry Criteria, Process Steps, Outputs and Exit Criteria, Measures, Verification Steps, Job Aids (templates, fill-in guides, samples, tools, training, etc.). ML3 is focused at the organization level – Similar projects do similar things similarly, with allowable tailoring for the specific needs of projects.

Institutionalization is not how long the processes have been documented, but rather how ingrained into the organization’s operational methodology.

Institutionalization is codified in the CMMI’s Generic Practices. No matter how many smart people you have doing good stuff (Specific Practices), unless you Institutionalize (embrace the Generic Practices) you’ll never be other than ML1/CL1.

The Generic Practices of the CMMI lead to institutionalization of the following concepts.

Policy

• It is not enough for senior management to just talk about process
• Senior management must be continuous, visible, advocates for doing it the right way
• If senior management does not get out in front of the process improvement initiative, people will soon figure out that it is not that important and it will fall by the wayside until the next new best thing comes along; it migh be perceived as the flavor of the month phenomenon we see in organizations that are struggling for a magical solution

Planing

• Who (actor and role)
• What (what the project is to accomplish)
• How (the process steps for converting the project's inputs into outputs)
• When (schedule)
• How Well (objective quality standards and measures)
• Where (do they stage their intermediate and final work products)
• With What (samples, tools, checklists, guidelines, etc.)
• Where (do they get help)

Resources

• People (empowered, trained, available)
• Tools (hardware, software, job aids)
• Environments (proper working spaces, labs, etc.)

Training

• Availability is not a skill (process training and appropriate skills are needed)
• Organizational (what process training is the responsibility of the organization)
• Home Organization (what specific skills training is ones home organization responsible for)
• Project Training (new tools or processes that are for the primary benefit of the project)
• Individual (what must the individual do to make themselves a valuable, contributing member of the company)

Configuration Management

• All work products ought to be placed under some level of control
• Changes must be authorized
• Changes must be controlled
• Audits must be performed to ensure the documentation matches the work products, which may be documents themselves

Teamwork

• All work is accomplished by teams
• Teams must be managed to effectively develop the synergy that makes teams more powerful than the sum of their parts
• Teams must be empowered and authorized to do what they are tasked to do

Measure

• Without developing a quantitative understanding of project status it is difficult to determine if a project is on track
• Without developing a quantitative understanding of what your processes are doing for you, it is easy to fool yourself into thinking process change is the same as process improvement 
• Measures may be both qualitative and quantitative

Control

• Project and senior management must effectively steer the project according to the data they are collecting, both qualitative and quantitative data should be used
• When the data indicate the current state is significantly different from the predicted or desired state, the project manager must take control
• Typically the project manager may change the staffing, scoping, or scheduling

 Process Assurance

• The CMMI focuses on process improvement as the basis for improving the results of projects
• It is, therefore, essential that the processes are executed as documented.
• If a different process is followed each time, it is impossible to gather data on how following the process is really helping you

Product Quality

• It is not enough just to follow the processes, it is also essential that following the processes enables you to do better work

Status

•  Gathering data from multiple sources allows a project manager to develop a composite picture of the project that can be viewed from many vantage points
• This multiple view picture gives the project manager a better understanding of the project and the ability to make projections about its future

Standardization

• Following standards allows the project to do the same things the same way over and over again.
• This repeatability permits the projects and organization to develop data over a period of time that allows for prediction of future similar projects

Improvement

• Using the collected data collected over a period of time allows the organization to improve their processes to ensure the best possible process is being used for each project in the present and for the future

The CMMI Appraisal Waiting Game

In my discussions with software managers and practitioners I often hear, “We’re just getting started with process improvement; or, we’re not ready for the CMMI yet; or, we don’t need an appraisal to know we’re at level 1.” These are all valid issues; however, there are several reasons to consider having an appraisal early on in the process improvement effort:

• Being aware of weaknesses within the organization isn’t always enough to start generating corrective action,
• Just knowing the “score” for your organization is not enough to effectively direct the process improvement efforts at the highest priority problems, and
• The organization, although staffed with experienced and skilled people, may not have sufficiently detailed knowledge of the model to effectively interpret and use it. 

It’s not coincidental that a CMMI based process improvement effort is often referred to as a journey. To make an effective start on the road to process improvement it’s essential to know several things:

• Where you’re starting from,
• Where you’re heading to,
• How great a distance is to be traveled,
• How much time the journey should take,
• What useful things might be discovered along the way, and
• What route to take?

An up front look at the current state of the organization can help you create a map for the upcoming journey. The first step in creating your map is through a relatively quick and informal appraisal of the organization level and project level processes, activities and deliverables.

An informal appraisal provides much more than just a score; it’s an effective diagnostic and intervention tool for taking the process pulse of the organization. The major benefits of an informal appraisal are to:

• Support, enable and encourage an organization wide commitment to the software process improvement effort,
• Initiate a collaborative effort among the members of a group who may not have an opportunity to work closely together before. This is NOT something being done TO anyone, it is being done BY us FOR us,
• Develop an accurate consensus opinion as to the strengths and weaknesses of the organization’s processes and process adherence (What our processes tell us to do, what we say we do, what we can prove we do) all with respect to a reference model, the CMMI, and
• Create a public demonstration of commitment by upper management, and show the depth of awareness of the problems by the members of the organization. 

In my experience as both an internal and external process improvement team member, it takes an external catalyst to reach a broad enough segment of the organization to attain critical mass and create a self-sustaining effort.

Why can’t we just knuckle down, charge harder, or put our noses to the grindstone, just like we’ve always done in the past?

We could, but we’d still just be doing the same things in the same ways all over again.

As an example, NO project manager in recorded history has EVER said, “Hmmmm, I think I’ll just repeat all the mistakes of my predecessors, that’s the ticket!!” Statistics abound, but essentially, software projects are late, exceed the budget and fail to meet all of the customer’s expectations.

If you always do what you always did, you’ll always get what you always got.  I vote for trying something new.

The CMMI IS Right for You; However . . .

As more organizations adopt the CMMI, the breadth of adopters is moving from large DoD/Aerospace companies to more midsize and smaller organizations in a much wider variety of market sectors. Additionally, these adopters from different market sectors don’t all have the same safety-of-life critical applications that the original DoD/Aerospace folks had.

This begs the question; if I’m not a large organization with safety-of-life critical applications, how can I best use the CMMI without overly burdening my organization with many potentially great but often unnecessarily cumbersome practices, or rearchitecting my entire process set?

Finally, some common sense solutions.

Follow the Intent of the Model.  You don’t have to re-architect your entire process set to comply with the CMMI, nor do you have to adopt CMMI specific language. In fact, you can be CMMI compliant without even knowing the CMMI exists (not likely, but possible). If you’re running your software business as a business, you’ve probably got most of the basics covered already. Your challenge is to: map your processes onto the CMMI to ensure you’ve addressed all the best practices, create or enhance processes where they are missing or weak, implement those new or newly enhanced processes, and follow them long enough to demonstrate that they are working better than the previous ones.

Go Slowly. We know you’ve got the task of achieving CMMI MLX by Tuesday, regardless of what all the industry data show. The SEI is beginning to take a very serious look at organizations that show Maturity Level Progress that is out of alignment with what the statistics show as a reasonable time to improve. Go Slowly. Conduct a baseline appraisal to show what your real starting point is. Plot a path from that point to your target. Defend the plan as reasonable, based on the available data.

Start Small. Don’t bite of more than you can chew. People, in general, fear the unknown, and are resistant to change. Carefully explain what your are attempting and why you are attempting it.  Don't overburden them with too many new ideas at once. We frequently find that the most successful strategy is to start off with only a couple of improvements. I like to tackle the most significant challenge, and an easy challenge. The most significant challenge must be addressed due to its potential for great positive impact. The easy challenge will demonstrate the validity of process improvement at making things better, and it will typically do so, quickly.

Run the Process Improvement Initiative as a Project. The Process Improvement Project needs Sponsorship, Support, a Plan, Resources, Schedules, Deliverables, Quality Checkpoints, Measures, etc. just like any other project. To not give it the benefit of all of these project characteristics, is to all but doom it to failure.

Availability is NOT a skill. If you are considering allocating 5% - 7% (typical) of your software development budget to achieving CMMI compliance to be both more successful, and to be able to bid on future work, don’t just take whoever is available on the bench to staff this activity. Software Process Engineering is a discipline with required skills and qualifications just like any other engineering discipline. The biggest challenge in software process improvement projects is that it is a people challenge in a technical environment. Make sure the resources assigned have both the technical and soft skills required to make the project a success.

Is There a Business Case for Appraisals / Process Improvement ?

The prime and probably only justifiable business driver for starting a process improvement effort is if it will adequately answer the question, “HOW WILL THIS POSITIVELY IMPACT MY BUSINESS?” In other words, will this effort save/earn my organization more than it costs? The answer to this simple question comes in many shades of gray.

• Your prime customer says you must start a process improvement effort and achieve a certain objective measure of success to be considered for future work,
• Your management wants to become more efficient so you can more easily beat out the competition for lucrative contracts,
• There is dissatisfaction in your external customer base due to the number of errors produced and delivered or because the delivered product fails to meet the customer's expectations,
• There is dissatisfaction within your own organization due to smokestack organization and lack of communication within the organization.

A model framework based software process improvement effort produces real bottom line results in all of the areas mentioned above. The available data show:

• Decreases in delivered defects,
• Increases in the number of defects caught earlier in the life cycle,
• Productivity gains in the number of delivered source lines of code,
• Improved internal communication,
• Decreases in estimated delivery schedules without increasing schedule risk, and
• Business Value Ratios (Return On Investment (ROI)) of anywhere between 4 & 8.8 to 1. Conservatively 5 to 1

To get that kind of return you’d probably have to invest a huge amount of the organization’s assets. Not So.

On average, in a one hundred person IT Organization, you’d need to do the following:

• Initially allocate funding in the amount of about a mid level developer’s starting salary, and
• Free up just three to six people or their equivalent to maintain the effort on an ongoing basis.

Managed just like any other project in your organization, your Process Improvement Project will produce positive business results for your organization and will continue to do so for all of the other projects in your organization as well.

Let’s Have an Appraisal – or – Won’t That Make Us Later / More Over Budget (Time, Money)

Although misplaced, the first question I usually hear from project managers concerning a software process improvement effort or more particularly a software process appraisal is how much later will it make us, or how much over budget will it put us.

Internal Costs

Budget Assumptions:

• An average time for a first time appraisal, inclusive of all training, briefings, interviews, etc. takes about 7 calendar weeks of labor from start to finish.
• The appraisal team will be made up of 6 members of the organization being appraised. Each of these people will be dedicated to the effort for 200 hours over the 7 weeks.
• Approximately 50 people from the organization will be interviewed. These people will need to allocate about 4 hours to the appraisal distributed during weeks 6 and 7.
• The “average” organization has 100 people that develop or manage software.
• An “average” software development project or release delivery is 6 - 9 months long, or around 7 ½ months on average.

Computations:

• 7 1/2 months of effort for 100 people is 130,000 hours of organizational effort.
• The effort required for the appraisal is 6 people x 200 hours + 50 people x 4 hours = 1,400 hours.
• 1400 hours / 130,000 hours = 1.08% (That’s right, just a shade over 1 %).

Conclusion:

During the 6 - 9 months of a typical organization’s project schedule, the organization will be impacted to the tune of around 1% of its project-based effort. The cost of an appraisal is in the noise level with respect to the estimation accuracy of both cost and schedule. Therefore it can be concluded that an appraisal will not cause a project to be any later than, or cost any more than if no appraisal were performed. Most project managers I know wouldn’t bet their paychecks that their cost or schedule estimates were accurate to within 1%.

How To Calculate The Total Cost Of An Appraisal:

• Lead Appraiser fees vary; however, $75,000 can be used as a working figure.
• Six weeks of expenses (Airfare, Room, Board, Vehicle, Supplies, Shipping)
• Fully burdened costs for 1400 hours of effort from the organization. This number might increase depending on how many people over the minimum attend the Introduction to Software Process Improvement and CMMI class.

This works out to not more than $100,000 for the Appraiser plus your organization’s costs

How About The Costs Of The Continuing Software Process Improvement Effort:

General rule of thumb data puts the costs of a continuing software process improvement effort at somewhere around 5 percent of the software budget. In terms of people, you would need a Software Engineering Process Group (SEPG) or equivalent that is 5 people per 100 developers and managers. The SEPG is composed of process people, software quality assurance people, configuration management specialists, measurement folks, etc. These are the types of people developing, improving and assuring that the proper processes and procedures are being followed during the project’s lifecycle phases.
How Do I Get Started?

With a potential ROI of 5:1, the best question might be, how can I get started right away?

The easiest way to get started is to conduct an informal appraisal of a selection of your organization’s projects to see if they are in compliance with your own organization’s standards for the management and development of projects. Secondly, compare the organization’s standards with an industry benchmark such as the CMMI. Finally, compare the differences and put together a plan for closing any gaps.

Intro to CMMI_DEV in Sacramento, CA in March 16-18

I will be teaching the SEI's Introduction to CMMI-DEV in Sacramento, CA in March.  There are seats available.  The dates will likely be March 16-18.

Please contact me if you are interested.

David Greer
510-717-9655
david.greer@americansystems.com