The Keys to Key Performance Indicators

Executives often ask me for the impossible and what they want is best exemplified by an often used phrase, “A sip from the firehose.” In most organizations today there is no shortage of data and in the project and portfolio management realm in particular we have a plethora of systems and processes that can generate data but no manager can possibly sift through thousands or even millions of records in order to determine what decisions they should make.

The result is that most significant decisions about projects including the decision to start the project in the first place is made on an ad-hoc basis.

The technique to avoid this is well known. Organizations should choose “Key Performance Indicators” (KPIs )which will surface information that is critical to making those business decisions. But, how does one choose those KPIs? How do you know that the KPIs you’re using are producing the result you require? Let’s take a look.

Choosing KPIs

First of all, where do we look for choosing the right Key Performance Indicator and how will we measure it? These metrics will be used in a variety of ways. We can use these metrics for distinguishing one project from another. We can use them in the project selection process, the stage gate process, the resource priority allocation process and the project review process. There is no limit to the number of things we could measure.

One place to look it through the main elements of PMI’s PMBOK (Project Management Body of Knowledge). In each section you might have possible measure that would lead to better management of the project.

In the Scope area, for example, you might have scope creep or scope change as a measure. In the Time area you might look at delays in the schedule. In the Cost area you might look at resource costs or Return on Investment. In the Communications area you might look at the timeliness of schedule updates or the status of project reporting. In the HR area you might look at Resource Variance or Resource Load/Overload or Resource balance. In the Risk area you might look at the volume of risk items and their severity or urgency and so on and so on.

Not every metric will make sense in every context. For example, in a private organization, projected Return on Investment might be a critical measure. In a public organization this might be completely inappropriate. There, citizen satisfaction or alignment with legislative requirements might be much more appropriate.

Some of the criteria for a good KPI would be this:

Ø The KPI reflects some element of corporate strategy

So, measuring ontime delivery might make sense in most organizations, but where corporate strategy is more focused on quality or safety, focusing on ontime delivery might result in management making decisions that are diametrically opposed to the corporate mission.

Ø The KPI is actionable

All too often we see organizations with spectacular dash boards containing lovely graphics and charts and, when we ask what action is supposed to happen if this indicator turns red or if the needle on that guage goes to the danger zone, no one knows the answer. The whole point of a measuring and reporting on a KPI is to ultimately take action.

Ø The KPI should result in obtainable objectives

Creating a metric which no one ever has a chance of getting out of the “red” and into the “green” makes no sense. It results in urgent emergency action day after day after day and ultimately is self-defeating. The objective that the KPI represents (overload less than 110% of availability for example) must be attainable.

Ø The KPI should be measureable

It’s tempting to make KPIs which are wholly subjective, where someone just plugs in a number or a color. If this is based on a feeling rather than some empirical measure, the KPI typically has little value and yet the decisions made on the basis of the KPI may be just as significant as those based on other empical measures.

Ø The KPI should not be duplicated

Just because we have a dozen different ways to display late schedules, showing all those KPIs doesn’t add value. In fact, it can be confusing and disruptive.

Ø And finally, the KPI and the actions associated with it should be understood by all decision makers. Having a common understanding among all the decision makers of what a particular measure and indicator means is critical to using KPIs effectively.

KPI Challenges

There are a number of easy mistakes that people doing this exercise for the first time face. Here are some of the most common:

Ø “Great news, I have hundreds of KPIs!”

This isn’t great news at all. Remember, what we’re after is a sip from the firehose. If you open the tap too far; if you let too much water through the hose, you’ll knock management from its feet and they won’t be able to absorb the information. Typically we look for a handful of KPIs. As Chris Iervolino, an expert in Business Performance Management (BPM) said, “Somewhere in the extensive negotiations of creating the KPIs, the ‘K’ got lost.” So, just because you can measure a thing, you don’t need to measure it.

Ø “Great news, we have only one KPI!”

Having too few KPIs is also not useful. There are always a couple of opposing forces and results that can be identified in the project management business and the whole point here is to make business decisions. Those opposing forces need to be identified and displayed in some way.

Ø KPIs are too subjective

There’s a strong incentive to invent KPIs for which there is no measure; no metric. The result is an indicator behind which is just a subjective decisions based on someone’s feeling or intuition. While many business decisions are made on a manager’s intuition, when we put subjectivity into a Key Performance Indicator, we take a subjective perspective and display it as an empirical or data-sourced result. The effect of this can be negative.

Ø The KPI has questionable completeness or quality

Imagine that we have a Key Performance Indicator that shows our resource capacity. The graphic shows the organization’s current status of all work and all resource availability and projects the expected over or under load of resources against our project and nonproject tasks in the future. But the indicator only has current project data for 20% of the projects. Another 60% of the project data is at least 30 days old and 20% of the data is missing altogether. Imagine what business decision might be made on the basis of this indicator alone and how potentially damaging that would be. When KPIs are displayed, we always recommend that there is some indicator of the indicator’s quality and the completeness of the underlying data.

Wrap up

The great news about applying the Key Performance Indicator paradigm to project and portfolio management data is that there’s typically lots of great data to choose from and the nature of the project management process results in a lot of that data being of very high formalized quality. That’s perfect for a productive KPI exercise.

If you’re getting started on choosing your KPIs, the best advice I can give is to start small. Start with 3 or 4 indicators and get the measurements of them, the display of them and the actions associated to them right. The benefits of doing this can be profound and don’t worry, you can always expand the number of indicators in the future.

Happy measuring!

That Elusive Proof of Concept

At least once a week I must get a request from an organization to advise them on how to proceed with a Proof of Concept for epm software. It can be for a wide variety of epm tools and the particular flavour isn’t relevant to today’s conversation. My approach to all of these requests is the same and it often makes the requestor take pause. “We’d be delighted to help with a Proof of Concept,” I’ll say. “Please describe to me exactly what you wish to prove.”

It’s an obvious question isn’t it? Yet, in less than 10% of the cases, does the requestor have an articulate answer. I’ll get a hundred variants on “We want to see if it will work for us.” or something to that effect. When I poke in a little deeper to find out what the person really wants to accomplish, there is a disturbing but remarkably common theme. In many, many cases, the person who is trying to do a proof of concept is hoping that they can quickly establish a working project system with the enterprise tool and that management will be able to see this miraculous tool working and providing such remarkable value that they will grant the hitherto unavailable budget for a complete deployment.

Unfortunately, trying to use a Proof of Concept deployment to convince management to commit to epm or an epm tool is almost never successful. There are a couple of problems that may not seem obvious.

First of all an epm deployment, even a tiny one that you may be thinking of for this Proof of Concept requires the support of management to have any chance of success. A deployment of epm is a tiny bit about technology and a huge amount about changing behaviour or culture change. Without the direct and unequivocal support of management, you may get the technology installed but you have no chance of effecting a culture change. This is true for a 10 user deployment as much as it is for a 1000 user deployment.

Also, some of the most popular objectives for an epm deployment such as Resource Capacity Planning simply can’t be delivered until all data is in the system. Oh, you can simulate it of course, and you can certainly show an example report but you can’t see even 1/100^th of the possible value with only a tiny percentage of the data. Moreover, without management ready to demand that all users commit to the system, there is little chance of getting access to all the data anyway.

Doing an epm deployment which has a chance of producing any real value for the organization, is going to first require a period of time where senior management examines and then commits to the results they hope for from the system. We usually do this work in a vision session of some kind early on in the deployment. Without knowing what business drivers the system is expected to affect, we are in the dark as to what the epm deployment is hoped to accomplish. At best, we must use guess work or hit-and-miss targeting to hope that we find a function in the system that somehow touches a business driver an executive is committed to.

No, using a Proof of Concept for this purpose may be one of the least effective manners of getting management to commit to epm. There are numerous other alternatives. If you want to get management to get on board with your epm system plans, here are a couple of suggestions:

One of my favorite methods is to do a facilitated vision session. Both senior management and the senior project personnel usually attend although sometimes not together. The beauty of this plan is that the investment in actual money is usually very small. A vision session for epm can usually be accomplished in a couple of days. There is often some preparation work and always some post-meeting documentation but still, the investment of both time and money is comparatively minimal. The other great thing about this strategy is that no matter who the ultimate system vendor is, the exercise with senior management is never lost time. Working through the business drivers for the epm system is a critical first step and, the worst case scenario is that management and the project personnel discover that they’re not ready to commit to epm at this time.

Another strategy is to use one of the many Executive Circle types of events for management to find out more about these systems. There are events of this type hosted by almost every epm system vendor but also from associations like the PMI or others. For example, the American Management Association has recently taken a more significant interest in project management and executive seminars by this group would be a great way to introduce the concept.

Visits to other organizations that already have epm where your executives can meet their executives are also tremendously effective in this area. Not only can your executives hear about how the epm system worked for another company but they are often quite gratified in being able to network and share concerns of all types. EPM system vendors can almost always broker this kind of visit for you.

Now, if you’ve gotten this far into the column and you’re a little ticked because you don’t actually fall into this category and your proof of concept plan falls into that 10% of cases where you do know what you want to prove, well fear not, I haven’t forgotten you.

If you actually are proceeding with a Proof of Concept, here are a couple of tips to making it successful.

First, make it a project. I know it sounds silly saying this to project people, but you’d be surprised how often this kind of thing isn’t managed as a project. Do everything you’d do with any other project. Have an executive sponsor, get a budget, make a schedule, have fixed objectives. If you don’t do this, you can end up with a proof of concept implementation that drags on for months or even years. (I’m not kidding. I’ve seen such implementations go on for more than 2 years!)

Next, make sure you have not only created some objectives but also some measurements that determine what will constitute a pass or fail of the implementation. Did it do what you needed? If so, move on. If not, once you’ve exhausted the avenues available to you in making it work, close that aspect down and get to the next step.

Regardless of whether you’re looking at convincing management or proving technical functionality, make sure your Proof of Concept has a beginning and an end. Remember to use the right tool for the job. If you are using a Proof of Concept as a tool, it works best at proving something specific rather than convincing someone of a general concept.

Hold onto your hats – Project2010 now arriving

Well, we told you it was coming and now the release of Microsoft Project and Project Server is just about upon us. On May 1st, Microsoft’s Enterprise clients and its partners will be able to access the official release of Microsoft Project 2010 and Microsoft Project Server 2010. While the official name of the products has dropped the word “Office” (It was quite a mouthful), both products continue to be part of the Microsoft Office family of products and will be released at the same time.

That’s not an accident.

Microsoft has made it quite clear that its strategy is to leverage client’s interest in one product line by tying it to another. The Office group has done this very successfully. You don’t think anymore of just buying Word or just buying Excel. No, you buy one of the bundles of Office and those products are contained within it.

Project and Project Server will continue to have their own licneces and aren’t available as part of any of the Office bubdles for now but there a bunch of other things that have changed.

Say bye-bye to Office Project Portfolio Server
One big change is how Microsoft has rolled some of the most popular Portfolio Selection functionality from Portfolio Server 2007 into Project Server 2010. There will not be a Portfolio Server 2010 product and, while existing users of Portfolio Server 2007 will see updates of that stand-alone product for some time, any new development on Portfolio management will focus on Project Server.

Did you say 32bit? Bite your tongue!
It’s all 64 all the time for server installations now. Installing Project Server will no longer be possible on a 32bit version of Windows Server. The new 64bit architecture makes a lot more memory available to Project Server but may cause some companies to take pause as the consider the costs of hardware.

Project Standard and Project Professional will still work on both 64bit and 32 bit versions of Windows Vista and Windows7.

Gotta love SharePoint
Previous versions of Project Server could be installed with either the free Windows SharePoint Services (WSS) of the licensed Microsoft Office SharePoint Server (MOSS). (Don’t you love how many acronyms we use in this industry?) Project Server will require users to have a MOSS license. The system will now longer support just the free WSS. This may be a license cost issue for some users.

So that’s the bad news. What’s the good news?
There’s good news in this release in a couple of areas. First of all, the underlying architecture of Project Server and Project Desktop hasn’t changed since 2007 and that will almost certainly mean the stability of the product at release time will be better than 2007 was.

Rather than rewriting the system’s architecture, there are new features that should be popular including Project Data Pages that allow Project data questionaires to be gathered online and then woven into a workflow, some improvements in the timesheet that will be great for those tring to do a full timesheet week and then update the project progress, some easier dashboarding tools that should allow people with intermediate skills to create great graphical representations of their data and, of course, the aforementioned Portfolio Selection functionality.

On the Desktop look for a distinction between Project Standard and Project Professional that will no doubt result in some kind of push to have people upgrade to the more powerful version.

Project Professional will have a new Timeline View and a new Team Management view which I think will go over well.
The scheduling tool will not default to automatic scheduling and you’ll be able to enter a description rather than a duration iin the duraton column as a sort of placeholder for the data to be entered later. Purists may howl at this. It’s another sign of how Microsoft Project has become so ubiquitous and how the product must now cater much more to non professional project managers.

Expect the Microsoft marketing beast to crank up to a fever pitch in the coming weeks. It’s already been moving forward at a pedestrian speed for months but as the product hits the market, Microsoft will do what it does best as it announces everything new in the Office 2010 family of products.

Resiloed – Matrix management gone awry

In a remarkable number of companies I visit, fledgling internal project organizations revert towards hierarchical structures. The organization becomes managed through the department or organizational structure and project managers become attached to that structure rather than an independent project office. The PMO devolves to a reporting function and in this modern era of cost cutting becomes a prime target for evaporating altogether.

How can this happen? After all, those of us in the project management industry tend to think of project management as most critical in difficult times and there is plenty of evidence to show how projectized organizations generate tremendous levels of efficiency.

To figure this out, it’s important to think of how organizations come to be in the first place.

Organizations usually get started from an entrepreneurial spirit. Company founders use family or trusted advisors to take over key elements of the company because it’s too big to do alone. This historic perspective continues into modern day accounting and business management. Organizations are traditionally organized by function, not by result. Look at any standard accounting chart of accounts and you’ll see department oriented thinking. Look at standard budgeting and it is department organized.

So department heads have traditionally had ultimate power, thinking of themselves as the heads of their own companies within companies. That’s great right up to the point where the products the company needs to create must cross these departments in order to be completed. And it is in answer to that challenge that today’s matrix organization came about.

The Matrix organization theory is relatively new but has been almost universally accepted in organizations that do projects. The idea is that projects move across departments. The power in the organization is now shared in some manner. Department heads are still responsible for their people but the project office is responsible for producing the result. Many modern day managers have grown up in world where they’ve only experienced Matrix structures in the organizations they’ve worked in. But, is matrix management automatically the most effective path? Opinions vary.

In a perfect world, the two axes of the matrix are intended to balance each other out. We’d expect that an organization that is completely oriented to the organizational axis of the matrix would have very happy employees who were well taken care of, highly trained and well compensated but would not get a lot of project throughput. Conversely, we’d expect that an organizations that is completely oriented to the project axis of the matrix would have lots of projects being completed but would be burning out staff on a regular basis, having lots of turnover and ultimately would lose its key resources. The idea of a matrix organization is that it is conflict by design. In a matrix organization we expect the two axis to push against each other with the intent being that we will find a perfect balance.

There are two big problems with this assumption however. First, not all people are created equal. Every organizations has some managers who are more articulate, more experience more forceful, more networked, better at negotiating and better at internal politics than others. Those people who are less experienced, less connected and ultimately less powerful will suffer by comparison. The second challenge is much more significant. The traditional position of the PMO is often one with lots of responsibility but little authority. Authority rests almost always with the organizational aspect of the matrix. So, when an organization is put under pressure (as happens in a tough economy) those people in authority are pressed to protect themselves. The result is often the exact opposite of what would be good for the organization.

In the economic pressures of the last couple of years, we’ve seen a number of organizations where the role of the PMO has been degraded. It’s exactly the opposite of what you’d expect in a challenging economy. We’d expect that this would be the time where the most throughput possible would be the incentive and, to be sure, in some companies that’s the case. But, in some organizations, the pressure on the organizational department managers is to grab back as much authority as possible. These groups create their own project management groups that are specific to their departments thus paying lip service to the project management requirement but at the same time removing the most powerful benefit to the organization overall: the ability to manage work across departments.

This is most common in the largest of organizations where the size of the matrix is tough to comprehend at the best of times. The end result is a weaker and weaker PMO until in the end, the PMO is, in some cases, dissolved completely in favour of “department PMO’s” which, of course manage projects only from a hierarchical perspective; that is to say, only for their own departments.

It’s a complete return to silo management by department.

If this is something you’re experiencing and you’re responsible for the project aspect of the matrix, there are a few things you can do.

First, market, market, market. Communications from the PMO are critical. We’ve always said this to our clients but selling the PMO shouldn’t ever stop once the PMO is established. An organization with lots of responsibility and little authority needs to lobby for its position forever. It’s just part of a good PMO. So, create a blog, a collaborative workspace or a website and start pitching your benefits to the organization straight away.

Next, keep your friends close but your enemies closer. It’s an old expression but it holds true today. Make sure your portfolio selection process or your project prioritization process includes those very department heads who feel they might become disenfranchised. Making sure that people in authority don’t feel left out can eliminate a huge risk to the PMO.

Finally, become Switzerland. Being the facilitator of portfolio meetings puts you one level beyond inter-department rivalry and makes you the neutral territory on which departments can negotiate. Having a neutral territory somewhere which prides itself on its neutrality rather than an organization which is looking for authority of its own is something everyone can commit to.

Maintaining a level of balance between the organizational and project axes of your matrix organization is essential if you don’t want to risk being returned to a silo organization.

Archive: April 28, 2010

The Keys to Key Performance Indicators

That Elusive Proof of Concept

Hold onto your hats – Project2010 now arriving

Resiloed – Matrix management gone awry