Posts filed under ‘Measuring HR’

The problem with ‘evidence’ in HR

I admit it: I like the evidence based approach to HR. I like it because it appeals to my sense of trying to do the right thing in the right way. And I like the idea of using other people’s findings and applying them to my own work.

The evidence based approach essentially tries to apply scientific standards of causality and evidence to demonstrate how intangible human capital can be observed and shown to add tangible business results. It questions common beliefs and demands that the practitioner use methods which is proven to work. Bingo. It is an approach taken from Evidence Based Medicine (EBM), which has been around for more than 20 years.

Evidence in HR

The problem I have found with a lot of the “evidence” in HR – especially in emerging areas (such as HR analytics) – is that it is produced by people who have a vested interest in the results.

Imagine for example that you were reading a study, which concludes that a particular drug is really effective. The article highlights all the benefits (and a couple of irrelevant disadvantages) and shows how this is a new important step in curing this particular illness. It compares  the product with similar products and shows that the effect is much better. Patients report higher satisfaction levels.  Everybody is happy.

Would it make a difference if the study had been made by the producers of the drug?  Of course it would. You would question the findings. You wouldn’t necessarily think that they were lying or making up stuff. You just wouldn’t trust it completely. And so you shouldn’t. The conflict of interest is obvious.

This is a major problem in HR. Take talent management. I have seen many studies showing high ROI on TM, correlation studies which show that companies with great TM processes have higher profit, lower employee turnover and higher engagement levels. I have seen studies showing TM leads to higher market share etc. The problem is that all of these studies are made by people and companies who have a clear interest in the results; consultants, software vendors, authors/publishers etc. I have only seen very few really good research (double-blinded, longitudinal etc.) about Talent Management.

Why is that? I believe there are three primary reasons;

  1. It takes time (and resources) to make great research. A great piece of double-blinded, longitudinal research takes many years to produce. Unfortunately many don’t have the patience for this.
  2. There is too little. Because clients like data and research and there is too little available from neutral researchers, people and companies with vested interest will have to produce it themselves.
  3. The conclusions are ‘better’ if produced by consultants. When consultants conclude something it goes something like this “we can conclude that better talent processes lead to the creation of significant shareholder value”. When a university professor concludes something, it goes something like this: “Our research point to several processes which may benefit the talent as well as the company but the effect depends on the context and other unknown factors”. Which one sells?

The fact that much of the evidence is produced by people with a vested interest does not make the studies or conclusions wrong. They may be as correct as they come. The problem is that as long as there is too little ‘proper’ evidence it is difficult to really trust it.

This is a problem because if we want people outside of this profession to take us serious – such as CEOs and CFOs – we must be able to show them data which cannot be questioned. So lets make those surveys and that data which can do that.

26/11/2012 at 20:56 5 comments

When bad data happen to good HR people!

You are an HR executive and you are sitting in your monthly strategy meeting with the top executives of your organization. You are about to present your monthly workforce data and updates on your KPI’s and strategic initiatives. This is the moment when the people who matters are really listening to you. You have the floor. But you dread this moment, because you know that they know that the data is bad. You hand out the status reports and you begin your presentation. And everybody in the room – including yourself – are thinking the same thing: “this report looks nice, but we don’t trust the data. It just doesn’t look and feel right”. And actually you know that the data isn’t right, but it is the best that you and your analytics department can find. It is not that the data is completely wrong – it is just not right. But you know that if you were to try to make the data right it would require so much work and resources. Resources you don’t have. So you continue with your presentation and hope that nobody asks. They usually don’t.

This is a problem for many in HR (as well as all other functions). You are using workforce data which doesn’t quite feel right but you use it because that is what is available and you or your analytics department have no alternative.

Bad HR analytics data

Bad data is a big problem and it affects every part of an organization, from sales to HR. Many studies have shown that bad data quality cost a lot for organizations. A Gartner survey revealed that in US:

  • 140 companies surveyed lost an average of $8.2M annually due to bad data
  • 30 companies surveyed estimated their losses at $20M
  • 6 companies surveyed estimated their losses to be more than $100M annually

Why does it cost so much? There are three reasons; bad data quality lead to bad decision, bad processes and ultimately bad data can lead to mistrust with your customers.

All organizations have bad data to some degree, but it seems that some have much more than others. There are four reasons behind why bad data happens:

  1. Lack of a coherent data strategy. Having a purely operational approach to your data is probably the biggest reason behind bad data. Data is suppose to support you in your strategic decision making and a lack of a coherent data strategy to support your organization’s strategy means that you approach becomes random and often meaningless.
  2. Assume that analytics software is the answer. HR analytics software is great, but it is simply just another data collection tool, albeit one with more potential than most. To get the most out of HR analytics you must go through a strategic data process and decide what data is of strategic importance to you and how they ideally look like.
  3. Garbage in, garbage out. This one is often overlooked although it should be clear to everyone. Your data is only as good as the component inputs.
  4. Lack of critical data sources. While the quality of the data is critical, what data sources you incorporate is equally important.

So how to avoid bad data? Thomas Redman, writes in his blog that “data creators must create data correctly, the first time, with full understanding of what that means to customers, those who use data they create. Data customers must communicate their data requirements to sources of data, and they must provide feedback when data are wrong. Virtually everyone recognizes they are at once data creators and data customers. There is, of course, a lot more to data quality management. But let’s not make this any more complicated than it needs to be.”

I agree. It is not that difficult, but once you are using bad data in your reporting, ROI’s and updates it is so difficult to change it for primarily psychological and political reasons. So 1) get it right first time and 2) when you observe bad data correct it immediately.

Ultimately, I believe that many executives are sitting with status reports, KPI’s and ROI calculations based upon bad data. And many know this to be true but it takes too much effort to correct it. The bad news is that you have to change it. There is simply no excuse to continue to use bad data. If you don’t have the resources to make them better at least stop using them. The good news is that good data makes a big difference. The quality of the decisions, processes and programs based on good data is worth so much more than it cost to find them – even if it means that you have to change many existing processes and disregard existing data sources. You (and your data) will be so much more trustworthy.

So what to do when bad data happen to good HR people? Fix it.

20/08/2012 at 16:47 13 comments

Why HR KPIs still matter but why they still fail to deliver

How to use KPIs in HR

Key Performance Indicators – or KPIs to everyone these days – continue to be a hotly debated topic in HR. There are those who are simply against them. They see no use of them. This group is divided into two sub-groups; those who simply don’t belive that you can or should attempt to put figures, numbers or monetary value on people. Figures and charts say nothing about people is the argument. To them, it does not matter  how well they are formulated or how they are used – they simply don’t like numbers, metrics and KPIs on people related matters at all.

The second sub-group in this first group is a mix of some HR analytics, some senior HR executives and some top management executives. Their argument against KPIs is that it just doesn’t work in practice. The senior HR executives have used KPIs for so long without success that they now disregard them. The top management has seen HR use KPIs but still feel that HR has not delivered over the years. And some in HR analytics believe that it is analytics and not metrics and/or KPI’s which is the answer.

The second group of people believe KPIs can add significant value.  This group also have two sub-groups; one believe that KPI’s are they answer to all and everything and use them as much as possible. In fact, their departments are almost entirely run by KPIs.

The second sub-group of this second group also believe that KPIs can add significant value but with a caveat namely that they should be used very carefully, only in small doses and if in doubt not at all. I admit that I belong to this group . I believe that in 95% of all organizations, HR KPIs are truly ugly (see definition) – non-strategic, there are too many of them, they are poorly measured and with no real impact and frankly they have not been given the attention they should. So I am left with a strange admission; I belief HR KPIs are a great and effective tool but I have very little evidence to back up this claim as I see as many if not more evidence in practice of the opposite.

I don’t think you can come up with clear definitive rules as to how to make good KPIs. They simply have to be adapted to each organization – what may work in one may not in another. However, I do believe that the best HR KPI’s I have seen follow these rules of thumb:

  • Aligned with the strategy and business plan of the organisation. The targets of the HR KPI should be linked directly to the strategy of the organisation
  • Personally owned. The HR KPI should be owned in two ways; firstly it should be linked to one person who is accountable for its success. This means that it is not falling between roles and people can argue about fault etc. Secondly the HR KPI should be meaningful for that person.
  • Actionable. Every HR KPI should have a project or a set of actions which will lead to meeting the target. It should be within the circle of influence.
  • Well defined. Every KPI should be precisely defined. An exact definition, which data are involved, where the data is collected from and delivered by whom. It should be formulated in a way so an outsider will be able to look at it and find the result.
  • Relevant. It must be relevant in the specific context of this HR department in this particular company.
  • Timely. There must be a specific time when the target should be met.
  • End KPIs (compared to Mean KPIs). Consider a KPI which is about the number of people who had an annual review. This is a classic ‘mean’ goal. It is not an end in itself to hold annual reviews. It is the desired results of the annual review which are interesting. All HR KPI targets should be end-goals not mean-goals.
  • Predictive (i.e. leading indicators).
  • Few.  It is better to meet the target of three or four  KPIs than to meet six of ten. When you have too many KPIs you tend to select the ones you feel are the ones to meet and consciously or unconsciously  not even try to meet the others. This subjective section of KPIs are bad for an organisation. Better select a few and meet them all.
  • Linked to bonus. It should make a difference to the person if he/she meets the HR KPI target or not.

So why are most HR KPIs so bad? I think it is down to three things; 1) Some just copy from other organizations, consultants or books. As I stated above – it simply doesn’t work. 2) Some don’t think strategic. Only strategic KPI’s will ever have the chance of being good. Unfortunately, creating strategic KPIs is more difficult than non-strategic ones. 3) They don’t matter. If there is no consequence of meeting the KPI target they will not have any effect.

My conclusion about HR KPIs is that it is a powerful tool to manage an HR department. However they still fail and do not deliver their promise and consequently are increasingly getting a bad reputation because they are poorly formulated and used. In practice this happens in most places. I believe that bad HR KPIs are a lot worse than not having any at all. Take the time to formulate a few really outstanding ones or don’t use them at all. The problem is that KPI’s do work in as much as people tend to follow them. If they are badly formulated employees will behave equally bad.

25/07/2012 at 11:43 7 comments

HR should measure against the pacebo effect – you will be surprised…

I believe HR can learn a lot from psychology – not just in terms of how to develop and manage people but also how to think about its own existence, which activities to do and approach to take. For example, I have recently argued that the concept of cognitive dissonance could make us understand why we (HR) make poor decisions even face with good data and what implications this may have on HR analytics.

Another psychological concept – the placebo effect – is useful to consider. I believe it should be the benchmark for all HR activities, and that HR should measure some of its activities against the placebo effect once in a while.

Measure HCM against placebo effect

The placebo effect can be defined as “the physiological or psychological response to an inert substance or procedure”. This means that you can give somebody an inactive stimuli or treatment and it can have an effect. For instance, if you give a sugar pill (which has no effect) to a person with a headache and tell him that it is a pill which relieves pain for headache (such as aspirin), he is likely to experience no or a lower level of headache despite the fact that the pill has no physical impact.

The placebo effect has been proven in many experiments and not just medicine. One experiment measured the response of humans when under the influence of alcohol. Some subjects were given successive doses of alcohol and their responses were measured after each dose. Other subjects were instead given a placebo meant which mimicked the taste of alcohol and they were told that they were drinking alcohol. As expected, the group that was given actual alcohol exhibited signs of drunkenness and lack of coordination. What was surprising was that the placebo group exhibited the same signs, with some even seeming drunk. It seemed that the mere suggestion of drinking alcohol produced inebriated behavior.

Why does the inactive pill, the fake alcohol and other placebos work? First of all, for the placebo effect to occur, the subject must believe that he/she is given effective treatment and that it must be suggested that the treatment is effective. It works probably as a result of classical conditioning – people are conditioned to associate a particular stimulus with a particular response. Another reason may be that they are more motivated to feel better and which to cooperate with an experimenter.

Placebos are highly used in medical research. In fact the FDA will not approve any new drug unless it can show a significant effect over and above the placebo effect – something many consider to be the biggest barrier for the approval of new medicine. Why not set the same criteria for HR? Why not test HR activities against the placebo effect before they are approved internally?

Let me propose a few examples

  • Recruitment – an expensive recruitment process with multiple tests and many rounds of interviews must produce a better job/person fit, higher performance and lower new employee turnover than a placebo recruitment process
  • Coaching – an expensive coach using the right techniques must be able to deliver a better result than a placebo coach
  • Teambuilding – a teambuilding program promise to create better performance, fewer conflicts and lower employee turnover. But such an event is expensive. It should be evaluated against a (very cheap) placebo event

The problem with experiments like these are of course the ethical aspect – you just cannot do experiments on people without their consent. However you can do experiments and tests which are ethical, easy and valuable.

HR activities should be effective, they must be measured and their performance/effectiveness must be better than the placebo effect. Lets measure smarter.

22/06/2012 at 12:01 1 comment

Top 5 Talent Management KPI’s

I was reading an excellent White Paper by among others Jeff Higgins from Human Capital Management Institute which is called “Top Five Metrics for Workforce Analytics“. The White Paper lists five metrics of which one of them is an index they call ‘Talent Management Index’ – something which alerts me when I read it.

While the report does not go into too much detail, it does outline their suggestions for top 5 metrics for Talent Management.

That got me thinking about my suggestion for top five metrics for Talent Management. I would argue that you really shouldn’t have more than five at the most. KPI’s can overwhelm you and they must be used with care (see here for the pitfalls of using HR KPI’s). My suggestions are;

  1. Talent Retention (this is more positive than its negative cousin ‘Talent Turnover’). This is measured by taking the number of identified talents leaving the company during the year divided by the number of identified talents at the beginning of the year. For me, retention rates are not always very interesting. Many times you do want to get rid of low performers and you should be able to that without messing up your Talent Management KPI’s. But it must be imperative to keep talents – otherwise they shouldn’t be identified as such.
  2. Talent Performance. This is measured by taking the performance score for your identified talents from your Performance Management System. An effective talent program should be able to develop talents in such a way that their performance score improves.
  3. Time to Hire for Critical Roles. This one a bit tricky, because this KPI actually can work against you. The best candidate for a critical role may not be the one which is just available (there may be a 3 month notice period). However, a successful Talent Management program should be able to fill critical roles quicker from within. This should be measured.
  4. Skills gab filling process. This is measured by taking your talents and measure their talent gab at the beginning and at the end of the year. The talent gap will be individual from company to company but be based upon your individual assessment made at the beginning of the year. It may also come from your Succession Planning tool. In any event you must have some way of measuring how well the skills gab is changing.
  5. Talent Engagement Levels. This is measured by taking the engagement survey and identifying the level for your talents.

I have written many times, that I think Talent Management is the most important strategic process for HR to get right. This is simply because the potential pay-off is phenomenal compared to many other HR processes.  Therefore it is too important not to measure and evaluate properly. Let’s get that right.

I will be very interested in hearing about alternatives to these Talent Management KPI’s. Which ones do you used and how do they work for you?

30/04/2012 at 20:21 15 comments

HR KPIs: The good, the bad and the ugly

This week I have had meetings with two HR executives about their HR KPIs (Key Performance Indicators).  They both complained about two things I hear a lot; 1) we have too many and 2) they are not very good. While recognising that they have too many they both found it hard to actually get rid of any. At the same time, while they felt that they were not pushing the business in the right direction, they didn’t know what the difference between a good and a bad HR KPI is.

The problem with KPIs is that they actually work most times. That is, if you start to measure people in certain ways and you link their pay to meeting those measures they will in most cases try to meet these goals (KPIs) at the cost of other things.  This is a problem because there is always a consequence with any KPI and if they are the wrong ones the consequences may be dire.

This can be simply illustrated here:

  • If you are in the recruitment department and your KPI is ‘Time To Fill’, you will be focused on filling a position with a person who may not be right for the job as long as it is done fast. The right person may have a three month notice period whereas another (not fitting the job as well) may only have one month notice period.
  • If your KPI is ‘Performance of the new employee after 6 months’ you will want to spend (a significant) extra time and money on finding the right person who can deliver performance fast, which means that the position may be vacant for many months thus disrupting the workplace in the meantime.
  • If your KPI is ‘Cost Per Hire’, you may use the cheapest channels, the fastest processes with the cheapest personality-tests because this will make you hit your target. The result may be that is the not the right person but it is the cheapest hire. Maybe you need to buy out the right person but this will ruin your KPI

 

The Good HR KPI
The good ones (i.e. the best ones) are the ones which are
  • Aligned with the strategy and business plan of the organisation. The targets of the HR KPI should be linked directly to the strategy of the organisation
  • Personally owned. The HR KPI should be owned in two ways; firstly it should be linked to one person who is accountable for its success. This means that it is not falling between roles and people can argue about fault etc. Secondly the HR KPI should be meaningful for that person.
  • Actionable. Every HR KPI should have a project or a set of actions which will lead to meeting the target. It should be within the circle of influence.
  • Well defined. Every KPI should be precisely defined. An exact definition, which data are involved, where the data is collected from and delivered by whom. It should be formulated in a way so an outsider will be able to look at it and find the result.
  • Relevant. It must be relevant in the specific context of this HR department in this particular company.
  • Timely. There must be a specific time when the target should be met.
  • End KPIs (compared to Mean KPIs). Consider a KPI which is about the number of people who had an annual review. This is a classic ‘mean’ goal. It is not an end in itself to hold annual reviews. It is the desired results of the annual review which are interesting. All HR KPI targets should be end-goals not mean-goals.
  • Predictive (i.e. leading indicators). Meeting the target of the KPI should lead to meeting business goals.
  • Few.  It is better to meet the target of five of five  KPI’s than to meet six of ten. When you have too many KPI’s you tend to select the ones you feel are the ones to meet and consciously or unconsciously  not even try to meet the others. This subjective section of KPI’s are bad for an organisation. Better select a few and meet them all.
  • Linked to bonus. It should make a difference to the person if he/she meets the HR KPI target or not.

The Bad HR KPI
Bad ones are the ones which appear good because they follow the characterises of good KPI but they are not strategic and relevant. So at the end of the year you and the rest of the organisation congratulate yourself on meeting your targets only to discover that you failed to deliver on your strategy.

You may have a really well defined KPI on recruitment, but if your issue is a high turnover of talented employees your KPI may be well defined, actionable, personally owned etc. but it is not relevant nor strategic.

The Ugly HR KPI
The truly ugly ones are the ones which are not strategic, relevant or cannot even be measured. An example could be:

  • Title: Most managers perform annual reviews
  • Description: % of managers who perform annual review
  • Target: Higher than last year

The trouble with this one is that it is a mean-KPI (see above), it is poorly defined and most likely not relevant. This is an ugly HR KPI.

It is not difficult to find HR KPI’s (see for example this library). It is a little more difficult to define them well and have processes in place to meet them . But this is certainly something most can do. But it requires work to make the to be one of The Good ones.

30/03/2012 at 12:46 12 comments

5 things to avoid when measuring HR

Measuring HR is important because it is the only way to make HR more effective and efficient. The only way! It does a lot of other things too – such as creating an objective tool to making better investment decisions and it makes HR more credible as a strategic partner for the rest of the business.

The evaluation method must be objective and tangible and preferable use tools such as HR analytics.

KPI’s are important but ultimately I believe that all strategic HR initiatives should be measured using an ROI tool. If you want to work strategic, you need a tool which can tell you if you are creating value. ROI does that. It is not without complications (read about ROI dangers) but I think it is the most appropriate one to use.

There are five common pitfalls when measuring HR using ROI:

  1. Too complicated. Be careful about making this too complex. Avoid lots of paperwork and too many meetings. ROI is a simple tool. It requires careful thinking about its assumption and what goes into the equation, but it is not complicated.
  2. No before measurement. Evaluation requires before and after measurements. If you only measure after the activity has taken place you don’t get the full picture and you cannot assess the value creation (here are 3 reasons why not to measure retrospectively).
  3. Using a standard system. Every organisation is unique – don’t use one-fit-all template. While it is tempting to copy one from another department or company it simply will not work.
  4. Making measuring HR an end in itself. To measure is not the end – it is a mean. Measuring HR is a mean to create better HR. Sometimes ROI measurements can develop a life on its own and itself become the purpose of the project (read here for why measuring HR is not an end).
  5. Measure too much. I believe you should only measure a few initiatives. Those which mattes and which have an impact on strategy and overall efficiency. ROI measurements are laborious but potentially very value added. Only measure the few parameters which matters. Apply the 80/20-rule

Measuring HR is an important element of working with HR. Measuring HR is not complicated but also not easy. It requires strategic thinking, good data and a willingness to use the results.

06/01/2012 at 12:27 2 comments

Newer Posts


Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 1,803 other followers

Latest Tweets

Feeds


%d bloggers like this: