Posts tagged ‘Evidence-based HR’
Succeeding with workforce analytics is difficult. It requires a mix of skills not found in one person only, and you should not assume, that you can do it on your own. We are all decent at most things but really only good in a few. You should therefore assemble a team, which has a multiple of superheroes each with a superpower of their own.
I described this in a previous post, where I suggested six competencies a superhero analytics team should have:
- Strong data management skills
- Captivating storyteller
- Understand the business
- Ability to visualize your results
- Strong psychological skills
- Excellent statistics and numbers skills
But what happens if just one of those skills are not present? Can’t we manage anyway? My answer is no. If just one of the skills is missing from the team, six outcomes are possible – each with a disastrous outcome – as shown in the figure below:
In essence, if you:
- have no good data, you will not be able to perform analytics. It is as the old saying goes: crap in – crap out. If you do not have good data, it is sometimes better not to do analytics.
- lack of storytelling abilities, the message will nog. As Tom Davenport describes: “Narrative is the way we simplify and make sense of a complex world” and it is the way messages are most effectively conveyed and the best way to get people to change (which is the ultimate goal of analytics).
- have no business acumen will mean that your team will perform excellent analytics on the wrong issues. Workforce Analytics should help decision making on vital must win battles for your organization. Understanding the business is vital to understand what those must win battles are.
- are not able to do visualizations you will bore your audience. Data and numbers are boring (and I am a numbers guy), but data and numbers effectively conveyed through visualization
- lack psychological skills you will misunderstand your findings, be unable to convert your information to knowledge and be subject to important challenges such as bias, cognitive dissonance, imposter syndrome etc.
- have poor numbers and statistics abilities, your analysis will just be plain poor. You can get really far with simple regression-, factor- and t-test analysis skills but at other times, you will need skills in more advanced statistics when the data set become really big or you are looking for more predictive analysis.
Analytics require a lot of skills and abilities – superpowers if you like. The best way to ensure that you have the right ones to deliver on your task is to assemble the best team. An analytics superhero team.
Overconfidence is a term used in psychology to describe a person bias for being right. The overconfidence effect is a well-established bias and describes a person’s subjective confidence in his or her judgments is reliably greater than the objective accuracy of those judgments, especially when confidence is relatively high.
Daniel Kahneman writes in his epic book ‘thinking fast and slow’ that “neither the quantity nor the quality of the evidence count for much in subjective confidence. The confidence that individuals have in their beliefs depends mostly on the quality of the story they can tell about what they see, even if they see little…our associative system tends to settle on a coherent pattern of activation and suppress doubts and ambiguity”. What this means in practical terms is that one would assume that
How is this relevant to HR Analytics you may ask? In many important ways. Imagine that you present a fantastic ‘Predictive Employee Turnover Analysis’ to your boss who is the head of HR. You have collected a ton of data, used complex algorithms and found the following conclusion: female leaders at level 3 and 4 who have been performing among the 10% best leaders (at those levels) for three straight years and have not received a promotion are 60% likely to leave the organization within one year. Wow. And based upon this analysis you suggest a targeted effort towards that group of leaders. You calculate that this effort alone will save the organization more than $3m a year.
Your leader is truly impressed and now ask you how confident you are that your result is correct. That this group of employees are that likely of leaving and that they effort will save that much money. This is a fair question because your boss has to make a decision based in part on your findings. What do you respond?
According to the theory of the overconfidence effect, you are likely to say a very high number. Perhaps even 90%. In any case, the number will be much higher than an objective accuracy of likelihood. Frankly this number is much lower than we think it is. And this is a problem because your boss then makes a decision based on wrong input from you. If he thinks you are right, he will underestimate the risk of the project – the risk of being wrong – hence estimating a too high ROI.
Why will you make a higher estimate of the correctness of your finding? According to the theory and what I have experienced many times in reality is that because when we work with data over a stretch of time, as such a project mentioned above would imply, we are slowly developing a story in our minds. A working hypothesis if you will. And slowly we are only finding data which supports our hypothesis. This is a natural inclination. We confirm our hypothesis when we work with data.
The solution to this problem, which is quite a significant problem; always have a steering group for your projects with many types of people, continuously present your findings and assign someone to be the devil’s advocate and make it that persons responsibility to ask the stupid questions and challenge your beliefs and hypothesis’.
Overconfidence is by the way only one of many biases we carry with us and which work against us. A couple of others such as cognitive dissonance, illusory correlation, availability and anchoring. They are subtle but important and possible devastating to good analytics.
My main message here is as follows; we would like to believe that because we work with data and facts we then automatically make decisions that are more rational. We do not. We have a fantastic ability as human beings to disregard facts and make decisions based on irrational biases. If we can overcome this natural tendency then working with data and facts will make our decisions so much better. Exponentially better. But we must continue to work on our psychology.
How important is leadership for business success? Soccer may provide an answer. Workforce Analytics should do the rest
How important is leadership really for business success? It is obviously difficult to answer. Perhaps we could start by asking; how important are the top leaders for a company’s success? While more specific, this question is also difficult to give a brief and unambiguous answer to. However, we can begin to approach an answer by starting a completely different place; in the world of football (or soccer to you US readers).
Many sports have for several years been working quite intense with numbers, data and analysis in order to make the best decisions and to prove (or disprove) common myths in sports. Michael Lewis’ fantastic book “Moneyball”, which has been made into a movie with Brat Pitt in the lead role, describes how Billy Beane, the manager of the small team Oakland A’s used figures, facts and analysis to produce results in baseball beyond what one could and should expect. The same has happened to some degree over the past 10 years in football (soccer).
In their equally great book “Soccernomics” by Simon Kuper and Stefan Szymanski, the two authors show that the correlation between the wage expenditure of each club in England compared to average and the average league position is a staggering 91%. This is illustrated in the figure below from the same book. In the conclusion of the full study, the authors says that “the size of the wage bills explained a massive 92% of variation in the league positions, if you took each club’s average for the entire period”. In other words, if you know how much each club spends on salaries compared to the average, you can pretty much predict where in the table the team will finish on average.
If 92% of the variation of the league postion is explained alone by the salary it goes without saying that the rest do not mean that much. The rest in football (soccer) is a lot of things like training facilities, the size and quality of the field, tactics, medical staff, fans, coaching staff and of course the manager (which is equivalent to the CEO). Only some 8 variance percentage points are left.
And before the obvious point is raised; you cannot just take a random group of players and double their salaries and then win the league the following season. The high explanatory effect is present because there is a fundamental mechanism at play in international football; you know who the best players are and the best players get the highest pay. Over the last 10 years the transparency level has increase a lot about how good a player is and therefore the salary that player deserves. In the business world, there is also a significant difference between different CEO’s pay but it is more questionable if the best are also the ones who are paid the most and visa versa.
If 92% is explained chiefly by the level of the total remuneration of the squad, you can – with a couple of assumptions – measure how good the managers are relative to each other. Kuper and Szymanski make that analysis by measuring managers are at achieving the positioning in relation to the statistical position that the team should reach based on the total amount of wages they pay. If you take a squad of players who statistically should finish in 6th place, but due to the manager’s motivation, tactics, gut feeling, management and everything else can get the team to end up in 3rd place, then you can conclude that the manager performed better than expected. This analysis shows that there are some football (soccer) managers who have performed on that metric much better than others. These include Bob Paisley, Bobby Robson, Alex Ferguson and Arsene Wenger. In those and other cases, one can statistically prove that the coach has added value, by how much and how consistent.
So to summarize; the manager of a professional football (soccer) club does not matter that much and the game is best explained as Kuper and Szymanski cites Jamie Carragher (a British footballer) for; “The bottom line is this: if you assemble a squad of players with talent and the right attitude and character, you will win more football matches than you lose, no matter how inventive your training sessions, what system you play or what team-talk you make.”. However some managers are significantly better than others and this can be measured and evaluated.
What about manages, leaders and CEO’s of companies? Is the same true as with football (soccer), that the impact of the leadership team has little explanatory power in relation to the overall business performance? That it is ‘just’ about finding the right employees? We must take an evidence-based approach. Which is not without problems.
I believe that there are many reasons why the wage of the employees (or talent as popular lingo is) do not predict a company’s performance to the same extend as is the case in football (soccer). Firstly, the fundamental mechanism around pay is not as efficient in companies as in global sports; the transparency of global talent is lower and it is not always the case that the best person gets paid the most. Performance Management systems are simply not that effective and efficient. At the same time, I expect that the IT infrastructure, processes, products and brand contributes significantly more to business success than it is the case with a football clubs – although it is probably much lower than business books in general assume.
So the question is how much? How much does a leadership team mean for the business outcome. I would like for Workforce Analytics to come up with the answer to that. We should have the data. We have the software. We have the clever data people. But do we have the insight and the incentive to find the right answer?
Analytics is not easy. Or to be clear; getting the most business value from your data is not easy. There is so much to get right before you can unlock the hidden gems which are unquestionable lying deep inside your databases.
Just consider the journey: First you must work on something which is highly important and valuable to the business. Then you must use all available data to make analysis which gives new insights and knowledge upon which decisions can be made. Finally, and most importantly, you must convince the decision makers to make these decisions based upon your analysis and to do that you must present it right, show them the value, consequences and risk of failure. Only then will your work bear any fruit.
But be careful – don’t assume that decision makers will believe you capable of doing all of that on your own. This reminds me when my daughter was younger, I one day tried to convince her, that I was a superhero. She verged on believing me but when I told her that I could do anything, she said “Now I know you are not telling the truth, dad. Everybody knows that superheros only have one superpower”.
I was not able to fool my daughter and neither will you be able to convince your head of HR or your CEO, that you possess all workforce analytics abilities at expert level.
I therefore propose that you assemble a team for you analytics which has a multiple of superheroes each with a superpower of their own. Specifically I suggest six competencies (in random order);
1. Excellent statistics and numbers skills
There is no getting around that good analytics requires excellent statistics and numbers skills. You can get far with doing simple regression-, factor- and t-test analysis but at other times, you will need skills in more advanced statistics when the data set become really big or you are looking for more predictive analysis.
2. Strong data management skills
Let’s be frank; you will get nowhere in your analytics journey if your data is not clean, good and have a strong governance structure around it. Those and many other data management issues are essential for good analytics. For some it is mundane work, for others it is a passion. If it is the former for you, get somebody on board for whom it is a passion. It is that important.
3. Captivating storyteller
Analytics – even predictive – will only add value if a decision is made on the back of it. It sounds trivial, but data does not speak for itself and to move a decision maker into making a decision you must create a compelling story around it. Sounds easy? For some it is for others it is not. Find somebody who does this well. It will make a big difference to the value of your analysis.
4. Ability to visualize your results
Studies on ex. cognitive load show that if you give a decision makers too much data, they will either not make any decision or make the wrong one. Visualization techniques is a powerful tool to present complex data in a simple and easy-to-understand way. This is not about making your pie charts 3D. It is a whole different category and an art more than a skill.
5. Strong psychological skills
There are so many reasons why I feel that strong psychological skills may be the most essential of all six skills. Just to name two reasons here; it is partly because you will understand how to make more impact with you data if you understand terms such as cognitive dissonance, bias, over-conficence etc. And also because your data has not meaning if you don’t understand how to convert information to knowledge which in essence requires a deep understanding of psychology.
6. Understand the business
A final skill which I find is most often not present as much as it should is the simple but powerful skill of actually understanding the business. This requires you to fully understand what is the customer value proposition is, what the strategy is (the must win battles), key differentiating factors, financial situation and more. I mean really understand the business.
Analytics require a lot of skills and abilities – superpowers if you like. The best way to ensure that you have the right ones to deliver on your task is to assemble the best team. An analytics superhero team.
If you read any article, book or blog post about workforce analytics, HR data or evidence based HR you will eventually read the following sentence “…but remember that correlation is not the same as causality”. And this is factually correct. Just because two items correlate does not mean that they are connected or that there is causality.
The problem is that it does not help you with your communication with top management or your recommendations for action. Take this example;
Imagine that you have worked really hard with your analytics project, and you have build a model which shows connections between various training programs and profits. You identify that one particular training program correlate very strongly with profits. You then produce your fine PowerPoint slides and present your findings to the management board. You say “I have identified this strong correlation between this particular program and bottom line profits BUT as you all know, correlation does not mean causality”. The managers look at each other and one say “So what do you say, should we roll this program out to all of our sales people or not?”. You look confused down in your papers and then respond “Well, I can’t actually say for sure that attending this training program will make your sales people produce higher profits or if it is the sales people who are very profitable who happens to attend this particular training program for unknown reasons. The only thing I can say is that the two variables appear to correlate”. The meeting ends and you wonder why your next analytics project don’t get funded.
You see the problem? On one hand you are right. You cannot for sure say which way the arrow turns between the two variables. But on the other hand the management board does not care about correlations (nor should they), they only care about solid and value-added recommendations.
What to do? My advise is; pretend that correlation is the same as causality. Test your hypothesis with qualitative data and present a recommendation based on your findings. So in the above example, interview five sales managers and ask them about their view on the causality; do they believe that the particular training program has led to significant increase in profits from the sales people attending the program? If it has they that’s it. I know it is not ‘correct’ or ‘true’ but it is good enough. And most likely you will be right.
I know there are a lot of fancy and even simple ways to test the hypothesis with time-lagging-tests or blind-tests or something similar. I have just experience a lot of places where even that kind of tests are not possible, too expensive or not available due to poor data. Make it easy. Do a qualitative test and present your findings as if correlation is the same as causality
(just don’t tell anybody I told you so – this approach is not looked well upon in analytics circles. Leaders don’t mind – trust me)
The hype surrounding Workforce analytics, metrics and Big Data in HR has really increased over the last 12 months. Every conference, article, blog and strategic initiative is filled with buzz words around data and fact-based HR. So much so that Workforce analytics now is in danger of overselling itself. To outsiders it may appear that HR is becoming more evidence-based in its approach. Unfortunately this is not the case.
When I completed my master in psychology 10 years ago I read a book called “Evidence-Based Practices in Mental Health” by Norcross, Beutler and Levant. It is a great book, which argues for a more evidence-based approach for psychology. Because to be frank, it really isn’t that evidence based. Take an example. If a person has mild depression there are many potential approaches to take. Lets take just a few; therapy (cognitive), medication (ex. Prozac), therapy (behavioral), meditation, physical exercise, therapy (psychodynamic), mindfulness therapy, self-help books and many more. You would think – and hope – that the advise and subsequent treatment this person would get would be based upon evidence. What works. For example, there is a lot of evidence which suggests that everything else being equal that cognitive behavioral therapy is significantly more effective than both medication and psychodynamic therapy for treating mild depression on its own. But if a patient happens to stumble upon a therapist who focus primarily on psychodynamic therapy – that’s what the patient will get. Psychology is a lot of things, but evidence based it is not.
With HR it is the same. The options we chose and our design of solutions are not based upon evidence but instead on intuition, personal preferences and habits. And often not the right ones. This is very problematic and with HR there is the added problem that we still don’t even know what works (in medicine and health care some evidence is available). What works and what works best are two questions w cannot answer.
Why is HR not more evidence based? I think there are four reasons;
- We don’t share data. There is too little data and evidence out there. Some is being produced but very little shared. A study from 2006 published in American Psychologist, showed that almost three-quarters of researchers who had published a paper in a high-impact psychology journal had not shared their data. This is not just an issue at universities. When I see Google and other leading companies in Workforce Analytics talk about great findings they never share their data. At conference when speakers talk about their great internal studies they never share data. And frankly it is not that sensitive. It really isn’t. I hope they don’t share because they wrongly believe their data is sensitive rather than the studies are really not that good. We must produce better evidence.
- Lack of the right competencies. Working with EB-HR mean that you have to understand what evidence is and how to get it. Many in HR wrongly believe that evidence means 100% certainty or proof of something. That is not correct. Evidence is always about probabilities and assumptions. Always. Even in natural science. Also, it is also not just about quantitative data but also includes more fluffy things such as qualitative data (my favorite) and experience. But too few Being able to design and implement a executive leadership program using an evidence based approach is something too few in HR can do.
- Lack of the right mindset. As with EB-mental health, many in HR don’t really know why this is important. “We have a talent program, it works, people are happy about it and talents are staying at the company, why should we used another approach to our program?”. While it may sound tempting to think like this – and most in HR do – it is missing the point completely. HR must – as any other support function or organization – be as effective and efficient as possible. The only way it is possible to tell if HR is that is to measure and use evidence to improve. The only way!
- We can get away with not being evidence based. Our primary stakeholders (managers, employees and shareholders) do not demand us to be evidence based. We can many times get away with presenting a talent management program with little or no proof that this is the best way to develop talents.
BUT it is not all bad. I think there are many small movements which suggests that the interest is evidence-based HR is growing, our knowledge of what works in HR is also improving, we have the technology to help us find facts, our basic data is better and there are more people with a broader mindset entering HR. Perhaps things will change. But for now please don’t pretend that HR is evidence-based, because it is not.
Tom Peters gave an excellent advise when he in 1982 said ‘Formula for success: under promise and over deliver’. This goes very much hand-in-hand with the truism in service management (and is HR not one big service organization?) that excellent service = customer perception – customer expectations. The more you raise the expectations the harder it is to deliver excellent service.
My point is; deliver the best product and service possible but just don’t over promise.
The field of Workforce Analytics is in danger of over selling itself. I think stating something like “HR has good data but needs to use them better in order to make better decisions” is a sensible statement and does not over-promise. It is a sort of statement which can lead to a discussion about which decisions we make from pure guess-work (most) and which decisions are so important that we need good data and clever analysis to improve those decisions.
What I do think HR, consultants and vendors should stop saying are things such as:
- “I believe that well thought out predictive HR analytics could become as important to the CEO as the balance sheet and P&L statement,”
- “Workforce Analytics can measure these things [engagement, profit, turnover and customer service] and show a direct, causal relationship to prove the bottom line impact”
- “Workforce analytics will enable us to reach our goal which is to reinvent work”
- “Workforce analytics will provide the language for HR executives to speak the language of business: numbers. This will give HR a seat at the table”
Many of these statements come from people who have a vested interest in the hype surrounding workforce analytics. People such as myself – consultants and vendors. But they are actually doing a disservice to the whole field.
Workforce analytics will contribute a lot over the coming years; it will make use of big data, provide new insights about the workforce and make HR more efficient and effective. But it is not a game-changer. It is not something radically new. And its limitations lies not with the software and the tools themselves – they lie with the mindset of the HR executive, the biases of the people using it and with the level of trust HR has within the organization.
But to get to the place where we can add the value from Workforce Analytics we need to stop over-selling it.