People + Strategy Journal

Winter 2022

The Big Question

People + Strategy asked three leaders who have first-hand experience working on data strategies for their organizations to share their lessons learned about the limitations of data.

The Big Question

​What Are the Limitations of Data?

There is no disputing that data is, and will continue to be, a powerful tool for organizations to capture insights that will help them refine their strategies for building strong cultures and better serving their customers. But data is not always the silver-bullet answer that many people hope. We asked three leaders who have first-hand experience working on data strategies for their organizations to share their lessons learned about the limitations of data. 

Understanding the Human Side of Analytics

Like every industry, HR is awash with data. Workforce analytics such as time-to-hire, retention, demographic, performance and survey data help chief people officers and chief HR officers prioritize, but it can’t be our only guide. Quantitative data has its limitations and you can’t automate everything. For example, you can’t use artificial intelligence to do performance reviews or for all facets of the hiring process. There is so much nuance around people—their motivations and reactions—that you will miss all kinds of things if you try to make those decisions exclusively based on AI’s understanding.

I’ve seen this firsthand. At a previous company where I worked, we were trying to improve our performance review process, and a Ph.D. on the team who took a very data-driven approach to his work, said, “Let me tackle that.” He came back to me 30 days later and said, “I can’t do it.” I wasn’t surprised. I know there are a number of different companies and organizations that are attempting something similar, but you can only get so far with it because it’s hard to completely understand employee motivation by looking at data alone.

It’s important for organizations to be clear about how data can be used to make decisions. Hiring is a good example. At InfluxData, for example, we don’t use a traditional coding test when we’re hiring developers. We’ve broadened that into an exercise that captures not only their coding skills, but also their ability to collaborate. Because we all know that, from a developer perspective, the days of sitting back and shutting the door and not talking to anybody for three days while you’re working on something are long over. To work in an agile environment, you need to build collaboration skills, and that’s difficult to capture on a test.

Another challenge for HR is that it can be hard to find people who can gather data and build the systems you need to create a great dashboard. Those skills are not readily available in the HR community. That’s changing, but when you’re in an organization like mine, with about 200 employees, you don’t have the luxury to have somebody on your team dedicated to that. There are new systems and tools out there that will help but they take a big investment of time and expertise to make sure they work well.

My focus is on building a dashboard that will provide a general view of the culture to the leadership team as well as the board. Some of the data points we focus on include the attrition rate. Any changes from month to month may tell me very little, but if I look at the trend over a quarter or a year, then you can see some patterns and understand where you need more data.

Another important area, which can be difficult to put your finger on, is recruiting data. It’s still a challenge to get reliable data about cost per hire and how long it takes to hire somebody, but that data is useful for everyone on the leadership team to know how quickly we are filling these roles. It’s surprising to me that the data around hiring is not as clean as it could be even at this point. 

And we need those insights quickly. If I’m suddenly asked to hire 100 more people, how do I know how many recruiters I need in order to do that? It’s critical to understand the load for each recruiter because recruiters themselves are hard to hire right now. You want to keep them happy, and you want to know about their pipeline and workload to understand if they’re struggling.

Hand-in-hand with that is engagement scores, which should give you a better understanding of why people may be leaving. In the tech industry, a 25 percent annual attrition rate is not uncommon right now, and you need to understand if people are leaving because of salary or something else. Pulse surveys are a start, but you have to drill down from there to better understand what people are thinking and feeling. You have to have conversations with employees to give you a fuller picture of what the data may be telling you.

The final point is that getting data and setting up systems so that they provide meaningful insights is an investment. You need to understand what your ROI is going to be, so do your homework about what will work and what won’t work and how quickly you can get your systems and approach up and running because it generally is an expensive proposition. Whether you’re going to be buying a platform or a tool or you’re going to have your data team build something for you, it’s going to take work and time. So understand what you’re trying to do, have a clear plan, and understand that your plan could involve a significant investment.

Ask the Right Questions About Your Data

In today’s world, we tend to be overloaded with data, but through effective data collection, analysis and reporting, we’re able to utilize internal and external data to create and mature business models, build conclusions, and drive insights and decision-making. These capabilities allow us to focus not only on the day-to-day operations but also to gain strategic insights to drive big agendas and increase organizational value. The big question is, what are the limitations of data? 

Often, there are misalignments between the data being collected and analyzed and the questions we are asking. For example, we may have questions regarding leadership impact that may include cross-team collaboration, customer engagement and insights, staff development and mentoring, business results, etc. However, we may simply be collecting logistical data on position title and level, revenue, sales, number of direct reports, etc. 

When we look at leadership impact, we may not be collecting and analyzing the right data across the organization to answer the pertinent questions. Consequently, a critical area of focus needs to be asking the right strategic and operational questions and collecting and analyzing the right data to effectively answer those questions. 

The next area of potential limitation and focus is, where is the data that can be integrated and analyzed to begin to answer pertinent questions? More fundamentally, are we organized in a way that we’re able to collect, align and integrate data to answer important questions? In that regard, the question is, do we have a strategic and organization-wide approach to data, or are we disconnected in terms of how we collect data based on a siloed organizational view? 

Do we have insights on how each division and business unit is operationally aligned to the broader strategy, so that when data is collected in each part of the organization, it can be connected across the enterprise? Without that broader lens, the usefulness of data will be limited. It is important to get people to think comprehensively about how their organization is strategically linked and to not fall into the trap of thinking that only data in their unit is relevant to them.

Another area of potential limitation of data comes from the quality and integrity of the underlying data. Without appropriate tools to control, segment and analyze data formats and other structural and logical differences, the analytical integrity of the data can be compromised. Additionally, when data is being collected, there needs to be validation and editing techniques and capabilities to ensure the quality of the data. Essentially, data quality and integrity are foundations to data value. 

Furthermore, in today’s world, all of the above can be compromised without leading-edge cybersecurity and privacy protection. Every organization must be able to protect their data, but equally important, they must be able to know when the data has suffered a security incident. Too often, we find that organizations discover security breaches many months after they’ve occurred.

Consequently, the monitoring of data security across the enterprise is essential to employees, customers, and intellectual capital. The privacy and security of data is not only dependent on security systems, tools, processes, and regulations but it must also be top-of-mind for employees. Do you know what ongoing capabilities are in place to protect your data and what role you must play in that regard? Do you know what role you must play if you suspect a security breach? Do you know what role you must play to recover from a breach? These are some of the questions to be answered and are essential to ensuring data security.

The other limitation is more of a cultural and ethical nature and that is the potential bias and misuse of the data. As we gather and analyze data and publish reports, one of the questions we must be mindful of—especially when it’s employee data or healthcare data or racial data—is what may be the potential biases and/or unethical use of the data that will limit its value to the organization.

Data is a significant asset. Being able to collect and analyze data to increase the value and effectiveness of your organization is critical. However, there are some limitations that must be kept in mind if an organization is going to get the best use of its data. Recognizing these limitations, the ongoing question for HR is: are you asking the right questions and how do you use data to create the best value in today’s organizations?

Valuing People as Investments, Not Expenses

When I look back over my career, there were many times when I too was data-obsessed, and I discovered the limitations of data. For example, I remember when technology really took off, and everyone wanted to use data to make better predictions. Instead of figuring out what had happened, businesses were focusing more on what was likely to happen. In hindsight, I realize that by moving with such focus on using data to better understand the future, rather than the past, people started overlooking the present. We were missing the signals about what was happening right now. 

Another limitation of data, in terms of data we need that we don’t have, stems from the fact that the business community continues to see people costs as expenses. While companies take a holistic view of their finances through a balance sheet—with assets and liabilities—we have nothing similar on the people side. There is no way we differentiate in practice between people costs and people investments. The current system doesn’t tell the whole picture. 

We invest in people and build their capabilities, which are going to benefit the company long-term. But all those investments are seen solely as an expense. I know there is no global accounting standard that’s available to show that on the balance sheet, but perhaps there should be. This is a blind spot even when we look at enterprise valuation. While all directors and CEOs will back “our people” as the differentiator and competitive sustainable asset, we account for them as expenses with no provision for acknowledging them as assets on the balance sheets. This should be the real big debate as the true business valuation of the enterprise resides not in the physical form but in the individual employees and collective teams. 

When you’re trying to build an organization of the future, capability building through learning, training and development is backed by the leadership. But during downturns, learning and training are often among the first things to be cut. When the business hits a slump, most times the leadership rallies around the CFOs who start calling the shots to bring expenses and discretionary spends down. Leadership often succumbs to this pressure and treats capability and skill building as discretionary spends. 

It’s all about seeing it with a different lens. People need to think about the data in a new way in the world of today.

A third limitation shows up in talent acquisition. Everybody spends time on the issues like cost per hire, fulfillment time and whether they have talent for the roles that are open. I think those questions are too narrow. Shouldn’t we be focusing more on the quality of hire, the stickiness of the hire and whether we are really hiring people who are going to drive performance for the enterprise? Are we really thinking about how they are going to be assimilated in the organization as true assets? 

I don’t think the data we capture and analyze does justice to the entire employee life cycle. There are also some fundamental shifts underway that should be broadening our perspective about how we think about and measure work. 

For decades we have practiced balanced work-life. In today’s context, should it not be about a balanced lifework? The whole meaning and purpose changes if we treat work as a subset of life instead of the other way around. People are also abandoning traditional thinking about career progressions. Instead they are focusing more on experiences and purpose. CHROs often monitor data to manage people through traditional career life cycles. But talent is seeking a shift from conventional life cycle to lifestyle. That creates a need to be more flexible. 

Granted, these ideas are not so easily measured but a space will emerge in the future where culture, capabilities, contribution and credibility will need to be shown as shifting along with the businesses. There is scope for organizations to develop better measures to track culture and whether the needle is moving in the right direction. 

Another area that needs new thinking is performance and performance measurements. We need a system that takes a more complete look at the combination of effort and outcome. For some roles, there is an outsized focus on outcome, when there should be a better way to account for effort. In other instances, the systems seem overly focused on tracking effort instead of the outcome. There is some traditional thinking there that needs to be challenged. Effort and contributions have to co-relate and exist as “two in a box.”

There are so many opportunities for rethinking the role of data in terms of what’s happening on the ground in the HR function. HR leaders have so much responsibility right now. You have to be digitally and technologically savvy, a strong communicator, have a deep understanding of business, and you need to understand the entire people value chain—the aspirations, expectations, reality and deliverables. Many different fields are coming together in the HR role, and it creates an enormous opportunity to find creative new ways to overcome the current limitations in the data we capture.

HR management is not just a function anymore. It’s evolved to become a functional conglomerate with so many asks being made of the CHRO. Therein lies the opportunity for HR to make an even greater impact in their organizations.