Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vivamus convallis sem tellus, vitae egestas felis vestibule ut.

Error message details.

Reuse Permissions

Request permission to republish or redistribute SHRM content and materials.

Cover Story: The Benefits of Big Data

Daunting to acquire and interpret but rich in rewards, 'big data' is the latest chapter in workforce analytics.

Page Content

Have you wondered how HR can use "big data"? The following examples illustrate its potential:

Defense Acquisition University, which trains thousands of military and civilian professionals in defense acquisition, logistics and technology, analyzes internal and external data to determine the least expensive places to conduct classroom training based on total cost. Variables include room cost, instructor salary and travel, expected attendance, and travel costs for students.

Silicon Valley-based Juniper Networks, which develops network infrastructure products, uses LinkedIn (the career-oriented social networking site) to track and analyze the skills, knowledge, experience and career paths of employees, former employees and potential employees.

Before FedEx Corp. acquires a company, its HR department analyzes aggregate employee data from the acquisition target, including employee engagement survey results, and compares them with FedEx data. "Our analysis provides management with another data point before they make their decision," says Bob Bennett, chief learning officer and vice president of HR.

Big data is the latest chapter in the story of data-driven HR. The story began in the 1970s with the creation of the first human capital metrics, continued on to benchmarks and dashboards, and advanced in the past decade to more-predictive workforce analytics. Big data represents a big leap forward, however, and most HR organizations are still in the early stages of applying metrics.

"I try not to use the term ‘big data.’ It scares people away," Bennett says. "The important message is that now, more than ever, deriving value from data is critical in the business environment. HR has an important role because it has to use data to drive employee behaviors, making sure those behaviors are measured, monitored and shaped to achieve business goals."

Needle and Haystack

Big data is not just the buzzword du jour; it is a new reality defined by new sources of data. It applies methods and tools across the enterprise, especially in marketing, customer service, product development and HR. The goal is to make human capital decisions based on data, not intuition or conventional wisdom. Challenges include capturing the data, finding and using appropriate analytical tools, and putting the data to good use.

If anything, the term "big data" understates the tsunami of data available from transactional systems, relational databases and business applications that are available from a growing number of outside sources, including e-mails, social media and more. The amount of data businesses have has increased because most information is now digitized and comes from many new sources, such as smartphones and sensors embedded in furniture and badges. EMC Corp., a data storage vendor, estimates that the amount of worldwide digital information doubles every two years. Standard software tools and databases are not designed to manage that kind of deluge.

"We have access to more information about people than ever, and it lets us be more precise," says Bill McKinney, vice president of talent and long-term development at Thrivent Financial for Lutherans, a faith-based financial services organization with 3,000 employees and 2,500 independent agents. "In HR, we can find people we never knew existed who might be interested in working here. Big data is the haystack and helps you find the needle."

Besides sheer volume, the variety of the information and the velocity with which it becomes available also fuel big data.

The variety includes data sources internal to HR and the enterprise as well as external sources, such as salary studies, industry benchmarks and workforce demographics. Variety also refers to structured and unstructured data, each of which places different demands on technology and users. Structured data is the standard stuff of relational databases, including HR information systems (HRIS), accounting systems and enterprise resource planning systems. Unstructured data covers an array of free-form information, including narrative answers on engagement surveys, social media posts, blogs, wikis, e-mails, and even images and videos—all of which require new technologies for organization and analysis.

Velocity refers to the increase in streaming data arriving in real time and to the speed at which data must be evaluated and made actionable for business value.

Coming to Terms

“Data will set you free,” says Steven Rice, executive vice president for HR at Juniper Networks based in Sunnyvale, Calif.

But first it might drown you. Many basic issues about “big data” existed before the concept of big data itself existed—such as availability, accuracy and definitions of data.

Consider Rice’s skirmishes over how to measure head count. “There was always a battle over who owned the data and the definition of head count,” Rice recalls. “People fought me tooth and nail, insisting that it was owned by the line businesses and wasn’t company data.”

Consultants, analysts and practitioners agree that defining what a metric means is one of the big unresolved challenges for HR metrics—and that challenge existed long before the big data era arrived. If enterprises do not use standard definitions of the data points that create these metrics, they are essentially meaningless. Imagine if the finance department had to use different definitions for “revenue” from each business unit. That’s the basic problem many HR organizations face with head count, attrition and other key terms.

Eventually, Rice successfully argued that the company needed to agree on one calculation to use for business planning, forecasting operational expenses and resource planning that would comply with the legal definition of “employee.” It still took a couple of quarters after his team “owned” head count before business-unit leaders accepted HR’s number as accurate. Now, it’s no longer a question of who owns the data but of how to unleash the potential brought on by having so much information at one’s fingertips.

Disputes like the one Rice describes may someday be a thing of the past. The 3-year-old Center for Talent Reporting, a consortium with 60 member companies, has proposed a set of standard definitions to be used for HR metrics.

David Vance, the center’s executive director, says the goal is to bring standard measures and standard reports on HR metrics the way the accounting profession has standard measures and reports for financial statements. More than 500 proposed definitions can be viewed on the center’s website.

The aim is to persuade HR organizations to adopt the definitions voluntarily and for software vendors to incorporate them into their products. This fall, the group expects to post an accreditation process for HR organizations, vendors and consultants, Vance says. Now, with big data, “it is more important than ever to have clarity and ground rules on how you report these measures, because companies are buried in data.”

Data Integration

Of the three dimensions of big data, variety presents the greatest challenge to HR.

HR has two internal sources: data it owns and data it collects from other enterprise systems.

HR data comes from systems such as payroll and HRIS. While HRIS and payroll systems have seen an increase in data volume, this by itself is not big data.

Sometimes an HRIS is a module in an enterprise application suite, which can make accessing data from other modules relatively easy. But often an HRIS stands alone, not linked to other systems in or outside HR. Point solutions have proliferated for talent management, recruiting, performance management, learning management and other functions, adding to the challenge of accessing and managing this rich data vein.

Surveys by Josh Bersin, principal and founder of Bersin by Deloitte, a talent management subsidiary of Deloitte Consulting, have found that "the average large company has more than 10 HR applications and its core HR system is over six years old," he wrote in a recent blog post. "It takes effort and energy to bring this data together and make sense of it."

HR often needs a data mart to tie its data together before subjecting it to analytics. A data mart is a software layer on top of independent databases that gives users access to some or all of the data in each layer.

A decade ago, when Defense Acquisition University got serious about metrics and analysis, it brought in an expert from industry as its new CFO, Mark Whiteside. He led a team that quickly put into place an integrated set of business systems that included a data mart. All these systems were mapped into the data mart,says Christopher R. Hardy, director of the university’s global learning and technology center.

Dashboards on desktop computers now give managers a direct view into the data mart and the metrics they use to run their operations. "We have monthly meetings to specifically look at trends and results," Hardy says. "Data analysis has changed our whole culture."

An early exercise convinced Hardy of the power of data analysis. The leadership team at the university had earlier accepted the conventional wisdom that an instructor needs four hours to prepare for one hour of class instruction. Data analysis revealed that after the instructor had taught the class for a while, prep time fell to 30 minutes for each hour, says Hardy, co-author of Leading a Learning Revolution: The Story Behind Defense Acquisition University’s Reinvention of Training (Pfeiffer, 2008). "The finding had a huge impact on staffing, budgeting and instructor performance expectations, and other resource decisions," he says.

Peeling Back the Layers

Analysis that combines HR data and financial data can provide human capital insights that lead to decisions and programs with business impact. Sales databases and customer relationship management systems are fertile repositories for information that can be used in advanced workforce analysis. Due to technical difficulties and silo mentalities, however, gaining access to data outside of HR’s control can be a hurdle. This is especially true of companies that have decentralized structures with business units that use different systems.

"We have all the people data we need because we did a serious overhaul of HRIS a couple of years ago, but the people data by itself isn’t enough for strategic analysis," says Andrew Ermon, HR analyst at Superior Energy Services Inc., a Houston-based oil equipment and services business with 14,000 employees worldwide. Ermon needs additional data to prepare a strategic workforce analysis when a business unit considers opening operations in a new domestic location. This data includes internal and external HR and financial data about the labor pool—supply, demand, skills and hiring costs.

In this highly decentralized company, however, "getting access to financial and operational data is hit or miss. The data is either hard to get to or hard to find," Ermon says. "And it often includes confidential data that people are hesitant to let others have access to."

Once HR gets permission to use financial data, there is more to do. At Juniper, which has more than 9,000 employees worldwide, HR wanted to quantify how quickly it took new salespeople to generate revenue at expected levels. "A lot of our backend systems were not aligned to capture this kind of information," says Steven Rice, executive vice president of HR. "We had to do a lot of architecting around what data items we capture. That is hard work."

In the past decade, HR at FedEx Express, an operating company of FedEx Corp. with 155,000 employees worldwide, has increased its use of analytics and fought similar battles. Today it has access to all kinds of data. "We always look at HR metrics in hiring, recruitment, training, ROI, efficiencies, retention, talent, skill sets needed, succession planning and individual planning," Bennett says. "To support business goals, we also look at business metrics when we evaluate our talent—cost of salary and benefits for talent per revenue, for example."

Unstructured Data

For years, HR has used unstructured narrative answers from engagement surveys and performance reviews for human capital insights. In the era of big data, the internal and external sources of unstructured data are multiplying to include social media, blogs, wikis, e-mails and more. These sources are likely to offer many actionable insights about employee engagement.

Starbucks gained insights about employee motivation from a survey with many open-ended questions the company used in an employee segmentation study. It hired graduate students to assist with a manual content analysis of the massive amount of information. Tools to automate this type of process are still evolving.

"Automated analysis techniques of unstructured data are still in their infancy," says Ranjan Dutta, director of the metrics and predictive analytics practice at PwC Saratoga, a consulting firm. "But the capabilities will grow over the next few years." Oracle, SAP, Workday and other software vendors are working to make it easier for customers to manage and use multisource big data.

A few HR organizations already mine social networking sites for data. Juniper, for example, uses LinkedIn. In addition to housing resumes, the site puts a lot of research and development into analytics. Over the years, it has rolled out more and more analytical features for both individual users and businesses. Today, a business that subscribes to the LinkedIn Recruiter service can access every LinkedIn profile and search for any number of criteria using standard LinkedIn filters.

Rice had developed an internal database for tracking employees’ skills, knowledge and experiences, as well as voluntary departures and potential candidates, but LinkedIn has become a superior solution because most professionals use it as the go-to place to post and update their career profiles. "People are much better about updating their profiles on LinkedIn than in our database," he says.

In addition to recruiting, Rice’s team uses LinkedIn data to better understand where Juniper hires from and where departing employees go.

Juniper is also among a group of corporate early adopters and power users that collaborate with LinkedIn to identify emerging talent practices and innovative ways the service might evolve. "LinkedIn is moving deliberately and thoughtfully on the corporate talent solutions beyond recruiting, which will be more applicable for our needs," Rice says.

He would like to use LinkedIn to better understand career paths and then apply that knowledge to influence recruiting, learning and other programs. But "we’re still figuring out the methodology to tackle this," he acknowledges.

Thrivent uses LinkedIn, too. Based on the company’s earlier analysis of HR and financial data, McKinney knows that a predictor of success for independent agents is having a family member or friend who works for Thrivent or elsewhere in financial services. Thrivent seeks agents who are entrepreneurial and want their own business but who also like the idea of being connected to a big company. It seeks individuals with strong ties to their communities and a desire to serve others. "We are figuring out LinkedIn as a mechanism to find these candidates," McKinney says.

Social networking sites such as Facebook also hold promise.

FedEx, which has conducted correlation studies between employee data and unstructured customer satisfaction data, is now exploring how it might compare what employees say about FedEx on public social networking and career sites with what they say on engagement surveys. "Employees are your brand ambassadors, and social media and network sites are megaphones about your company," Bennett says. "Today employee satisfaction transcends the workplace into the social space. If we see that employees are dissatisfied and putting that on social media, we have a problem."

But McKinney is reluctant to tap into a social networking site such as Facebook because of privacy concerns. He argues that the trust-based relationship with employees could be at risk if HR starts perusing Facebook pages. "What data you decide to look at and why is a tricky line to walk."

The Power of Predictability

For HR, the crown jewel of big data is data-based models that predict the probability of an outcome—job candidates who are most likely to succeed, for example. Predictive models can be fine-tuned over time to become more accurate.

Here’s an example: According to Bersin, Deloitte Consulting created an attrition model "that looks at 30 or 40 dimensions. HR can now predict months in advance with high correlation who is going to leave."

Since the federal budget sequestration, Defense Acquisition University has used predictive modeling to understand the resources it will need and the training its clientele will require as a result of the budget cuts, Hardy says. The total-cost model for classrooms cited earlier in this article is one example.

Thrivent’s McKinney says every HR organization would like to be able to predict which job candidates and employees will succeed. "That is an industrywide challenge we’re all working on," he says. "It is similar to how you would do customer predictive modeling in marketing." To understand the success profile and the predictive model constructed from it, you need a lot of data from the past, McKinney adds.

Predictive modeling requires historical data, PwC Saratoga’s Dutta confirms. Even if an HR organization is not yet ready to dive into analytics, he urges leaders to start collecting data. "If you set up a system today to collect it, in a couple of years you will be able to start testing hypotheses."

Most HR organizations are not currently using predictive modeling. They’re still writing the earlier chapters of their own data-driven stories. Bersin developed a maturity model he says HR organizations follow on the way to more-sophisticated use of big data.

In a survey of 435 U.S. and Canadian companies with 1,000 or more employees conducted by Bersin earlier this year, the respondents identified their organizations’ maturity levels in adopting talent analytics. About half of the respondents (56 percent) were using operational reporting, 30 percent were using proactive advanced reporting, 10 percent were using strategic analytics, and only 4 percent were using predictive modeling.

In the 11 years since Bersin started his consulting firm, he has seen slow progress toward the adoption of data-driven HR. "It hasn’t gone off like a rocket, but there is an increasing level of awareness that this is an important part of HR," he says. "More HR people have analytical jobs now, and if you offer an analytics track at an HR conference, people will attend."

For FedEx’s Bennett, the overriding challenge of big data, like the earlier data-driven HR phases before it, is to understand the business value of the data and, whatever its structure or source, to understand its relevance to business strategy. "That needs to be the starting point," he explains. "That way, we have a better understanding of the direction we need to go."

Bill Roberts is technology contributing editor for HR Magazine and is based in Silicon Valley in California.

Web Extras

Additional Content


​An organization run by AI is not a futuristic concept. Such technology is already a part of many workplaces and will continue to shape the labor market and HR. Here's how employers and employees can successfully manage generative AI and other AI-powered systems.