Big Data Challenge

Storing, integrating and analyzing HR data is a top priority for HRIS leaders

By Dave Zielinski Apr 2, 2012

Between 2005 and 2010 the amount of HR data Mike Harmer was tasked with storing in his organization’s data warehouse grew almost 200 percent. On top of that core HR information, Harmer, director of HR analytics and technology for Salt Lake City-based Intermountain Healthcare Inc., has added talent management data from recruiting, performance management, succession planning and compensation systems that promises to double the information volume.

Transforming that raw data into a structured format that can produce timely, meaningful business reports is no quick or easy task. “It takes about a year to completely add a new source system to our data warehouse,” Harmer said, “and at the same time the business is moving forward and demanding new data, and analysis of that data, while we’re in that process.”

This big data challenge is one faced by more HR leaders as the volume and variety of data grows and business leaders increasingly use analytics, not hunches or prior experience, to make strategic talent decisions. As more data sources are added under the HR umbrella—with talent management, workforce management, social media and business performance data often joining payroll and benefits information—finding ways to store, integrate and create user-friendly reports from this mother lode has become a top priority for HRIS leaders.

“It is rapidly becoming a question of when you’ll have to deal with big data, not if you’ll have to,” said Nathaniel Rowe, a research analyst with the Aberdeen Group, a Boston-based firm that specializes in human capital research. Data is growing at a 36 percent annual rate across organizations Aberdeen surveyed.

“The average organization of 2,500 employees or above has 47 different data sources that make up its HR technology system,” said Jason Averbook, CEO of Knowledge Infusion, a Minneapolis-based human resource technology consulting firm. “Combine that with other enterprise data gathered to measure HR’s impact, as well as external data from the likes of Facebook [and] LinkedIn, and it starts to become an overwhelming situation.”

The big data problem is not faced only by Fortune 500 companies with petabytes of data or instantaneous processing needs. “A smaller organization might only have several dozen terabytes of information, but if it’s complex data in many different formats and the demand is high to access or analyze it quickly, [the firm] also faces a big data challenge,” Rowe said.

The Three V’s

Big data isn’t just about managing volume effectively, Rowe said. It’s also about managing variety and velocity issues. Variety refers to data not typically found in traditional databases.

“You look at the growing amount of unstructured data like social media comments, e-mail, text messages, blogs, video and podcasts, and it doesn’t fit neatly in the rows of a relational database,” Rowe said. “Being able to extend an organization’s information architecture to not only handle this unstructured data in a variety of formats but to take it to the next step and create actionable information from it to supplement traditional reporting is a crucial part of the big data challenge.”

Velocity refers to collecting and analyzing data quickly. “Data is coming into organizations so much faster today, and managers want reports and insight faster than ever before,” Rowe said.

Tools to Manage Big Data

There are many tools to help HRIS leaders manage and mine growing data sets for actionable insights. Business intelligence (BI) software, for example, provides sophisticated data analysis and visualization, but Rowe said these platforms are applied to HR data infrequently.

Integrating HR technology systems can take data analysis to another level. According to Aberdeen’s December 2011 study The Role of Big Data Analytics in HR: Speed, Satisfaction and Scale, the organizations that performed best in speed, productivity and reducing cost of managing HR data were those that invested in integrating as much data from as many sources as possible. Almost three-quarters of companies with those “heavily integrated” HR systems in the study reported using BI tools to crunch data.

Rowe said two other tools valuable in managing big data are data quality and data enrichment software tools. Data quality tools help to standardize records as well as detect and correct errors. Data enrichment tools can identify when HR records have missing or incomplete information.

“Big data is essentially an issue of scale and manual processes … having tools that automatically go through HR data to standardize it, normalize it and make sure the proper people are creating and editing it is critical,” Rowe said.

On the productivity side, integrating HR systems can create faster, more efficient data searches. In the Aberdeen study, organizations that integrated their HR data took just seven minutes on average for a typical data query, while those with segmented systems took 64 minutes for such queries.

“When you think about how many people are querying each day and how much time they’re saving they can devote to other tasks when data is integrated, that’s a dramatic impact on productivity,” Rowe said.

Wanted: Analytical Skills

Structuring and integrating HR data is one thing; generating timely reports from it that can help line managers make better talent decisions is another. Some experts say that the ability to perform quantitative data analysis and deliver relevant workforce data on metrics is among the competencies that line leaders value most in HR. Unfortunately, the skill is a scarce one.

“For any big data initiative to truly pay off, it’s going to require HR to develop a new capability of telling good stories with data,” Averbook said. “Data arrives in raw and unusable forms, and HR’s job is to turn it into something that can be used and understood for decision-making. You can spend all the money in the world on BI tools, but if you can’t tell a story with data in the language of the business, not in HR language, it won’t matter to anyone.”

The need for more sophisticated data analysis has changed the way Harmer staffs his HRIS team at Intermountain Healthcare. “It used to be we could just … pull key data and create a few graphs on a page that would constitute a dashboard,” Harmer said. “But we’ve grown from that need for basic analysts to a need for data scientists, where we require math, statistics, programming, data visualization and domain expertise all in one blended skill set on our team.”

At Nissan North America, a re-engineering initiative that consolidated eight core HR systems into one system as well as integrated data from talent management has led to improved data analysis, said Anish Baijal, director of HR and talent management for the Nashville, Tenn.-based company.

Baijal, who was an engineer before moving to HR, said that rather than trying to develop high-level quantitative analysis skills of his HR staff, he’s found it more efficient to “borrow” that talent from other Nissan functions. He brings in experts from engineering and marketing on rotational assignments in part for their analytical capabilities.

“When they work side by side with HR staff, they help them with data analysis. And when they rotate back to their departments, they have a better understanding of the issues facing HR,” Baijal said.

The effort to integrate talent management data with core HR information paid dividends for Baijal in 2011 when senior leaders came to him with an urgent request following the tsunami in Japan that caused a supply chain crisis at Nissan.

“Leadership came to us and said they needed to find people in the company who could challenge the status quo,” Baijal said. “They wanted people who could generate new ideas for keeping plants running and rebuilding vehicle inventory. We were able to quickly identify 10 people from across the organization with the competency of an entrepreneurial or ‘challenger’ mind-set, which wouldn’t have been as easy without the data integration.”

Dave Zielinski is a freelance business journalist in Minneapolis, Minn.​​


Job Finder

Find an HR Job Near You
Post a Job


Find the Right Vendor for Your HR Needs

SHRM’s HR Vendor Directory contains over 10,000 companies

Search & Connect