From payroll to recruiting to learning and development, the number of technologies built to help run organizations has mushroomed, making it more challenging for HR buyers to choose the right one.
SHRM Online spoke with HR technology leaders, analysts and consultants about how to use one key step in the vendor selection process—the product demonstration—to make more-informed decisions.
Use Cases Should Drive Demos
Experts say one of the most important things HR buyers can do is adopt a proactive mindset when going into a product demo. Maggie Rutz, an executive advisor with research and advisory firm Gartner based in Atlanta, said too often HR leaders take the approach of "you educate me" in vendor demos rather than controlling the agenda themselves.
"You don't want to give technology vendors an open door to show you every bell and whistle they have," Rutz said. "Rather, come prepared with specific use cases around pain points in your current HR processes or practices and ask vendors to show how their solution specifically addresses them."
For example, a current HR process might force staff to create and manipulate many different Excel spreadsheets to generate a talent analytics report. Try asking a prospective vendor how its reporting and analytics capabilities could solve for that unwieldy process.
Michael Rochelle, chief strategy officer and principal human capital management analyst with the Brandon Hall Group in Delray Beach, Fla., recommends against sending finalist vendors a long list of prioritized requirements, and instead suggests focusing on specific use cases. "Usually three to four use cases will cover about 80 percent of the way companies will need to use a vendor's software," he said.
Elaine Orler, senior vice president of technology consulting for Talent Function, a San Diego-based company owned by global recruitment process outsourcing firm Cielo, said HR and recruiting leaders need to ask different questions of vendors during product demos than they have in the past.
"We've moved past asking only to see how a system will perform work transactions to asking to see the experiences different users will have when using a technology," Orler said. In the demo of an applicant tracking system, for example, that might mean asking to experience a simulated day in the life of a recruiter or job candidate, such as demonstrating how an applicant would self-schedule a phone screen.
"I encourage organizations to define what they want that day-in-the-life to look like so those participating in a demo can follow the logic," Orler said.
Leave Time for the 'Wow' Factor
Orler also encourages HR buyers to allot some time for the "wow" factor in product demos. "Give the provider some time to show you something you might not have thought of or a feature they think is the best fit for your needs," she said.
George LaRocque, founder of WorkTech, an HR technology advisory company in New York City, said HR leaders sometimes make the mistake of assessing prospective vendors against their current HR work processes rather than against improvements to those workflows that a new technology might help to enable.
"Give vendors a chance within a demo script to show you innovative ways they might have to solve problems and expose you to solutions you may not have considered yet," LaRocque said.
Orler also believes some factors are better addressed through a request for proposal (RFP) document or other conversations with a vendor. One such area is assessing how well a vendor's system integrates with other existing HR technologies.
"Avoid going into deep conversations about integrations," Orler said. "They're difficult to show in a demo, and they require a deeper conversation than what can be accomplished in that setting."
Create the Right Evaluation Team, Scoring System
Experts say it's important the right people are involved in evaluating product demos and the right scoring system is used to compare vendor finalists. Rochelle believes it's crucial to get multiple user perspectives on a vendor's platform.
"Often times a blind spot is evaluating a system only through the lens of the HR administrator," he said. "You also need to have use cases that assess a system from the employee's viewpoint or the recruiter's viewpoint or a line manager's viewpoint. Each of those users has different requirements of a system."
Rochelle recommends breaking a demo evaluation team into groups of specialists. "Rather than just having one generic team, a better approach is to have certain small groups of experts assess how a technology handles certain use cases," he said. "That really helps you get more granular in the evaluation."
For example, information technology staff could ensure a prospective system aligns with existing organizational architectures, and payroll specialists could examine how well a system handles the tax compliance requirements of employees working remotely from different states or countries.
Rochelle suggests demo evaluation teams rate the various aspects of a vendor's system on a scale from one to five, with five being the highest rating.
"One of the biggest problems with demos is evaluations by HR are too qualitative," he said. "Evaluators might say they don't like something, but what does that really mean? People are just offering opinions. Without a more scientific scoring system, it often creates deadlock in making decisions on vendors."
It's also important to allot the right amount of time for demos, experts say. "In-person demos usually take longer than people realize, often a day and a half to do them right," said Marc S. Miller, a longtime HR technology consultant in White Plains, NY. "I've had some vendors tell me they can demo their product in three hours, which usually isn't realistic. There is much that needs to go on behind the scenes in addition to the actual demo time."
Time frames should be modified for demos conducted via Zoom, Miller said, a process used more frequently during the pandemic. "It's difficult to keep people engaged in a demo for hours when it's conducted over a videoconferencing call," he said. "You should have shorter segments and smaller groups of demo evaluators in that scenario."
Evaluating Artificial Intelligence Tools
When evaluating professed AI or machine learning capabilities in a prospective vendor's system, experts say look for transparency in how the technology works and how the tools protect against bias in talent decisions.
"Ask the vendor what kind of AI specialists they used to build the tools, how those tools are audited or tested, and how much they've invested in them," Rutz said. "Receiving halting answers to those questions should cause you to think twice."
Stacia Garr, co-founder and principal analyst of RedThread Research in Woodside, Calif., said HR buyers should assess how vendors' AI tools mitigate against bias in their algorithms and data sets when those tools are involved in hiring or promotion decisions. "If a vendor can't answer that question, it's a big red flag," Garr said. "It should never be a black box, and vendors should always be able to give you a detailed and satisfactory answer to that question."
Dave Zielinski is a freelance business writer and editor in Minneapolis.