People + Strategy Journal

Winter 2021

Applying Artificial Intelligence in Fluid Environments

As the capabilities of artificial intelligence and machine learning continue to grow, leaders need to provide the framework and reality check to truly harness the problem-solving possibilities.

By Matthew Beers
Applying Artificial Intelligence in Fuid Environments

​Artificial intelligence (AI) has seen surge of interest—and not a little hype—primarily due to the prevalence of larger and larger data sets, increases in computing power relative to cost and development of new theoretical understandings of how to more effectively parse the subtleties underlying our complex world. 

The last 50 years has seen computing power increase by a factor over 2 billion, with a significant portion of that exponential growth happening in the last decade. Along with increased computing has come increasing quantities of data. By 2025, humans are expected to generate 463 exabytes of data (463 million terabytes), each day.1 While the volume of information generated by an individual business is much smaller, the scale of available data is truly mind-boggling. At the same time that increased density of transistors enable the high performance of modern smart phones, they also create the capacity for every business to apply the latest advancements in AI to the data it generates.

Despite such power, the vagaries of a global pandemic and related geopolitical, social and market events in 2020 helped remind leaders of nearly all organizations of the challenges that confront leaders in making informed decisions in the face of ambiguous data. Can AI and machine learning (ML) help improve our ability to navigate in the fog? Maybe. But the field is moving fast, its limitations are often misunderstood, and every day there seems to a new breakthrough that will change the playbook for how we interact with machines. (For instance, the recently developed GPT-3, a research project from OpenAI, is a natural language generator.2 Trained with nearly 175 billion parameters, it is able to generate sensible language that, in most instances, is indistinguishable from that written by a human.) My own role has traditionally been in technical leadership, connecting the needs of the business to the details of the implementation of technology. This article is meant to provide a guide for how leaders of an organization can harness advances in the field of AI, anticipate pitfalls along the way and help bridge the gap between the needs of the business and the ability of technology implementors to most effectively meet those needs. 

Defining AI and ML

​Artificial intelligence (AI) as a discipline is focused on the creation of autonomous agents that are able to perceive their environment and take actions to maximize the chances of succeeding at their goals. Practically speaking, AI is the application of input data to a predictive model to optimize an outcome. 

This definition has wide-ranging applicability, from the computer-controlled characters in a video game, to the choice of positions in a game of chess or Go, to the movements of an autonomous vehicle as it navigates roadways. At the inception of the field of AI, most models were selected and optimized by software developers, based on the goals of the particular software. Many of these models as applied in business were based in “expert systems,” a symbolic representation of the data connected with rules that were processed to make a decision. 

Rather than construct large systems of interconnected rules to make a decision, another approach is to borrow from statistics and probability, and to allow the computer itself to infer the model based on a training dataset. Machine learning (ML) focuses on allowing computers to use data and a specified objective to build such models. 

One of the best examples of this comes from the early application of ML to automatically filtering spam e-mail. The problem set: Marketers were developing methods for delivering spam faster than traditionally designed word and pattern recognition could match. Using a training set of data obtained from the e-mail recipients themselves, a Bayesian classifier (a type of ML program) could identify the unwanted messages for a given user with their own mail based upon that person’s filtering habits. The outcome: a reasonably personalized spam blocker that could evolve. 

Applying AI in 2021

The key in applying AI and ML techniques in any business context, but especially when attempting to parse ambiguous and fluid environments, is to start with the right question. This frames the analysis and provides a guide for judging whether the answer makes sense. The first step is to identify the kind of question the organization would like to answer. These fall under a few categories:
  1. Outcome prediction: This category of analysis builds a model based on historical data and then allows leadership to vary the inputs in a series of what-if predictions. There are natural limitations to these models, and most models built with data from 2019 probably did not fare well in 2020—just as most mid-pandemic data sets might have limited value for planning in late 2021 or 2022.
  2. Optimize an outcome: These kinds of questions involve minimization and maximization of an objective. Instead of varying the inputs to predict an outcome, the models built from these kinds of questions seek to identify the right inputs (i.e. decisions) that will optimize for a desired outcome. The famed routing algorithm of delivery drivers avoiding left turns, which leads to a significant reduction in delivery time as well as minimization of fuel costs, is an example of a problem where this category applies.
  3. Finding clusters: As new data enters the system, these kinds of analyses attempt to categorize and bucket the data into clusters that can be reasoned about. Many fraud detection systems use clustering to identify the characteristics of suspicious transactions and to determine if a given transaction belongs to the category of fraud. 
  4. Dimensionality reduction: The core idea here is try and reduce the complexity of the incoming data and select the key indicators that your leadership can use to drive effective decision-making. The classifying of email spam discussed at left is an example of this, as the algorithm must consider a large number of features of incoming text and reduce them to only the information necessary to classify a message as spam. 
With a sense of the right kind of question, the next task is to identify the objective function you are attempting to optimize and the set of data that would be ideal to include. This information may be readily available, or available but not easily digestible, or not yet existent in your organization. Identifying the crucial data set is a significant step and drives the actual sourcing of the right information. Achieving clarity at each step—question definition and data required to adequately explore that question— is critical.

Consider a hypothetical case where the leadership team would like to understand the impact of communication on productivity. This type of question is one of optimizing an outcome (productivity) and the discussion of necessary input data ends up encompassing a wide range of possibilities. For example, scheduled and impromptu meetings, email volume, incoming phone calls and instant message conversations all seem like reasonable starts to understand interruptions that could affect productivity.

However, there is an immediate clarity problem in this hypothetical: how do you measure productivity? This question as framed is currently ambiguous and practically speaking, unmeasurable. At the same time, the problem the leadership team is attempting to analyze is a fair one. The model would require a quantifiable metric, such as number of projects delivered or deadlines met. A better metric may be the amount of available working time that is not dedicated to meetings, email, instant messages, etc. The core point here is that you must be able to measure your objective in order to build effective AI models. 

ML is a conceptually simple concept but can break down in two places. First, if the objective is not defined in a measurable way, then the model can become excessively complex, and the conclusions from that model unreliable or simply incomprehensible. Second, models break down when the input data is not relevant to answering the question at hand, or when that data is “noisy”—that is, corrupted or has additional, non-meaningful attributes. Both places lead to similar issues, where the model ends up with confounding factors (inputs to the model that shouldn’t affect the outcome, but do), and possible false correlations between the inputs the output. 

Imprecision in the question and improper inputs create a model where you are rolling a die. Given the magnitude of data that can be processed in reasonable amounts of time and for reasonable amounts of money, it is tempting to throw all of the possibly relevant data into the mix and let the machines sort it out. While this can lead to AI modes that have surprisingly high correlations in training, you may end with a non-sensical real-world answer, like the divorce rate in Maine having a 99.6 percent correlation with the per capita consumption of margarine.

Big Circles, Little Circles

None of this should be taken to mean that you can’t start with an imprecise question and input and iterate to an answer, but rather that you should be aware of the places where ML models are stressed and work with the engineers to seek the right approach for your business need. With the clarity of the question, and some sense of the data that you would want to start with, comes the time to engage with engineers and data scientists to iterate further on the requirements and refine the objective you are trying to execute.

Big Circles, Little Circles

I think of this process as a cycling through little circles and big circles. The little circles are the fast iterations of the engineering effort where the engineers take the problem and the data and work to find an awesome solution. The big circles come as the leadership considers the answers being provided by the model, and with their deep domain expertise, determine if that makes sense for the business.

As an example, consider the traveling salesperson problem, which is a famous optimization challenge in computer science. A company’s sales rep needs to visit a list of cities distributed across a wide geography. Given the distances that must be covered, the business wants to plot the shortest possible route for visiting all cities exactly once and returning to the start. While it seems simple, this problem is nearly impossible to solve.

In considering the route for the wandering sales rep, engineers will likely find a solution stating that these cities should be visited in a specific order. But the leadership team, upon reviewing the route, will be equally likely to determine that it is invalid, because, for example, you can’t visit customer A before you visit customer B. This is the domain knowledge held by the leaders. 

The little circle of engineering produced a solution, the big circle of leadership provides a sanity check and potentially further constraints on the solution, and that interplay allows for improvement in the model, the data set or the question itself. These iterative loops have applications to many fields of planning, logistics and even DNA sequencing. 

Pitfalls

Applying AI techniques to business strategy starts with a good, measurable question, and an eye toward the right data. As a project proceeds, here are a few pitfalls leaders should understand.

1. Garbage In = Garbage Out
It’s a cliché in the development of algorithms and the use of data to support decision-making, but the quantity of data is not the only factor to consider when building decision models. Consider the trouble of teaching a machine to multiply from a simple times table. If the input told the machine that 9 x 9 was 99, the resulting model will encode the wrong data. The quality of your input data, and how that data is categorized and tagged, will have significant impacts on the quality of the both the model and the output.

One mitigation is to have humans perform the cleanup and normalization of the data. Depending on the sources, and the noise of the incoming data stream, this may be a time consuming and expensive operation, and one that generally can’t be outsourced. In a survey by Figure Eight in 2018, 55 percent of data scientists cited data quality as the largest challenge in their jobs,4 and the 2017 survey conducted by the same organization found that 51 percent of the time is spent “collecting, cleaning, labeling and organizing” the data.5 For organizations without a specialized role for managing the data, these cleaning and management tasks would fall on other employees.

Another way to address this problem is, perhaps ironically, ML. There is a growing market selling solutions that can validate, and in some cases, correct bad data as it enters the data warehouse. When considering the kind of data the model needs to train on, these solutions utilize ML to build a model to help sort, categorize and label the information. This is not the panacea of complexity reduction but can dramatically reduce the costs to start with better data.

2. Be Wary of Complexity in the Model
Earlier in my career I worked on a system that used an ML model for predicting the quality of a company’s intellectual property assets. The model used attributes of a particular type of asset, along with external data related to that type. The model utilized more than 50 discrete attributes, with over 100 factors ultimately contributing to a single numerical score for a customer’s assets. The challenge embedded in that system ultimately stemmed from the complexity of the model and the inability of a person to have an intuitive sense for how it worked. If a score changed from month to month, no one was able to explain why, and without that intuition, customers had difficulty trusting the analysis. The suggestion here is not that the number of factors was inappropriate, but that the complexity of the model reduced its real-world usefulness. 

This idea of simplifying may seem counterintuitive when a key attraction of AI and ML is their ability to quickly analyze seemingly infinitely complex amounts of data. The temptation to have a model account for all possible factors is alluring. But AI or ML problem-solving and datasets require disciplined simplification to support effective execution.

AI models are non-trivial, and, depending on the learning algorithm being applied, completely opaque, even to experienced AI practitioners. While the inner workings of the model may be cloudy, and the process the software followed to arrive at the model may seem like magic, the factors and inputs to the model should still make sense to you, when considered through the lens of your domain expertise.

3. Machines Can Learn Bias
One of the greatest achievements of ML is the efficient construction of models that emulate human decision-making. However, as HR leaders know well, the machine will use all of the available inputs to optimize the model to those decisions and will encode any of the bias in those decisions. 

In 2016, Pro Publica published an in-depth analysis of risk assessments that were being used in criminal justice systems to predict recidivism rates.6 They found, after controlling for socio-economic factors, that the risk assessment algorithm was biased against black defendants—that it predicted higher rates of re-offence—and biased for white defendants. Objective analyses of actual recidivism data did not support the model’s predictions.

Social media algorithms are good, ongoing examples of ML models’ susceptibility to bias. The most valuable commodity in most social media is engagement—keeping a user’s eyes on the screen—and a platform’s algorithms will present content that seems statistically most likely to keep users engaged. (Essentially, “If you like this cat video or political meme, click on this image next.”)7 Understanding the ML model behind such suggestions, savvy marketers game the system using content farms, where thousands of “clicks” bias algorithms to link to content that may have little to do with a user’s actual interests. At its most benign, this leads to sales pitches for products you have no interest in. Less benign manipulations can take users down rabbit holes of misinformation. 

The most important understanding here is that models can be biased to begin with or can learn biases over time. When considering the crucial questions your organization seeks to answer, and the kinds of inputs that query will require, consider also potential biases. Define additional metrics that can help to measure if there is an unintended bias in the model and perform corrections to minimize it.

brain

Conclusion

There is an apocryphal AI story relating to the invention in the 1970s of a robot that, with computer vision and advanced AI, could play table tennis with a pool cue. Researchers organized a press conference to demonstrate this feat to reporters, with a presentation hyping the advances of AI. 

At the key moment, one of the researchers served to the machine and… nothing happened. The ball bounced right past it and off the table. As it turns out, the robot failed to respond because at that precise moment, the algorithm had paused to free up memory and resources in order to be ready to play.

The field of AI has come a long way from those days (and we do today have robots that play table tennis with pool cues). At the same time, to strain the metaphor, our organizations and leaders face random serves coming at them from all directions. 

The availability of low-cost, high performance computers, the immense amount of accessible data and continued development in the fields of AI and ML can make available a whole new toolset for leaders. But the complimentary skills organizations can develop for maximizing this potential include simplifying problem-sets, addressing them with crisp questions, and embedding a deliberate interplay between data, engineering and subject-matter expertise.  

Matthew Beers, Ph.D., has served in Lead Engineer and Chief Technology Officer roles. He can be reached at mbeers@gmail.com.

References
1 How much data is generated each day? World Economic Forum. https://www.weforum.org/agenda/2019/04/how-much-data-is-generated-each-day-cf4bddf29f/(2019).
2 Brown, T. B. et al. Language Models are Few-Shot Learners. (2020).
3 Vigen, T. Spurious Correlations. Spurious Correlations. https://www.tylervigen.com/spurious-correlations.
4 Figure Eight. Data Scientist Report 2018.
5 Figure Eight. Data Scientist Report 2017.
6 Angwin, J., Larson, J., Mattu, S. & Kirchener, L. Machine Bias. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing (2016).
7 Many recent articles discuss the challenges this presents to users and platforms across a full range of social medial. See https://www.nytimes.com/2020/11/24/technology/facebook-election-misinformation.html and https://thenextweb.com/google/2019/06/14/youtube-recommendations-toxic-algorithm-google-ai/