Malcolm Gladwell on big data and digital transformation

ORLANDO – It’s no secret that technology has drastically changed the way humans make decisions and solve problems. As Canadian journalist and author Malcolm Gladwell tells the crowd at Citrix Systems, Inc.’s 2017 Synergy conference, before the advent of big data and analytics, automation, and connected devices, problems were like puzzles: information had to be gathered and put together before a decision could be made. But in our new digital age, problems have evolved into mysteries – there is often too much information available, and it all needs to be sorted through and understood before a conclusion can be drawn.

Malcolm Gladwell

“We’ve set ourselves up to think of every problem in the world as a puzzle, which is how it used to be, but we’re not a puzzle world anymore; we’re living in a mystery world with an overload of information,” Gladwell tells the Synergy audience. “Problems are fundamentally different now and we need to be mystery solvers in this new digital world that we live in.”

But what does this mean for the future of work? He explains that many professions have had a difficult time understanding the transition happening in the workplace and adapting to the changes. As computers and machines are increasingly taking over many traditional jobs, people need to reevaluate their roles and re-position themselves to fill the voids left by technology.

“With automation taking over many menial, everyday tasks, people now need to move their focus to occupying the holes in the decision making process that technology can’t replace. It’s easier said than done, however,” he adds, saying that society and workplaces still need to better understand exactly what and where those holes are.

He uses sports, and in particular basketball, as an example. Calculating win shares, which is a metric used to estimate the number of wins a player produces for his team based on their yearly statistics, using a computer algorithm seems like a great way to judge a player’s impact on the surface, but after comparing the numbers of four current players, Gladwell says its not that simple.

While it’s difficult to see the exact numbers in the below picture from Gladwell’s keynote presentation, its clear that three of the four players saw their highest win share numbers in their fourth year playing in the league, and declined from there. This can be attributed to a number of different reasons, Gladwell points out, such as physical prowess generally peaking at the age of 24 (the age all these players were in their fourth seasons), and signing new (very large) contracts, which can often make a player complacent.

Gladwell explaining win shares in the NBA and comparing the scores of four players.

But the last player, Gordon Hayward, saw his win shares steadily (albeit inconsistently) climb and peaked in his most recent 2016-2017 season.

“Hayward beat the system – he didn’t decline in his fourth season like the other players on that list, and frankly, like many other players in the NBA. Instead, he continued to improve his abilities through hard work and practice and got better over time,” Gladwell says. “Any coach and general manager wants a highly motivated guy like that on their team, but how do you find such players? Statistics won’t show you what kind of person someone is or what their work ethic is like, and that’s where human judgment comes in to fill the holes that computer algorithms leave.”

Gladwell also brings up another example of a transforming profession: the role of a teacher in the education system. One of the most common ways to evaluate teachers is with the value-added model (VAM) system, which uses statistical data to measure the value that a teacher brings to their classroom based on the performance of their students. The kids write two tests, one in the beginning of the year and one at the end, and if the classroom average goes up, the teacher gets a high VAM score, but if the average goes down, a teachers’ VAM scores will drop.

However, as Gladwell highlights, a teacher’s VAM score is incredibly volatile and changes drastically across other measurement systems, courses, and years.

“This system is used to measure and hire teachers, but it’s assuming that being a good teacher is inherent in the individual, and that’s wrong,” he explains. “Being a good teacher is a matter of fit in the classroom, how well they know a particular topic, and the interactions between them and a child.”

He asks the crowd to imagine a teacher who has a really low VAM score after teaching a predominantly low income English-as-a-second-language class one year, but then witnesses their VAM score triple the next when they teach more middle-class and largely white students.

“The measurement system may be a good base, but it can’t be the only method of hiring teachers – you need the opinions of principals and other teachers who get to know their peers on a personal level. A teacher’s ability to form bonds with their children is subjective and can’t be predicted by computer algorithm scores,” Gladwell stresses.

He goes on to add that as a result, new technology will not replace humans; it will clarify what the role of humans ought to be moving forward. People have social abilities and will need to evolve into a more analytical, contextual position while tech does the work of collecting data, according to Gladwell.

“There’s both good news and bad news when it comes to this digital transformation: the bad news is that it will be very difficult for some, like a basketball coach needing to become a psychologist to truly understand his players or a principal that can rearrange classes months after school starts based on where a teacher really fits in. But the good news is that humans and human judgment is not going away – we need it more than ever,” he concludes.

Would you recommend this article?


Thanks for taking the time to let us know what you think of this article!
We'd love to hear your opinion about this or any other story you read in our publication.

Jim Love, Chief Content Officer, IT World Canada

Featured Download

Mandy Kovacs
Mandy Kovacs
Mandy is a lineup editor at CTV News. A former staffer at IT World Canada, she's now contributing as a part-time podcast host on Hashtag Trending. She is a Carleton University journalism graduate with extensive experience in the B2B market. When not writing about tech, you can find her active on Twitter following political news and sports, and preparing for her future as a cat lady.

Related Tech News

Featured Tech Jobs


CDN in your inbox

CDN delivers a critical analysis of the competitive landscape detailing both the challenges and opportunities facing solution providers. CDN's email newsletter details the most important news and commentary from the channel.