This Algorithm Is Better At Predicting Human Behavior Than Humans Are

Analyzing big data sets in order to forecast trends or predict customer behavior usually relies on both computers and humans. Computer algorithms are advanced enough to rapidly comb through numbers and find useful patterns, and humans are still necessary for setting the parameters and analyzing the results. But an algorithm created by two MIT researchers suggest we could take out the human factor all together.

Conceived by Max Kanter, a MIT graduate student in computer science, and his advisor, Kalyan Veeramachaneni, the Data Science Machine can approximate human “intuition” when it comes to data analysis. Using raw datasets to make models that predict things like when a student is most at risk of dropping a course, or whether a retail customer will turn into a repeat buyer, its creators claim it can do it faster and with more accuracy than its human counterparts.

To test the system prototype, the researchers pitted the Data Science Machine against human teams at three data science competitions. While the algorithm didn’t get the top score in any of the competitions, it did beat out a whopping 615 of the 906 human teams competing. In two of the competitions, it created models that were 94% and 96% as accurate as the winning teams. Whereas the teams of humans required months to build their prediction algorithms, the Data Science Machine did it in 2 to 12 hours.

The researchers don’t view the algorithm as a replacement for human intelligence, but do recognize that it could prove useful for helping analyze the huge amount of data with less manpower. It could also be an important tool for user-centered design—if a machine can comb through massive amounts of data with much less manpower and in record time, it could also help companies better understand their customer base and design with future behavior in mind.

Source: Fast Company, Meg Miller

Meg Miller is an assistant editor at Co.Design covering art, technology, and design.