Is This Weed-Spotting, Yield-Predicting Rover The Future of Farming?
Source: Smithsonian Magazine, Elizabeth Gamillo
Photo: an a machine be taught to understand the plant world? (X, the Moonshot Factory)
The robot, developed by Alphabet Inc.’s X, will make its public debut at the Smithsonian
By the year 2050, Earth’s population is expected to reach nearly ten billion people. With this growth comes a staggering demand for food resources, particularly drought, heat, pest and disease resistant crop varieties that give high yields in the face of climate change.
Enter X, Alphabet Inc.’s so-called “moonshot factory,” where innovators face the world’s biggest challenges head-on and develop ground-breaking technology at a startup pace. Project Mineral, one of X’s current efforts, is focused on finding an effective way to address the global food security crisis through “computational agriculture,” a term coined by X to describe new technologies that will further increase understanding about the plant world.
“The agriculture industry has digitized,” says Project Mineral lead Elliot Grant. Farmers today use sensors, GPS and spreadsheets to collect data on crops and generate satellite imagery of their fields. “But it hasn’t led to more understanding. So the next step beyond digitization, is the science of making sense of this very complex plant world by combining multiple technologies such as robotics, sensors, data modeling, machine learning and simulation. The subtle difference is that computational agriculture is the sense making of all the data,” Grant explains.
Since the project launched in 2016, Mineral team innovators have been focused on answering one critical question: Can a machine be taught to understand the plant world?
After years of tweaking, Grant and his team’s latest prototype—a plant-scanning, rover-like robot powered by artificial intelligence—will make its public debut at the Smithsonian’s “Futures” exhibition, an expansive exploration of the future through art, history, design and technology opening at the Arts & Industries Building in Washington, D.C. later this year. Capable of syncing up with satellite imagery, weather data and soil information, the sleek, four-wheeled plant rover, about as tall as a shipping container and as wide as a car, uses various cameras and machine algorithms to monitor and spot potential issues with plants. As it rolls through farmland, it can identify weeds, measure the ripeness of fruit and predict crop yields. The Mineral rover can also adjust its width, length and height to accommodate crops in numerous stages of development. For example, it can be taller to image towering, mature wheat plants, or widen to scan a broad bed of lettuce.
But it didn’t start out quite so chic and impressive: The first prototype was made with two bikes, some scaffolding, a roll of duct tape and several Google Pixel phones. To put their Franken-machine to the test, Mineral’s diverse team, consisting of engineers, biologists, agronomists and more, whisked it away to a nearby strawberry field and pulled it through rows of red fruit to see if it could capture enough plant images to use for machine learning.
“So, after a few hours of pushing and pulling this contraption, through the mud and a bunch of squashed berries, we came back to the lab, looked at the imagery we had, and concluded that although there were a couple hundred things we still needed to improve, there was a glimmer of hope that this was going to work,” Grant explains.
After their initial experiment, and discussions with farmers and plant breeders, the Mineral team built, scrapped and reimagined their rover. This burn-and-churn, momentum-building phase is part of X’s rapid iteration methodology. If an experiment is simply not working out, X project leaders learn from errors and move on. “The essence of a rapid iteration is to move quickly, take risks, take smart risks, but do it in a way that continually leads to learning,” says Grant.
In one experiment, Mineral used a machine learning algorithm called CycleGAN, or cycle generative adversarial networks, to see if they could create simulated plant images of strawberries. CycleGAN generates realistic images, which Mineral can then use to diversify the rover’s image library. This way, when the rover encounters various scenarios out in the field, it can accurately identify specific crops, traits or ailments.
A.I. like this is useful for simulating plant diseases, pests or pathogens, especially when a robot needs to recognize it without having ever seen it before. (This approach prevents the detrimental alternative of purposefully inoculating fields with diseases.)