Researchers design robot that can find, pick hidden strawberries

(Press-News.org) PULLMAN, Wash. -- Strawberries are delicate and hard to harvest—easily bruised and often hidden under a canopy of leaves.

This creates headaches for scientists trying to design robotic harvesters. Now a Washington State University-led team has designed one that combines an artificial-intelligence vision system, soft silicone “fingers,” and a fan that gently move leaves out of the way to get at hidden fruit.

Experiments in the lab and in outdoor fields showed that the harvester correctly detected strawberries 80% of the time, on average, and could classify whether the berries were hidden 93% of the time.

The design, development and testing of the robotic harvester marks a step toward making open-field robotic harvesting more commercially viable; it was the subject of a paper published in July in the journal Computers and Electronics in Agriculture.

The lead author was Zixuan He, who completed the work and earned his PhD at WSU before taking a position as a post-doctoral researcher at Aarhus University in Denmark. Co-authors included Manoj Karkee, a former WSU professor studying agricultural automation engineering who recently took a position at Cornell University; and Qin Zhang, professor emeritus and former director of WSU’s Center for Precision & Automated Agricultural Systems in Prosser.

              The $20 billion strawberry market is projected to grow by 6% a year over the next decade, but labor shortages beleaguer the industry, driving up costs and potentially limiting harvests.

University and private researchers have pursued robotic harvesting technology for years, but the efforts to make them commercially workable have so far fallen short. Among the problems: many of the berries on strawberry plants are hidden under leaves, which makes it hard for robotic systems to identify and pick the fruit.

“Novel approaches like these are essential to improve the fruit accessibility and picking efficiency, and thus enhance the practical applicability of harvesting robots,” Karkee said.

Much of the previous work on robotic harvesting has focused on greenhouse-grown berries or “table-top” systems, in which the berries droop below the plant, making occlusion less of a problem and allowing simpler robotic harvesters to successfully pick the fruit.

The WSU team developed a more complex harvester targeting conditions in the field. The robot—a four-wheeled cart with a platform for the computer system that rolls above the row of strawberries—uses a vision system that applies artificial intelligence methods to locate the fruit

“We have a 3-D camera that takes two images – one is the color image and one shows the depth,” Zixuan He said. “We then use the machine learning model to find the strawberry based on the  fruit feature information matched to the depth information, and then we know its location.”

The harvester calculates how the robotic arms and grippers should approach and pick the fruit. A key feature is its fan system, which blows a steady stream of air through tubes located near soft, silicone grippers.

The airstreams move the leaves for the grippers to reach the strawberry. The picking efficiency of the system was substantially improved by the fan system. In tests without the fans, the harvesters picked 58% of the ripe fruit; using the fan improved the picking rate to 74%.

Improving the efficiency and effectiveness of the system would be important to make it work in the fields. In particular, the speed would need to be improved—it took the harvester 20 seconds, on average, to identify and pick a berry with the fan system.

Zixuan He said that the fan system could be useful in developing robotic harvesters for other crops where occlusion is a challenge, such as grapes. He said that it is unlikely that robotic harvesters would fully replace human workers in the foreseeable future—it’s more likely that strawberry growers could use robotic systems to supplement human labor.

“Right now, it won’t totally replace the manual labor but it could be a very promising supplement when you don’t have enough people in the field,” he said.

END