Robots: still science fiction a few decades ago, but now we can't imagine living without them, both at home and in the workplace. Greenhouses are also becoming increasingly automated: climate computers, potting machines, sorting systems, harvest robots... the list goes on and on.
Given the continuing issues of getting workers in the greenhouse, automated pickers are a particularly interesting development. In this article, we'll take a look at what's happening in the world of harvest robots.
Tomatoes
Priva Kompano
While it's not a harvest robot, it does make the life of a tomato grower much easier: Priva's deleafing robot Kompano.
Crop operations often represent a great physical burden for employees, and may in fact be quite monotonous. Together with a large group of tomato growers, Priva developed a robot to remove the leaves of tomato plants independently and efficiently. The first prototype was tested in 2014. It had been in development for much longer though: 15 years of research and development went into the robot. When it was introduced during the GreenTech in 2016, it promptly won the Innovation Award.
The robot initially became available to a consortium of tomato growers who had invested in the project. This article makes clear how necessary those investments were, given the amount of research needed to develop the required vision system.
And deleafing tomatoes is just the beginning, according to Priva. "This first generation of automated crop labour will lead to more automated solutions, such as different crops and tasks, for example harvesting cucumbers."
Agronauta
Agronauta is a project that unites seven Spanish entities as well as seven research organizations. Together, they focus on the experimental development of the collection and preparation of vine tomato in greenhouses using a mobile robotic system.
The system, in which Tecnova is one of the participants, will be completed with the development of new packaging systems, to be installed at the farm, as well as global production management software able to manage and forecast demand.
Panasonic
It's not just horticulture companies that are interested in getting robotic hands into the greenhouse. In 2015, the Nikkei Asian Review reported that Panasonic had developed a tomato-picking robot.
The Tomato Harvesting Robot moves along a rail that is laid between the tomato seedling rows to harvest the tomatoes. The robot is equipped with a camera that has an image recognition function. It uses this to find the tomatoes and determine whether they should be harvested or not. In this way, it decides the route for operation and brings an end effector close to the tomato for harvesting. It passes the targeted tomato through a ring, then pulls on it. By pressing the primary limb (called the peduncle), the tomato is picked as if it were being taken by a hand, and dropped into a pocket below.
"Harvesting is presently done at the pace of one tomato every six seconds. Since people can pick one tomato every 2-3 seconds, people are faster, but they can only work for about 3-4 hours. On the other hand, the Tomato Harvesting Robots can work for 10 consecutive hours or more, and they can also work at night", according to the Japanese corporation.
The robot, which was unveiled in January 2018 in Tokyo, still has a long way to go though. When Gizmodo visited Panasonic's robotics lab, they found out that human pickers don't have anything to fear just yet.
Squse
Panasonic isn't the only Japanese company working on a mechanized tomato-picker: Squse has also been developing one.
With a camera and range image sensor attached to each of the main body and the tips of the two robotic arms, the robot recognizes the bunches and fruits of tomatoes. When it detects a targeted fruit, two units of the "UR5" multi-joint robot, which was developed by Denmark-based Universal Robots A/S, function as arms and collect the fruit. At this point, the time it takes from searching to harvesting is 20 seconds.
"Even 20 seconds is slow as operation time," Squse told Japan Today in 2015. "We want to reduce it to 10 seconds or faster, specifically about six seconds, eventually."
University of Guelph
On the other side of the planet, Canadian scientists are working on the Guelph Intelligent Greenhouse Automation System. In December 2015, Medhat Moussa announced long-term testing of its robot with three Leamington area greenhouse vegetable growers using various greenhouse crops, conditions, and sizes.
In order to get to that stage, certain challenges had to be overcome. “You are dealing with non-rigid objects like tomatoes and you will end up with ketchup if you’re not careful, so you need a very sensitive gripper that is not high cost but is also fast,” Medhat explained.
In addition to harvesting tomatoes, peppers and cucumbers, Medhat Moussa is working on developing a system that will allow the robot to more frequently collect data on all plants in the greenhouse to monitor for disease. Once the fully developed robot hits the market, it has the potential to reduce labour costs and labour shortages, and pay for itself in five years, according to the University of Guelph.
Cornell University
Another North American university is also working on robotic technology that could bring automated tomato pickers closer. Cornell University's Organic Robotics Lab has developed a robotic hand that mimics the movements of a human one.
Toward the end of 2016, it passed the tomato test: the hand was able to scan three tomatoes and determine, by softness, which was the ripest.
Suzhou Botian Automatic Technology
The Chinese have also been busy in the field of agricultural robots. One example is the harvesting robot that Suzhou Botian Automatic Technology showed at Macfrut in 2017. According to manager Pengbo Wang, the robot can harvest not only tomatoes, but also other vegetables, as well as strawberries. "It is equipped with a camera that can assess ripening with a precision of 92%. The cutting system has a precision of 90%. The bionic hand is coated with a soft material so as not to damage the produce."
GRoW
One of the recent, and also one of the most versatile robots, is GRow (Greenhouse Robotic Worker). The first application of this robot, developed by Israeli startup MetoMotion, is selective robotic harvesting of greenhouse tomatoes. But that's not all: it also adapts to perform a range of greenhouse tasks, including harvesting of additional vegetables, pruning, monitoring, pollinating, and more.
The robot works with a 3D Vision System to detect ripe fruit and calculate their location. The system is designed to locate a stem without the need for exact data and to cut and catch in a single operation. While harvesting in the greenhouse, the system will be generating data on the crop. This way the system can inform a grower on stress in the plant, forecast the yield and help create work plans, as CEO Adi Nir explains.
In August 2018, MetoMotion published the above video, showing the robot in action during an alpha test in a tomato greenhouse. Beta testing and commercial introduction is slated for 2019.
EPCO
Another recent video comes from Malaysia, showing a tomato-harvesting robot in action at a trade show. As with all robots, it remains to be seen how accurate they are in the field, but one thing is clear: robotization is an issue everywhere on the planet.
Peppers
SWEEPER
Pepper picker. Apart from being impossible to say five times fast, it's also the subject of an EU-funded project to develop a robot that harvests peppers: SWEEPER.
In April 2012, the first prototype, then still under the CROPS project (Clever Robots for Crops), was tested in a greenhouse environment. Developed by researchers from Wageningen University & Research, the prototype was equipped with a gripper arm with nine pivot points and a number of 3D camera systems. Jochen Hemming of WUR explained the challenge of building a successful pepper-harvesting robot: "We have asked growers the hourly rate of their employees and developed our robot according to these numbers." That means that the machine has to reap a pepper every six seconds, day and night.
In 2014, the CROPS project showed the first positive results: it was able to pick peppers in the greenhouse autonomously. The project was promising enough to spin it off into its own international research program, called SWEEPER. Wageningen University & Research Centre was appointed to coordinate the international network in which partners from Sweden, Israel, Belgium and the Netherlands take part. The partnership includes universities, research institutes, a system integrator and a large grower of sweet pepper.
After testing and selection of individual modules, the first model of the robot was integrated by Dutch company Irmato (Veghel). It was placed at the laboratory of Wageningen UR Greenhouse Horticulture where, in 2016, it was used to further test manipulator control and fruit detection software from Umeå University (Sweden) and Ben Gurion University (Israel). Soon after the initial tests, it would be transferred to a commercial greenhouse (De Tuindershoek) at IJsselmuiden, the Netherlands. The year after, the robot was tested further in a Belgian greenhouse.
Finally, in July 2018, all that developing and testing culminated in the first live test of the SWEEPER. The live test, conducted at De Tuindershoek, drew excited reactions, although it was evident that a 100% score is still a utopia. But then, a 100% score is not a goal for the project - the robot is to be an addition to a team of human workers in the greenhouse.
Soft fruit
Agrobot
They're small, they're numerous, and soft by nature: strawberries and other soft fruit are particularly finicky when it comes to picking. But that hasn't stopped robot makers from trying. One of the first robot strawberry harvesters is the Agrobot.
The commercial version of this Spanish strawberry-picking robot was tested in 2012, after having been through six years of development. "The US is in dire need of able workers, and our technology seems to be the perfect solution," said designer/engineer Juan Bravo. When exactly it will be ready for commercial use remains to be seen.
Shibuya Seiki
Automated soft fruit harvesters, are of interest to both greenhouse growers and open-field berry farmers. A Japanese robot, demonstrated in 2013 by agricultural machinery maker Shibuya Seiki and the National Agriculture and Food Research Organisation, is no exception.
The two-metre (6ft 6ins) robot moves on rails between rows of strawberries. It calculates the degree of ripeness from the colour of the strawberry, which it observes with two digital cameras. It also uses the images from the two cameras to calculate the distance from the target, then approaches the strawberry it is aiming at.
Harvest Croo
Harvest Croo Robotics is a big player in open-field agricultural automation. In 2016, their strawberry harvest robot got its first U.S. patent approval.
The picking wheel uses conservation of motion as opposed to a “pick and move” motion. It picks the initial berry, then rotates to expose the next claw to pick another berry at a rapid rate. Once a strawberry is picked by one of the six food-grade silicon claws, it is rotated to the top of the wheel. From here, the berry is carefully placed into a cup that will then be transferred to a central location on the machine, to be inspected for a second time, before placing into consumer packs. This is all done without human intervention.
In December of that year, Harvest Croo got another major boost when the National Science Foundation awarded a grant of up to $1 million to the company.
As we saw already with other harvesting robots, Harvest Croo's strawberry-picking robot also still has a way to go before it can rival human pickers. In March 2018, Bob Pitzer, the company's co-founder, was confident they'll get there soon though. "There's quirks to work out, but it's getting there. We're close," he told NPR, expecting the robot would be out in the fields within two years. The berry picker is currently being tested at Wish Farms, one of the driving companies behind the project.
Octinion
Meanwhile, in Belgium, R&D company Octinion is also working on a strawberry harvest robot of their own. After several years of work, their prototype was ready to be unveiled in September 2016.
Following this first demonstration, the robot was also showcased at the International Strawberry Congress held in Antwerp in 2017. Shortly afterwards, Fast Company wrote up an in-depth report on Octinion's robot.
Dogtooth
Across the channel, in Cambridgeshire, a team of researchers is also working on a strawberry robot, under the name Dogtooth. In 2016, the Eastern Agri-Tech Growth Initiative granted the project £60,000 to help them develop their robot.
Octinion’s robot plucks the strawberries by twisting them 90 degrees, leaving each berry without any of the stem attached. This is the method most consumers in Europe prefer, as keeping the stalks attached risks them bruising other strawberries in the punnet. Dogtooth takes the opposite approach, snipping the berry off by the stem, leaving just a little bit of it intact. Doing so helps extend the fruit’s shelf life, so both methods have benefits and trade-offs, as The Nerd Stash reported.
Fieldworks Robotics
The British haven't been sitting still in the face of a troublesome labor market. In August 2018, University of Plymouth spinout Fieldwork Robotics signed an agreement with a leading UK soft-fruit grower to expand development of its harvesting robots.
Dr Martin Stoelen of Plymouth University, who is also the founder of the company, is also working on a tomato harvesting project with the Sunqiao and Shanghai Jiaotong University in China. He said: “The collaboration agreement we’ve signed with Hall Hunter is a big step forward for Fieldwork and the team at the University of Plymouth. I’m looking forward to seeing our robots operating in the field.”
The Hall Hunter Partnership grows 14,000 tonnes of raspberries, strawberries, blackberries and blueberries and its major customers include Waitrose, Marks & Spencer and Tesco. The family-run business owns farms and glass houses in Berkshire, Surrey and West Sussex and also runs a farm in Portugal.
University of Essex
Another British university is also working on a berry-picking robot in partnership with a grower. Robotics experts at the University of Essex are working with jam makers Wilkin & Sons of Tiptree to look at ways of picking strawberries with robots.
“The challenge is that no two berries are the same - they come in different shapes, sizes, order of ripeness and many are hidden in the foliage,” explained Dr Vishuu Mohan, from the School of Computer Science and Electronic Engineering. “Also the environment keeps changing constantly - sunny, windy, rainy - in contrast to a typical industrial environment. Hence, dextrous manipulation in unstructured environments is a big challenge for robotics today.”
BSF chairman Nick Marston said the robot was “not a realistic proposal to help with Brexit in the short and medium term.” Warning about a lack of clarity around seasonal workers’ rights once Britain leaves the EU, he told The Independent: “Picking robots are very much at the trial stage. They are starting to pick strawberries but it will be quite some years before they are a commercial reality.”
Strawberry Fields
At the World Robot Olympiad, an international competition for talented school and university students from more than 60 countries, a robotics team from ITMO University in Saint Petersburg (Russia) took first prize in the Senior High age group.
The ITMO team competed with a project called Strawberry Fields. This is a smart fruit patch like no other, being handled by a robot which uses a camera and a neural network to locate the berries, identify how ripe they are, and uploads the map of the patch to the Internet. Connecting through a special app, fruit lovers then choose any berry they would like to feast on, after which the robot readily picks it and dispatches it on a quadcopter which delivers the berry to the consumer.
According to Igor Lositsky, what makes the project stand out is its use of the cutting-edge, Soft Robotics technologies, such as a soft capture mechanism and a system capable of independently growing all kinds of plants with the help of neural networks and real-time databases. It was ITMO’s Biochemistry Cluster specialists that helped the team members master the complexities of the soft capture mechanism. They also provided the team with all the materials necessary to create the Soft Robotics elements, which enable the green-thumb robot to work with fragile things such as strawberries.
Cucumbers
Crux
Cucumbers are a different animal. That hasn't stopped a team from Eindhoven to try their hand at a cucumber harvest robot. In 2017, after ten years of work, Crux Agribotics developed a robot that visualizes, recognizes, qualifies, harvests, transports, selects and packs ready to ship cucumbers.
According to Crux' Richard Vialle, the hardest part of the development of the robot is not just about the machine, but also about the data that is being collected. The concept is fully automated and based on Robotics, Vision and Machine Learning technology. It means that data from every step is available for the grower and that the machine will learn over time to work better.
That it’s going to succeed, is a certainty for Richard. Simultaneously with the harvest robot, Crux Agribotics is also developing a testing robot and a packing robot. This system can ultimately test and sort 30,000 cucumbers per hour. The first has already been sold to a Canadian grower. "By selling it, we actually proved ourselves," says Richard. "And in addition to all the millions of investments that it has taken so far, we have also generated the first financial value with our start-up."
CATCH
Open-field cucumber growers are pinning their hopes on a European project: CATCH. As part of this project, the Fraunhofer Institute for Production Systems and Design Technology IPK is developing and testing a dual-arm robot for the automated harvesting of cucumbers. This lightweight solution has the potential to keep crop cultivation commercially viable in Germany.
In Germany, cucumbers destined for pickle jars are harvested by hand with the aid of “cucumber flyers” – farm vehicles with wing-like attachments. Seasonal workers lie on their stomachs on the vehicle’s wings and pluck the ripe cucumbers. This labor-intensive and energy-sapping type of manual harvesting is increasingly becoming uneconomical. In addition, the per-unit costs of harvesting have risen since Germany introduced a minimum wage. Many of the country’s agricultural regions consequently face an uncertain future; cucumber farming has already begun relocating to Eastern Europe and India. There is thus an urgent need for improved harvesting technologies to maintain the economic viability of cucumber farming in Germany.
Read more about how European researchers are trying to solve this conundrum.
Lettuce
Spread
Vertical farms are perhaps the most futuristic of all agricultural enterprises, with a purple glow illuminating row upon row of lettuce grown with the help of computers. One such vertical farm chain is Spread, in Japan. With the help of robots, already in 2015 it was able to grow and harvest 30,000 heads of lettuce a day. The concept caught on, and in 2016 Spread extended its existing Kameoka farm, to create a facility that could grow 50,000 heads of lettuce per day.
Using robotics in indoor farming is complicated, however. Perhaps the thorniest problem is the variability of each, say, head of lettuce. "It's not like you're making a car and every item is the same," says Brandon Alexander, co-founder of Iron Ox, a Silicon Valley startup building a robotic environment for greenhouses growing lettuce and other leafy greens. Thus, robots must be programmed to be able to grab, remove and transplant things that are not exactly the same shape.
Ultimately, robotic experts predict that robotic indoor farming systems won't replace conventional methods. Instead, they'll serve as supplements--enhancing agriculture grown the old-fashioned way with a more efficient, sustainable and weather-resistant method for producing crops.