Mechanizations final frontier: harvesting robotization

Harvesting of many fresh fruit and vegetables is still exclusively carried out by hand to better maintain its organoleptic characteristics and, above all, its storage life. Although somewhat a little slowly, robotics begins to offer interesting alternative solutions

In horticultural and fruit crops, both in protected and open fields, phytosanitary treatments, harvesting and sometimes pruning are the costliest processes in economic terms, essentially because of the need of labor.

As far as in particular, harvesting, if the product to be harvested is destined to be transformed (for example the grapes into wine, the olives in oil, the fruit in juices) the relative mechanization has come a long way, and it is possible to say that, in most cases, the quantitative and qualitative parameters that can be obtained are more than satisfactory.

Conversely, if the product is intended for fresh consumption, things change radically and, apart from sporadic exceptions, manual intervention is still widely generalized.

The most logic evolution of this operation concerns the harvesting robotization, a challenge on which we have been working for some time, on a global level. In general, we are talking about reproducing at best what the human being is easily able to do, namely: identifying, discriminating, picking, manipulating, storing.

Picking a ripe fruit or vegetable from a tree or a plant can seem a simple operation, literally that even a child could do, but this gesture reproduced by a machine (that is the robot) requires a very advanced and sophisticated technology, which has been developed for several decades but it has not yet reached a level of reliability and efficiency to be usefully considered as a substitute of human intervention.

In other sectors, such as the industrial one, robotics has now established itself as a valid alternative to man in the execution of repetitive and heavy operations, carried out, however, under extremely controlled conditions; when it goes to “en plein air” “the question gets complicated, sometimes quite a lot...

 

Identifying the product on the tree

Most plant organisms are made up of several hypogeal parts (below the ground level) and epigeal parts (above the ground level). If the examination is restricted to tree crops, there are trunk and branches, and then leaves, flowers, fruits It is, therefore, necessary to identify the fruits inside a variegated vegetal mass, where often the leaves can hide and screen the object of the harvest. Fruit detection techniques often refer to the fruit’s typical shape and size, and are based on specific algorithms, more and more often supported by the application of artificial intelligence, which in this case “learns” from previous experience to perceive more and more promptly and precisely the target on which to act.

The methods for identifying fruits (or vegetables) are varied, but they basically refer to vision techniques (often in 3D, and sometimes with ultrasound) and are able to understand the difference between the “fruit” and the rest of the vegetation.

An excellent support to the more efficient identification of the products of the robotized collection can be provided by the adaptation of the crop, that is to say, the possibility to concentrate to the greatest extent possible, the productive area, for example through appropriate pruning (for those species that adapt well) and/or with interventions of a chemical nature or, more often, physical defoliation, which can clear away from the “obstacles” the area where the fruit is located.

 

Assessing the degree of ripeness

An additional complication in the picking of fruit and vegetable products intended for fresh consumption concerns the evaluation of the correct degree of ripeness. If from a purely human point of view it is an almost obvious consideration, given that it is based not only on the observation of the color of the fruit but also on the tactile perception of the consistency of its pulp, with the robot all this becomes considerably more complicated, considering the various and often changing daylight conditions, a parameter that is highly critical in open field situations.

Obviously then, the tactile evaluation of the consistency with the robot is impossible to pursue (think of strawberries for example...), since the storage life of the fruit could easily be compromised.

Therefore, having to assess the level of ripeness of the fruit only on its color (and sometimes on its nuances), it becomes essential to be able to count on constant intensity light sources, and with standardized chromatic component, which among other things must be little or ideally not at all influenced by natural lighting. 

A further complication arises from those fruits whose external surface is not uniform both as color and intensity of each single color (a typical example is the tomato, partly red and partly green): in this case, it is essential to establish with specific algorithms, different thresholds of discrimination, thanks to which decide whether or not to pick the fruit.

Here again, the agronomic research can be of considerable help, selecting those cultivars that, in addition to desirable organoleptic characteristics, can show a ripeness as much contextual as possible, combined with an external colorimetric aspect of the highest degree of homogeneity.

 

Picking, handling and storing the product

 After having identified with certainty the “target” on which to intervene, it is necessary to act mechanically, separating the fruit or the vegetable from the plant, with the double objective of preserving the best conditions of both.

For these phases, the techniques developed have necessarily differentiated according to the physical characteristics of the fruits, but all substantially refer to more or less sophisticated mechanical arms, electrically, hydraulically or pneumatically driven.

What makes a considerable difference is the final tool, that is to say, what physically comes into contact with the product, for which the conformation, the surface, and the movements must be aimed, at the highest possible level, at avoiding any trauma.

Finally, the (temporary) storage phase should not be underestimated and should prioritize the utmost delicacy, for example, when the fruit or the vegetable falls into the temporary container. This might appear to be a minor issue, but it can affect the timing of the routine, considering also that these are single movements repeated several thousand times.

 

The performance of the vehicle

There are several current manufacturing of fruit and vegetable harvesting robots, concentrated in particular on large scale productions, such as apples, citrus fruits, tomatoes, strawberries, etc.

In this case, the discriminating factor is not so much the product to be picked, but rather whether the area of ​​intervention concerns protected cultivation or the open field.

In the first case, in fact, the robot usually moves on well-identified and predefined paths, above all always the same in space and time. Another much more complex issue is the movement of the robotic vehicle in the open field, where in addition to the variability of the vegetation there are the irregularities and the changeability of the vehicle’s transit surface to be dealt with.

The solutions adopted are extremely varied, ranging from the classic tractor-driven operator to the self-propelled vehicle based on pneumatic wheels or on tracks (often made of rubber) as propulsion systems, or on machines borrowed from the industrial sector, such as lifting platforms.

The following is a quick (and certainly not exhaustive) review of the most recent achievements, most of which are still at the prototype stage.

 

Metomotion 

The Israeli Metomotion has developed an autonomous robot for picking fresh tomatoes grown in greenhouses (Greenhouse Robotic Worker - GRoW), based on a 4-wheel vehicle with autonomous driving, equipped with two robotic arms, an advanced system of vision and detection of ripe berries to be picked as well as an original solution (mechanical as well) for picking the product. Basically, the terminal tool of the arm is a pantograph mechanism that operates in the horizontal plane, identifies the bunch with the berries, goes at the level of the stalk, and tightening the pantograph gently tends it towards the machine body by cutting and grasping, and then placing the stalk on a conveyor belt, which conveys it into the storage hopper.

Manufacturers claim that GRoW reduces tomato damage compared to manual picking while increasing picking efficiency. As for costs, they envisage that one operator, up to 5 robots could control up to 5 robots at the same time, with a reduction of up to 50% of the manual picking cost.

Among other things, the Metomotion robot can be adapted for picking other vegetables, but also for other cultivation operations, such as pruning, pollination and cultivation monitoring.

 

Agrobot

The Spaniards of Agrobot have manufactured the SW6010, a robot based on a “gantry” (that is to say, a vehicle with a portal equipped with tires) designed to pick strawberries for fresh consumption, both in the field and in protected areas.

Strawberries are a very delicate fruit: they have to be picked, unlike apples and bananas when they are perfectly ripe. Even the slightest pressure exerted on the fruit triggers a process of deterioration that makes them unsaleable. Strawberries are a very delicate fruit: they have to be picked, unlike apples and bananas when they are perfectly ripe. Even the slightest pressure exerted on the fruit triggers a process of deterioration that makes them unsaleable.

For this reason, to protect them from excessive pressures or falls, the fruits (which by their nature are pendulous with respect to the foliage, therefore easily intercepted) are separated from the stems with a pincer tool equipped with two sharp blades, which after taking the single strawberry lay it on a conveyor belt directed to the packaging area.

A large series of robotic arms control the interactions of blades and baskets with the fruits, while a sophisticated camera-based vision system analyzes each fruit individually, examining its shape and color and activating its picking if considered ripe.

For field operation, the machinery is equipped with a motion control based on ultrasonic sensors, which in addition to preventing any contact between the arm and the ground also ensure an adequate safety distance from the vegetation, thus ensuring the correct trajectory between the rows of the crop. The whole is further assisted by an inductive sensor for angular measurement of the absolute position, which intervenes continuously on the steering device.

 

FFRobot

Still from Israel, FFRobotics proposes a robot for picking apples, where the identification of the fruits to be picked is based, as in many other cases, on a sophisticated vision system working for advanced algorithms for the detection of ripe fruits, but also of those unsuitable, because damaged. http://www.ffrobotics.com/

The robot, placed on a machinery towed by a tractor, picks the single apple through a linear movement arm at the end of which there is a 3-finger gripper, which from the position of maximum opening, gently grabs the fruit, then making it perform a 90° rotation, so as to cause the stem to detach. Afterward, the apples are placed on a conveyor belt equipped with individual containers and then conveyed to the harvesting basket.  Also, according to the type of plant cultivation, the robot is usually able to collect up to 80-90% of the fruit, as it is not able, at the moment, to detect those completely hidden by the foliage or the branches. The evolution of the robot will be to be adapted so as to be used for harvesting peaches and lemons, in order to extend the period of annual use, so as to further reduce costs.

 

Abundant robotics

Still on the subject of picking apple robots, the Californian Abundant Robotics has built a piece of machinery, driven by the tractor, which stands out from the others because it picks the fruit by “sucking” it. The alternate vertical movement of the robotic arm, combined with the forward motion of the vehicle, continuously scans the row, in order to identify the apples ready for the harvest. The extraction takes place by suction, bringing the suction side close to the fruit, creating a depression. The apple is then conveyed by the air stream into the collection basket.

 

Sweeper EU

An interesting project funded by the European Union, involving 6 partners from the Netherlands, Belgium, Sweden, and Israel, is developing the Sweeper, a robot currently designated to harvest peppers grown in greenhouses. On the frame of a small panto­graph elevating plat­form, powered by a battery-powered electric motor, a robotic arm is installed which includes both the product detection system, which uses powerful LED flashes to make itself independent from the variations of the intensity of natural light and to be able to operate even in conditions of darkness, and the system to pick up the individual pepper, whose petiole is first blocked and then cut with a small double oscillating blade. The pepper then falls gently on a support with fixed curved fingers, which then lays it in the collection box, positioned on board. At the moment, the robot is characterized by still low productivity: in fact, it takes 24 seconds in the greenhouse to harvest a single pepper, and in any case, even under controlled laboratory conditions it has not been possible to go below 15 seconds.

A few words in conclusion

The current situation on the robotic harvesting of agricultural products does not actually bode for an imminent massive replacement of human labor. Studies in this field of research have been going on for many years, but the impression is that progress is less exciting than what is claimed. It is also true that automation (especially in the processes of vision), precision mechanics and artificial intelligence techniques have made great strides, but in any case, there are three fundamental obstacles to the spread of this technique: robot productivity is currently not yet fully competitive with human work; to motivate and increase the adoption of robots, much emphasis is given to the possible replacement of human labor in a task so boring, annoying and sometimes burdensome, but in some cases, there is the problem of employing the operating units differently; for some crops and certain types of cultivations, robotic harvesting does not yet provide the results of the human labor, especially concerning the discrimination of the product not suitable for harvesting and for the efficient harvesting of those fruits difficult to pick