Sbir phase i: trajectory optimizations and learned foliage manipulation to accelerate throughput in automated strawberry harvesting

This $275,000 SBIR Phase I project grant awarded by the National Science Foundation (NSF) under the NSF Technology, Innovation, and Partnerships program (CFDA 47.084) aims to develop software capabilities to improve automated harvesting of strawberries. The project will implement two key components:

1. A trajectory optimization module for a robot manipulator's camera to maximize information gain and reduce localization uncertainty for strawberries while respecting kinematic and collision constraints.

2. A trained neural network that estimates parameters to define a manipulation task plan for displacing foliage to maximize strawberry visibility and access for subsequent picking. The award period runs from September 15, 2023 to February 29, 2024 and is being carried out by L5 Automation Inc., a Hispanic American-owned for-profit small business.

Abstract

The broader/commercial impact of this small business innovation research (sbir) phase i project is to make automated harvesting of strawberries more efficient and effective and hence, more financially viable for growers to adopt. Automated harvesters currently deployed in conventional strawberry farms cannot reliably handle peak-season conditions when strawberries are hidden below a thick plant canopy and where plants must be displaced to view and pick the fruit. This project will develop software to expand the set of conditions whereby automation can increase productivity. Strawberries, the second most popular fruit in the united states, have the highest cost per acre to harvest because of their high touch harvesting process. Automating the harvesting process reduces labor needs, with the potential to either decrease costs and / or increase the quality of the fruit. Further, the project is expected to create high-skill jobs for american workers. This project?s main technical objective is to improve the quality and coverage of the map representation of strawberries and plants which will increase the number of harvested strawberries and the rate at which they are picked. The scope of the phase i activity is to implement two software capabilities and to test them in simulation, laboratory and field environments. The first capability is a trajectory optimization module for a camera mounted to a robot manipulator. This technology will be designed to maximize information gain and to reduce localization uncertainty for strawberries while respecting kinematic and collision constraints for the motion of the robot arm. Success is to be measured by the rate of information gain relative to a NA?VE precomputed scan. The second is a trained neural network which estimates the parameters that best define a manipulation task plan for displacing foliage to maximize strawberry visibility and access for subsequent picking. Training and inference will be done in an end-to-end fashion, allowing an estimate of the value of a given task plan from color and depth camera observations of the scene. This contrasts with a conventional pipeline which doesn?t make the most of the rich latent representations possible with neural networks. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are not planned for this award.