LAVA: Long-horizon Visual Action based Food Acquisition Study

Core Concepts

Introducing LAVA for efficient food acquisition using a hierarchical policy framework.

Abstract

The study introduces Long-horizon Visual Action (LAVA) for acquiring liquid, semisolid, and deformable foods. The framework employs a hierarchical policy with high-level, mid-level, and low-level policies. LAVA outperforms baselines in real-world trials across various food types with a success rate of 89 ± 4%. The paper details the system architecture, experimental setup, data collection process, baselines comparison, and zero-shot generalization results.
I. Introduction

Robotic Assisted Feeding (RAF) aims to restore independence in feeding for individuals with mobility impairments.
Existing RAF methods focus on solid foods, leaving a gap in manipulation strategies for semi-solid and deformable foods.
LAVA introduces Long-horizon Visual Action for acquiring diverse food types efficiently.
II. Related Work

Prior works have explored robot-assisted feeding focusing on bite acquisition and transfer.
Learning from Demonstration (LfD) is utilized for developing new skills by observing expert demonstrations.
Long-horizon planning frameworks separate high-level strategic decision-making from detailed motion planning.
III. Problem Statement

The challenge addressed is sequential bite acquisition to maximize efficiency in long-horizon food acquisition.
Access to bowl image observations and expert demonstration data is assumed to learn a policy for efficient food acquisition.
IV. Proposed Approach

A hierarchical policy framework is formalized into high-level, mid-level, and low-level sub-policies.
High-level policy selects manipulation primitives based on visual inputs; mid-level refines these primitives; low-level executes actions.
V. Experiments

Experimental setup includes a UR5e robot arm with custom spoon attachment and RealSense camera.
Data collection involves kinesthetic teaching focusing on cereals and tofu.
Baseline models include LAVA-low and Fixed Trajectory Scooping (FTS).
Results show LAVA's superior performance in efficiency, spillage reduction, breakage prevention compared to baselines across various food types.
VI. Conclusion & Future Work

LAVA demonstrates robust performance across varied configurations including soup with tofu chunks through zero-shot generalization.
Limitations exist in handling thin or irregular foods requiring specialized strategies.
Future work will focus on broadening action space for diverse food types and exploring efficient data acquisition methods.

Stats

Across 46 bowls, LAVA acquires much more efficiently than baselines with a success rate of 89 ± 4%.

Quotes

"LAVA adeptly adjusts to real-time changes in food depth."
"Our approach demonstrates robust performance across varied configurations."

Key Insights Distilled From

LAVA

by Amisha Bhask... at arxiv.org 03-20-2024

https://arxiv.org/pdf/2403.12876.pdf

Deeper Inquiries

How do all the methods handle the challenge of scooping liquids prone to spillage?

In the context provided, the baseline models and LAVA approach were evaluated in handling liquids like water and soup, which are prone to spillage. The baselines struggled with fluidity issues, leading to significant spillage due to their static approaches. FTS model's fixed end-effector orientation and height did not adjust well to liquid dynamics, resulting in inefficiency and spillage. LAVA-low coped initially but faced challenges as water levels decreased, showing inefficiency in maintaining scoop sizes. On the other hand, LAVA adeptly adjusted to real-time changes in food depth by achieving optimal scoop sizes and minimizing spillage for efficient bowl clearance.

What about the acquisition of more solid yet deformable food types like tofu?

The evaluation included testing how different methods handled solid yet deformable foods like tofu. Baseline models encountered difficulties with deformable foods such as tofu, often causing breakages during scooping attempts. The rigid scooping motion of FTS led to damage on the food items while Lava-low managed some successful scoops but caused accumulation and breakage due to a lack of strategic prioritization based on subregions. In contrast, LAVA prioritized tofu chunks based on their subregion positioning them for easier access which significantly reduced breakages compared to the baselines.

How does each method fare in preventing spillage with solid foods like fruit chunks?

When it comes to preventing spillage with solid foods such as fruit chunks, both baseline models showed less efficiency compared to LAVA approach. Solid irregularly shaped foods posed challenges for baselines leading them prone rolling or falling off spoons especially fruits with curved surfaces where they struggled even more. However, LAVA employing an align-then-scoop strategy ensured better alignment adapting well even with irregular shapes reducing spills by adjusting its strategy according to fruit shape ensuring secure scooping without compromising efficiency making it outperform both baseline models significantly when dealing with solid food types like fruit chunks.