Online Deep Neural Network-Driven Nonlinear Model Predictive Control for Stylistic Humanoid Robot Walking with Step Adjustment

Kernkonzepte

This research proposes a novel three-layered architecture for achieving stylistic, human-like walking in humanoid robots, combining a Deep Neural Network (DNN) for trajectory generation with a Model Predictive Controller (MPC) for online step adjustment and dynamic feasibility.

Zusammenfassung

Bibliographic Information:

Romualdi, G., Viceconte, P. M., Moretti, L., Sorrentino, I., Dafarra, S., Traversaro, S., & Pucci, D. (2024). Online DNN-driven Nonlinear MPC for Stylistic Humanoid Robot Walking with Step Adjustment. 2024 IEEE-RAS International Conference on Humanoid Robots. IEEE.

Research Objective:

This research aims to develop a robust and adaptable locomotion control architecture for humanoid robots that enables stylistic, human-like walking while maintaining dynamic stability and handling external disturbances.

Methodology:

The researchers propose a three-layered hierarchical architecture:

1. Trajectory Generation Layer: A Mode-Adaptive Neural Network (MANN) trained on human motion capture data generates nominal joint trajectories, base poses, and footstep plans, mimicking human-like walking styles.
2. Trajectory Adjustment Layer: This layer, implemented as either a Receding Horizon Planner (RHP) or a Model Predictive Controller (MPC), ensures dynamic feasibility by adjusting the CoM trajectory and contact locations based on the nominal references from the DNN and real-time feedback. A Kalman Filter (KF) with genetically optimized parameters is used to reduce noise in joint velocity estimations for the MPC.
3. Trajectory Control Layer: This layer generates joint commands for the robot based on the adjusted trajectories, utilizing a CoM-ZMP controller, a swing foot planner, and a QP-based inverse kinematics solver.

Key Findings:

• The proposed architecture successfully enabled the ergoCub humanoid robot to achieve stylistic walking patterns resembling human motion capture data.
• Both RHP and MPC implementations of the trajectory adjustment layer demonstrated the ability to adjust footsteps online to maintain balance under external disturbances.
• The MPC implementation, coupled with the GA-tuned KF, proved more effective in handling disturbances due to its closed-loop nature and reduced noise sensitivity.

Main Conclusions:

The research demonstrates the effectiveness of integrating data-driven and model-based approaches for achieving robust and stylistic humanoid robot locomotion. The proposed architecture, particularly the MPC implementation, shows promise for real-world applications requiring adaptability and disturbance rejection.

Significance:

This research contributes to the advancement of humanoid robot locomotion control by bridging the gap between stylistic motion generation and dynamic stability. The proposed architecture offers a promising solution for developing robots capable of navigating complex environments with human-like agility and adaptability.

Limitations and Future Research:

• The current system lacks a dedicated base estimator, potentially limiting its robustness to stronger disturbances.
• The use of position control instead of torque control might hinder the robot's ability to respond naturally to perturbations.
• Future research will focus on integrating a base estimator and transitioning to torque control to enhance the system's robustness and responsiveness.

Statistiken

The humanoid robot used is 160 cm tall and weighs 56 kg.
The DNN was trained on 20 minutes of human walking motion capture data, resulting in 150k training points.
The MPC operates with a sampling time of 50 ms and a 1.2 s horizon.
The RHP operates with a sampling time of 60 ms and a 1.2 s horizon.
The trajectory control layer runs at 500 Hz.
The robot withstood disturbances up to 68 N.

Zitate

Wichtige Erkenntnisse aus

Online DNN-driven Nonlinear MPC for Stylistic Humanoid Robot Walking with Step Adjustment

by Giulio Romua... um arxiv.org 10-11-2024

https://arxiv.org/pdf/2410.07849.pdf

Tiefere Fragen

How could the integration of different sensory modalities, such as vision or tactile sensing, further enhance the robot's ability to adapt to complex and dynamic environments?

Integrating diverse sensory modalities like vision and tactile sensing can significantly bolster the robot's adaptability in complex and dynamic environments. Here's how:
Vision:

Enhanced environmental perception: Vision systems, using RGB-D cameras or LiDAR, can provide a richer understanding of the terrain. This allows the robot to perceive obstacles, uneven surfaces, stairs, and other environmental features that are not explicitly provided as input to the system.
Predictive footstep planning: By "seeing" the terrain ahead, the robot can predictively adjust its footstep placement and timing. This is particularly crucial for traversing challenging terrains where precise foot placement is critical for stability. The vision data can be integrated into the trajectory generation or trajectory adjustment layers to modify the nominal trajectory based on the perceived environment.
Robustness to external disturbances: Vision can help detect and react to external disturbances more effectively. For instance, seeing a pushing force approaching the robot can trigger preemptive adjustments in the CoM trajectory or footstep planning to mitigate the disturbance's impact.
Tactile Sensing:

Refined contact force control: Tactile sensors embedded in the robot's feet can provide valuable feedback on the contact forces experienced during locomotion. This information can be used to refine the contact force regularization in the MPC formulation, leading to more stable and compliant interactions with the environment.
Slip detection and recovery: Tactile sensing can detect foot slippage in real-time, a crucial capability for maintaining balance, especially on slippery surfaces. Upon detecting slippage, the control architecture can trigger immediate corrective actions, such as adjusting the ZMP or replanning the footstep, to prevent falls.
Improved terrain adaptability: By sensing the ground's texture and compliance, tactile feedback can help the robot adapt its gait parameters, such as step length and stiffness, for optimal performance on different surfaces (e.g., soft ground, gravel, etc.).
Incorporating these sensory modalities would involve fusing their data with the existing DNN-driven MPC framework. This could be achieved by using the sensory information to update the robot's internal state estimation, which in turn influences the MPC's predictions and control outputs.

While stylistic motion is desirable, could it potentially compromise the robot's efficiency or stability compared to more traditional, optimized gaits?

Yes, prioritizing stylistic motion derived from human MoCap data can potentially lead to trade-offs in efficiency and stability compared to traditional, optimized gaits for humanoid robots.
Efficiency:

Increased energy consumption: Human gaits, while natural-looking, are not necessarily optimized for energy efficiency. Replicating stylistic nuances might involve more complex joint movements and activations, leading to higher energy expenditure compared to gaits specifically designed to minimize energy consumption.
Reduced speed and agility: Stylistic gaits might prioritize aesthetic qualities over maximizing speed or agility. The constraints imposed by mimicking human-like motion could limit the robot's ability to achieve the same speed or rapid direction changes as gaits optimized for these performance metrics.
Stability:

Reduced stability margins: Traditional gaits often prioritize stability by maintaining a larger Zero Moment Point (ZMP) within the support polygon. Stylistic gaits, in contrast, might involve movements that temporarily reduce stability margins to achieve a more human-like appearance.
Sensitivity to disturbances: The complex joint coordination and potentially reduced stability margins associated with stylistic gaits could make the robot more susceptible to external disturbances. The control system would need to compensate effectively to maintain balance.
However, it's important to note that:

Advancements in control algorithms: Sophisticated control techniques, such as the DNN-driven MPC presented in the paper, can help mitigate some of these trade-offs. These methods can optimize for both style and stability/efficiency to a certain extent.
Task-specificity: The importance of efficiency and stability versus style depends heavily on the robot's intended application. For tasks where natural appearance and human-like motion are paramount (e.g., social robotics, entertainment), the trade-offs might be acceptable.
Therefore, finding a balance between stylistic appeal and practical considerations is crucial. Future research could explore optimizing stylistic gaits for improved efficiency and robustness while retaining their human-like qualities.

How might this research contribute to the development of assistive robotic devices or exoskeletons designed to augment human locomotion and balance?

This research holds significant promise for advancing assistive robotic devices and exoskeletons designed to enhance human locomotion and balance. Here's how:
Personalized Gait Assistance:

Adapting to individual walking styles: The use of DNNs trained on MoCap data allows the system to learn and adapt to different walking patterns. This is crucial for assistive devices, as they need to work seamlessly with the user's unique gait and preferences rather than forcing a predefined, rigid motion pattern.
Providing natural and intuitive assistance: By mimicking human-like motion, the assistive device can provide more natural and intuitive assistance, leading to greater user comfort and acceptance. This is particularly relevant for individuals with mobility impairments who might find traditional, robotic-looking movements jarring or unnatural.
Enhanced Balance and Stability:

Predictive fall prevention: The MPC-based trajectory adjustment framework can anticipate potential loss of balance based on the user's motion and environmental interactions. This allows the assistive device to proactively provide corrective forces or torques, preventing falls and improving user safety.
Adaptive support during dynamic activities: The system's ability to handle external disturbances and adjust footstep placement in real-time can be invaluable in assisting users during dynamic activities like walking on uneven terrain, climbing stairs, or navigating crowded spaces.
Improved User Experience:

Reduced cognitive burden: The intuitive nature of the assistance provided by the system can reduce the cognitive burden on the user. They can focus on their intended movements rather than constantly adjusting to the device's operation.
Increased mobility and independence: By augmenting human capabilities, these devices can enhance mobility and independence for individuals with walking difficulties, improving their overall quality of life.
Furthermore, the research on joint velocity estimation using Kalman filters can be directly applied to exoskeletons to improve the accuracy and responsiveness of motion tracking and assistance.
This research lays the groundwork for developing assistive devices that are not only functional but also user-centered, promoting natural movement and enhancing safety for individuals with mobility challenges.