Core Concepts
A hybrid strategy that leverages the strengths of large language models (LLMs) and structured safety engineering models to enable interactive, concept-guided analysis and codesign of complex system safety.
Abstract
The article presents a concept-guided approach to enhance the capabilities of LLMs for graph analysis and manipulation, particularly in the context of safety-relevant system development. The key components are:
-
System Model and Intermediate Representation (IR):
- The system model is created using the OSATE tool and exported as an ECore file, which is then verbalized into an intuitive list-based IR.
- The IR represents the system architecture, including components and their interactions, as well as safety-related information such as fault propagation logic.
-
LLM Agent:
- The custom LLM agent deploys a hybrid strategy, combining prompt engineering, heuristic reasoning, and retrieval-augmented generation (RAG) techniques.
- The agent runs a cascading decision layer to identify the task type (e.g., safety question answering, system safety analysis, suggestions for fault-tolerance) and a subsequent layer to formulate the task for information retrieval.
- The agent leverages external tools and a database of system descriptions, relevant documents, and safety concepts to perform tasks such as fault propagation analysis, critical path calculation, single point of failure detection, and graph manipulation for fault-tolerance.
-
Experiments and Results:
- The approach is tested on a simplified automated driving system model, demonstrating the agent's ability to accurately identify tasks, retrieve relevant information, and provide meaningful insights and suggestions for improving system safety.
- The agent can make suggestions for modifying the system graph to improve fault tolerance, based on the concept of redundancy and the identification of single points of failure.
The proposed framework provides the basis for an interactive, LLM-based Human-AI safety codesign approach, where the LLM agent assists engineers in analyzing and enhancing the safety of complex systems.
Stats
If IMU, Radar1, and Radar2 have a fault, it can lead to inaccurate measurements, missed or incorrect object detections, and incorrect data processing, which can degrade the system's performance and safety.
The critical path includes components such as Camera1, Camera2, CollisionAvoidance, GPS, IMU, ImageProcessor, Lidar1, Map, PathPlanner, PointCloudProcessor, SensorFusion, and VehicleController.
The single points of failure in the system are PathPlanner, VehicleController, Map, SensorFusion, CollisionAvoidance, and GPS.
Quotes
"When these components have a fault, it can lead to degraded performance, reduced safety, and potentially compromised functionality of the system."
"The critical path represents the sequence of components and processes that are essential for the system's operation and performance. Any delay or failure in these components can significantly impact the overall functionality and reliability of the system."
"These components are considered single points of failure because if any of them were to fail, it could result in a complete system failure or significant degradation in the system's performance."