Comprehensive Classification and Challenges of Non-Functional Requirements in Machine Learning-Enabled Software Systems

المفاهيم الأساسية

This systematic literature review provides a comprehensive classification of 30 non-functional requirements affecting machine learning-enabled software systems, and identifies over 23 key challenges faced by researchers and practitioners when dealing with these non-functional requirements.

الملخص

The study conducted a systematic literature review to address two key objectives:

Classify the non-functional requirements of machine learning-enabled software systems identified in the research literature.

The review identified a total of 30 distinct non-functional requirements, which were grouped into 6 main classes: Accuracy, Resiliency, Performance, Interpretability, Sustainability, and Fairness.
The non-functional requirements span across various domains where machine learning is applied, including environmental, healthcare, and financial domains.

Identify the key challenges faced when dealing with non-functional requirements in machine learning-enabled systems.

The review compiled a catalog of over 23 software engineering challenges, including issues with requirements elicitation, modeling, testing, and optimization of non-functional attributes.
The challenges cover aspects such as the inherent trade-offs between different non-functional requirements, the difficulty in specifying and measuring certain non-functional properties, and the need for automated approaches to handle the complexity.
The findings provide a comprehensive overview of the state of research on non-functional requirements in machine learning-enabled systems, and highlight important directions for future work to better support practitioners in developing reliable and trustworthy AI-powered software.

الإحصائيات

"memory problems and battery drain" [S1]
"accuracy" [S2-S54]
"robustness" [S2-S7, S9, S10, S12, S19, S20, S22-S24, S27-S31, S40, S45, S46, S48, S49, S51, S53, S55-S64]
"security" [S2, S3, S7, S9, S12, S14, S19, S22-S24, S27, S31-S33, S36, S45, S49, S55, S56, S58-S60, S62]
"performance" [S1, S10, S13, S15-S17, S28-S30, S32, S37, S39, S40, S46, S50, S52, S53, S58-S60, S65]
"fairness" [S13, S15, S17, S21, S26, S33-S35, S41, S42, S54, S61, S63-S69]
"behavioral" [S7, S14, S20, S28, S43, S47-S49, S59, S61, S63]
"interpretability" [S14, S17, S27, S33, S38, S39, S45, S59]
"transferability" [S24, S31, S49, S55, S57, S58, S60]
"safety" [S6, S7, S14, S24, S47, S50, S57]
"reliability" [S20, S27, S28, S33, S48, S59, S62]
"explainability" [S9, S21, S38, S39, S61, S68]
"retrainability" [S3, S5, S23, S46, S62]

اقتباسات

"memory problems and battery drain" [S1]
"accuracy" [S2-S54]
"robustness" [S2-S7, S9, S10, S12, S19, S20, S22-S24, S27-S31, S40, S45, S46, S48, S49, S51, S53, S55-S64]
"security" [S2, S3, S7, S9, S12, S14, S19, S22-S24, S27, S31-S33, S36, S45, S49, S55, S56, S58-S60, S62]

الرؤى الأساسية المستخلصة من

Classification, Challenges, and Automated Approaches to Handle Non-Functional Requirements in ML-Enabled Systems

by Vincenzo De ... في arxiv.org 04-11-2024

https://arxiv.org/pdf/2311.17483.pdf

استفسارات أعمق

How can the identified non-functional requirements be effectively prioritized and balanced during the development of machine learning-enabled systems?

In the context of machine learning-enabled systems, prioritizing and balancing non-functional requirements is crucial to ensure the overall success and effectiveness of the system. Here are some strategies to achieve this:

Stakeholder Involvement: Engage with stakeholders to understand their priorities and expectations regarding non-functional requirements. This will help in aligning the development process with the most critical requirements.

Impact Analysis: Conduct an impact analysis to assess the potential consequences of not meeting specific non-functional requirements. This will help in prioritizing requirements based on their impact on the system.

Risk Assessment: Evaluate the risks associated with each non-functional requirement and prioritize them based on the level of risk they pose to the system.

Trade-off Analysis: Identify potential trade-offs between different non-functional requirements and work with stakeholders to make informed decisions on how to balance conflicting requirements.

Continuous Monitoring: Regularly monitor the performance of the system against the identified non-functional requirements and adjust priorities as needed based on real-world performance data.

Automated Tools: Utilize automated tools and techniques to track, analyze, and prioritize non-functional requirements throughout the development lifecycle.

By following these strategies, development teams can effectively prioritize and balance non-functional requirements to ensure the successful development of machine learning-enabled systems.

How can the lessons learned from addressing non-functional requirements in traditional software systems be adapted and applied to the unique challenges posed by machine learning-enabled systems?

Lessons learned from addressing non-functional requirements in traditional software systems can be adapted and applied to machine learning-enabled systems in the following ways:

Establish Clear Requirements: Just like in traditional software systems, it is essential to establish clear and measurable non-functional requirements for machine learning-enabled systems. This includes defining requirements related to performance, security, reliability, and other key aspects.

Consider Domain-specific Challenges: Machine learning systems have unique challenges such as model interpretability, fairness, and bias. Lessons learned from traditional systems can be adapted to address these domain-specific challenges effectively.

Testing and Validation: Lessons learned from traditional software testing and validation processes can be applied to machine learning systems to ensure that non-functional requirements are met. This includes rigorous testing, validation, and performance monitoring.

Risk Management: Lessons learned from traditional software risk management practices can be adapted to identify and mitigate risks associated with non-functional requirements in machine learning systems.

Collaboration and Communication: Effective collaboration and communication between stakeholders, developers, and data scientists are essential for addressing non-functional requirements in machine learning systems. Lessons learned from traditional software development in this area can be valuable.

By adapting and applying lessons learned from traditional software systems, development teams can navigate the unique challenges posed by machine learning-enabled systems more effectively.

What automated techniques can be developed to systematically analyze and optimize multiple, potentially conflicting non-functional requirements in machine learning-enabled systems?

Automated techniques can play a crucial role in systematically analyzing and optimizing multiple, potentially conflicting non-functional requirements in machine learning-enabled systems. Here are some techniques that can be developed:

Multi-Objective Optimization: Develop algorithms that can optimize multiple non-functional requirements simultaneously by treating them as competing objectives. Techniques like Pareto optimization can help in finding optimal solutions that balance conflicting requirements.

Constraint Satisfaction: Use constraint satisfaction techniques to ensure that all non-functional requirements are met while considering their interdependencies and potential conflicts.

Machine Learning Models: Develop machine learning models that can predict the impact of different configurations on non-functional requirements. These models can help in identifying optimal solutions that satisfy multiple requirements.

Automated Monitoring and Adjustment: Implement automated monitoring systems that continuously track the performance of the system against non-functional requirements and automatically adjust configurations to optimize performance.

Simulation and Modeling: Use simulation and modeling techniques to evaluate the impact of different configurations on non-functional requirements before implementation. This can help in identifying potential conflicts and finding optimal solutions.

Feedback-driven Optimization: Implement feedback loops that collect data on system performance and use this information to iteratively optimize non-functional requirements.

By developing and implementing these automated techniques, development teams can effectively analyze and optimize multiple non-functional requirements in machine learning-enabled systems, ensuring optimal performance and reliability.