Integrating Computational Creativity Principles to Enable Creative Problem Solving in Large Language and Vision Models

Core Concepts

Computational Creativity principles can be integrated with research in large language and vision models to address their key limitation in creative problem solving.

Abstract

This paper discusses approaches for integrating Computational Creativity (CC) with research in large language and vision models (LLVMs) to address the key limitation of these models in creative problem solving. The authors first provide an overview of how LLVMs are typically used in task planning, highlighting the potential entry points for introducing creative problem solving capabilities. They then discuss how principles from CC literature, specifically Boden's three forms of creativity (exploratory, combinational, and transformational), can be extended to augment the embedding spaces of LLVMs for enabling creative problem solving. The authors present preliminary experiments demonstrating the application of transformational creativity, where they show that providing information about object affordances in prompts can improve the ability of LLVMs to creatively replace missing objects. However, they note that the full integration of all three forms of creativity is likely necessary for effective creative problem solving in LLVMs. The paper emphasizes the need for a deeper integration of CC and ML research, as creative problem solving is not only a key limitation of current LLVMs but also potentially linked to the broader goal of Artificial General Intelligence (AGI). The authors hope this work will encourage discussions on creative problem solving and CC within the ML community.

Stats

"Creativity is "...the ability to come up with an idea which, relative to the pre-existing domain-space in one's mind, one could not have had before." "Intelligence is the ability to work and adapt to the environment with insufficient knowledge and resources." "Some exploration merely shows us the nature of the relevant conceptual space that we had not explicitly noticed before." "To overcome a limitation in the conceptual space, one must change it in some way."

Quotes

"Creativity is "...the ability to come up with an idea which, relative to the pre-existing domain-space in one's mind, one could not have had before. Whether any other person (or system) has already come up with it on an earlier occasion is irrelevant." "Intelligence is the ability to work and adapt to the environment with insufficient knowledge and resources." "Some exploration merely shows us the nature of the relevant conceptual space that we had not explicitly noticed before." "To overcome a limitation in the conceptual space, one must change it in some way."

Key Insights Distilled From

Creative Problem Solving in Large Language and Vision Models -- What Would it Take?

by Lakshmi Nair... at arxiv.org 05-03-2024

https://arxiv.org/pdf/2405.01453.pdf

Deeper Inquiries

How can the three forms of creativity (exploratory, combinational, and transformational) be effectively integrated within a unified framework for enabling creative problem solving in LLVMs?

To integrate the three forms of creativity within a unified framework for enabling creative problem solving in Large Language and Vision Models (LLVMs), a systematic approach is required. Here's how each form can be effectively integrated: Exploratory Creativity: This form involves exploration within the conceptual or embedding space of the model, akin to search. To integrate exploratory creativity, LLVMs can be designed to explore the embedding space beyond the typical output space. This can involve heuristic or non-heuristic-based search within the model's embedding space to uncover novel solutions that go beyond the model's standard outputs. Combinational Creativity: Combinational approaches entail combining two concepts to create something new. In LLVMs, this can be achieved by incorporating cross-attention layers that allow the model to combine information from different modalities or sources within its embedding space. By enabling the model to blend concepts within its own embedding space, novel solutions can be generated through creative combinations. Transformational Creativity: Transformational creativity involves transforming existing conceptual spaces to produce new ones. In LLVMs, this can be realized through prompt engineering and in-context learning techniques. By providing prompts that reframe the problem or introduce new perspectives, the model's embedding space can be transformed to facilitate creative problem solving. By integrating these three forms of creativity within a unified framework, LLVMs can leverage a comprehensive set of cognitive processes to tackle creative problem-solving tasks. This approach allows for a holistic exploration of the model's capabilities and enhances its ingenuity in generating novel solutions.

How can the potential limitations or drawbacks of relying solely on prompting techniques for transformational creativity in LLVMs be addressed?

While prompting techniques are valuable for guiding models in transformational creativity, there are potential limitations and drawbacks that need to be addressed: Prompt Bias: Relying solely on predefined prompts can introduce bias and limit the model's ability to explore diverse solutions. To address this, prompts should be carefully designed to encourage open-ended exploration and avoid steering the model towards specific solutions. Limited Flexibility: Predefined prompts may restrict the model's flexibility in redefining the problem space. To overcome this limitation, dynamic prompting strategies can be implemented, where prompts evolve based on the model's responses and feedback, allowing for adaptive problem re-representation. Lack of Novelty: Prompting techniques alone may not always lead to truly novel or creative solutions, as they rely on existing information provided in the prompts. To enhance transformational creativity, prompts should be designed to encourage out-of-the-box thinking and challenge the model to explore unconventional problem-solving approaches. Overfitting to Prompts: Models may become overly reliant on specific prompts, leading to overfitting and limited generalization to new problem domains. To mitigate this, a diverse set of prompts should be used, and the model should be trained on a wide range of problem-solving scenarios to foster adaptability and creativity. By addressing these limitations through dynamic and diverse prompting strategies, LLVMs can enhance their transformational creativity capabilities and effectively tackle complex and ill-defined problem-solving tasks.

Given the potential link between creative problem solving and Artificial General Intelligence, what other cognitive capabilities beyond creativity should be targeted to advance the development of AGI systems?

In advancing the development of Artificial General Intelligence (AGI) systems, beyond creativity, several other cognitive capabilities should be targeted to achieve comprehensive intelligence: Reasoning and Logic: AGI systems should possess robust reasoning and logical capabilities to analyze complex problems, infer relationships, and make informed decisions based on available information. Incorporating formal logic and probabilistic reasoning can enhance the system's problem-solving abilities. Memory and Learning: Memory mechanisms, including short-term and long-term memory, are essential for retaining information, learning from past experiences, and adapting to new tasks. AGI systems should exhibit efficient learning algorithms and memory architectures to facilitate continuous improvement and knowledge retention. Adaptability and Generalization: AGI systems need to demonstrate adaptability to varying environments and tasks, as well as the ability to generalize knowledge across different domains. Developing algorithms for transfer learning, meta-learning, and domain adaptation can enhance the system's versatility and applicability. Social and Emotional Intelligence: Integrating social and emotional intelligence into AGI systems can enable them to interact effectively with humans, understand emotions, and exhibit empathy. Emulating human-like social behaviors and emotional responses can enhance the system's communication and collaboration capabilities. Ethical and Moral Reasoning: AGI systems should be equipped with ethical reasoning frameworks to make morally sound decisions and adhere to ethical principles. Incorporating ethical guidelines and value systems into the system's decision-making processes is crucial for responsible AI development. Sensorimotor Skills: Integrating sensorimotor skills, such as perception, manipulation, and control, is essential for AGI systems to interact with the physical world. Developing algorithms for sensor fusion, motor control, and dexterous manipulation can enhance the system's ability to perform real-world tasks. By targeting these cognitive capabilities in addition to creativity, AGI systems can progress towards achieving human-like intelligence and exhibit a comprehensive range of cognitive functions necessary for autonomous and adaptive behavior in diverse contexts.