Kernkonzepte
The Neural-Symbolic Recursive Machine (NSR) is a principled framework that integrates neural perception, syntactic parsing, and semantic reasoning to achieve human-like systematic generalization across diverse sequence-to-sequence tasks.
Zusammenfassung
The paper introduces the Neural-Symbolic Recursive Machine (NSR), a model designed to achieve systematic generalization in sequence-to-sequence tasks. At the core of NSR is a Grounded Symbol System (GSS), which emerges directly from training data without the need for domain-specific knowledge.
The NSR model has a modular design, consisting of three trainable components:
- Neural Perception: Converts raw input (e.g., handwritten expressions) into a symbolic sequence.
- Dependency Parsing: Infers dependencies between the symbols using a transition-based neural parser.
- Program Induction: Deduces the semantic meaning of symbols using functional programs.
The key aspects of NSR are:
- Inductive biases of equivariance and compositionality, which enable the decomposition, sequential processing, and recomposition of complex inputs.
- A novel deduction-abduction algorithm for end-to-end training of the model without intermediate supervision for the GSS.
NSR is evaluated on three challenging benchmarks that test systematic generalization:
- SCAN: Translating natural language commands to action sequences.
- PCFG: Predicting output sequences from string manipulation commands.
- HINT: Computing results of handwritten arithmetic expressions.
NSR outperforms state-of-the-art models on these benchmarks, achieving 100% generalization accuracy on SCAN and PCFG, and surpassing the previous best accuracy on HINT by 23%. The analyses show that NSR's modular design and intrinsic inductive biases lead to stronger generalization and enhanced transferability compared to traditional neural networks and existing neural-symbolic models.
Statistiken
NSR achieves 100% generalization accuracy on the SCAN and PCFG benchmarks.
NSR surpasses the previous best accuracy on the HINT benchmark by 23%.
NSR demonstrates 100% generalization accuracy on a compositional machine translation task.
Zitate
"NSR's capacity for various sequence-to-sequence tasks, underpinned by the inductive biases of equivariance and compositionality, allows for the decomposition of complex inputs, sequential processing of components, and their recomposition, thus facilitating the acquisition of meaningful symbols and compositional rules."
"NSR establishes new records on these benchmarks, achieving 100% generalization accuracy on SCAN and PCFG, and surpassing the previous best accuracy on HINT by 23%."