Evolution Transformer: In-Context Evolutionary Optimization Unveiled

The author introduces the Evolution Transformer, a causal Transformer architecture for Evolution Strategies, trained through Evolutionary Algorithm Distillation to optimize in-context learning. The approach aims to discover powerful optimization principles via meta-optimization.