By Jacob Schrum schrum2@cs.utexas.edu
Motivation Intelligent agents are needed - Search-and-rescue robots
- Mars exploration
- Training simulations
- Video games
Insight into nature of intelligence - Sufficient conditions for emergence of:
- Cooperation
- Communication
- Multimodal behavior
Talk Outline Bio-inspired learning methods - Neural networks
- Evolutionary computation
My research Future work Conclusion
Artificial Neural Networks Brain = network of neurons ANN = abstraction of brain - Neurons organized into layers
What Can Neural Networks Do? In theory, anything! - Universal Approximation
- Theorem
-
Can’t program: too complicated In practice, learning/training is hard - Supervised: Backpropagation
- Unsupervised: Self-Organizing Maps
- Reinforcement Learning: Temporal-Difference
- and Evolutionary Computation
Evolutionary Computation Computational abstraction of evolution - Descent with modification (mutation)
- Sexual reproduction (crossover)
- Survival of the fittest (natural selection)
Evolution + Neural Nets = Neuroevolution - Population of neural networks
- Mutation and crossover modify networks
- Net used as control policy to evaluate fitness
Neuroevolution Example
Neuroevolution Example
Neuroevolution Example
Neuroevolution Example
Neuroevolution Applications
Neuroevolution Applications
Neuroevolution Applications
Neuroevolution Applications
What is Missing? NERO agents are specialists - Sniping from a distance
- Aggressively rushing in
Humans can do all of this, and more Multimodal behavior - Different behaviors for different situations
Human-like behavior
What I do With Neuroevolution Discover complex agent behavior Discover multimodal behavior Contributions: Use multi-objective evolution - Different objectives for different modes
Evolve modular networks - Networks with modules for
- each mode
Human-like behavior
Pareto-based Multiobjective Optimization
Non-dominated Sorting Genetic Algorithm II Population P with size N; Evaluate P Use mutation (& crossover) to get P´ size N; Evaluate P´ Calculate non-dominated fronts of P P´ size 2N New population size N from highest fronts of P P´
Ms. Pac-Man Popular classic game Predator-prey scenario - Ghosts are predators
- Until power pill is eaten
Multimodal behavior needed - Running from threats
- Chasing edible ghosts
- More?
Modular Networks Different areas of brain specialize - Structural modularity → functional modularity
Apply to evolved neural networks - Separate module → behavioral mode
Preference neurons (grey) arbitrate between modules Use module with highest preference output
Module Mutation Let evolution decide how many modules
Intelligent Module Usage Evolution discovers a novel task division Dedicates one module to luring (cyan) Improves ghost eating when using other module
Comparison With Other Work
Types of Intelligence Evolved intelligent Ms. Pac-Man behavior - Surprising module usage
- Evolution discovers the unexpected
- Diverse collection of solutions
Still not human-like - Human-like vs. optimal
- Human intelligence
Modern Game: Unreal Tournament 3D world with simulated physics Agents attack, retreat, explore, etc. Multimodal behavior required to succeed
Human-like Behavior: BotPrize International competition at CIG conference A Turing Test for video game bots - Judge as human over 50% of time to win
- After 5 years, we won in 2012
Evolved combat behavior - Constrained to
- be human-like
Guessing Game Coleman: ???? Milford: ???? Moises: ???? Lawerence: ???? Clifford: ???? Kathe: ???? Tristan: ???? Jackie: ????
Judging Game
Player Identities Coleman: UT^2 (Our winning bot) Milford: ICE-2010 (bot) Moises: Discordia (bot) Lawerence: Native UT2004 bot Clifford: w00t (bot) Kathe: Human Tristan: Human Jackie: Native UT2004 bot
Human Subject Study Six participants played the judging game Recorded extensive post-game interviews What criteria to humans claim to judge by?
Lessons Learned - Evolved with accuracy restrictions
- Disable elaborate dodging
Humans are “tenacious” - Opponent-relative actions
- Encourage “focusing” on opponent
Don’t repeat mistakes - Database of human traces to get unstuck
Bot Architecture
Future Work Evolving teamwork - Ghosts must cooperate to eat Ms. Pac-Man
- Unreal Tournament supports team play
- Domination, Capture the Flag, etc.
Interactive evolution - Evolve in response to human interaction
- Adaptive opponents/assistants
- Evolutionary art
- Content generation
Conclusion Evolution discovers unexpected behavior Modular networks learn multimodal behavior Human behavior not optimal - Evolution can be constrained to be
- more human-like
Many directions for future research
Questions? contact Jacob Schrum schrum2@cs.utexas.edu
Dostları ilə paylaş: |