Adaptive AI



Description

My first bachelor thesis for the bachelor course MultiMediaTechnology is called

"Managing Adaptive High Level Behavior of Computer Controlled Agents in Virtual Games using Neuroevolution"

and deals with an approach to AI that is able to adapt its behavior to an unknown or ever-changing environment. This video shows an example of my approach to an adaptive AI during the process of adjusting itself to the circumstances of a previously unknown environment.


AI Setup:

The AI utilizes an Evolutionary Artificial Neural Network to adaptively classify the virtual character's current situation in combination with a Finite State Machine that stores the NPC's potential reactions to occurring situations.

In this concept the ANN's task is to analyze the current situation and match one of the states stored in the AI's FSM as an appropriate reaction to this situation. The AI's ANN has a separate input node for each of the parameters that form the NPC's current situation. Furthermore the ANN operates several output nodes, each representing one of the different states stored in the FSM. During runtime the AI's ANN analyzes the current situation and in this manner computes output values in each of its output nodes. Subsequently the FSM is ordered to chance the NPC's current state to the state with the highest value in its respective output node of the ANN.
In the experiments this setup is applied to control the high level behavior of a non-player character.


Experimental Setup:

In the experimental environment piles of food (red dots) are randomly distributed among its rectangular surface which is also populated by the NPCs (moving dots). The aim of the NPCs is to gather the food. When a pile is absorbed a new one appears at a random position. A small house which serves as the NPCs' spawn point is positioned in the center of the area. Additionally the environment runs a weather simulation by modifying a raininess-parameter.
During rain an NPC becomes wet (indicated by a bluish coloring) when it stays outside the house. After a while a wet NPC becomes ill (greenish coloring). When it is not located in the rain it dries. To convalesce the NPC has to stay dry for a certain amount of time.
In this experiment three different inputs feed into the ANN:

  • The environment's raininess-parameter: Since the model of the NPC perceives its environment with its eyes it is able to watch the sky and detect whether it is clear, overcast, drizzling or raining heavily.
  • The NPC's wetness-value: This parameter is ranged between 0 and 1 and serves the AI with information on the NPC's wetness. A higher value indicates that the NPC will become ill in the near future.
  • The NPC's illness-value: This value measures the NPC's illness. Like the wetness-value it is ranged between 0 and 1 where 0 indicates the NPC to be in good health.
The AI's FSM stores only two different states:
  • GoHome: In this state the NPC moves straight to the house in the center of the area and stays there until the state is changed.
  • GatherFood: The NPC wanders around until it detects a pile of food in its range of vision. When this occurs the NPC approaches the food to gather it.


Results:

The initial AI acts randomly but after only 10 generations all the entities in the population converge to a similar behavior. They have learned that it is best to stay inside the house during rain and go out to gather food when the sun is shining. Subsequent generations stick to this policy.