Artificial Neural Network

 After completing the MatLab tutorial and looking at some of the included reference text I used the O'Reilly "AI for Game Developers" book to research the topic of neural networks in games. 

The book talks about the benefits of using neural networks in games, the most intriguing of which is the idea that they can learn so as to adapt the game as you play. They discuss four ways that they can be used effectively;

Control

Used for controlling machines in particular to games racing cars. An example the book gives is Colin McRae Rally as the network was trained by observing game developers race around the courses.

Threat Assessment

Training a neural network in game using validation mainly, but in certain games such as strategy simulations they can be attuned to a players particular style of play.

Attack or Flee

You can also use them to control how certain NPCs behave towards others, such as attack or flee. They can learn by positive and negative reinforcement, an example of which they are told to attack and they die they have then learnt not to attack in that situation.

Anticipation

An example of anticipating the next move of your opponent in a fighting game, they discuss the fact that training for this type of gameplay would happen dynamically in game as each game can greatly differ.

The book goes in to greater detail about teaching neural networks but the above ways of using them seem to be the most common.

Intelligent Agents

 Intelligent agents try to achieve goals by using sensors to calculate their reaction to the environment. Nikola Kasabov defines intelligent agents with certain characteristics; 

• accommodate new problem solving rules incrementally
• adapt online and in real time
• be able to analyze itself in terms of behavior, error and success.
• learn and improve through interaction with the environment
• learn quickly from large amounts of data
• have memory-based exemplar storage and retrieval capacities
• have parameters to represent short and long term memory, age, forgetting, etc.

Russel and Norvig groups intelligent agents into five classes that represent their intelligence and capability. These range from simple reflex agents, where they act on if condition then action, to learning agents, which can expand their basic knowledge due to feedback and from that they can make informed decisions. There are other agents apart from the five classes defined by Russel and Norvig.

An common example of intelligent agents in games is Wumpus World(Hunt the Wumpus). It is a simple block world that can be used to represent reason and knowledge. The goal is to get the gold without dying and the wumpus has a set of rules to follow.

Ant, Bee, and Swarm Particle Algorithms

Ant Colony Optimization (ACO)
When ants are searching for food they release pheromones along their routes to guide other ants to the food source. The stronger the pheromone trail the more ants that will follow it and make the trail stronger. As time passes pheromones will disappear and the shortest route will be the only route left.
Example of games that use this algorithm;
    Tetris,
     and an optimization of Pacman, Ms Pacman.

Artificial Bee Colony (ABC)
Scout bees search random flower patches and return to the hive where they evaluate the patches they visited by certain qualities. While at the hive they perform the “waggle dance” which tells the rest of the hive the location of the food source. The onlookers determine what food source they will collect from based on profitability. When they are finished with that food source the bee then scouts for other sources.
Implemented example games;
    Sudoku,
    The Hexxagon Game.

Particle Swarm Optimization (PSO)
This algorithm was based on birds seeking food, where there is a piece of food and many birds in that area. All of the birds know their distance from the food and the birds closest to the food.
Game examples;
    TicTacToe,
    Checkers.

Pattern Movement in Alice:Madness Returns

In Madness Returns pattern movements are seen in the environment, and NPC actions. 
This game has a lot of different environments from caves, derelict buildings, to the sea. In all these environments there is visible pattern movement in background objects and creatures, e.g. flags, fire, water, and Nutterflys(creatures). As this game is a 3D platformer it is necessary to discuss the pattern movement of the platforms. There are three types of platforms, pattern movement wise, in the game, there are solid non moving, action triggered, and continual movement platforms. Apart from the completely still platforms they use predictable pattern movement once activated if they need to be. While the environment and platforms do not necessarily show intelligence they create atmosphere and challenge.
Pattern Movement in NPCs as in most games are clearly visible in enemies to non-enemies. Non combat NPCs mainly stay stationary and only interacts with the player by speaking set lines. The main appearance of pattern movement in NPCs is in combat. An example of this is noticing when in a battle the direction that the enemies chase after you, if you die and decide on your next try to go a different way the enemies will adapt and change their movement. This is an example of pattern movement that gives the illusion of intelligence.

Improv

Improv: A System for Scripting Interactive Actors in Virtual Worlds

What

In this paper Improv is stated as “An Expert System For Authors”, what it is is an interactive system that lets authors of various abilities create lifelike animated character interactions.

Why

Improv was created with the intention of creating tools that would be able to convey a sense of psychological engagement. There are other tools which the paper discusses but the general consensus is that they were trying to improve on these tools to make it easier for non programmers to create these believable interactive animations.

How

The high-level description of how Improv works is that they have used an “english-style” scripting language and a network distribution model. However this does not describe the architecture, the diagram below is from the paper and shows the three main elements in the architecture. 

The Behaviour Engine simply put is the mind of the actor, it is where decisions about triggering animations and higher-level capabilities are taken care of. 
The Animation Engine controls the body of the actor, layering and blending animations to convey the mood of the actor.
The Geometry is the model that is manipulated into the expressions and actions.
Underneath these engines there are two other parts of the system that gives the actor a more believeable lifelike feel. Degrees of Freedom(DOF) values are what control the actor over time and the smooth transitions between actions. Coherent Noise is also a technique used, it is applied so as to avoid complex simulation models. It adds the slight movements that living beings make when doing anything. An example is the controlled randomness of eye blinking, wandering gazes, or the small motions of a character trying to maintain balance.
While that is the basic underlying of the system, how to use the system is by using the aforementioned “english-style” scripting language. The first example below shows the script needed for an actor to make a curtain call showing triggering actions. While the second example shows the actor making a choice, once a choice has been made it is executed as if it was an action chosen by the author.

Examples:

define SCRIPT "Curtain Call"
{ "walk to center" }
{ continue until { my location equals center } }
{ "turn to camera" }
{ continue until { "turn to camera" is done } }
{ "bow" }
{ continue for 3 seconds }
{ "walk offstage" }

SCRIPT "Rock Paper Scissors"
{ choose from { "Rock" "Paper" "Scissors" } }

There is also the ability to control more than one actor, the diagram below shows a blackboard between one actors system and the others. The blackboard gives the impression to users that all the actors know what all the other actors are doing and gives the ability of being able to respond quickly and in a meaningful way.
 


The second diagram included here demonstrates the how the user can interact with the actor, the levels of abstraction are needed to allow the author to control what the user can do. 
 

There are many ways to control the actors another important technique is the implementation of Decision Rules. The system uses the decision rules to weight each object that has been passed to it allowing the actor to “intelligently” make the right choice.

So What

In the paper the authors finish discussing the possibilities of Improv. Their goal is to make it into a commercial tool set with plenty of documentation for ease of use. They also plan to add voice recognition and behavioural sub-systems which allows the user to control goals, moods, attitudes, and relationships between the actors using spoken English. They finish by stating a big list of areas where they believe this system would be useful, including Role Playing Games, shared virtual worlds, digital puppetry, computer guides and companions among more. 

Improv is an interesting system that looks like it would be beneficial in many areas as the authors stated. However the ability for non programmers to use it effectively is still a point for me as demonstrated by the script above it is simple looking  but as it gets more complex and larger will non programmers be able to understand what they are doing and how to achieve their goal in an effective way?

JESS Language

Provide a brief summary for your manager as to whether your company could be using JESS language as a platform for an online entertainment product. You can use the brief introduction as a starting point for your investigation.

JESS is an expert system language which uses the Rete algorithm to match patterns efficiently.  This can be used with our online entertainment product in many ways. 
As a rule based system it will store our catalog and use pattern matching to intelligently filter the products for each customer based on similar people and their previous purchases. It does this by having a set of rules that relate to the relationship between the products and customers which will deduct what the customer might prefer.
JESS can also be used with our resource planning systems and at the order validation side of the production. 
However there we would have to use JESS as a servlet as it does not always work as an internet applet and some of the functionality may be lost when used with certain browsers.  
I would still recommend the use of it as it can continually be built on and customised with ease.

Recursive Transition Networks

Can you use Recursive Transition Networks as Smoke and Mirrors for apparent Intelligence in the context of Game AI, if so how, if not why?

Recursive Transition Networks(RTNs) are used for language parsing to create “well formed” sentences. RTNs can be used outside of games to generate essays and papers that have sensible sentences but don’t necessarily make sense when read together. In games it is used as a speech language parser to give NPCs a sense of intelligence when speaking to the player. Not only can they be used for speech but also the generation of appropriate rules and equipment.
As an example of speech parsing in games Bot Colony, the player can speak freely and the game will try to respond in a human intelligent manner. Compared to popular games being released now which use dialogue trees, voice commands, or other fixed ways of answering, Bot Colony is superior in technology using deeper processing than the other techniques above. 
How well do the Bots in Bot Colony actually talk?

By the technology you would expect that they could converse well however they may get confused as English is a complicated language to follow.
Despite that Bot Colony shows that RTNs can in fact be used in games.