Placeholder Image

Subtitles section Play video

  • Hi I'm Tommy Thompson and this is AI and Games: a series on research and applications of Artificial

  • Intelligence in video games. Since the beginning of this channel I've explored a variety of

  • AI techniques to solve specific problems within video game design that require both immediate,

  • reactive behaviour in conjunction with long-term and strategic decision making. First-person

  • shooters are certainly a common genre from which interesting AI can emerge given its

  • predicated on fast, reactive and engaging gameplay. But if we're to consider the need

  • for effective systems that balance both short and long term decision making in games, there

  • is no greater problem space to explore than Real-Time Strategy (RTS) games.

  • Both the RTS and FPS genres have continued to innovate the state of the art in AAA video

  • gaming in an effort to satisfy their design. But RTS games have much larger considerations

  • for the long-term impact of quick and reactive decisions, as well as the need to manage resources

  • at varying levels of abstraction and complexity over multi-hour gameplay sessions. When considering

  • the challenges and innovations faced in AI for strategy games, as a player, developer,

  • or researcher there's one franchise you simply cannot go without discussing, Creative Assembly's

  • Total War.

  • Total War has grown from strength to strength since the launch of the franchise in the year

  • 2000. As each game becomes more ambitious, the challenges faced by the AI systems continue

  • to increase in scope and complexity. This has resulted in a development team that continues

  • to push the state of the art in game AI unlike any other in video game history. But the journey

  • to reach more recent entries at the time of this video, such as Total War: Attila and

  • Warhammer has had more than a few bumps in the road along the way. So in this first in

  • a series, I will be looking at not just how the core tenets of the AI of Total War, but

  • how each entry in the series aims to solidify, tweak or completely rewrite parts of these

  • systems. In addition, I'll look at the response to these changes from the fanbase, the modding

  • community that has thrived within Total War and how The Creative Assembly adopted groundbreaking

  • academic research to help innovate from the very first entry all the way to more recent

  • releases.

  • So let us begin our march to total war.

  • But first let's get into the Real-Time Strategy genre and understand where Total War sits

  • within this space. Real-Time Strategy games are focussed primarily on the control of territory

  • and assets: whereby two or more opposing forces will appear in the same map in an effort to

  • assert dominance by securing resources or locations within the existing defined area.

  • Often in order for this to be achieved, each force will build collections of structures

  • and units that either improve the resource collection within a given area, defend existing

  • structures and land or scout and secure areas of the map by force. The latter often requiring

  • the destruction of opposing forces units in the process. This process typically requires

  • players to explore a previously hidden environment by piercing what is referred to as the 'Fog

  • of War' in order to establish enemy and resource locations. In addition, over time each faction

  • is expected to build upon the structures and units previously established by enhancing

  • their existing capabilities and resource acquisition.

  • While arguably not the first RTS game ever made, Westwood's 1992 release Dune II: The

  • Building of a Dynasty acted as the archetype for future games in this genre such as Westwood's

  • own Command and Conquer series, but also Age of Empires, Total Annihilation, Dungeon Keeper,

  • Homeworld and Blizzard's Warcraft and Starcraft franchises. The genre has continued to evolve

  • and branch off since the late 1990's, with the likes of real-time tactics games like

  • Dawn of War and Star Wars: Empire at War emphasising tactics over resource management. Meanwhile

  • games such as 1994's UFO: Enemy Unknown have explored a balance between the real-time and

  • turn-based decision making, with it still proving popular to this day with the franchise

  • being successfully rebooted as XCOM in 2012. Arguably the largest deviation is the the

  • rise of the Multiplayer Online Battle Arena or MOBA genre - popularised by the Warcraft

  • III mod Defense of the Ancients and typified by games such as Dota 2 and League of Legends

  • - where players assume the role of hero units on the ground to define and assist the larger

  • strategies playing out during a match.

  • AI applications within real-time strategy games have existed since the inception of

  • the genre, with players responsible for managing the movement of faction units by telling them

  • where to go, but without direct control of their actual movement. As such, players are

  • expected to interact with these systems in order to yield desired outcomes. In addition,

  • AI is critical to the single-player aspects of the game, given that players play off against

  • opposing commanders responsible for managing their own units and construction strategies.

  • The role of the AI player is one that is tremendously challenging and is considered to be one of

  • the larger challenges facing academic research. This has led to the forming of numerous academic

  • benchmarks and projects, including the Starcraft AI Competition hosted at the AAAI conference

  • on Artificial Intelligence and Interactive Digital Entertainment. This is a video in

  • and of itself, but don't worry: I'll be talking about that on AI and Games in the near future.

  • So what makes Total War so special and challenging for an AI system? Creative Assembly's strategy

  • series is interesting in that it combines multiple modes of strategic gameplay into

  • a single game: with players managing turn-based resource management and political strategy

  • alongside real-time combat with large scale armies. In addition, the combat juggles between

  • micro and macro levels: requiring not only AI control of individual units, but more abstract

  • control of unit types and managing the combat formations and troop deployments on the battlefield

  • at runtime.

  • The first entry in the franchise, Shogun: Total War, balances the combat simulations

  • that strive for realism and authenticity, alongside the political strategy that is aimed

  • to give context and stakes to each conflict. The original Total War is set in 1530, during

  • the Sengoku Jidai period of feudal Japan: a time largely popularised in contemporary

  • fiction by the works of Japanese film director Akira Kurosawa, with films such as Kagemusha,

  • Seven Samurai and Ran proving influential on the design and development of the game

  • - with clips of the latter being used as part of the games cinematics. Both the player and

  • opposing AI assume the role of the Daimyo: local-lords who control provinces of Japan

  • with a need to conduct both diplomatic strategy alongside military movements. When rival factions

  • are drawn into conflict, players take control of the Taisho or general and move hundreds

  • if not thousands of troops across the battlefield. Sengoku Jidai made for an ideal period of

  • history for the game, given the politics and even the economics of the period was built

  • around the logistics of fielding armies in defence of the Daimyo and his ambitions.

  • Total War deviates from many traditional real-time strategy games in that it removes mechanics

  • such as Fog of War from combat gameplay as well as the need for resource management outside

  • of unit counts from combat decision making. Given the nature of the game itself, the AI

  • player is comprised of two distinct AI systems: the Diplomacy AI and the combat AI.

  • The Diplomacy AI manages the turn-based strategy of the game and is responsible for moving

  • armies around the map, conducting diplomacy (be it by sending envoys or assassins to forge

  • alliances or eliminate opposition) and building the agriculture and infrastructure of owned

  • provinces.

  • Meanwhile the combat AI dictates combat unit formations, strategies and attack patterns.

  • Much like human players, this is only responsible for managing the strategy of specific groups

  • of units, given the units themselves are already controlled by AI techniques.

  • So to get to grips with the AI of Total War, I'm going to explore the AI systems from lowest

  • to highest level: from individual troop control systems all the way strategic systems that

  • seek to conquer all of feudal Japan.

  • Total War combat is driven by units: groups of specific troop types that can be deployed

  • in formation ranging from melee types to archers to cavalry. These combat units are not only

  • expected to keep formation, but to move around and conduct combat as a unit. Movement is

  • often a tricky task, especially when asked to navigate through or around a variety of

  • terrains such as mountains and forests. To achieve this, Total War adopts artificial

  • neural networks for managing individual units. Neural nets are a fast and effective means

  • to achieve quick and reactive responses to a pre-defined objective. We typically train

  • them using machine learning methods but they can also be tweaked by hand if the network

  • is small enough. What makes this so effective is that once they are trained, the processing

  • time to make decisions is lightning fast. In addition, a well-trained neural net is

  • able to generalise its decision making process: meaning it can recognise the similarities

  • in numerous individual circumstances and in each case give a similar answer. Despite this,

  • neural networks are typically not great when being given multiple objectives to complete

  • at once, especially when they oppose one another. In this instance, the troops have different

  • neural nets they can trigger in depending on the objective they're trying to solve,

  • whether it's moving formation, avoiding oncoming enemy fire or taking position to attack themselves,

  • be it up close or from afar. Each of these networks is pre-defined and is not tweaked

  • or optimised during gameplay, so it's not learning to be better at the game as you're

  • playing it.

  • These Unit AI systems are in effect regardless of whether a human or AI is in control of

  • the army, as each requires these neural net systems to manage the individual troop behaviours.

  • The combat level is the AI system that controls opposing forces akin to players. This AI needs

  • to decide the group formations and actions of units. Moving segments of its army together

  • in order to achieve a specific tactic or micromanage movement of units given the terrain or weather

  • that is currently in play.

  • To achieve this, Shogun employs a logic system that allows for specific decisions to be made

  • based upon the current state of the world. These decisions are derived from 'The Art

  • of War': a collection of strategies by Chinese military strategist Sun Tzu that dates back

  • to around 500 BC. The use of a logic system is well suited to adapting the texts, given

  • that in many instances a set of conditions are in play that should result in a particular

  • response from the system. These rules considers the type, condition and size of both our units

  • and our opponents, the current terrain, the weather and state of troop morale. It's been

  • stated that there are 220 rules embedded in the combat AI system across three distinct

  • versions, each with their slightly different sets of behaviours.

  • A driving force behind this was to ensure that combat was as realistic as possible,

  • deviating away from the tactics that had driven many real time strategy games of the era.

  • As Mike Simpson, Development Director of Creative Assembly stated:

  • "Most [games] use simple scripts of canned behaviour that fire when you bump in to them,

  • and very simple swarming behaviours. They're limited, and aregameyrather than real

  • world. What I mean by that is that the tactics you use to beat them are something that you

  • have to learn for each game or sometimes each scenario/level, and bear no relation to reality.

  • What we strive for is a game where real world tactics actually work. It's not the easiest

  • path to take, but it is the most rewarding."

  • The final layer of Shogun's AI systems is the diplomacy system. This is in control of

  • all decision making on the main campaign map: managing army movement, conducting diplomancy

  • between factions and building infrastructure within provinces. This uses a mixture of classic

  • state-machine decision making alongside the use of genetic algorithms in order to customise

  • the tactics made by each Daimyo. Genetic algorithms are a form of machine learning where we create

  • large quantities of candidate solutions to a given problem that learn to solve it by

  • breeding the best ones with each other. These candidates aren't code, they're an encoding

  • of how the AI behaves typically as a string of numbers, which is then translated into

  • the AI itself to dictate how it behaves. These allows for us to create a collection of diverse

  • solutions for a given problem: they don't all behave in the exact same way and have

  • their own slight characteristics. As a result, each Daimyo has a unique combat behaviour.

  • From a players perspective, this gives them each some sort of personality: allowing for

  • Daimyo's that are more willing to conduct diplomacy to others that will take a more

  • aggressive stance in expanding their territory: attacking both AI and human player alike.

  • With these three systems, the core tenets of the AI of Total War are in place: to manage

  • individual units whilst retaining cohesive behaviour, to group units of troops together