CASE STUDIES

Team Training in Commercial Massive-Multiplayer Gaming Environments

AOC-PAT

The Problem

The Defense Advanced Research Projects Agency (DARPA) is sponsoring an initiative to establish warfighter superiority through a new generation of training systems. Massive Multi-Player On-Line Role Playing Games (MMPOLRPG, or MMPG) are influencing the systems’ design. However, there is little evidence suggesting what skills are best taught in this fashion.

The Solution

A hypothesis that emerged from DARPA's DARWARS initiative is that teamwork skills can be taught effectively using modern commercial off-the-shelf MMPGs, as there is no need for the game to be realistic with regard to modern military operations.

Aptima, in conjunction with BBN Technologies, tested this hypothesis by creating a training exercise within a popular fantasy game which would both challenge and require critical teamwork skills.

Background Research – We reviewed the teamwork literature to identify key teamwork behaviors. Such behaviors include monitoring, back-up, communication, and leadership. We asked whether these teamwork skills could be observed in MMPG environments and how best to support the training of such skills in these environments.

Game Selection – We selected a fantasy-based gaming environment. Similar to many MMPG games, it afforded the study of a large number of users (up to 64) working in collaboration. However, unlike many MMPGs, it also afforded relatively fast and simple scenario structuring and extension.

Exercise Development – We created a capture-the-flag game in which two platoons would compete against one another. Each platoon was divided into three squads, composed of individuals possessing different qualities and abilities (e.g., strength, ability to heal others, etc.). Such an exercise required both inter-squad and intra-squad teamwork to succeed.

Evaluation – Forty members of a U.S. Army Infantry Platoon participated in the exercise. We recorded three different types of data. First, observers recorded instances of teamwork on evaluation forms. Second, we videotaped team debriefing and post-session After Action Reviews (AARs). Lastly, participants completed a questionnaire that addressed game experience, character knowledge, and team experience.

The Results

Aptima created a series of lessons learned that address critical technological, logistical, and pedagogical issues discovered in exercise, including:

Teamwork skills can be observed in MMPGs.

Targeted scenario design is critical for training effectiveness and operational acceptance.

Low operational realism does have benefits, despite some drawbacks.