Download PDF →
LTC Zachary Vogt served in various positions including assignments in 5th Ranger Training Battalion, the 101st Airborne Division (Air Assault), Fort Benning Georgia; Iraq; 2d Infantry Division, 10th Special Forces Group (Airborne), Special Operations Command Forward-West Africa; and Fort Gordon Georgia. He is currently assigned as the Cyber Electromagnetic Activities Branch Chief at the USARPAC.
Dr. Jeffrey Huisingh, PhD, LTC, USA (R) is a former active duty field artillery, aviation, and FA49 officer with assignments in the 9th Infantry Division (Motorized), 199th Separate Motorized Brigade, 2d Infantry Division, TRADOC Analysis Center, United States Air Force Academy, US Army Cadet Command, I Corps, and Army Test and Evaluation Command. Post-retirement, he has supported multiple test and evaluation projects and experiments in the land, air, sea, cyber, and space domains.
Dr. Randall Zimmerman, PhD, LTC, USA (R) served in a variety of infantry assignments with the 82nd Airborne Division and as an operations research analyst for the Defense Logistics Agency and the TRADOC Analysis Center. Since retirement from active duty, he has served as the lead analyst for a variety of Army experiments, examining issues focused on mission command, network architectures, and cyber-related challenges.
The authors thank the C5ISR staff for its assistance with this article. Special thanks go to Dr. Richard Wittstruck and Mr. Ramon Almodovar for their thoughtful review of and comments on the draft.
Traditionally, operational forces have conducted readiness training independently from research and development efforts. The result of this approach allows commanders and researchers to tailor individual events to meet their objectives. This conventional approach produced the results each group needed and worked in the fiscally permissive environment. In the early 2000’s, the Army used an integrated approach when it combined experimentation with the Task Force XXI and Future Combat Systems concepts during unit rotations at National Training Center and Mission Command Training Program exercises. Many of those early E-E events were focused on emerging command and control concepts and equipment used at the brigade and division levels.
In 2018, the Department of the Army (DA) identified an exercise, ORIENT SHIELD 19 (OS19), and an experiment, CYBER BLITZ 19 (CB19), as candidates for a pilot program to assess the efficacy of the E-E concept in the current and anticipated modernization and readiness environments. OS19 was a United States (US) Army, Pacific Command (USARPAC)-sponsored, bilateral exercise; while CB19 was an experiment, co-sponsored by the US Army Combat Capabilities Development Command’s (CCDC’s) Command, Control, Communications, Computers, Cyber, Intelligence, Surveillance, and Reconnaissance (C5ISR) Center and US Army Cyber Center of Excellence (CCoE).
Figure 1 highlights the relationships of the units. USARPAC tasked US Army Japan (USARJ) to plan and execute OS19, which had been a relatively small scale, company and below, bilateral training exercise with US Army National Guard forces and Japan Ground Self-Defense Force (JGSDF) counterparts.
As the operational headquarters responsible for the Multi-Domain Task Force (MDTF) pilot program that includes the Intelligence, Information, Cyberspace, Electronic Warfare and Space Detachment (I2CEWS), USARPAC identified two main training objectives for the OS19 and CB19 E-E. They are: for the I2CEWS to train as a unit and to integrate the I2CEWS into OS19 as much as possible.
Figure 1. Organizational Relationships
Parallel with the OS19 planning efforts, the CCDC C5ISR Center and the CCoE (the organizations responsible for leading the CB19 event) identified 17 unique experimental objectives that spanned the operational, institutional, and research and development communities of interest. Each of the experimental objectives was aligned against the multi-domain problem set. Each of the objectives for OS19 and CB19 was considered part of the E-E design.
As seen in figure 1, the three main organizations planning and preparing for OS19 and CB19 execution were operating in a collegial manner and following USARJ’s overarching plan. This relationship enabled additional input to USARJ staff planning efforts.
Combining this training exercise with a discovery experiment represents a nontraditional execution model that created challenges and opportunities for the leaders of both efforts. The blended experiment and training objective approach allowed the participating units time to work on specific, mission-essential tasks while accomplishing experiment objectives; including evaluating technologies. The remainder of this article highlights the differences between exercises and experiments, identifies planning and execution challenges during OS19 and CB19, and provides recommendations for the planning and executing subsequent E-Es to maximize the benefits for the Soldier, the joint warfighter, the US Army, and the multi-Service force.
The Army is organized along operational and institutional lines of effort with the institutional mission supporting the operational mission. To support these missions, the Army developed tailored organizational frameworks for each domain. These organizations and missions differ significantly enough to create potential friction points when the institutional mission that requires experimentation is run concurrently with the operational mission of conducting exercises and training units. Table 1 highlights some of the differences between experiments and exercises.
Table 1. Experimentation vs. Exercise Comparison
Army Mission Category
Laboratory, technology development center, center of excellence
Combatant commands, numbered armies, and echelons corps and below
Civilian or military study lead operating under a directive
Military commander operating under the authority of AR 600-20, Army Command Policy
Military, civilian, and contractor
Analysis and Memorialization
Data-rich environment; a holistic, detailed analysis process; a comprehensive final report
Individual observations equal: Green Book AARs, Joint Lessons Learned Information System entries/AAR Slides
Prepare for “war in the future”
Prepare to “fight tonight”
Safe to fail
Must not fail
AAR—after action report
Organization. Army organizations charged to conduct experiments are structured with civilian leadership/involvement, equipped with laboratory and range facilities, and staffed by DA civilians and contractors to design, execute, collect and analyze data, and report results to enable acquisition decisions. Combatant commands, numbered armies, and echelons at corps and below levels are led by a commander with reporting responsibility to a higher-level commander and are staffed and equipped to deliver combat power. The effectiveness of combat power is gained through training and exercises designed to improve individual and collective skills.
Leadership. Civilian directors of institutional organizations may report to other civilians or military leaders while operational units are commanded by officers under the authority of Army Regulation (AR) 600-20, Army Command Policy. Commanders operate under an unambiguous chain of command and report to other commanders also governed by either AR 600-20, or comparable authorities, issued by the Departments of the Navy or Air Force.
Participants. Experiment participants can include commissioned officers, noncommissioned officers, enlisted members, and civilians from every Service and are often supported by contractors for the experiment’s planning and execution, including data collection and analysis. Exercise participants are predominantly military who plan, execute, and evaluate training and exercise events.
Analysis and Memorialization. Experiments follow a detailed data collection and analysis plan that takes a holistic look at data collected over the period of the experiment, gathered from multiple data streams. For exercises, observers compare what they are seeing to their perceptions of a standard and report on it in an after-action review (AAR). Experiments produce reports published after the experiment while the informal (Green Book) AAR, PowerPoint presentation, or entries into the Joint Lessons Learned Information System are immediately produced following an exercise.
Immediacy. The US Army Training and Doctrine Command (TRADOC) is modernizing the Army by developing and testing new organizational structures and technologies with the implication that this effort will take time. Conversely, the Army places a requirement on commanders to train to a level of credible collective readiness (DA, 2017) to respond to global contingency operations (Milley, 2016). This “fight tonight” mindset requires commanders to collectively train their units with organic equipment and manpower. This difference in immediacy is a fundamental difference between experiments and exercises and represents significantly different mission sets for experiment directors and troop commanders.
Success Philosophy. In a discovery experiment, technologies; tactics, techniques, and procedures (TTP); concepts of employment (CONEMP); and concepts of operations (CONOP) are iteratively attempted and modified to produce the best possible outcome. In exercises, established battle drills are rehearsed continuously and perfected based on a foundation of institutional training and doctrine and evaluated against standards or best practices. Experiments are conducted with the understanding that systems or TTP, CONEMPs, and CONOPs will not work as intended and require refinement or, in some cases, wholesale change. In contrast, exercises are conducted to prepare units to execute combat operations for which failure is not an option.
Army Chief of Staff, General James C. McConville emphasized this fact in the following statement. “Winning Matters. When we send the US Army somewhere, we don’t go to participate, we don’t go to try hard, we go to win. There is no second place or honorable mention in combat” (https://www.usafmcom.army.mil/News/Article-Display/Article/1930793/new-chief-of-staff-taking-care-of-people-key-to-winning-the-fight/).
Two characteristics, immediacy and success philosophy, represent significant potential friction points between experimentation and exercise objectives and leaders’ perceptions of success and failure. All of the identified characteristics and their potential impacts are described in the following paragraphs.
Lessons learned from OS19 and CB19
Discussion. Traditional OS19 exercises were small, tactical, bilateral events focused on company- and platoon-level operations. Once the decision was made by Headquarters, Department of the Army (HQDA) to combine CB19 with OS19, the two planning teams initiated a series of video teleconferences (VTCs), conference calls, and in-person meetings hosted by USARPAC, the OS19 sponsor. The USARPAC vision for OS19 was significantly more expansive in scope than previous exercises. A number of challenges occurred early in the planning process. For example, separate orders were issued for OS19 and CB19 execution, rather than having both combined in an initial order and addressing the questions of primacy or parity. This led to an inefficient start in identifying relationships and expectations.
Recommendation. HQDA issue a single order to all involved organizations before planning and preparations begin, that establishes the roles and responsibilities for all participants and defines the scope and objectives for the effort.
Discussion. Unity of command is an essential principle of military operations. The US Army Field Manual 3-0, Operations, defines it as one of nine “Principles of War”, and reads: “For every objective, ensure unity of command [is] under one responsible commander”. This guarantees one person has ultimate responsibility for the objectives (and people) that fall under his or her purview, and at the same time, makes clear to everyone who is ultimately responsible.
Unity of effort implies a lack of responsibility because one person is not ultimately in charge; however, unity of effort requires significant coordination. The joint, multinational, and interagency nature of unified action creates situations where the military commander does not directly control all elements in the area of operations. In the absence of command authority, commanders cooperate, negotiate, and build consensus to achieve unity of effort.”
The OS19 and CB19 organizational structures (illustrated in figure 2) highlight the principle of unity of effort for the E-E pilot. The OS19 and CB19 teams established exercise/experiment control teams (EXCON) and used twice-daily meetings to coordinate activities of both operations. During execution, the EXCON synchronization meetings, at the beginning and end of each day, proved crucial when expected actions did not take place within the training unit. When that occurred, the CB19 EXCON had to stimulate the training audience again to achieve an experiment objective.
Figure 2. Unity of Effort
Recommendation. Communicate with the designated lead and partners early and often.
Discussion. Representatives from the following organizations participated in the CB19 portion of OS19/CB19.
- The USARPAC.
- The C5ISR Center.
- The CCoE.
- The Intelligence Center of Excellence.
- The Intelligence Capability Development Integration Directorate.
- The US Army Cyber Command.
- The 151st Theater Information Operations Group.
- The US Army Cyber Protection Brigade.
- The JGSDF.
- The USARJ.
- The I Corps.
- The 25th Infantry Division.
- The 4th Infantry Division.
- The 335th Theater Signal Command.
- The 359th Signal Brigade.
- The 500th Military Intelligence Brigade.
- The US Army Special Operations Command.
- The 1st Special Forces Group (Airborne).
- The New Jersey Army National Guard.
- The Network Cross-Functional Team.
- The John F. Kennedy Special Warfare Center and School.
- The US Army Civil Affairs and Psychological Operations Command.
- The US Army Acquisition Support Center.
- The US Army Cryptologic Office.
- The US Army Cyber Institute.
- The 25th Air Force.
- The 780th Military Intelligence Brigade.
- The 1st Information Operations Command (Land).
- The Army Service Forces.
- The 75th Innovations Command.
- The Air Force Space and Missile Systems Center.
- Numerous technology providers.
CB19 required this large coalition of organizations to execute the experiment for a variety of reasons. One of the most significant challenges for the 2019 I2CEWS formation was manning and equipment availability. The unit was manned at ~50% of its Table of Distribution and Allowances (TDA) and had almost none of its assigned equipment. To address these shortfalls, the CB19 planning team worked with HQDA, and units from across the Army, to provide the necessary personnel and equipment for the I2CEWS to have a full complement to conduct its operations.
To support the Multi-Domain Task Force (MDTF) and Intelligence, Information, Cyberspace, Electronic Warfare and Space Detachment training objectives; E-E planners developed an overarching joint task force scenario that consisted of a three-phase operation. The overall intent of the scenario was to provide the MDTF and I2CEWS regional peer challenges from competition, through conflict, in all domains. For the I2CEWS, this resulted in the five master scenario event list (MSEL) events highlighted in figure 3. These events were coordinated and synchronized between the CB19 and OS19 E-E controllers to ensure training and experiment objectives were stimulated appropriately.
Figure 3. Operational Phases
In an exercise, the training audience plans and reacts to the provided stimulus and continues its mission. If the training unit made an unexpected reaction, a future MSEL is planned to ensure training objectives are achieved. Most training exercises are designed using contemporary equipment and doctrine and follow a sequential progression of actions that result in a US or coalition force victory at the end of the exercise.
During experiments, deliberate attempts are made to control variables and identify changes in outcomes. Frequently, comparisons between technologies, new doctrine, or planned capabilities are required to achieve experimental objectives. This results in the experimental unit “losing” engagements with the threat forces. While this approach can be “unsettling” for US forces, it affords the experiment staff the opportunity to create measurement space for the experiment objectives. Reconciling the need for measurement space to achieve experiment objectives with training objectives can be challenging in the E-E construct.
In contrast to an experiment, troops conducting a field training or command post exercise finish their scenario and are adjusted, by the EXCON, to get them back on track, rather than stop during the event and redoing it. To redo actions during an exercise would not make sense to most training audiences because they operate from the Army aphorism, “train like you fight and fight like you train”. Continually stopping or adjusting variables in an exercise (if the troops did not reach the expected outcome) would violate that aphorism and become a negative training value for the force.
Recommendation. Establish, in the base order, the E-E the priority of effort. The priority of effort will provide a framework for subordinate units when making decisions regarding resources or E-E design.
Discussion. Experiments like CB19, develop detailed analysis and data collection plans that account for all the stakeholder objectives. Experiments collect quantitative and qualitative data from multiple data streams. Each of the experiment objectives is addressed through an experiment design that focuses on creating the conditions necessary for answering the stated objectives.
By comparison, exercises rely, primarily, on qualitative observations made by subject matter experts (e.g., observer/coach trainers at the National Training Center or in the mission command training program) who are informed by Mission Essential Task Lists (METL), Army Training and Evaluation Program (ARTEP) standards, and experience. Those observations are aggregated, compared to the doctrinal norms, and provided to training units as part of an AAR process. Experiments produce reports (published after the experiment) while the Green Book AAR, PowerPoint presentation, or entries into the Joint Lessons Learned Information System are immediately produced following an exercise.
During CB19, multiple pieces of experimental equipment and software were provided to the I2CEWS Soldiers for training and conducting their operations. Each piece of equipment had specific measures and analysis objectives that required constant monitoring and assessments. At the conclusion of CB19, the Soldiers provided direct feedback to the developers of each piece of software and equipment for them to use for making additional improvements. By comparison, during training events, the emphasis is on using the equipment that is provided to the unit, in addition to mission success. There is no mechanism for the Soldiers to provide direct improvement feedback to the equipment developers.
Recommendation. Continue to populate E-E events with experimental equipment to facilitate developing new equipment and concepts. This is even more important regarding electronic warfare equipment since the force is using quick-reaction capabilities and not using programs of record that will not be fielded for a significant amount of time.
Discussion. The Army requires commanders to train to a level of credible collective readiness (DA, 2017) and be prepared to respond to global contingency operations (Milley, 2016). This fight tonight mindset compels commanders to train their units with existing, organic equipment and manpower. Parallel with the fight tonight training mindset, the TRADOC is tasked with modernizing the Army by developing and testing new organizational structures and technologies with an eye toward Army requirements that are five years beyond the present. These juxtaposed missions frame different challenges for Army leaders.
Recommendation. Specify a balanced objective approach in the E-E order to the participating organizations.
Discussion. US Army experiments are designed to test unit capabilities and emerging doctrine, and to push units and equipment to their breaking point. In other words, cause them to fail. In contrast, exercises increase the challenges of the threat force in response to the performance of the training audience and, ultimately, end when the US forces have achieved their training objectives. The perception that commanders have “failed” in preparing and training their units must be overcome for experiments to have meaningful results.
Recommendation. Leaders must create a “safe to fail” environment for experimentation to be successful.
Summary: The Recommended Way Ahead
Since the Army plans to combine exercises and experiments for the foreseeable future, planners may wish to consider the following lessons learned from OS19/CB19:
- Acknowledge duality of purpose. Without a common commander, Field Manual (FM) 3-0, Operations, suggests the leader of the experimentation effort and the commander of the unit being exercised adopt a unity of effort mindset. A unified effort is required for the experiment and exercise to succeed as a linked E-E. This mindset is one of a partnership of equals that promotes a non-hierarchical relationship similar to multinational operations with coalition partners. To ensure a common understanding, primary and secondary objectives for each organization should be recorded in a memorandum of understanding and signed by organizational leaders with rank or positional parity.
- Conduct a concept development meeting that results in a common understanding of how both organizations intend to meet their objectives and support the other.
- Conduct full-scale systems tests prior to the E-E to ensure interoperability.
- Convene a common E-E control cell to coordinate all events and establish priorities.
- Schedule regular coordination meetings prior to the E-E to ensure leaders from both organizations know each other prior to execution.
- Plan for face-to-face or VTC meetings at regular intervals to share information relevant to E-E objectives.
- Establish priorities early in the planning process to shape events leading up to the E-E and during it.
- Designate staff functional leads for planning (i.e., Personnel (G/S1), Intelligence (G/S2), Operations and Training (G/S37), Signal Operations (G/S6), Financial Management (G/S8), or Civil Affairs Operations (G/S9).
- Integrate the E-E participating units into the planning process early so they understand the intent of the E-E and can identify their training objectives and get them integrated into the overall plan. This process could start with a warning order and a unit back brief.
Endnotes and References
 Jones, J., Unity of Command and Unity of Effort in Complex Operations: Implications for Leadership. Institute for National Strategic Studies (2010). https://inssblog.wordpress.com/
Alberts, D.A.; & Hayes, R.E., Code of best practice for experimentation. Washington, D.C.: Command and Control Research Program (2002).
Army Futures Command (2019). Army Futures Command. Retrieved from https://armyfuturescommand.com/.
Department of the Army (DA) (2017, October). FM 3-0, Operations. Washington, DC: Author. Retrieved from https://www.hsdl.org/?abstract&did=805076.
DA (n.d.). Who We Are: Organization. Retrieved from https://www.army.mil/info/organization/.
McConville, J.C., New Chief of Staff: Taking Care of People Key to Winning The Fight, August 9, 2019, Retrieved from https://www.usafmcom.army.mil/News/Article-Display/Article/1930793/new-chief-of-staff-taking-care-of-people-key-to-winning-the-fight/.
Milley, M.A., Army Readiness Guidance, Calendar Year 2016–17 [Memorandum]. Washington, DC: Department of the Army (January 20, 2016). Retrieved from https://www.army.mil/e2/downloads/rv7/standto/docs/army_readiness_guidance.pdf.
Disclaimer. The opinions, conclusions, and recommendations expressed or implied within are those of the contributors and do not necessarily reflect the views of the Department of Defense or any other agency of the Federal Government.
Originally released 1 May 2021 at https://www.alsa.mil/News/Article/2585478/the-exercise-experiment-e-e-a-new-reality/