New research by Embry-Riddle Aeronautical University Presidential Fellow Dr. Barbara Holder examines pilot training performance at five major airlines to identify strengths and opportunities for improvement.

Holder, an associate professor in the College of Aviation, School of Graduate Studies, found that airlines struggle to train and assess the nontechnical aspects of pilot performance, known as Crew Resource Management (CRM) and Threat and Error Management (TEM).

Headshot of Dr. Barbara Holder
Dr. Barbara Holder has developed a framework in which the nontechnical aspects of performance are formulated as actions, or verbs, that pilots do on every flight.

The research, published in the journal associated with Applied Human Factors and Ergonomics International, introduces a new action-oriented training framework that goes beyond CRM and TEM by making explicit the nontechnical actions and strategies that are foundational for flight safety, she said.

In the framework, the nontechnical aspects of performance are formulated as actions, or verbs, that pilots do on every flight. These include communicate; plan; lead and participate; manage time and tasks; and assure. Holder said the actions apply broadly across tasks and roles, making them recognizable, understandable and trainable.

For example, an effective communication strategy is to “think out loud” when something comes to mind that may affect the safety of flight. The value of doing so is to engage the voice to make intentions and concerns available to other crewmembers. This allows them the opportunity to provide another perspective, countermand, creatively problem-solve and error-check.

Holder said the actions and strategies in the framework are not intended to be exhaustive but are foundational to effective performance and can be reliably taught and assessed.

“These were derived from effective nontechnical training approaches, safety analyses, observations of well-performing pilots and modern cognitive science,” she said. “We are changing the pilot training paradigm, based on new knowledge.”

Holder said the framework has been tested and validated at two major U.S. airlines and one regional airline. The validations were conducted in three phases, she said, and the results indicate airline consensus on the framework's direction and relevance.

The research comes at a time when the aviation industry is reevaluating nontechnical training. As part of the research validation, the nontechnical actions and associated strategies in the new framework were applied to two airlines’ active line-oriented flight training (LOFT) simulator lessons, demonstrating the feasibility of their application to existing pilot training programs. The pilot participants successfully applied the framework’s actions and strategies during the simulator scenarios, and instructors reliably and consistently identified and assessed them.

Dr. Alan Stolzer, dean of Embry-Riddle’s College of Aviation, called Holder's work a great example of applied research that will improve aviation safety.

“She's taken something long considered vague or unteachable and is turning it into a clear, practical framework instructors and pilots can actually use,” he said.

Grant Marsh, who is earning a Ph.D. in Aviation, assisted Holder during interviews with instructors and check pilots at airlines that participated in the research.

“As aircraft are becoming more advanced, the skills required to operate them in a safe manner are evolving,” Marsh said. “The nontechnical skills required need to be defined and need to be trained.”

Marsh is beginning a position at the Federal Aviation Administration this month, where he will continue conducting human factors research on the flight deck. He described the work as “my ideal career.”

Holder, whose doctoral degree is in cognitive science, studies pilot thinking “in the wild” to develop operationally relevant and impactful safety enhancements to complex sociotechnical systems.

She said the new framework will “advance pilot training holistically by focusing on what the pilots should do, providing context-relevant strategies for action and integrating technical and nontechnical pilot performance, with the aim of developing in pilots the foundational skills needed to effectively manage operational variability and build resilience.”