1-10 of 30 results
-
Development of a Safety Performance Decision-Making Tool for Flight Training Organization
PI Marisa Aguiar
CO-I Carolina Anderson
Title 14 of the Code of Federal Regulations (CFR) Part 141 flight training organizations are actively pursuing ways to increase operational safety by introducing advanced risk assessment and decision-making techniques. The purpose of the dissertation was to create and validate a safety performance decision-making tool to transform a reactive safety model into a predictive, safety performance decision-making tool, specific to large, collegiate Title 14 CFR Part 141 flight training organizations, to increase safety and aid in operational decision-making. The validated safety decision-making tool uses what-if scenarios to assess how changes to the controllable input variables impact the overall level of operational risk within an organization’s flight department.
Utilizing SPIs determined to be most indicative of flight risk within large, collegiate flight training organizations, a predictive, safety performance decision-making tool was developed utilizing Monte Carlo simulation. In a high-risk system beset with uncertainty, applying Monte Carlo simulation addresses the need to accommodate uncontrollable inputs into the model in a manner that enables the model to produce meaningful output data. This research utilizes the validated equations drawn from the non-statistical model developed by Anderson, Aguiar, Truong, Friend, Williams, & Dickson (2020) for the mathematical inputs driving the computational nodes, including the SPIs, as the foundation to develop the safety performance decision-making tool.
The probability distributions of the uncontrollable inputs were drawn from a sample of operational data from September 2017 to September 2019 from a large, collegiate 14 CFR Part 141 flight training organization in the southeastern United States. The study conducted simulation runs based on true operational ranges to simulate the operating conditions possible within large, collegiate CFR Part 141 flight training organizations with varying levels of controllable resources including personnel (Aviation Maintenance Technicians and Instructor Pilots) and expenditures (active flight students and available aircraft).
The study compared the output from three different Verification Scenarios—each using a unique seed value to ensure a different sample of random numbers for the uncontrollable inputs. ANOVA testing indicated no significant differences appeared among the three different groups, indicating the results are statistically reliable.
Four What-if Scenarios were conducted by manipulating the controllable inputs. Mean probability was the key output and represents the forecasted level of operational risk on a standardized 0-5 risk scale for the Flight Score, Maintenance Score, Damage and Related Impact, and an Overall Risk Score. Results indicate the lowest Overall Risk Score occurred when the level of personnel was high yet expenditures were moderate.
Changes to the controllable inputs are reflected by variations to the outputs demonstrating the utility and potential for the safety performance decision-making tool. The outputs could be utilized by safety personnel and administrators to make more informed safety-related decisions without expending unnecessary resources. The model could be adapted for use in any CFR Part 141 flight training organization with data collection capabilities and an SMS by modifying the input value probability distributions to reflect the operating conditions of the selected 14 CFR Part 141 flight training organization.
Categories: Graduate
-
Organizational Design of Secondary Aviation/Aerospace/Engineering Career Education Programs
PI Susan Archer
CO-I David Esser
Modern nations operate within a global economy, relying heavily on the aviation industry for efficient and effective transportation of passengers and goods. The Boeing 2018 Pilot and Technical Outlook Report indicated that over the next 20 years, the aviation industry will need almost two and a half million new aircrew and maintenance employees to meet anticipated global demand. The industry will also need engineers, aviation managers, and workers in other aviation and aerospace disciplines. Aviation and aerospace jobs require solid backgrounds in mathematics, science, and technology; the development of pre-college aviation / aerospace / engineering career education programs would presumably enhance student preparation in these areas and increase the workforce pipeline for the industry. The goal of this study was to identify and evaluate the underlying organizational factors of successful secondary aviation / aerospace / engineering career education programs, through application of measures traditionally associated with organizational theory.
Analysis of collected data involved exploratory factor analysis to identify underlying factors, confirmatory factor analysis to verify significant relationships between manifest variables and latent constructs and to ensure a good-fitting measurement model, and structural equation modeling to identify significant relationships between latent constructs and achieve the best-fitting model of these relationships for the collected data. Variables were Likert-scale responses to literature-based survey items associated with organizational vision, leadership, communication, collaboration, decision-making, flexibility, accountability, resource availability, motivation, and learning. Additionally, participants were invited to provide comments related to any of the survey items to explain or add detail to their response selection. These comments were reviewed both as they related to individual survey items and for detection of underlying themes. Participants in the study comprised stakeholders associated with career education programs in the disciplines of interest, including students, parents, alumni, school / program faculty and staff, industry members, and advisory board members.
Hypothesis testing results suggested that the most important factor in predicting success for an aviation / aerospace / engineering academy or program is personal motivation related to learning. Though other underlying factors, including leadership / collaborative environment, organizational accountability, and resource availability were clearly related to perceived program success, they appeared to have indirect relationships with success. It is also important to recognize that a paired qualitative analysis of participant comments generated themes that transcended survey item topics, and the identification of these themes supported the conclusions from hypothesis testing regarding underlying factors. Personal motivation was the most commonly recurring theme in comments, supporting the hypothesis testing result indicating its predictive strength for an organization’s success.
Understanding the constructs that are most closely related to an organization’s success, as they are perceived by its stakeholders, offers current program leaders and groups interested in creating new programs evidence they can use to design the frameworks for their programs. Anticipated workforce shortages warrant study of how to increase the number of candidates not only in post-secondary academic and training programs, but to shift recruiting earlier through implementation of quality secondary-level programs that are established on a foundation of research-based strategies for success.
Categories: Graduate
-
Predicting General Aviation Accidents Using Machine Learning Algorithms
PI Bradley Baugh
CO-I Bruce Conway
Aviation safety management is implemented through reactive, proactive, and predictive methodologies. Unlike reactive and proactive safety, predictive safety can predict the next accident and enable prevention before an actual occurrence. The study outlined here promotes predictive safety management through machine learning technologies using large amounts of data to facilitate predictive modeling.
The study addresses efforts to reduce General Aviation accidents, an effort that was renewed in earnest with the Federal Aviation Administration’s 1998 Safer Skies Initiative. Over the past 22 years, the General Aviation fatality rate has decreased. However, accidents still happen, and there is some evidence showing the number of accidents, representing hazard exposure, is increasing. The accident data suggest that the aviation community still has more to learn about the variables involved in an accident sequence.
The purpose of the study was to conduct an exploratory data-driven examination of General Aviation accidents in the United States from January 1, 1998, to December 31, 2018, using machine learning and data mining techniques. The goal was to determine what model best predicts fatal and severe injury aviation accidents and further, what variables were most important in the prediction model.
The study sample comprised 26,387 fixed-wing general aviation accidents accessed through the publicly accessible National Transportation Safety Board Aviation Accident Database and Synopses archive. Using a mixed-methods approach, the study employed both unstructured narrative text and structured tabular data within the predictive modeling. First, the accident narratives were culled using text mining algorithms to develop text-based quantitative variables. Next, data mining algorithms were used to develop models based on both text- and data-based variables derived from the accident reports.
Five types of machine learning models were created using SAS® Enterprise Miner™, including the Decision Tree, Gradient Boosting, Logistic Regression, Neural Network, and Random Forest. Additionally, three broad sets of variables were used in modeling, including text-only, data-only, and a combination of text and data variables. Three models, Logistic Regression (text-only variables), Random Forest (text-only variables), and Gradient Boosting (text and data variables), emerged with a similar prediction capability. The top six variables within the models were all text-based covering Medical, Slow-flight and stalls, Flight control, IMC flight, Weather factors, and Flight hours topics. The Logistic Regression (Text) model was selected as the champion model: Misclassification Rate = 0.098, ROC Index = 0.945, and Cumulative Lift = 3.46.
The results of the study provide insights to the entire General Aviation community, including government, industry, flight training, and the operational pilot. Specific recommendations include the following areas: 1) improve the quality and usefulness of accident reports for machine learning applications, 2) investigate ways to capture and publish more open-source flight data for use in safety modeling, 3) invest in additional medical education and find ways to address impairing medications and high risk medical conditions, 4) renew efforts on improving flight skills and combatting decision-based errors, 5) emphasize the importance of weather briefings, pre-flight planning, and weather-based risk management, and 6) create an aviation-specific corpus for text mining to improve text analysis and transformation.
Categories: Graduate
-
Gold Standards Training and Evaluator Calibration of Pilot School Check Instructors
PI Paul Cairns
CO-I Andrew Dattel
A key component of air carrier advanced qualification programs is the calibration and training of instructors and evaluators and assurance of reliable and valid data in support of such programs. A significant amount of research is available concerning the calibration of air carrier evaluators, but no research exists regarding the calibration of pilot school check instructors. This study was designed to determine if pilot school check instructors can be calibrated against a gold standard to perform reliable and accurate evaluations.
Calibration followed the principles and theories of andragogy and adult learning and teaching, including an emphasis on the cognitive domain of learning, learner-centered instruction, and human resource development. These in combination with methods commonly used in aviation instruction aimed to increase the effectiveness of the calibration. Discussion of these combinations is included. A specific method for delivery of the calibration was provided along with a complete lesson plan. This study used a one-group pretest-posttest design. A group of 10 pilot school check instructors was measured before and after receiving rater calibration training. Statistical measures included raw inter- and referent-rater agreement percentages, Cohen’s kappa and kappa-like statistics for inter- and referent-rater reliability, Pearson product-moment correlations for sensitivity to true changes in pilot performance, and a standardized mean absolute difference for grading accuracy. Improvement in all the measurements from pretest to posttest was expected, but actual results were mixed. However, a holistic interpretation of the results combined with feedback from the check instructors showed promise in calibration training for pilot school check instructors. A thorough discussion of the limitations and lessons learned from the study, recommendations for pilot schools, and recommendations for future research is included.Categories: Graduate
-
Examining Unstable Approach Predictors Using Flight Data Monitoring Information
PI David Carroll
CO-I David Esser
The approach and landing phase of flight is statistically the most dangerous part of flying. While it only accounts for 4% of flight time, it represents 49% of commercial jet mishaps. One key to mitigating the risks involved in this flight segment is the stabilized approach. A stabilized approach requires meeting rigorous standards for many flight parameters as the aircraft nears landing. Exceeding any of these parameters results in an unstable approach (UA). The energy management (EM) accomplished by the flight crew, represented by the EM variables in the study, influences the execution of a stabilized approach.
While EM is a critical element of executing a stabilized approach, there appears to be a lack of studies that identify specific EM variables that contribute to UA probability. Additionally, several possible moderating variables (MVs) may affect the probability of a UA. Fortunately, modern jet transport aircraft have Flight Data Monitoring (FDM) systems that capture a wealth of information that enable the analysis of these EM variables. This study used FDM data to answer the questions about what influence a set of EM variables has on the probability of a UA event. The analysis also determined what impact a set of possible MVs, not directly related to EM, has on these EM variables influence.
The analysis used logistic regression (LR) to investigate FDM information. The LR provided estimations of odds ratios for each of the variables and the interaction factors for the MVs. These statistics defined a model to evaluate the influences of the EM and MVs, providing answers to the research questions posed. The results determined the model was a good fit to the data but had poor discrimination. The model supported three of the original seven EM hypotheses and none of the 28 MV hypotheses.
The study identified three specific EM variables that significantly influenced the probability of a UA event. Of the MVs, only one significant influence was revealed but was opposite that hypothesized. Identifying the EM variables, and examining their impacts, shows their importance in preventing UAs. Further, the results help prevent future UAs by informing the design of training programs. Additionally, the current effort fills gaps in the current body of knowledge, as there appears to be a lack of studies in the areas investigated.
A gap in the body of knowledge filled by investigating an area of limited research and the results provide practical application in the analysis of EM-related events. Aviation safety practitioners now have additional information to identify trend issues that may lead to the increased probability of a UA event. Finally, this study was one of very few granted access to actual operational FDM information by an air carrier. The data were crucial in evaluating the proposed model against real-world flight operations, comparing theory to reality. Without access to such closely held information, the research for this dissertation would not have been possible.
Categories: Graduate
-
The Development and Validation of the Game User Experience Satisfaction Scale (GUESS)
PI Barbara Chaparro
PI Joseph Keebler
CO-I Mikki Phan, User Experience Researcher, Google
The purpose of this research is to develop and psychometrically validate a new instrument that comprehensively measures video game satisfaction based on key factors. The video game industry often conducts playtesting sessions in order to provide insight into players’ attitudes and preferences. However, quality feedback is difficult to obtain from playtesting sessions without a quality gaming assessment tool.
A new instrument measuring video game satisfaction, called the Game User Experience Satisfaction Scale (GUESS), with nine subscales emerged. These scales included Usability/Playability, Narratives, Play Engrossment, Enjoyment, Creative Freedom, Audio Aesthetics, Personal Gratification, Social Connectivity, and Visual Aesthetics.
The GUESS was developed and validated based on the assessments of over 450 unique video game titles across many popular genres. Thus, it can be applied across many types of video games in the industry both as a way to assess what aspects of a game contribute to user satisfaction and as a tool to aid in debriefing users on their gaming experience. Based on current best practices of scale development and validation, the project used a mixed-method design that consisted of item pool generation, expert review, questionnaire pilot study, exploratory factor analysis (N = 629), and confirmatory factor analysis (N = 729).
Results from this multistage process demonstrate that the GUESS can be administered to video game players with various gaming experience (e.g., newbie/novice, hardcore/expert) playing a variety of game genres. Ratings of all the items per factor can be averaged to obtain a score of each subscale and average scores for each subscale can be added together in order to obtain a composite score of video game satisfaction. This can aid game designers in determining aspects of a game to improve as well as emphasize to target gaming markets.
Since its development and validation, the GUESS continues to be used to understand video game satisfaction across platforms and user groups by both the GEARS and User Experience Research labs at ERAU as well as gaming labs from other universities.
This was dissertation research by Dr. Mikki Phan supervised by Dr. Barbara Chaparro and Dr. Joseph Keebler. Mikki is now a User Experience Researcher at Google in Mountain View, CA. For more information, see Phan, M. H., Keebler, J. R., & Chaparro, B. S. (2016). The Development and Validation of the Game User Experience Satisfaction Scale (GUESS). Human Factors, 58(8), 1217-1247 and http://www.wired.co.uk/article/science-can-now-determine-how-good-any-video-game-is . Contact Barbara.Chaparro@erau.edu to obtain the GUESS instrument.
Mikki Phan, PhDCategories: Graduate
-
Cost Optimization Modeling for Airport Capacity Expansion Problems in Metropolitan Areas
PI Woo Jin Choi
CO-I Dothang Truong
The purpose of this research was to develop a cost optimization model to identify an optimal solution to expand airport capacity in metropolitan areas in consideration of demand uncertainties. The study first analyzed four airport capacity expansion cases from different regions of the world to identify possible solutions to expand airport capacity and key cost functions which are highly related to airport capacity problems. Using mixedinteger nonlinear programming (MINLP), a deterministic optimization model was developed with the inclusion of six cost functions: capital cost, operation cost, delay cost, noise cost, operation readiness, and airport transfer (ORAT) cost, and passenger access cost. These six cost functions can be used to consider a possible trade-off between airport capacity and congestion and address multiple stakeholders’ cost concerns.
This deterministic model was validated using an example case of the Sydney metropolitan area in Australia, which presented an optimal solution of a dual airport system along with scalable outcomes for a 50-year timeline. The study also tested alternative input values to the discount rate, operation cost, and passenger access costs to review the reliability of the deterministic model. Six additional experimental models were tested, and all models successfully yielded optimal solutions. The moderating effects of financial discount rate, airport operation cost, and passenger access costs on the optimal solution were quantitatively the same in presence of a deterministic demand profile.
This deterministic model was then transformed into a stochastic optimization model to address concerns with the uncertainty of future traffic demand, which was further reviewed with three what-if demand scenarios of the Sydney Model: random and positive growth of traffic demand, normal distribution of traffic demand changes based on the historical traffic record of the Sydney region, and reflection of the current COVID- 19 pandemic situation. This study used a Monte Carlo simulation to address the uncertainty of future traffic demand as an uncontrollable input. The Sydney Model and three What-if Models successfully presented objective model outcomes and identified the optimal solutions to expand airport capacity while minimizing overall costs. The results of this work indicated that the moderating effect of traffic uncertainties can make a difference with an optimal solution. Therefore, airport decision-makers and airport planners should carefully consider the uncertainty factors that would influence the airport capacity expansion solution.
This research demonstrated the effectiveness of combining MINLP and the Monte Carlo simulation to support a long-term strategic decision for airport capacity problems in metropolitan areas at the early stages of the planning process while addressing future traffic demand uncertainty. Other uncertainty factors, such as political events, new technologies, alternative modes of transport, financial crisis, technological innovation, and demographic changes might also be treated as uncontrollable variables to augment this optimization model.
Categories: Graduate
-
A Comparison Of Safety Management Systems Training Methods At A Collegiate Flight Training Institution
PI Mackenzie Dickson
The purpose of this thesis was to compare how two different types of Safety Management Systems (SMS) training affect SMS knowledge in instructors and students in a university flight program. Additionally, the research sought to determine whether a correlation exists between safety knowledge and safety culture perception. An experimental research design was used to study two independent variables, training method and person type, and their effect of SMS knowledge. A non-experimental design was used to study the correlation between safety knowledge and safety culture perception. Research has shown that a safety-training program’s engagement level has an effect on the safety knowledge retained by trainees. This study sought to determine if higher-engagement, live- presentation training is a better approach to SMS training than a computer-based training module currently in use by the university studied. The results of this study can provide the university with useful guidance in constructing its SMS training program, an essential element to an effective SMS. Additionally, this study can demonstrate the importance of safety training in establishing positive perceptions of the university’s safety culture among students and instructors.
This study employed an experimental method, using quantitative data, to answer whether different training methods differ in SMS knowledge retained by students and instructors at a collegiate flight school. Additionally, a non-experimental design was used to find a correlation between SMS knowledge and safety culture perception at the same flight school.
Categories: Graduate
-
Student Engagement in Aviation MOOCs: Identifying Subgroups and Their Differences
PI Jennifer Edwards
CO-I Mark Friend
The purpose of this study was to expand the current understanding of learner engagement in aviation-related Massive Open Online Courses (MOOCs) through cluster analysis.
MOOCs, regarded for their low- or no-cost educational content, often attract thousands of students who are free to engage with the provided content to the extent of their choosing. As online training for pilots, flight attendants, mechanics, and small unmanned aerial system operators continues to expand, understanding how learners engage in optional aviation-focused, online course material may help inform course design and instruction in the aviation industry. In this study, Moore’s theory of transactional distance, which posits psychological or communicative distance can impede learning and success, was used as a descriptive framework for analysis. Archived learning analytics datasets from two 2018 iterations of the same small unmanned aerial systems MOOC were cluster-analyzed (N = 1,032 and N = 4,037). The enrolled students included individuals worldwide; some were affiliated with the host institution, but most were not. The data sets were cluster analyzed separately to categorize participants into common subpopulations based on discussion post pages viewed and posts written, video pages viewed, and quiz grades. Subgroup differences were examined in days of activity and record of completion. Pre- and postcourse survey data provided additional variables for analysis of subgroup differences in demographics (age, geographic location, education level, employment in the aviation industry) and learning goals. Analysis of engagement variables revealed three significantly different subgroups for each MOOC. Engagement patterns were similar between MOOCs for the most and least engaged groups, but differences were noted in the middle groups; MOOC 1’s middle group had a broader interest in optional content (both in discussions and videos); whereas MOOC 2’s middle group had a narrower interest in optional discussions. Mandatory items (Mandatory Discussion or Quizzes) were the best predictors in classifying subgroups for both MOOCs. Significant associations were found between subgroups and education levels, days of activity, and total quiz scores. This study addressed two known problems: a lack of information on student engagement in aviation-related MOOCs, and more broadly, a growing imperative to examine learners who utilize MOOCs but do not complete them. This study served as an important first step for course developers and instructors who aim to meet the diverse needs of the aviation-education community.Categories: Graduate
-
Determinants of Aviation Students’ Intentions to Use Virtual Reality for Flight Training
PI Stephanie Fussell, Ph.D.
CO-I Dothang Truong
The goal of this research was to determine the factors that influence aviation students’ intention to use VR for flight training. An extended Technology Acceptance Model (TAM) was developed that incorporates elements of the Theory of Planned Behavior (TPB); factors derived from relevant, validated extended TAMs; and new factors that are theorized to impact use intention. These factors are related to aviation education, the use of VR technology in training environments, and using VR for flight training. The new model may explain flight students’ acceptance of VR for flight training as well as their intent to use the technology. A quantitative research method with a cross-sectional survey design was utilized. Descriptive statistical analysis, a confirmatory factor analysis (CFA), and a structural equation modeling (SEM) process were employed. Data were collected from aviation students enrolled in FAA-approved Part 141 pilot schools in early 2020 using a survey design. Results indicated a good model fit to answer the three research questions of the study. There were 14 hypotheses in the original model. Although one was removed, an additional relationship was discovered, validated, and added to the model. Nine of the hypotheses were supported. Eight of the nine predictor factors of the model were determined to directly or indirectly impact behavioral intention (BI). The original TAM factors had the strongest relationships. Relationships between factors particularly relevant to VR technology and aviation training were also supported.
Immersive simulation technology has been incorporated into numerous training environments, including medicine, engineering, and marketing. The aviation industry, in particular, has a history of embracing technology to enhance training and has especially regulated the requirements of devices for flight training. Virtual reality (VR) is the newest technology being adapted for training purposes. Many educational institutions training providers are incorporating virtual environments (VE) and VR systems into curricula and training programs to expand educational opportunities, enhance learning, promote deep cognitive learning, and leverage the abilities of a generation of students who have adopted technology from an early age.
As VR is adopted for educational purposes, researchers are conducting experiments to learning with the VE occurs at an equal or greater level than in the real world. However, research surrounding students’ perceptions of the technology and intentions to use it for training has been neglected. This is especially true in the realm of aviation and flight training. The goal of this research was to determine the factors that influence aviation students’ intention to use VR for flight training. An extended Technology Acceptance Model (TAM) was developed that incorporates elements of the Theory of Planned Behavior (TPB); factors derived from relevant, validated extended TAMs; and new factors that are theorized to impact use intention. These factors are related to aviation education, the use of VR technology in training environments, and using VR for flight training. The new model may explain flight students’ acceptance of VR for flight training as well as their intent to use the technology.
A quantitative research method with a cross-sectional survey design was utilized. Descriptive statistical analysis, a confirmatory factor analysis (CFA), and a structural equation modeling (SEM) process were employed. Data were collected from aviation students enrolled in FAA-approved Part 141 pilot schools in early 2020 using a survey design. Results indicated a good model fit to answer the three research questions of the study. There were 14 hypotheses in the original model. Although one was removed, an additional relationship was discovered, validated, and added to the model. Nine of the hypotheses were supported. Eight of the nine predictor factors of the model were determined to directly or indirectly impact behavioral intention (BI). The original TAM factors had the strongest relationships. Relationships between factors particularly relevant to VR technology and aviation training were also supported.
The results of the study fill a gap in the research surrounding the use of VR for flight training and the influencing factors of behavioral intention. The model may also be modified for other educational and training environments as well as other forms of immersive simulation technology.
Categories: Graduate
1-10 of 30 results