Skip navigation

Systems Engineering


Understanding Failures in Systems Engineering

The long-term objective of this research is to contribute to the fundamental understanding of why failures occur in systems engineering. Despite our best efforts, systems engineering continues to fail, and the rate of failures shows no sign of decreasing. Current approaches based on methods, tools and processes are not working. We need to do more than propose better processes—we need a foundational basis that is informative and can be adapted to a broad range of circumstances and industries (e.g., mining, oil and gas, chemical, and aerospace) to guide design and operational choices that help prevent or mitigate failures.

The overarching goal for this research project is to improve understanding of systems engineering project failures, such as budget and schedule exceedances and quality concerns, to help mitigate them.

Goal 1: Identify problems in past accidents and project failures and link these problems to remediation measures.

Summary: We studied 63 systems engineering failures (30 accidents + 33 project failures) and identified similar causes, such as problems with testing, project management, and requirements engineering. We subsequently linked these causes to recommendations experts made in accident reports and built a “cause-recommendation” network, which we are deploying in JavaScript—stay tuned for us to post the network on this website.

Goal 2: Gather data from large-scale aerospace companies on how they conduct systems engineering, with a focus on how they study and report their own systems engineering failures and relate these findings to our study on past project failures.

Summary: We surveyed and interviewed systems engineers at 2 large-scale aerospace companies, and found that their failure investigations focus on technical problems that systems engineers can solve. In our investigation of 63 systems engineering failures, we found causes related to technical problems, as well as “people” problems—mismanagement, human factors problems, and others. In their internal reports, these aerospace companies did not report any “people” problems.

Goal 3: Determine whether students who have taken more systems engineering courses at Purdue are better able to identify “pain points” in systems engineering as identified by accident investigators.

Summary: In our study of 63 systems engineering failures, we found “pain points”-- decision points made before the accident took place that accident-causation experts identified as significantly contributing to the accident. We wrote 8 survey questions on some of these pain points and distributed them to the current Purdue AAE students. We are currently analyzing results from this survey to determine whether performance in systems engineering classes correlates with the students’ ability to identify these pain points.

Click here to view the data we have generated in our research.

Click here to view and use our failure analysis tool. Please give it a try, and let us know what you think!



Design Process Waivers and Risk

Many organizations use standard design processes to formulate their missions. When teams need to deviate from this process, they can request a waiver. In some cases, deviations may contribute to some of the science objectives not being met. Our partner organization has extensive, mission-specific data regarding how and why deviations have affected different aspects of missions, including how well the science objectives were met. However, these data have not been systematized and comprehensively analyzed across missions. Therefore, there is an opportunity to analyze the history of waivers and missions to better understand how future missions may be affected by different types of waivers.

The objective of this study is to determine how deviations from the design process affect our partner organization's missions. Using data on waivers and mission outcomes, we will investigate how waivers affect the qualitative probabilities of meeting each mission’s science objectives, and what the mechanisms of these effects might be. In previous work in our group, we have identified common causes of systems engineering failures—here we will use these causes as a starting point to understanding how waivers might reduce the probability of overall mission success.