Practice based research in large-scale organisations

Challenges in complex and safety critical complex adaptive systems

A stream of research by Dr. Paola Amaldi focuses on challenges raised by major information technology and information systems innovation projects in complex and safety critical socio-technical systems, like transport, aerospace, defence, industrial process plants, and national public health systems.

While ever-increasingly complex technologies, AIs, and expert systems, allow achievement of targets unthinkable in the recent past, these advances have introduced a complexity that often goes beyond our ability to master them.

The operation of these systems is getting so complex that it defies the understanding of all but a few experts, and in some cases even they struggle to diagnose the problem (see for example, the software problem with the Boeing 757 that was recently involved in two fatal aircraft crashes). One of the challenges is that we as a society, are expecting to build systems that are more and more ambitious and whose complexity, ultimately, goes beyond our ability intellectually to manage them. These challenges might remain undetected or poorly addressed leading to major catastrophes.

So far, they have been mostly addressed from the operator-system interaction point of view, for example when the underlying software logic is not clear to those operating the system, or a warning device becomes counter-productive because of a high rate of false alarms. However, the rapid development of automation in our delay life (drones, driverless cars, super-smart personal devices, wearable technologies) suggests that we need a system-wide approach to envision the long-term consequences and design resiliency in these systems before it becomes too late.

Meeting the challenges

Challenges might be addressed at all levels of societal involvement by targeting inadequate or biased decision-making strategies, normalised risk perception and risk assessment; hierarchical levels of control, and system archetypes are but a few tools to be employed to gauge the multi-layered interactions and to address the non-linear causality of these systems.

Challenges will also be addressed at organisational/societal level where the way an application is conceived is critically examined through an historical/cultural analysis of the processes underlying its development and implementation.

Critical interactions associated with increasing software and more generally with operational system complexity are to be identified at all levels of analysis. This is at the heart of our systemic approach in the study of major project failures.

A number of undergoing or just completed projects address these challenges.

• Human system interaction and adaptive behaviour: Through an analysis of thousands of incident reports, we identified functional, rather than erroneous, patterns of behaviour.
• Project management and software failures: Historical analysis of documents highlighting decision making around the adoption and evolution of systems have shown cultural biases, policy pressures, rationalising biases underlying decisions and trends.
• Organisational drift into software failures: Ongoing work uses ethnographic methods to investigate organisational drifts while conceiving and adopting new safety critical applications.
• Organisational resilience engineering: Through interviewing and through focus groups, we are studying how an organisation prepares to cope with challenges introduced by increasingly complex automated systems.
• We use system archetypes and dynamic-loop modelling to represent the development of Public Health crisis like Antimicrobial Resistance (AMR) by raising the spotlights from the front-liners to top managers of major organisations, involved in the development of the problem.

All of the above will contribute to addressing the following question: what is a ‘Learning Organisation’? when the planned or effected changes are suggestive of a systemic rather than modular and fragmented approach to large scale socio-technical organisation?