Presenters:
Dr Kelvin Ross
Software engineering is undergoing its most significant shift since the move from waterfall to agile: agentic AI tools that don’t just assist with code, but plan, act, test and correct with increasing autonomy. Drawing on hands-on experience deploying these tools for coding, test automation and delivery, this talk examines how the software development lifecycle itself is changing, and what that means for systems safety.
We’ll unpack how large language models and agentic frameworks actually work, and where they can potentially strengthen safety practice - faster hazard analysis, continuous verification, automated traceability and safety case assembly at a scale not previously possible. But agentic autonomy also raises hard questions for a discipline built on determinism, independence and repeatable evidence. Can a verification agent ever be independent of the development agent it shares a lineage with? How do certification and tool-qualification regimes cope with tools that update themselves overnight? Who is accountable when an agent’s decision contributes to a the development, testing and assurance of a product?
This talk offers a practitioner’s view of where agentic AI is heading, the opportunities for systems safety, and the assurance gaps our profession needs to close before the technology outruns our frameworks.
Dr Clive Boughton
The best AI systems provide very significant, positive effectiveness at every phase of systems/software engineering - not just coding. The engineering of highly safe and/or secure systems requires the application of an end-to-end set of stringent phases/processes that help guarantee the creation and proper operation of products with high integrity and availability. Appropriate system/software engineering (SSE) lifecycles (of course) exist and are certainly used where operational safety/security is a high priority. Although the essential phases of the SSE lifecycle haven’t changed much, the fulfilment of necessary activities and use of specialized (sometimes automated) tools/techniques within each phase are constantly challenged with the ever-increasing demands on both safety and security. The ultimate effect is more effort, time and cost for development and the increasing need for highly skilled and experienced systems/software engineers and project managers.
Ah! But AI can save us! Can’t it? Maybe! However, it is best to consider a holistic approach in applying AI to the SSE lifecycle and associated customer/developer project management activities. This presentation focusses on applying AI to the SSE lifecycle based on experiments that I’ve performed across key phases.
Daniel Grivicic
This paper considers how engineers assess designs with the heart, drawing on felt experience of what has gone wrong before. However, when confronted with a new and unfamiliar design, the head takes over, and the written requirements and others’ experience through standards become all they have. Yet the deepest understanding of a design lives in that felt experience, not in the requirements or engineering standards. So, can a design be called good when only the head was used?
Engineers check a design in two ways: against the written requirements, and against their own experience of what has gone wrong before. The second is the one that catches the deep problems; however, it is seldom documented. It lives within the individual, drifting and fading as people move between companies and industries, and organisations seldom capture it. When a design is genuinely new, the kind we meet more and more as capabilities evolve, the engineer has no past pattern to draw on, so the written requirements and associated standards become the whole of the check. They then sign off that the design is “designed to requirements”. But that confirms only the written half. The lived experience that would tell us the design is sound was never on the page, so its absence cannot be seen. Compliance with the requirement quietly stands in for the knowledge that the design is good, and no one notices, because the missing knowledge left no trace.
Daryl Wheatley
There is no doubt that Artificial Intelligence (AI) is now being used to assist with the coding of non-safety related systems. It is only a matter of time, if it is not already occurring, that AI might participate in some aspects Safety Application Program development. This could be software module development or creating code that is based on existing software structures that are easily repeatable. It must be accepted that AI, if only in somewhat different ways, makes mistakes just like humans. If it is considered acceptable that AI assist with code development could there be potential to use it to assist with the functional validation of the safety application program. There is potential that diverse types of AI could be used to identify and highlight misalignment with the stated system requirements as well as miscellaneous errors to be more thoroughly reviewed by an experienced human checker. This could increase the timeliness, scale and complexity of the software review that could be conducted. Any tools that could reduce systematic errors should be examined and critically considered. However, given the nature of AI how do we ensure that the assistance rendered by AI adds value to the validation. How do we ensure that safety is improved and systematic errors reduced rather than using AI for the sake of it and giving ourselves a misplaced, overinflated or worse completely false sense of security?
Barry Brown
Integration is often presented as an unforeseeable and insurmountable obstacle to the delivery of Engineering projects. Moreover, it is often perceived to be an Engineering Problem. There is a clear contradiction in this logic of this conclusion.
This paper aims to challenge the above perception through consideration of Causation particularly in terms of the Management Systems that an organisation implements to successfully deliver a project. To achieve this the following steps will be followed:
One might question the value of the above evolution and whether it comes close to addressing the unsurmountable Integration obstacle. However, we can now categorise all Safety specific undesirable Events as generic Threats to a generic visual representation of the SMS through its Safety Claims. We have in effect created a universal visualisation to Assure project Safety Success. This paper aims to go further through the following steps:
This paper aims to deliver Integrated project success. Similar visual representations alone do not achieve this, but they come close. This following steps complete the circle of evolution:
The result of the above approach is to deliver Systems Integrated through Logical Claims (SILC-Tie representation).
The implicit conclusion from the above approach is that effective Integration is not a discrete activity for which the Engineering function is wholly responsible or a discrete phase of a project. It requires effective Integration of all functions throughout the entire project life-cycle, Integration of all Management Systems and an understanding of the common touch-points that impact each aspect of project Success.
Effective and Efficient Integration is a conjunction of different worlds and different philosophies. These worlds can coexist, but to harmoniously deliver common success mutual dependencies and common touch-points need to be managed as factors that are critical to project success.
Collin Song & Zoltan David
Safety cases are intended to be living documents containing safety arguments and evidence, remaining valid throughout the full lifecycle of a system. However, many rail projects treat them as one-off deliverables, producing static safety cases only to obtain system acceptance. These safety cases are often not meaningfully updated during operation and maintenance stage to maintain the validity, which creates a gap between the basis on which the original safety argument was made and the evolving operational reality.
Safety cases are expensive documents to produce. When prepared well, they preserve the valuable insights into what makes the system safe and on what basis safety decisions were made. Over time, this information may no longer be apparent to operators or even to the original system supplier after organisational changes.
Academic research has long advocated dynamic safety cases, where design-time safety arguments are extended to benefit operation and maintenance. They can serve as powerful tools for continuous run-time monitoring, feedback and improvements. They can also facilitate systematic management of change to identify the “ripple effects” that may not be obvious in emerging and complex socio-technical systems within their application environment.
Despite this potential, adoption of dynamic safety case for run-time assurance within the rail industry remains limited. This is increasingly problematic in an era of continuously evolving capabilities, where legacy systems are more frequently modified and where artificial intelligence, autonomous systems, and data-driven systems are increasingly introduced.
This research examines the practical considerations for applying dynamic safety cases in the railway context. It explores their potential benefits and how rail system safety practitioners can improve safety case development practices so that operators can use these safety cases to realise those benefits.
Dr Andrew Hussey, Joshua Cobb, Martin Hughes, Yacine Kharoufi, Hubert Grondin, & Padma Kodey
One of the key contributions of the Common Safety Method, also reflected in the EN50126:2017, is the use of Codes of Practice to identify hazards and their associated controls. The UNISIG standards for development of interoperable ETCS systems also incorporate hazard analysis and a hazard log, however this is mainly from the perspective of the components of an ETCS system and their contribution to the ETCS core and auxiliary hazards. Instead, at the overall ETCS System level, a functional analysis is desirable, also reflecting the balance achieved between system and human contributions to failure, so as to properly manage both the EN50126 process as well as demonstrate safety in accordance with SFAIRP. Since this functional analysis is missing from the UNISIG, it needs to be provided for each project where an ETCS system is delivered. In Hitachi Rail, previous projects have created functional analyses as needed to reflect the specific requirements applicable for each customer. However increasingly, the need for standardisation of the analysis, as well as re-use of previous best practice, has become apparent. To accommodate this, a Generic ETCS Hazard Analysis has been developed, which can also be considered to be an emerging Code of Practice for identification of hazards and their associated controls for ETCS systems. This paper is concerned with describing the approach that was taken to developing this analysis, as well as the benefits that this brings, and the remaining challenges to fully realise the complete potential. We describe two case studies where the Generic ETCS Hazard Analysis was used, the optimisations to our process that resulted, as well as the limitations that we observed. The first case study is a project hazard analysis, while the second case study is a training course, incorporating a SFAIRP summary for each hazard in the Generic ETCS Hazard Analysis.
Ray Feodoroff
The principle problem for safety assurance of any system has long been recognised as a burden, generally applied as a separate “argument”, more often by a non-associative third party author, essentially tasked with making the “argument” the system is safe by way of explaining that the “correct” requirements are implemented, in a manner akin to what otherwise has the look and feel of design rationale and over the means-end decomposition of the requirements into design and implementation.
This problem of requirements as the root of assurance (safety or otherwise) seems also to have evolved as a “solution” in the AI Coding Agent space with reference to Spec Driven Development (SDD) and/or Test Driven Development (TDD).
If we have spent so much time away from writing specifications to meet the WISCY1 urge of project management (and of customers really), and with noticeable flaws in the implementation of TDD over the decades, whether a mashup of SDD and TDD will resolve issues with “software” let alone “software safety” assurance for AI generated code begs asking. Going back to first principles may only beg more questions, but we need to go back to first principles.
Going back in time appears, therefore, a must to investigate what means are available to tackle software assurance for AI generated code.
This is otherwise a status report for an ongoing investigation towards tool smithing of a semiformal but approachable means to support AI coding agent driven SDD (and by extension TDD), to support a level of software assurance of code generated by AI agents.
Sarah Maslen & Jan Hayes
In high stakes technoscientific domains (i.e., aviation, healthcare, engineering of hazardous infrastructure), AI technologies are increasingly being designed for professional decision support. This raises significant questions regarding professional ethics and decision making. We examine these “narrow” forms of AI as epistemic (i.e. knowledge-related) technologies which call on us to grapple with the relation between human and nonhuman in reasoning, matters of trust, and responsibility for decisions in both forward- and backward-looking senses. We articulate four areas of particular concern, including the challenge that moral distancing poses to ethical decision making, the limitations of models in accounting for rare events, how users can become overly reliant and uncritical in their evaluation of AI outputs for reasons of human cognition as well as their work pressures, and lastly the critical role that AI whistleblowers play in ensuring ethical protections. Other research has called for AI ethics in practice driven by operationalized principles. We argue that effective AI-assisted decision making must also be grounded in the decision making moment. The goal must be to see not only what AI can or cannot do, but what professionals do currently and must continue to do, and how AI may or may not help.
Michael Johnson & Bart Jacobs
“Voting systems” can mean both
Historically voting systems in sense (1) have been defined by long procedural rules about sorting and shuffling ballot papers, and the counting systems (2) have had to be built to match those wordy specs . But voting systems in sense (1) have since the 18th century been based on mathematical descriptions, and the wordy procedures conflate the mathematically necessary steps and procedural steps intended to remove ambiguity (added so that observers can agree that the steps were followed and so the outcome is unarguable).
A recent analysis of voting systems (in both senses) (see [1]) has shown how changing the datatype from lists of ballot papers to multisets whose base elements are the possible votes (eg different preference sequences) leads to some remarkable simplifications aligning
This talk will likely teach even well-informed Australians significant things that they didn’t know about Australia’s voting systems and suggest improvements that would both simplify the public understanding of the systems and reinstate the mathematical intentions of STV vote counting, as well as providing examples of greatly simplified implementations. It is hoped that such data-type reformulation might be similarly useful in other safety- or mission-critical systems, and that seeing a remarkable, and remarkably simplifying, example might inspire similar work in other areas.
[1] Bart Jacobs, Michael Johnson and Richard Buckland, Counting votes with multisets, arxiv.org/abs/2606.05218
Professor Jill Slay
Space systems have quietly become load-bearing infrastructure for the Australian economy, national security and everyday life. Positioning, timing, communications and Earth observation services underpin energy grids, financial settlement, emergency response and the precision agriculture on which export revenues depend. Yet the governance frameworks designed to protect critical infrastructure were written before this dependency became acute, and the threat environment has evolved far faster than the regulatory response.
This keynote draws on two intersecting bodies of work. The first is the 2026 Independent Review of the Security of Critical Infrastructure Act 2018, which found that submissions overwhelmingly supported extending formal obligations to space assets and that current voluntary arrangements leave significant gaps in resilience planning. The second is original research into the security of Software-Defined Networking architectures in space - a technology that offers compelling operational advantages for large constellations but that concentrates control in ways that mirror the single points of failure safety engineers have spent decades designing out of terrestrial systems.
The talk introduces AUSCAN - the Assured Unified SPARTA-COSMOS2 Assessment Notation - a quantitative risk scoring methodology that integrates threat characterisation from the SPARTA framework with asset criticality weights from the COSMOS2 space mission architecture model. AUSCAN produces a governance-ready score compatible with Australia’s Critical Infrastructure Risk Management Program obligations and with established Continuous Risk Management practice in system safety. Three attack categories unique to the SDN-space intersection are examined: transferred threats with amplified consequences, amplified threats with constellation-scale blast radii, and novel threats that have no terrestrial analogue.
The central argument is systemic rather than technical: space failures share the irreversibility and cascading consequence that define safety-critical systems, and the engineering disciplines of system safety offer proven governance models that cyber regulation has yet to fully adopt. Closing the gap demands coordinated action across operators, regulators and standards bodies before the next significant space-related cyber incident makes the case for us.
Gary Gaskell
We cannot patch our way to a secure system state. Neither will patching give us a stable and safe system. This presentation will provide a background for discussions about the gaps between the current cyber security “industry’s” strategies and products compared to what is required to support safety critical systems. The ‘whack a mole’ approach of the cyber industry is a dangerous approach to systems protection. In many cases, this has evolved from our software programming practices - as soon as the software works for the expected scenarios - it is released. However, secure systems must handle edge cases and indeed unexpected abuse cases.
Many industry players are motivated by the confidentiality of personal data. While that is a worthy goal, it is not as important as the integrity and hence reliability of a control system. Similarly, the concept also applies to our national critical infrastructure. Safe and reliable systems depend on systems engineering disciplines and robust products. Unfortunately these concepts are not common in the cyber security ‘industry’. This presentation will discuss this gap between the engineering disciplines of control system engineering and the current cyber security ‘industry’. The talk will explore how we might close the gap with the cyber security community.
Matthew Squair
Fratricide risk in a modern Integrated Air and Missile Defence (IAMD) enterprise System of Systems (e-SoS) is an emergent property of the networked system of systems resulting from the integration, reconfiguration and interoperability aspects of the IAMD SoS. This paper reframes fratricide risk, and it’s prevention, as a system-of-systems design, governance, and operations problem. In order to derive effective hazard controls the IAMD eSoS is analysed to identify system of system hazard causal mechanisms in the context of the six enterprise mission level functions (i.e. tracking, identification, classification, airspace control, friendly force and engagement). Fratricide risk arises when those distributed functions diverge across the enterprise resulting in a breakdown in coordination and there is no control mechanism within the constituent systems of the SoS available to restore a consistent and correct picture before an irreversible decision is made. A broader socio-technical perspective is taken that includes the people, policy, processes and information as part of the SoS of interest. Classical safety analysis techniques such as event and fault trees are used to structure identified hazard controls to provide a defence in depth control architecture. The advantages and challenges of implementing this approach within an agile development framework are discussed.
Dr Andrew Hussey, Graziano Marino, Maria Hill, & Mohan Murari
The standard development lifecycle for railway projects typically assumes a linear progression from concept through definition, implementation, deployment, and maintenance, often spanning multiple years, with established standards and processes reflecting this approach. However, modern railway systems operate in an environment where functionality must evolve continuously while maintaining safe, uninterrupted operations.
This paper examines a case study of a mission-critical, 24/7 automated freight system, which requires not only ongoing maintenance but also the frequent introduction of new and modified functionalities. The continuous evolution of such systems challenges traditional lifecycle assumptions and necessitates the adaptation of established safety and engineering processes.
Key challenges are identified, including maintaining safety assurance under compressed deployment timelines, managing frequent system updates, and ensuring robust configuration and integration practices within an operational environment. The paper presents practical insights and lessons learned from implementing multiple safe deployments per year, demonstrating how tailored processes and strong stakeholder collaboration can enable effective, efficient, and safe delivery of system updates in a continuous development context, based on tailoring the standard processes, using key concepts from other methodologies, such as agile, lean and evolutionary prototyping.
Raj Chotai
The development of structured safety arguments is a critical yet resource-intensive activity in the assurance of safety-critical systems. Recent advances in generative artificial intelligence (AI), particularly large language models, present new opportunities to assist safety engineers in producing safety arguments and associated artefacts. However, uncritical adoption of AI poses significant risks, including false confidence, hidden assumptions, and inappropriate safety judgments. This paper presents a practitioner- focused examination of AI-assisted safety argument generation using a human-in-the-loop approach. Drawing on historical, human-developed safety arguments as reference cases, the study compares them with AI-generated equivalents created under controlled conditions. The analysis identifies areas where AI provides genuine value, areas where it introduces subtle safety risks, and activities that remain solely under human authority. Based on these findings, the paper proposes a responsibility boundary model and governance controls for the safe and effective use of AI in safety argument development.
Payal Tiwari
AI safety literature - including the most recent 2026 reports from PwC, McKinsey, IBM, Deloitte and BCG - focuses almost exclusively on system-level assurance and enterprise value. What is largely absent is a safety-critical analysis of how agentic AI operating models affect the people least equipped to contest automated decisions: migrant communities, family violence survivors, and people who interact with AI-mediated public services.
Australia’s Robodebt scheme is the most documented case study of what autonomous decision-making looks like without genuine human accountability - $1.76 billion raised against 526,000 people, with documented deaths. As agentic AI scales, the question of who is accountable to the citizen - not just to the system owner - becomes a safety-critical question, not merely an ethical one.
I bring a rare dual perspective to this topic: 15+ years as a technology executive, currently operating as a Fractional Technology Advisor & Chief AI Officer (CAIO), with prior roles at Deloitte, EY, Accenture and Wipro - and a practitioner background in family violence policy and multicultural community advocacy, including advisory work with Victoria Legal Aid’s Specialist Family Violence Courts project.