Chapter 77: Standards Evolution |
ISO/IEC TR 5469 Paves the Way for Safety-Related Systems Using AI Technologies |
Summary |
For decades, industrial robot safety followed a predictable script. A robot performed a task within a defined workspace. Engineers assessed the risks. They installed fences, light curtains, and interlocks. The robot repeated the same motion millions of times with deterministic precision. Safety was a matter of geometry and hardwired logic. Then artificial intelligence entered the factory floor, and the script began to change. AI systems do not simply execute predefined paths; they perceive, interpret, and decide. They learn from data. They respond to conditions their designers never explicitly anticipated. This capability offers remarkable flexibility, but it also fractures the foundational assumptions upon which traditional functional safety standards were built. ISO/IEC TR 5469, published in early 2024, represents the first serious international attempt to bridge this gap. It does not prescribe rigid rules. Instead, it describes the properties, risk factors, methods, and processes relevant to using AI inside safety-related functions, using non-AI safety functions to constrain AI-controlled equipment, and using AI to design safety functions themselves. For the industrial robotics community, this technical report serves as both a warning and a roadmap. This chapter explains what the standard addresses, why it matters, and how industries ranging from automotive manufacturing to warehouse logistics are beginning to apply its principles in practice. |
|
1. The Gap That TR 5469 Was Written to Fill |
1.1 Why Traditional Safety Standards Assume a Deterministic World |
Functional safety standards such as IEC 61508 and ISO 13849 were developed for systems whose behavior could be fully specified in advance. A safety function in a classical industrial robot might be 'if the light curtain is broken, remove power from the drives within a specified response time.' The logic is explicit, the failure modes are enumerable, and the performance level can be calculated with established techniques. The underlying philosophy is deterministic: given the same inputs, the system produces the same outputs, and any deviation is a fault to be detected and managed. |
AI systems violate this philosophy at a fundamental level. A neural network that has learned to recognize human figures in a camera feed does not follow a rule that can be traced line by line. Its behavior emerges from millions of numerical weights adjusted during training. The same input image, if slightly altered by noise or lighting, may produce a different classification. The system may encounter situations that were absent from its training data and respond in ways that are difficult to predict, let alone prove safe. The verification methods that functional safety engineers have relied upon for decades, such as exhaustive testing against a complete specification, do not map cleanly onto systems whose specifications are inherently incomplete. |
|
1.2 What the Technical Report Actually Contains |
ISO/IEC TR 5469 is structured around three modes of AI involvement in safety. The first is the use of AI inside a safety-related function to realize the functionality itself. Imagine a collaborative robot that uses a vision-based AI model to detect a human hand entering its workspace and adjust its trajectory accordingly. The safety function, namely preventing contact with the human, depends on the AI model's perception. The second mode is the use of non-AI safety-related functions to ensure safety for equipment controlled by AI. Here, the AI might handle productivity optimization, such as choosing which parts to pick first from a bin, while a separate, conventional safety system monitors for hazards and intervenes if the AI's decisions lead toward unsafe states. The third mode is the use of AI systems to design and develop safety-related functions, for example, using machine learning to explore design spaces for safety controllers that human engineers might not consider. |
The report does not declare any of these modes inherently acceptable or unacceptable. Rather, it catalogs the properties of AI that create risk and the methods available to manage those risks. It discusses the opacity of learned models, the sensitivity of AI to adversarial inputs, the challenges of verifying systems whose behavior changes with environmental conditions, and the limits of test coverage when the input space is effectively unbounded. It also emphasizes the need for continuous verification throughout the system lifecycle, recognizing that an AI model that was safe at deployment may drift or degrade as the world around it changes. |
|
1.3 The Relationship to Sector-Specific Standards |
TR 5469 is explicitly a guidance document, not a certification standard. It does not provide a checklist that a robot manufacturer can complete to claim compliance. Instead, it provides a conceptual vocabulary and a risk framework that other standards can reference. The forthcoming ISO/IEC TS 22440 is expected to translate the technical report's guidance into more detailed requirements. Meanwhile, the 2025 revision of ISO 10218-1, the foundational safety standard for industrial robots, has already begun to reflect the new reality. It now requires manufacturers to provide detailed documentation of safety functions, including their performance parameters and relationship to inputs, outputs, and response times, and it introduces cybersecurity requirements that acknowledge the networked and software-intensive nature of modern robot systems. The European Union's regulatory landscape has also shifted. The AI Omnibus, which entered into force in July 2026, restructured the compliance pathway for 'physical AI' by integrating high-risk AI requirements into the Machinery Regulation, with future delegated acts expected to add AI-specific health and safety requirements. TR 5469 sits at the technical foundation of these evolving frameworks. |
|
2. What AI Changes About Safety in Industrial Robots |
2.1 Perception Is Not Measurement |
Traditional robot safety relies on sensors that report measurable quantities: a light curtain reports whether its beam is interrupted, a force sensor reports a torque value, an encoder reports an angle. These are measurements with known error bounds and predictable failure modes. AI-based perception, by contrast, produces interpretations. A vision model does not measure 'human present'; it classifies an image region as containing a human with some confidence score. That score may be high in the training data distribution and drop sharply in unusual lighting or when the person is partially occluded. The safety function that depends on this interpretation must account for the possibility of confident but wrong outputs. |
2.2 Generalization Cuts Both Ways |
The great promise of AI in robotics is generalization: a model trained on thousands of examples can handle variations it has never seen. The same property creates safety uncertainty. A traditional safety system fails in predictable ways. An AI perception system may succeed on a thousand consecutive cycles and then fail on the thousand-and-first because the object configuration is subtly different from anything in its training set. The failure is not a broken sensor or a seized actuator; it is a gap in the model's learned representation of the world. TR 5469 identifies this as a core challenge, noting that the non-existence of a complete a priori specification makes conventional verification incomplete. |
2.3 The Environment Is Part of the System |
An AI model's behavior depends on the data distribution it encounters. When the lighting in a factory changes, when a new type of packaging material is introduced, when seasonal humidity alters camera lens clarity, the input distribution shifts. A model that performed safely at commissioning may become less reliable months later without any component failing in the conventional sense. This phenomenon, often called distribution shift, means that safety assurance cannot end at the factory gate. TR 5469 emphasizes lifecycle monitoring and continuous verification as essential complements to pre-deployment testing. |
2.4 Adversarial and Accidental Inputs |
AI models are vulnerable not only to natural variation but also to deliberately crafted inputs. An adversarial patch, a small pattern placed in a camera's field of view, can cause a vision model to misclassify objects with high confidence. While deliberate attacks may seem exotic in a fenced industrial cell, the broader category of out-of-distribution inputs is not. A reflective surface, an unusual shadow, or a maintenance worker wearing an unfamiliar garment can all produce inputs far from the training distribution. The safety function must be designed with the assumption that perception will sometimes be wrong, and it must fail in a way that does not endanger people. |
|
3. How Industries Are Applying the Principles |
3.1 Automotive Manufacturing: Constraining AI with Deterministic Safety Layers |
The automotive industry has been among the earliest adopters of AI-adjacent robotics safety practices, though often without labeling them as such. In body shops and assembly lines, AI is increasingly used for tasks such as adaptive bin picking, where a vision model identifies and localizes parts in a cluttered container, and for trajectory optimization in painting or sealing applications. The safety architecture in these deployments typically follows the second mode described in TR 5469: AI handles the productive task, while a separate safety-rated system enforces hard limits. |
Consider a robotic bin-picking cell feeding a stamping press. A 3D vision system powered by a learned model identifies the pose of each blank in a bin. The robot arm reaches in, grasps the part, and places it into the press. The AI model's decisions affect productivity, but they do not govern the boundary of the robot's safe workspace. A safety-rated laser scanner monitors the cell perimeter and a safety controller enforces speed and position limits independent of the AI. If the AI selects an unreachable pose or misidentifies a part, the robot may drop the part or collide with the bin, but it cannot enter the space where a human might be present. This separation of concerns, where AI optimizes within a safety envelope defined by conventional technology, is the most common pattern in industrial settings today. It aligns with TR 5469's guidance that non-AI safety functions can be used to ensure safety for AI-controlled equipment. |
3.2 Collaborative Robots: The Hard Problem of Speed and Separation Monitoring |
Collaborative robots introduce a more demanding scenario. In a speed and separation monitoring application, the robot must continuously adjust its speed based on the distance to a human. Traditional implementations use certified safety laser scanners that detect a person's presence and position. An AI-based approach might use a camera and a learned model to distinguish between a human, a forklift, and a falling box, allowing the robot to behave differently in each case. The appeal is clear: a system that understands context can be more productive than one that treats every intrusion as an emergency. |
The safety challenge is equally clear. If the AI model misclassifies a human as a stationary object, the robot may continue moving at full speed. The confidence score is not a certified measurement of distance. To address this, some developers have begun implementing what TR 5469 might describe as a three-stage realization: the AI perception provides an estimate of human presence and pose, a conventional safety-rated function independently verifies proximity using a certified sensor, and a safety controller enforces the speed limit based on whichever source reports the greater hazard. The AI adds value when it is correct; the conventional safety layer ensures that errors do not become accidents. This architecture accepts that AI will sometimes be wrong and designs the system so that wrongness is tolerable. |
3.3 Warehouse Logistics: AI for Navigation, Conventional Safety for Stopping |
Autonomous mobile robots and automated guided vehicles in warehouses present a different risk profile. The primary hazard is collision with people or other vehicles. Many AMR fleets use AI for navigation, mapping, and traffic management. A learned model might predict the likely path of a human walking through an aisle, allowing the robot to plan a smoother route. The safety function, however, remains largely conventional. Safety-rated lidars detect obstacles within a protective field, and the robot stops if an object enters that field, regardless of what the AI predicts. |
TR 5469's relevance here lies in the boundary between the two domains. If the AI navigation system's predictions influence the robot's speed, then the safety-rated stopping distance must account for the maximum speed the AI might command. If the AI can request a trajectory that brings the robot closer to a human than the conventional safety field would normally allow, the safety function must be designed to reject that request or to dynamically adjust the field based on certified inputs. The technical report's discussion of 'level of automation and control' and 'transparency' speaks directly to these design decisions: how much authority does the AI have, and how clearly can that authority be bounded and communicated to the safety engineer |
3.4 Electronics and Semiconductor Manufacturing: Fast Deployment with Guarded Boundaries |
The 3C electronics and semiconductor sectors have adopted AI-enabled robotics with particular speed, driven by short product cycles and the need for rapid reconfiguration. In semiconductor back-end processes, robots perform chip tray loading and unloading with requirements for high precision and minimal downtime. Some deployments have reported application orchestration times measured in minutes rather than days, with success rates exceeding 99.9 percent in structured environments. The AI in these systems often handles task sequencing and visual servoing, adapting to slight variations in tray position and component orientation. |
The safety architecture in such deployments tends to be conservative. The robot operates within a guarded cell with interlocked doors. The AI's freedom is confined to the space inside the guard. If the AI makes an error, it may scrap a tray of chips or collide with a fixture, but the interlock ensures that no person is exposed. This is the simplest application of TR 5469's second mode: the AI is productive but not safety-critical, and conventional safeguarding ensures that its failures remain contained. As these systems become more collaborative and the guards come down, the safety challenges will migrate toward the collaborative robot scenarios described earlier. |
3.5 Battery Manufacturing: High-Risk Processes as Early Adopters |
The battery industry offers a striking example of AI-controlled robots taking over tasks that are dangerous for humans. In battery pack assembly, high-voltage connect and disconnect operations and direct current internal resistance testing are hazardous. Human workers must follow strict procedures and wear protective equipment. Some manufacturers have begun deploying robots with learned manipulation policies to perform these tasks, reporting insertion success rates above 99 percent and cycle times comparable to skilled workers. The safety justification in these cases is compelling: the robot removes the human from the hazard entirely. The AI's errors may damage equipment or scrap a pack, but they do not injure a person because no person is present during the operation. |
TR 5469's framework is relevant even when humans are absent from the immediate hazard zone. The question shifts from 'can the AI hurt someone' to 'can the AI create a condition that later hurts someone' A robot that mishandles a high-voltage connection might create a latent fault in a battery pack that only manifests in the finished vehicle. The safety analysis must consider the AI's contribution to product integrity, not just immediate physical safety. This is a subtle point that the technical report's discussion of the three-stage realization principle and lifecycle verification helps to illuminate. |
3.6 Collaborative Assembly and the Human-in-the-Loop |
Perhaps the most demanding application of TR 5469's principles is the collaborative assembly cell where a human and a robot share a workspace and a task. In these settings, AI might be used to recognize the human's intended action and adjust the robot's behavior accordingly. A robot handing a tool to a human worker needs to know when the human has grasped it and when to release. A vision-based AI model can provide this awareness, but the safety function that prevents the robot from moving while the human's hand is near the gripper must be robust to the model's errors. |
Some research groups have explored architectures in which a learned model predicts human motion and a conventional safety monitor checks that prediction against the robot's certified safe limits. If the predicted motion would bring the human within a minimum separation distance, the robot slows or stops regardless of the AI's confidence. The AI's role is to make the interaction smoother, not to be the last line of defense. This aligns with the technical report's emphasis on using non-AI safety functions to bound the behavior of AI-controlled equipment, and with the broader principle that safety assurance should not depend on the correctness of a component whose failure modes are not fully understood. |
|
4. What Compliance Looks Like in Practice |
4.1 Documentation Beyond the Conventional |
ISO 10218-1:2025 now requires manufacturers to provide a functional safety clause that describes all implemented safety functions, including their performance parameters and relationship to inputs, outputs, and response times. When an AI model participates in a safety function, even indirectly, this documentation must describe the model's role, its performance characteristics, its known limitations, and the measures taken to ensure that its errors are detected or tolerated. A conventional safety function can be described with a circuit diagram and a calculated performance level. An AI-involved function requires a narrative that explains the data used for training, the validation and testing performed, the monitoring in place during operation, and the criteria for retraining or decommissioning. |
4.2 Continuous Monitoring as a Safety Requirement |
TR 5469's most consequential implication for industrial practice is that safety assurance for AI systems cannot be a one-time event. The report calls for continuous verification and improvement throughout the system lifecycle. In operational terms, this means that a robot using an AI perception model for a safety-adjacent function should have mechanisms to detect when its inputs have drifted outside the distribution on which the model was validated. If the camera view becomes obscured, if the lighting changes beyond a threshold, or if the objects in the scene no longer resemble the training data, the system should degrade gracefully, perhaps reverting to a slower speed or a more conservative separation distance. Some advanced systems log confidence scores and input statistics, using them to trigger maintenance alerts or automatic recalibration. These practices are not yet standardized, but they represent the direction in which the field is moving. |
4.3 The Role of Simulation and Digital Twins |
Verification of AI-based safety functions cannot rely solely on physical testing, because the input space is too large and the relevant scenarios may be rare. Simulation environments and digital twins offer a way to explore a wider range of conditions than physical testing allows. A vision model can be evaluated against synthetic images with varied lighting, occlusion, and object poses. A motion planning policy can be tested against simulated humans whose behavior spans a range of speeds and trajectories. These methods do not prove safety in the mathematical sense, but they provide evidence of robustness that complements real-world testing. TR 5469 acknowledges the limitations of test coverage and points toward simulation as one of the available methods for building confidence in AI systems. |
4.4 The Evolving Regulatory Landscape |
The European Union's AI Omnibus, effective from July 2026, has restructured the compliance pathway for AI-enabled machinery. High-risk AI requirements for machinery will be integrated into the Machinery Regulation through future delegated acts, rather than applying the AI Act directly. For robot manufacturers selling into Europe, this means that the safety file for a machine with AI-based safety functions will increasingly need to address AI-specific concerns within the machinery conformity assessment framework. TR 5469 provides the technical vocabulary for those discussions, even though it is not itself a harmonized standard. In other jurisdictions, similar developments are underway. The technical report's influence is likely to be felt through its adoption by standards bodies and its use as a reference in certification schemes, rather than through direct regulatory citation. |
|
5. What TR 5469 Does Not Solve |
5.1 It Is Guidance, Not a Prescription |
A technical report is not a normative standard. It describes and explains; it does not mandate. A robot manufacturer cannot claim compliance with TR 5469 in the way one claims compliance with ISO 10218-1. The report's value lies in helping engineers and safety assessors ask the right questions. What data was the model trained onHow does its performance degrade outside that distributionWhat monitoring is in placeWhat happens when the model is wrongThese questions do not have universal answers, but the report provides a structured way to approach them. |
5.2 The Verification Problem Remains Open |
The fundamental challenge that TR 5469 describes, namely that AI systems cannot be verified against a complete specification in the way deterministic systems can, has not been solved. The report catalogs methods and processes that can increase confidence, but it does not provide a formula for proving safety. For industrial robot applications where the consequences of failure are severe, this means that AI will likely remain confined to roles where its errors can be caught by conventional safety layers for the foreseeable future. The fully autonomous collaborative robot that relies on AI for its primary safety function remains a research frontier, not a commercial reality. |
5.3 Organizational and Cultural Barriers |
The separation between AI development teams and functional safety teams within many organizations creates practical obstacles. AI engineers optimize for accuracy, generalization, and performance. Safety engineers optimize for predictability, traceability, and risk reduction. These objectives are not inherently opposed, but they require translation and negotiation. TR 5469 provides a shared vocabulary, but it does not resolve the organizational tensions. Companies that have made progress, such as those deploying AI-enabled robots in battery manufacturing and electronics assembly, tend to have cross-functional teams that include safety expertise from the earliest design stages, rather than treating safety as a final checkpoint before deployment. |
|
6. Detailed Summary |
ISO/IEC TR 5469 marks a turning point in how the industrial robotics community thinks about safety. It acknowledges a reality that many engineers had already sensed: artificial intelligence is entering the factory floor, and the deterministic assumptions that underpinned functional safety for decades no longer hold in the same way. The technical report does not pretend to offer a complete solution. Instead, it provides a structured description of the problem, a vocabulary for discussing it, and a catalog of methods that can help. Its three modes of AI involvement in safety, namely AI inside the safety function, AI controlled within non-AI safety boundaries, and AI used to design safety functions, map onto the real architectures being deployed in automotive assembly, warehouse logistics, electronics manufacturing, and battery production. The most common pattern in industry today is the second mode: AI handles productivity and adaptation, while conventional safety-rated systems enforce hard limits. This pattern is pragmatic. It allows companies to benefit from AI's flexibility without betting human safety on the correctness of a learned model. More ambitious applications, such as AI-based speed and separation monitoring in collaborative cells, require architectures that can detect and tolerate AI errors, and these remain areas of active development and careful validation. The report's emphasis on lifecycle verification and continuous monitoring reflects the recognition that AI systems change over time, not because their code is modified, but because the world they perceive changes around them. For engineers and safety assessors working in industrial robotics, TR 5469 is best understood as a foundation rather than a finish line. It establishes the conceptual groundwork that sector-specific standards and forthcoming technical specifications will build upon. It signals to the industry that AI safety is not a problem to be solved once and forgotten, but an ongoing discipline that must be integrated into design, deployment, and operation. The factories of the coming decade will contain more AI, not less. The standards evolution that TR 5469 represents is the beginning of the industrial robotics field's answer to the question of how to make that AI safe, not by pretending it is something it is not, but by understanding what it is and designing systems that account for its limitations as carefully as they exploit its capabilities. |
cs@easiersoft.com