Trustworthy AI for Manufacturing

Joseph Cohen¹ and Xun Huan²

1 Department of Mechanical and Aerospace Engineering, Rutgers University, Piscataway, NJ, USA 2 Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI, USA

E-mail: joseph.cohen3@rutgers.edu, xhuan@umich.edu

Status

History has shown that trust in new technologies is rarely immediate. Industrial automation, aviation autopilots, and self-driving cars all faced initial skepticism, earning acceptance only after demonstrating reliability, safety, and integration with human oversight and regulations. Artificial intelligence (AI) in manufacturing stands at a similar crossroads: to be trusted, AI systems must not only perform well, but also be explainable, accountable, and situated within human and institutional contexts. AI is rapidly transforming manufacturing, advancing prognostics and health management [1], predictive maintenance [2], quality control [3], process optimization [4], and industrial digital twins [5]. These technologies improve throughput and yield, reduce downtime, and enhance quality in complex, multistage manufacturing environments. However, the opaque, black-box nature of AI raises critical concerns about transparency, reliability, and safety, especially in high-stakes domains such as aerospace and medical device manufacturing. These concerns can lead to resistance or overreliance, ultimately undermining the transformative potential of AI. Trust in AI extends beyond technical performance, encompassing also cultural and organizational elements. While organizations with strong innovation cultures may be more willing to adopt AI, others require stronger assurances of reliability and compliance. Even highly accurate models paired with explainable AI can fail to earn trust if their explanations do not meaningfully represent system behavior. After all, an AI “explanation” that humans cannot understand or act upon is not truly an explanation. The institutional dimension of trust is equally critical. Since AI systems cannot be held legally accountable, responsibility ultimately rests with people and organizations. This makes trustworthiness essential not only for adoption, but for assigning responsibility, mitigating liability risks, and enabling regulatory oversight. In high-consequence manufacturing domains, trust also depends on insurability and certifiability: systems must not only be reliable in terms of performance, but also legally and operationally dependable.

Current and future challenges

The path toward trustworthy AI for manufacturing faces interconnected challenges that we organize into two categories: computational and modeling and human and societal (see Figure 1).

Figure 1. Trustworthy AI framework for manufacturing. Technical foundations in computational

modeling (inner ring) enable human and societal factors (outer ring) that are essential for successful AI adoption in manufacturing environments.

1. Computational and modeling challenges Interpretability and explainability: Modern AI systems often function as black boxes with deep architectures creating abstract latent representations that are not readily interpretable, making it difficult for users to understand prediction processes. Most systems also lack mechanisms to articulate the reasoning behind their outputs, constraining their explainability. Mathematical foundations: Unlike established computational frameworks such as the finite element method and numerical integration, many modern AI models—especially those with complex architectures and foundation models—have relatively limited theoretical analysis [6,7] (e.g., on consistency, stability, and convergence). This theoretical gap means systems are often deployed based on empirical performance alone, making it difficult to generalize into unseen and sparse regimes, understand unexpected behavior, or make systematic improvements. Verification, validation, and uncertainty quantification (VVUQ): Despite operating with noisy, sparse data and potentially misspecified assumptions, AI systems typically provide predictions without meaningful confidence measures. This absence of principled uncertainty quantification (UQ) can lead to overconfident predictions—resulting in unexpected risks and failures—or overly cautious predictions, which may cause unnecessary safety margins and wasted resources. Robust deployment also requires formal verification (establishing mathematical correctness and numerical accuracy) and validation (ensuring accurate representation of real-world systems), under a comprehensive VVUQ framework [8].

Security and privacy: As AI systems become increasingly dependent on digital infrastructure, they also become increasingly vulnerable to adversarial attacks from malicious actors. Centralized data repositories, cloud services, and connected devices face threats including data poisoning, model inversion, and data leaks [9] that can degrade performance, compromise sensitive information, and erode trust. Generative AI: The emergence of large language models (LLMs) and agentic systems introduce novel concerns including hallucinated outputs, unpredictable behavior, and untraceable reasoning pathways. As such systems are deployed in sensitive manufacturing stages—process planning, design, maintenance, and operations—it becomes increasingly important to understand not only what they can do, but when, where, and why they fail.

2. Human and societal challenges Human-computer interaction (HCI): Trust requires effective communication of predictions, uncertainties, and explanations tailored to users’ roles, backgrounds, and decision contexts [10]. For example, a floor operator needs intuitive visualizations, while a manager requires aggregated summaries tied to strategic performance indicators. Trust also evolves over time, requiring systems to support iterative interaction, feedback, and adaptation as both users and the manufacturing environment change. Organizational integration: AI must facilitate consistent communication across diverse teams, including operations, engineering, and business strategy. Explanations must be both role-specific and translatable across functions, often needing specialized multi-agent systems rather than isolated models. AI adoption must also navigate organizational culture while upholding fairness, equity, and non-discrimination principles, particularly amid labor-management tensions and perceived job displacement. Additionally, successful integration of AI with legacy systems and domain expertise is essential to preserve valuable human knowledge and intuition. Objective and value misalignment: AI models typically optimize narrow mathematical loss functions that may fail to reflect real-world priorities, ethical standards, or user intentions. In manufacturing, this can create new failure modes or neglect long-term tradeoffs important to stakeholders. The saying “all models are wrong” takes new significance in AI, where this wrongness can manifest silently and catastrophically when misaligned with emergent domain realities. Sustainability: The growing energy demands of training and deploying large models strain power infrastructure. High computational costs also impede industrial adoption, particularly for smaller manufacturers. Without innovation in energy-efficient algorithms and infrastructure, the environmental footprint of AI may become a barrier inhibiting responsible long-term adoption.

Advances in science and technology to meet challenges

We organize the recent advances in trustworthy AI for manufacturing to mirror the earlier challenge categories: computational and modeling and human and societal.

1. Computational and modeling advances Explainable AI (XAI): Post-hoc explanation methods such as Shapley values [11], counterfactual explanations [12], and saliency-based attribution [13] continue progress to provide interpretable model outputs. Furthermore, developments in causal modeling [14] and counterfactual reasoning offer robust insights into cause-and-effect relationships and support “what-if” scenario analysis critical for manufacturing decision-making. Mathematical foundations: Researchers increasingly investigate AI models’ analytical properties including complex training dynamics such as phase transitions, double descent, and grokking [6,15]. Efforts to bridge AI with classical numerical analysis and scientific computing are essential for integrating AI into the manufacturing computational infrastructure. Physics-informed AI and hybrid modeling approaches, such

as neural operators [16] and physics-informed neural networks (PINNs) [17], combine data-driven learning with domain-specific physical laws, offering enhanced trust through constraint-based learning and improved extrapolation beyond training regimes. Advanced UQ: UQ provides frameworks to characterize and communicate both epistemic and aleatoric uncertainty [18]. Bayesian approaches provide principled frameworks for representing epistemic uncertainty (what models do not know due to limited knowledge), particularly valuable for updating uncertainty estimates by assimilating sparse, noisy, and indirectly observed data. Ensemble-based and non-Bayesian methods effectively capture aleatoric uncertainty (inherent variability or randomness), crucial in manufacturing where processes are often stochastic. These approaches together support decision-making by providing not only AI model predictions but also their confidence levels. Monitoring for AI lifecycle: Techniques to detect distributional shift, concept drift, and anomalies are enabling real-time awareness of model degradation [19]. These capabilities are essential to managing the full lifecycle of AI models, where they can be continuously validated, updated, or decommissioned in response to evolving real-world conditions.

2. Human and societal advances Human-centered design: Research increasingly emphasizes user-adaptive, interactive explanations over static output. Modern explanation interfaces support dialogue-based refinement where users can ask follow- up questions and steer explanations toward what they find meaningful [10]. This approach supports trust calibration, helping users develop appropriate mental models of when to rely on AI and when to challenge them. Human-robot collaboration: Advances in human-robot teaming [20] focus on settings where AI assists rather than replaces human decision-makers, such as factory floors with autonomous systems and human operators. These directions will enable smoother, safer coordination between human expertise and AI capabilities. Standards and community infrastructure: Growing momentum toward shared standards, regulations, and community infrastructure include benchmark datasets with labeled outcomes, metadata, and uncertainty; standardized metrics for evaluating performance as well as explainability and robustness; third-party certification and validation frameworks; and open-source platforms and documentation standards to promote transparency and reproducibility. Sustainable AI development: Energy-efficient AI advances including model compression, hardware-aware training, and carbon-conscious deployment aimed at reducing the resource demands of both development and operation. These methods become increasingly important as the scale and number of deployed models continue to grow, ensuring that AI adoption remains environmentally responsible.

Concluding remarks

Trustworthy AI is a fundamental requirement for next-generation manufacturing. As AI systems become more capable and ubiquitous, the risks of opacity, misalignment, and failure scale alongside their potential benefits. Meeting these challenges requires coordinated progress across computational and modeling as well as human and societal foundations. The maturation of AI mirrors traditional engineering disciplines’ evolution through rigorous theory, verifiable practices, and earned community trust. Achieving this vision requires bringing together mathematicians, computer scientists, engineers, social scientists, domain experts, and end users. Success will bring AI systems to manufacturing that are not only powerful, but also understandable, reliable, and worthy of our trust.

References

[1] Ochella, Shafiee and Dinmohammadi 2022 Artificial intelligence in prognostics and health management of engineering systems Engineering Applications of Artificial Intelligence 108104552 [2] Ucar Karakose and Kırımça 2024 Artificial intelligence for predictive maintenance applications: Key components, trustworthiness, and future trends Applied Sciences 14(2) 898 [3] Prezas Michalos Arkouli Katsikarelis and Makris 2022 AI-enhanced vision system for dispensing process monitoring and quality control in manufacturing of large parts Procedia CIRP 1071275–1280 [4] Senoner Netland and Feuerriegel 2022 Using explainable artificial intelligence to improve process quality: Evidence from semiconductor manufacturing Management Science 68(8) 5704–5723 [5] Huang Shen Li Fey and Brecher 2021 A survey on AI-driven digital twins in Industry 4.0: Smart manufacturing and advanced robotics Sensors 21(19) 6340 [6] Suh and Cheng 2025 A survey on statistical theory of deep learning: Approximation, training dynamics, and generative models. Annual Review of Statistics and Its Application 12(1) 177–207 [7] Colbrook Antun and Hansen 2022 The difficulty of computing stable and accurate neural networks: On the barriers of deep learning and Smale’s 18th problem Proceedings of the National Academy of Sciences 119(12) e2107151119 [8] Roy and Oberkampf 2011 A comprehensive framework for verification, validation, and uncertainty quantification in scientific computing Computer Methods in Applied Mechanics and Engineering 200(25–28) 2131–2144 [9] Biggio and Roli 2018 Wild patterns: Ten years after the rise of adversarial machine learning Pattern Recognition 84317–331 [10] Liao and Varshney 2021 Human-centered explainable AI (XAI): From algorithms to user experiences arXiv:2110.10790 [11] Lundberg and Lee 2017 A unified approach to interpreting model predictions Advances in Neural Information Processing Systems 304765–4774 [12] Mothilal, Sharma and Tan 2020 Explaining machine learning classifiers through diverse counterfactual explanations Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency 607–617 [13] Adebayo, Gilmer, Muelly, Goodfellow, Hardt and Kim 2018 Sanity checks for saliency maps Advances in Neural Information Processing Systems 319505–9515 [14] Pearl 2009 Causal inference in statistics: An overview Statistics Surveys 396–146 [15] Liu, Kitouni, Nolte, Michaud, Tegmark and Williams 2022 Towards understanding grokking: An effective theory of representation learning Advances in Neural Information Processing Systems 3534651–34663 [16] Azizzadenesheli, Kovachki, Li, Liu-Schiaffini, Kossaifi and Anandkumar 2024 Neural operators for accelerating scientific simulations and design Nature Reviews Physics 6(5) 320–328 [17] Raissi, Perdikaris and Karniadakis 2019 Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations Journal of Computational Physics 378686–707 [18] Nemani, Biggio, Huan, Hu, Fink, Tran, Wang, Zhang and Hu 2023 Uncertainty quantification in machine learning for engineering design and health prognostics: A tutorial Mechanical Systems and Signal Processing 205, 110796 [19] Lu, Liu, Dong, Gu, Gama and Zhang 2018 Learning under concept drift: A review IEEE Transactions on Knowledge and Data Engineering 31(12) 2346–2363 [20] Dhanda, Rogers, Hall, Dekoninck and Dhokia 2025 Reviewing human-robot collaboration in manufacturing: Opportunities and challenges in the context of industry 5.0 Robotics and Computer-Integrated Manufacturing 93102937