Artificial Intelligence Assisted Infrastructure Assessment using Mixed Reality Systems

Abstract

Conventional methods for visual assessment of civil infrastructures have certain limitations, such as subjectivity of the collected data, long inspection time, and high cost of labor. Although some new technologies (i.e., robotic techniques) that are currently in practice can collect objective, quantified data, the inspector’s own expertise is still critical in many instances because these technologies are not designed to work interactively with a human inspector. This study aims to create a smart, human-centered method that offers significant contributions to infrastructure inspection, maintenance, management practice, and safety for the bridge owners. By developing a smart mixed reality (MR) framework, which can be integrated into a wearable holographic headset device, a bridge inspector, for example, can automatically analyze a certain defect such as a crack that he or she sees on an element, and display its dimension information in real-time along with the condition state. Such systems can potentially decrease the time and cost of infrastructure inspections by accelerating essential tasks of the inspector such as defect measurement, condition assessment, and data processing to management systems. The human-centered artificial intelligence (AI) will help the inspector collect more quantified and objective data while incorporating the inspector’s professional judgment. This study explains in detail the described system and related methodologies of implementing attention guided semisupervised deep learning into MR technology, which interacts with the human inspector during assessment. Thereby, the inspector and the AI will collaborate/communicate for improved visual inspection.