AI object detection, holographic hybrid reality and haemodynamic response to construction site safety risks

Abstract

Construction practitioners make decisions about safety risks that can be subjective and prone to error. The trained computer object detection provides a standardised method to deal with this issue, but it typically relies on many photos to train one object, which is costly and time-consuming. This project proposes a new algorithm to train a computer to identify construction safety risks with fewer photos. In addition, holographic hybrid reality will be developed for safety training in the construction industry; Mercedes-Benz has used a similar approach to inform employees about collision zones. We will use the trained images to develop HoloLens hybrid reality to share construction site safety knowledge via wearable HoloLens for on-site safety risk detection. Lastly, although decision-making in various areas has been studied using neuroscience, how an individual’s brain makes decisions when different construction safety risks are perceived and the impact of holographic safety training on brain reaction and activities remains unknown. These issues will be studied via haemodynamic response and neuroimaging. In this research agenda, we plan to construct a photo library of 10,000 high-quality photos of various construction risks with different shading, size, and orientation by collecting Creative Commons construction photos and turning existing online Creative Commons videos into photos. To achieve this goal, we will input the online common creative photos into our image-based CAPTCHA system (similar to Google’s ReCaptcha). Each group will include 16 photos to allow construction practitioners to identify and click on those that include a safety risk. The identified photos with safety risks will be saved, and specific categories of safety risks will be uploaded to social media and sent via email. The trained figures will be deployed to the HoloLens in our laboratory. About 20 safety experts, including safety managers and trainers, will be invited to use the HoloLens for detecting hazards on-site and provide comments for improvement. Holographic hybrid reality will be built with Unity C# and HoloToolkit. The object detection results obtained will also be used so the research participants will see the real scene with not only hazards labelled by AI, but also some high-risk elements that cannot be included in ordinary safety training, such as open taps with water running into the ground and blasting. Four holographic hybrid reality training scenarios will be generated: a general construction site, and three specially designed scenarios for refurbishment, new building, and road construction settings. In the last stage, we will use functional near-infrared spectroscopy (fNIRS) to study construction practitioners’ brain responses when they see and identify various kinds of hazards. The first group will be exposed to holographic hybrid reality with some safety risks on the first day, and they will be asked on the tenth day to identify the safety risks. The control group will receive no safety training but will be asked to identify risks. All research participants will be monitored by fNIRS when they attempt to identify the safety risks. Haemodynamic response and neuroimaging tests will be used to study the effectiveness of the safety training.