Hololabo Nishimatsu Construction Develops drone work support using XR and 3D model superimposition technology on aerial footage
*View in browser* *Hololab*
Press release: April 18, 2024
**
Hololabo Nishimatsu Construction Develops drone work support using XR and 3D model superimposition technology on aerial footage
*-Fusing BIM/CIM and reality, verifying elemental technology for realizing an XR construction management platform-*
Hololab Co., Ltd. (Shinagawa-ku, Tokyo, Representative Director and CEO: Kaoru Nakamura, hereinafter referred to as “Hololab”) is Nishimatsu Construction Co., Ltd. (Minato-ku, Tokyo, Representative Director and CEO: Kaoru Takase)
As part of our joint research with Nobutoshi (hereinafter referred to as “Nishimatsu Construction”), we have developed technology that superimposes 3D models on aerial images taken by drones in real time, and works with a head-mounted display to visualize construction work and support drone operation. did.
At the construction stage, 3D models (CIM models) are being used to improve productivity and improve the efficiency of site management operations, and one way to utilize them is to use AR (Augmented Reality (Augmented Reality) technology is available, which can be expected to share images of the construction situation, identify issues early, propose improvement measures, and reduce construction rework. However, current AR technology is mainly used with
smartphones, tablets, head-mounted displays, etc., so it is limited to a subjective viewpoint from the ground, making it difficult to see the entire plan. The viewpoint was limited as I had to prepare a . Additionally, in order to check multiple locations, it was necessary to move around the site and align the AR at each location.
In order to solve these problems, we developed a 3D model
superimposition technology from a free viewpoint using a drone that can move at high speed without being affected by position or distance. (Joint patent pending)
Location information acquisition without the need for markers For AR superimposition, methods that use markers such as QR codes are common, but in this research, DJI Mavic 3 Enterprise
The RTK module is installed in the Global Navigation Satellite System (GNSS). By linking the position information provided by the Drone System) with the drone’s own attitude and camera information, real-time AR superimposition is performed without the need for markers. In addition, the latitude and longitude information of the design is used to place the 3D model, and it is possible to place it directly at the registered specified coordinates from a web browser.
Drone work support using XR devices
In order to solve the problem of complicating operations by performing these controls while piloting the drone, the pilot uses a transparent XR head-mounted display (Trimble
XR10 with HoloLens 2) has been installed, and we have developed a system that grasps various information and assists in operation without letting go of your hands.
On the spatial head-up display (HUD) projected on the XR10, in addition to the AR superimposed aerial footage, you can check the control UI for the 3D model that can be operated with hand tracking, shooting functions, and various statuses of the drone in flight. Masu. Also, based on location information, the drone’s flight position is displayed on a map projected with AR. There is no need to switch the camera direction to check, and you can get a bird’s-eye view of the drone’s position in the distance, which is difficult to grasp visually.
Trimble XR10 with HoloLens 2
DJI Mavic 3 Enterprise
Performance improvement and remote sharing system using PC as the core In order to avoid the processing load of AR rendering from affecting drone control and XR devices, these calculations are processed on high-performance PCs within the field network. By sharing images from a PC over the on-site network, high-resolution real-time rendering and communication speeds are achieved while reducing the burden on each device. By linking with a remote conference system, we have made it possible to share aerial footage with remote locations via the Internet.
System overview diagram
https://www.youtube.com/watch?v=ZLeY0Mizgxc
Conducting on-site verification
Field verification of this development was conducted at a dam construction site (Natori City, Miyagi Prefecture) currently being constructed by Nishimatsu Construction.
By superimposing a 3D model of the dam body and rock classification onto aerial footage taken by a drone, we confirmed that we could see the entire site from a free perspective and compare the current situation with the construction plan. In addition, the mobility of the drone allows it to move and check vast fields in a short time, leading to a significant reduction in on-site travel time compared to conventional AR technology.
The XR10 spatial head-up display (HUD) worn by the pilot displays superimposed images, internal information of the drone, and the location of the aircraft in flight on a map, so the pilot can It is possible to check multiple information at the same time, such as the aircraft status, flight position, and 3D model superimposition. Superimposed image of dam body model
Superimposed image of rock class classification model
On-site verification of development system
Future prospects
In this development verification, we were able to understand the situation in a wide area by superimposing a 3D model from a free viewpoint, and it was confirmed that time was saved. However, there were also differences in superimposition accuracy due to the difference between the transmission speed and rendering speed while the drone was moving. Now that we have confirmed this, we will continue to expand the functionality to further improve accuracy, optimize the system, and further improve operability and convenience, and proceed with development to develop it as a service that can be used universally.
About Hololab
Hololab conducts research, system/application development, and educational activities based on the themes of AR/VR technology, various three-dimensional spatial data such as BIM/CIM, point cloud/photogrammetry, and 3D city models, and the new experiences they create. We are a technology group. We will pioneer the cutting edge of an expanding field, disseminate it widely to the world, and enjoy it together.
Inquiries regarding this matter
Hololab Co., Ltd.
https://hololab.co.jp/#contact
Source: DJI official website https://www.dji.com/jp
Source: Nikon Trimble official website https://
www.nikon-trimble.co.jp/TrimbleXR10/
*About details about this release*
https://prtimes.jp/main/html/rd/p/000000096.000023638.html
Copyright © PR TIMES Corporation All Rights Reserved.