Fujitsu Limited
Developing technology to acquire 3D shape data of underwater creatures and structures using AI to realize ocean digital twins
Contribute to carbon neutrality and biodiversity conservation by supporting the planning of marine ecosystem conservation measures ……
As part of our research and development of ocean digital twins, which reproduce the state of the ocean with high precision in digital space and make it possible to predict changes in the environment that makes up the ocean and the effects of measures that utilize the ocean, through simulation, We have developed a technology that utilizes AI to obtain high-resolution 3D shape data of underwater creatures and structures using autonomous underwater vehicles (AUVs).
This technology utilizes AI to sharpen images, allowing it to identify objects and measure their shapes even in murky waters, and to provide stability from autonomous unmanned underwater vehicles even in waves and currents. It consists of real-time measurement technology that enables measurements. This technology makes it possible to visualize the status of target organisms and structures and estimate their volume, etc. during ocean surveys aimed at carbon neutrality and biodiversity conservation. Regarding these technologies, we conducted demonstration experiments in the sea near Ishigaki Island, Okinawa Prefecture, together with the Maritime Technology and Safety Research Institute (Note 1) (hereinafter referred to as the Maritime Technology and Safety Research Institute) of the National Maritime, Port and Aeronautical Research Institute, and investigated the effects of coral reefs. We succeeded in obtaining precise 3D shape data and confirmed the effectiveness of the technology.
Going forward, we aim to expand the measurement targets of the technology we have established to include seaweed, which absorbs a large amount of blue carbon (Note 2), and aim to establish an ocean digital twin of seaweed beds by the end of 2026. This will support companies and local governments in planning measures such as estimating carbon stored in seaweed beds, measures to conserve and create seaweed beds, and measures to conserve biodiversity in coral reefs, and will promote Sustainability Transformation (SX). ). The company plans to present detailed results of the demonstration experiment at the 2024 Spring Conference of the Japanese Society of Fisheries Science, which will be held from Wednesday, March 27, 2024 to Saturday, March 30, 2024.
[Image 1: https://prtimes.jp/i/93942/277/resize/d93942-277-cb63b2dad621d67ba6c8-0.png&s3=93942-277-9653c37ceeebf857c9c800fc6c1561ad-980×620.png ]
Figure 1 Concept of marine digital twin to realize SX
【background】
Our company has set the resolution of global environmental problems as one of the materialities for sustainable growth, and is working on climate change (carbon neutrality) and coexistence with nature (preservation of biodiversity). The ocean accounts for approximately 70% of the earth’s surface area and has a major impact on climate change.In order to support the formulation of measures such as carbon neutrality and biodiversity conservation in this ocean, we We are working on the development of an ocean digital twin that enables advance verification of ocean-related policies. The marine digital twin concept uses data such as vegetation distribution and
three-dimensional shapes of underwater organisms such as seaweed and coral reefs, as well as structures that affect marine resources and the marine environment, to be collected using autonomous unmanned underwater vehicles and satellites. We will use this data to collect digital data, build a model that numerically simulates changes in the environment and growth of organisms that make up the ocean, and use it for preliminary verification of ocean-related measures. For example, in order to pre-verify measures related to blue carbon, in addition to estimating the volume and plant species from three-dimensional shape data of sea creatures and calculating the amount of carbon dioxide absorbed, biological knowledge can also be incorporated to improve the growth of sea creatures. Perform a time series simulation. This makes it possible to simulate in digital space the changes in carbon dioxide absorption when a certain marine ecosystem conservation measure is taken, including the growth status of marine organisms, and to verify the effectiveness of global warming countermeasures in advance. We aim to
In order to preserve the marine ecosystem and understand the amount of carbon dioxide absorbed, it is necessary to acquire high-resolution 3D shape data with a resolution of several centimeters (Note 3), and identify and estimate the volume of underwater organisms. However, with existing technologies such as acoustic sonar, problems such as beam width limitations have resulted in coarse data with a resolution of about 10 cm.
[About the development technology]
In order to solve the above issues, we have developed two technologies that make it possible to obtain high-resolution 3D shape data of underwater organisms and structures, even in the difficult environment unique to the ocean where there are effects such as turbidity, waves, and ocean currents. Developed.
1. Image sharpening AI technology that restores the color and outline of underwater objects
Image sharpening AI that performs deep learning optimized for underwater subjects to create three-dimensional images of living things and structures with high resolution, even in images with blurred outlines and degraded colors due to images taken in murky waters. We have developed the technology. This technology consists of two AIs that remove turbidity and restore contours. It restores the original colors of the subject, generates an image with improved blurred contours, and then converts it into 3D. This prevents errors during 3D processing and object recognition, and makes it possible to measure the shape of each object.
[Image 2: https://prtimes.jp/i/93942/277/resize/d93942-277-3652606408f05e46cabb-1.png&s3=93942-277-1b21ee8d1432a7bd82e0c24aa936a90d-980×560.png ]
Figure 2 Precise 3D shape data of coral reefs using image sharpening AI technology
2. Undersea 3D measurement technology that enables real-time 3D measurement from a moving autonomous unmanned underwater vehicle High-speed sampling technology using short-period laser emission and high-speed scanning, cultivated through the Judging Support System, which supports gymnastics judgment, developed jointly with the International Gymnastics Federation (Note 4) to perform 3D
measurements underwater in real time. has been adopted. Furthermore, we have introduced underwater LiDAR (Light Detection And Ranging, Note 5), which can select the wavelength suitable for measurement depending on sea conditions from among three laser wavelengths. This will not only enable 3D measurements from a moving autonomous unmanned underwater vehicle, but it is also expected that by developing technology to track the movement of objects, it will also be possible to measure moving objects.
[Image 3: https://prtimes.jp/i/93942/277/resize/d93942-277-0b3a613a4ecab3046a8d-2.png&s3=93942-277-7d9f93705122d5ee9e57c035421868e1-980×320.png ]
Figure 3 3D measurement results using underwater LiDAR
In order to confirm the effectiveness of this technology, we worked with the Maritime Technology and Safety Research Institute to integrate cameras and LiDAR, which performs 3D measurement in real time, into the AUV-ASV coupling system (Note 6) being developed by the research institute. Equipped with a new underwater fusion sensor, we conducted a demonstration experiment (Note 7) to automatically acquire underwater data in real time. From this experiment, we succeeded in obtaining centimeter-order high-resolution three-dimensional shape data of underwater piping and coral reefs in real time.
[Image 4: https://prtimes.jp/i/93942/277/resize/d93942-277-ae37d6ea8d4e9b21d302-3.png&s3=93942-277-266567528bbcc7491c7675d92cfa83dc-980×560.png ]
Figure 4 AUV-ASV coupling system and data acquisition scene
[Future outlook]
In the future, we will continue to develop technologies that can stably acquire data even in various environments such as strong currents and uneven seabed topography, and we will continue to develop technologies that can stably acquire data in various environments such as strong currents and uneven seabed topography. We will expand the scope of measurement to include inspections of wind power generation equipment and accumulate use cases. We will also proceed with the development of an ocean digital twin that will perform simulations that incorporate knowledge from biology and environmental studies based on the measured 3D shape data. We will also build partnerships with companies, local governments, and organizations that are working on measures toward carbon neutrality, and aim to support customers in formulating measures using ocean digital twins.
[About trademark]
Proper nouns such as product names listed are trademarks or registered trademarks of each company.
[Note]
Note 1
Maritime Technology and Safety Research Institute, National Institute of Maritime, Port and Aviation Technology: Location: Mitaka City, Tokyo; Director: Kenmasa Minemoto
Note 2
Blue carbon: Carbon that is captured and stored in coastal and marine ecosystems.
Note 3
Resolution: The ability of a measuring instrument to identify physical quantities. It is expressed as the minimum distance between two distinguishable points among the measurement targets.
Note 4
International Gymnastics Federation: Headquarters Lausanne,
Switzerland, President Morinari Watanabe
Note 5
LiDAR: A technology that measures the distance to an object, its shape, etc. by irradiating an object with a laser and capturing the reflected light with an optical sensor.
Note 6
AUV-ASV connection system: Images taken by an autonomous underwater vehicle (AUV) underwater are sent via wired cable to an autonomous surface vehicle (ASV) at sea. An underwater survey system developed by the Maritime Technology and Safety Research Institute that transmits information to ships via Wi-Fi.
Note 7
Demonstration experiment: Conducted in the sea near Ishigaki Island, Okinawa Prefecture, between January 22, 2024 (Monday) and January 25, 2024 (Thursday).
【Related Links】
・International Gymnastics Federation begins using Fujitsu’s Judging Support System in all 10 events (Press release October 5, 2023) (https://pr.fujitsu.com/jp/news/2023/10/5-1.html)
・Fujitsu Group’s materiality
(https://www.fujitsu.com/jp/about/csr/materiality/)
[About our contribution to SDGs]
[Image 5: https://prtimes.jp/i/93942/277/resize/d93942-277-988fa62a8407a7964942-4.png&s3=93942-277-a7e1887ee008d4d05978a3400c2369bf-196×112.png ]
The Sustainable Development Goals (SDGs) adopted by the United Nations in 2015 are common goals that the entire world should achieve by 2030. Our purpose (raison d’être), “To bring trust to society and make the world more sustainable through innovation,” is our promise to contribute to the SDGs.
[Image 6: https://prtimes.jp/i/93942/277/resize/d93942-277-69d44370232aeedf9c77-5.png&s3=93942-277-ea4a7aae6b4525102f5a633cebb2893e-387×184.png ]
Main SDGs that this project aims to contribute to
[Inquiries regarding this matter]
Fujitsu Contact Line (General Counter)
0120-933-200 (toll free)
Reception hours: 9:00 to 12:00 and 13:00 to 17:30 (excluding Saturdays, Sundays, holidays, and Fujitsu designated holidays) Contact form (https://contactline.jp.fujitsu.com/customform/csque04802/873532/) Product prices, specifications, service details, etc. stated in the press release are as of the date of announcement. It is subject to change without prior notice. Please note.
More details about this release:
https://prtimes.jp/main/html/rd/p/000000277.000093942.html