AI mask generation system for professional use “MaskPilot”
“AI” was jointly developed by stu and Annex Digital Japan. Started use in masking work for the special effects TV drama “Kamen Rider Series”.
*View in browser* *stu Co., Ltd.*
Press release: May 13, 2024
**
AI mask generation system for professional use “MaskPilot”
“AI” was jointly developed by stu and Annex Digital Japan. Started use in masking work for the special effects TV drama “Kamen Rider Series”.
stu Co., Ltd. (Headquarters: Shibuya-ku, Tokyo, CEO: Takayasu Kuroda) is collaborating with Annex Digital Japan Co., Ltd. (Headquarters: Minato-ku, Tokyo, CEO: Teppei Wakabayashi) to develop the AI mask generation system “MaskPilot”.
We have developed AI. After verifying the effectiveness of this system through various use cases, we will begin its actual use in masking work for the special effects TV drama “Kamen Rider Series.”
((C)2024 Ishimori Pro, ADK EM, Bandai, Toei Video, Toei)
In recent years, advances in AI have led to greater efficiency and innovation in various industries. In this trend, stu aims to DX the production process of video production using AI and provide new experiences.
As part of this, in November 2023, we will closely examine video production workflows and launch the AI mask generation technology “Scene” that meets the needs of video production sites.
We have developed “Tracking Mask”.
Related link: stu develops automatic mask generation technology “Scene Tracking Mask” using AI
Providing technology for short film “Tonomura” starring Atsuko Maeda and Asuka Kudo | Press release from stu Inc.
This time, Annex Digital Japan Co., Ltd., which is responsible for many masking operations in the Japanese video production industry, has joined this project, and we are pleased to announce that we will be working on a new project for professional use.
We jointly developed the AI mask generation system “MaskPilot AI.” * ■Challenges in recent AI mask technology and “MaskPilot AI”*
The biggest issue with the AI mask technology that came up during joint development with Annex Digital Japan, which has extensive experience in masking operations, was data handling. Many of the powerful AI tools that have emerged in recent years operate on servers with ambiguous or unspecified data handling policies, making it difficult for many to handle unpublished data or data with highly confidential intellectual property rights. There are concerns. For this reason, AI technology could not be fully utilized for masking in the field.
Taking these circumstances into account, we created “MaskPilot “AI” was developed with the aim of creating a system comparable to recent advanced AI tools while ensuring the safety of data usage. * ▽Ensuring safety in data usage by supporting on-premise server* “MaskPilot
AI’s system was built to run on a local on-premises server (*). We will make maximum use of AI while ensuring safety in data usage. *Refers to a local server that does not use cloud services
* ▽Automatically generate rough masks with intuitive operations* “MaskPilot
AI’ uses intuitive UI operations to select objects to be masked and automatically generates a rough mask. By combining multiple object selection functions and automatic tracking of selected objects, we aimed to significantly reduce the effort required for rough masking. ((C)2024 Ishimori Pro, ADK EM, Bandai, Toei Video, Toei)
In order to verify the effectiveness of this system, we compared the total work time for manual work, using this system, and using other AI tools in various cases. As a result of comparative verification, compared to masking that is done entirely by hand, *
The total work time was reduced by an average of 40% when using this system and by 38% when using other AI tools. *
Despite being an on-premises server configuration, its effectiveness was demonstrated by the fact that it achieved results comparable to existing powerful AI tools.
* ■Future outlook for “MaskPilot AI”*
“MaskPilot AI” whose effectiveness has been confirmed through verification ‘ will now enter the operational phase. The system will be introduced for masking work in the special effects TV drama “Kamen Rider Series,” further improving the system’s ease of use and rough mask generation performance. In addition, although we are currently only automatically generating rough masks, in the future we will continue to work with stu and Annex Digital Japan, with an eye to automatically generating high-definition masks that could only be achieved manually.
We will continue joint development.
* ■stu’s AI/machine learning research and development*
At stu, we are working on research and development of AI and machine learning in the entertainment field, aiming to DX the production process of video production and provide new experiences using AI.
At stu, we are constantly exploring new possibilities with AI, and will continue to actively provide technology to companies and various projects. We are looking for entertainment companies and local governments who can cooperate with full-scale development and demonstration experiments. Please feel free to contact us.
Some examples of past AI utilization
* ・Proof experiment of “3D Gaussian Splatting” that easily implements 3D space construction * 3D Gaussian that makes digital twin easy We have started demonstration experiments on Splatting, and by combining it with WebGL, it can be easily used on web browsers, etc. * ・Generation AI workflow efficiency system “SyncFlow AI” *
Developed “SyncFlow AI”, a system that uses generative AI to improve the efficiency of video production and expand the range of expression, and provided video to the “Hundred Year Art Festival – Uchiboso Art Festival”
* ・Automatic mask generation/automatic scene detection technology “Scene Tracking Mask” *
“Scene Tracking” is a system that has automatic mask generation technology and automatic scene detection technology for captured videos.
Provided technology to filmmaker Kensaku Kakimoto’s short film “Tonomura” using “Mask”
* ・Real-time participation virtual live system “LAIV” *
Developed “LAIV”, a participatory virtual system that changes real-time production using generated AI based on participants’ comments, providing a new VR live experience that enables interactive production between artists and participants.
■stu Co., Ltd.
With the mission of “reinventing entertainment,” stu is centered on a creative team that creates borderless ideas, and is engaged in edge technology business in the 5G, We are a company that creates a new era of entertainment by combining content development businesses that incorporate workflow.
・Inquiries: contact@stu.inc
・Official website: https://stu.inc/
・Official X (old Twitter): https://twitter.com/stuinc_
■* Annex Digital Japan Co., Ltd.*
Annex Digital Japan performs a variety of video-related operations, including live-action video production, CG production, digital signage, and XR content production.
We are also a unique video production company that conducts planning and sales at our Tokyo head office, and actual production at our own subsidiary studios in China, Cambodia, and Poland, as well as studios around the world.
・Inquiries: contact@annex-digital.com
・Official website: https://annex-digital.com/
*About details about this release*
https://prtimes.jp/main/html/rd/p/000000033.000047133.html
*Download press release materials*
https://prtimes.jp/im/action.php?run=html&page=releaseimage&company_id=47133&release_id=33
Unsubscribe HTML email
Copyright © PR TIMES Corporation All Rights Reserved.