[TOPPAN Holdings Co., Ltd.] PKSHA Group and TOPPAN Edge jointly develop a new RAG service that improves response accuracy of generated AI

 With this service, when the generated AI refers to business manuals and administrative regulations held by companies, the original data is structured into a readable format that is easy for the AI ​​to read, and an algorithm optimized for the structured data is used. By reading it, you can improve the answer accuracy of the generated AI.

In developing this service, from May 7, 2024 to September 6, 2024, Shizuoka Bank, Ltd. (Head office: Shizuoka City, Shizuoka Prefecture, Representative Director and President: Minoru Yagi, hereinafter referred to as Shizuoka Bank) We conducted a demonstration experiment using the bank's internal regulations and manuals as materials.

In the demonstration experiment, Shizuoka Bank selected a total of 34 types of internal regulations and manuals as the source data to be verified, and created questions and model answers regarding the regulations. We used TOPPAN Edge's new generation AI dedicated tool to structure the original data into sentences optimized for PKSHA LLMS (*2), and combined the PKSHA Group's new character extraction/segmentation technology and search algorithm to create a database. . Obtained answers to questions asked by Shizuoka Bank employees to the “PKSHA AI Help Desk” that applied the database (total number of questions and answers to be verified: 261). Shizuoka Bank compared the answers with model answers and evaluated them by determining whether they were correct or incorrect.

As a result, the document accuracy rate (*3) improved from 87.3% to 95.4%, and the chunk accuracy rate (*4) increased from 63.2% to 84.6%. As a result, response accuracy has improved by approximately 20% compared to general RAG.

■ Background and purpose of service development

Currently, the use of generative AI is becoming commonplace in companies and organizations to improve productivity and operational efficiency. On the other hand, at RAG, which is indispensable for the advanced use of generative AI in internal operations, the generative AI is unable to process information correctly due to manuals created with the premise that humans can read them, resulting in incorrect answers and incorrect responses. nation (*5) is a factor in risk.

PKSHA Group and TOPPAN Edge will combine the following strengths to develop a new RAG service that significantly improves the response accuracy of generated AI, thereby further utilizing the intellectual assets of companies and organizations and improving the work efficiency of employees. We aim to realize this and provide an innovative employee experience.

■Features of this service

1 High-quality answers using “AI readable” of reference data and “algorithms optimized for structured data”

When using generative AI to improve business efficiency in a company, it is important to improve the accuracy of RAG responses by structuring the reference data and tuning the algorithm. Through the efforts of both companies, this service can be designed on both sides, making it possible to achieve higher response accuracy than when each process is designed independently.

2 Batch processing of large amounts of documents using proprietary tools developed by us

In the information society, the amount of documents such as manuals that are a company's intellectual assets is enormous. This service supports the batch structuring of such large amounts of documents by using a unique generation AI dedicated tool that specializes in AI readability.

In addition, TOPPAN Edge will give a presentation on the results of this demonstration experiment and an overview of the new jointly developed RAG service at the “PKSHA AI Summit for Workplace 2024” to be held in September.

“PKSHA AI Summit for Workplace 2024”

・Date and time: September 25, 2024 (Wednesday) 13:30-16:35 ・September 26 (Thursday) 13:30-16:15*2-day event

*TOPPAN Edge Lecture date and time: Thursday, September 26, 2024 15:35-15:45

・Holding format: Online

・Application: https://aisaas.pkshatech.com/wp-summit2024

・Lecture details

Title: Business innovation through interactive manuals ~The forefront of document structuring~

Speaker: Hiroaki Taka, General Manager, Business Innovation Division, TOPPAN Edge Co., Ltd.

Lecture summary: Hallucination, which is an issue when using manuals made for human reading as reference data.

To solve this problem, we are developing the RAG service jointly developed with the PKSHA Group and innovative business using it.

I would like to share some tips for your work experience.

*1 RAG: A framework that combines LLM (Large-Scale Language Model) text generation with the search for highly reliable external information to improve output accuracy, which is difficult to control with prompts alone. It is called “RAG'' by its acronym because it is capable of augmenting search (retrieval) functions and generating high-quality answers (generation).

*2: PKSHA LLMS: A solution released by PKSHA in March 2023 that provides an environment where multiple LLMs can be integrated and customized. (

*3 Document accuracy rate: The percentage of documents that were able to be found that contained the basis for the answer to the question.

*4 Chunk accuracy rate: The percentage of people who were able to search for chunks of text that included the basis for the answer to the question.

*5 Hallucination: Artificial intelligence (AI) responds with plausible false information, such as content that is different from the facts or irrelevant to the context.

■Comments from each company regarding this matter

・Representative director of PKSHA Workplace Co., Ltd.

 Comment from Tetsuya Sato, Executive Officer of PKSHA Technology Co., Ltd.

With the advent of generative AI, the traditional paper-based manual creation and reference work is undergoing a major transformation. Use cases for manual reference using RAG are already being trialled in various industries, but many remain at PoC (proof of concept). Under these circumstances, by combining TOPPAN Edge's “software-friendly” manual creation technology with our solution “PKSHA LLMS”, we expect to develop new solutions in the knowledge management field for companies. We will strive to create a new standard for work styles through the social implementation of “future software,” so please look forward to future developments.

・Comment from Toshiya Akiho, Executive Vice President and Executive Officer of TOPPAN Edge Co., Ltd.

By applying the document processing technology that TOPPAN Edge has cultivated through the production of manuals and administrative regulations for financial institutions, and data production for printed matter, we will create a future that is not only easy to read for humans, but also easy for generative AI to read. In addition to defining what manuals should be, the PKSHA Group will continue to develop high-quality services that stand out from previous generative AI services and contribute to the further social implementation of generative AI.

・Comment from Satoyasu Takizawa, Director and Managing Executive Officer, Shizuoka Bank, Ltd.

Shizuoka Bank introduced the “PKSHA AI Help Desk'' in July 2023 in order to improve the efficiency of responding to internal inquiries and promote the sharing and accumulation of knowledge related to internal administrative procedures.

Furthermore, starting in October 2023, we are trialling a “document-type chatbot” (automatically creating answers to inquiries about internal administrative procedures from manuals, etc. using generative AI) in some operations. Improving accuracy is an issue.

Therefore, through this demonstration experiment, we were able to confirm an effective method for improving response accuracy, which is an important first step toward solving the problem. We believe that it has the potential to become a solution for more efficient utilization of

In the future, we plan to expand the areas in which generative AI can be used while solving issues such as creating documents that are both easy to read by AI (manuals) and documents that are easy to read by humans.

■Company profile

・About PKSHA Workplace

Company name: PKSHA Technology Co., Ltd.

Address: Hongo Segawa Building 4F, 2-35-10 Hongo, Bunkyo-ku, Tokyo

Representative: Tetsuya Sato, Representative Director

URL: https://wp.pkshatech.com/

・About PKSHA Technology

Company name: PKSHA Technology Co., Ltd.

Address: Hongo Segawa Building 4F, 2-35-10 Hongo, Bunkyo-ku, Tokyo

Representative: Representative Director Katsuya Uenoyama

URL: https://www.pkshatech.com/

・About TOPPAN Edge

Company name: TOPPAN Edge Co., Ltd.

Head office location: 1-7-3 Higashi-Shinbashi, Minato-ku, Tokyo

Representative: Representative Director and President  Soeda Hideki

URL: https://www.edge.toppan.com/

*Product and service names mentioned in this news release are trademarks or registered trademarks of each company.

*The information contained in this news release is current as of the date of announcement. It is subject to change without prior notice.

Above  

 |   |