[miibo Co., Ltd.] Announcing an “analysis function” that detects lies and mistakes in generated AI! Providing a group of functions that strongly support PDCA in AI development
*View in browser* *miibo*
Press release: January 30, 2024
**
Announcing an “analysis function” that detects lies and mistakes in generated AI! Providing a group of functions that strongly support PDCA in AI development
* Conversational AI construction platform miibo has announced the beta version of a function that allows AI to analyze the accuracy of AI responses and the degree to which problems are solved. We are now recruiting early users. *
miibo new feature “AI analysis function”
Summary of presentation
miibo has released a new feature (beta version) that can detect and improve “hallucinations” and misspoken words, which are major hurdles in introducing generation AI. Analyzes the responses made by AI and provides advice as well as “reliability” and “resolution
prediction.”
Please watch the video below to see it in action.
https://www.youtube.com/watch?v=byJh9y3YJEw
Conversational AI construction platform miibo
miibo, a conversational AI construction platform developed by miibo Co., Ltd. – Beginners who want to easily create AI first
– Developers who develop products that incorporate AI in earnest This is a conversational AI construction platform that can approach both of these aspects. miibo is a conversational AI construction platform that combines these two aspects and has a unique position both domestically and internationally.
[Example of use case of AI created with miibo]
・Customer support 24-hour automatic response AI
・AI for internal help desk response
・AI character development
・Development of conversational AI for celebrities and artists ・Development of AI assistant
・Development of AI writer
Conversational AI that can be used for a variety of purposes, including those listed above, is being created every day on miibo, and is being introduced by listed companies and local governments.
Official site:
https://miibo.jp
Reference material: [Use Case Magazine] Conversational AI created with miibo (note)
https://note.com/miibo/m/ma82aceda74a8
An approach to reduce the problem of generative AI “hallucination” The introduction of chatbots using LLM is also increasing. However, there are many cases where issues such as “hallucination” of generative AI become a bottleneck, and people hesitate to introduce it.
*“AI Hallucination” = A phenomenon in which AI outputs false information in a plausible form.
Therefore, miibo will release an “AI analysis” function that evaluates and analyzes the comments of the developed AI from an objective perspective.
Image of AI analysis function
Actual situation when using AI analysis function
AI analysis items
With the AI analysis function, you can obtain the following three items about AI statements.
1. Reliability
2. Solution prediction
3. Advice
* Degree of reliability*
Determine how reliable the statements made by AI are. With miibo, you can give specialized knowledge to AI in advance and build an AI that can have conversations about that area. It is calculated by comparing it with the expertise you have provided and analyzing whether you are making up unnecessary facts or saying something inaccurate. In other words, you can check if hallucination is occurring.
The following is a test in which the name of miibo’s official Twitter account was intentionally made incorrect. They pointed out errors in the Twitter account and suggested future improvements.
Analysis results for unreliable responses
* Solution prediction *
Users predict whether they were able to solve the problem by talking with AI. This will be used to judge not only the accuracy of the information, but also whether the AI was able to provide a service that was tailored to the user.
For example, in the example below, the AI suggests that although the information is reliable, it is not providing enough value to the user due to lack of information. It is possible to lead to improvements beyond hallucination countermeasures.
*Advice*
AI will analyze from an objective perspective how to improve the AI response.
Knowing the user’s solution status will lead to further improvements For example, in a customer support chatbot, it is important to analyze whether the user actually solved the problem.
Therefore, many chatbots have buttons such as “solved” and “helpful”. On the other hand, many users do not click “solved” or “not resolved”. If they are presented with a link to information about their question, they will quickly be redirected to another screen, and once it is resolved, they will not return. Therefore, even if the issue is resolved, only a few kind users will receive feedback.
The above-mentioned “resolution prediction” using AI analysis is useful for exactly that measurement.
By letting the AI’s responses predict how well the user was able to solve the problem, it is possible to gain insights that cannot be obtained from the user’s active feedback.
Regarding the accuracy of these predictions, we have obtained very high results in our experimental stage. With this beta release, we aim to receive feedback from more users and make improvements before releasing it to the public in the future.
Recruiting early users
Starting today, miibo is recruiting early users of the above “AI analysis function (beta version)”.
Users with miibo accounts that meet the following conditions can apply.
[Application conditions]
・Those who have a miibo account
・For corporations with Enterprise Plan/Premium Plan
・Corporations that can cooperate with A user interview after use ・Corporations that can cooperate with accuracy verification
We will provide this feature in advance to a few companies from among those who apply.
Depending on the application status, we may not be able to provide this to all corporations. Please note.
[Provided content]
・Open AI analysis function to miibo account for 1 month (no additional charge required)
【Application method】
If you are a corporation that would like to use this service, please apply using the form below.
https://forms.gle/NadbTLPTG459e2Yr5
*About details about this release*
https://prtimes.jp/main/html/rd/p/000000016.000118546.html
*Download press release materials*
https://prtimes.jp/im/action.php?run=html&page=releaseimage&company_id=118546&release_id=16