Master in NLP (TAL/自然语言处理) Interested in Chatbot, Dialogue system, Machine Learning, Semantic technology and User experience.View the profile
About the talk
The beauty of a conversational AI is the natural way it communicates, which leads to the challenge of handling infinite spontaneous responses in certain contexts. We will discuss the process of continuous improvement of the Conversational AI in production, especially the parts of analyzing user feedback, data training and the lessons learned.
Presented by Jielei Li, Cognitive Software Engineer at Orange at the 2021 Rasa Summit (https://rasa.com/summit/).
- Learn more about Rasa: [https://rasa.com](https://www.youtube.com/redirect?even...)
- Rasa documentation: [http://rasa.com/docs](https://www.youtube.com/redirect?even...)
- Join the Rasa Community: [https://forum.rasa.com](https://www.youtube.com/redirect?even...)
- Twitter: [https://twitter.com/Rasa_HQ](https://www.youtube.com/redirect?even...)
- Facebook: [https://www.facebook.com/RasaHQ](https://www.youtube.com/redirect?even...)
- Linkedin: [https://www.linkedin.com/company/rasa](https://www.youtube.com/redirect?even...)
#conversationalAI #cicd #aichatbot
Hello everyone, thank you for having me here. So, the red the Senate and getting kind of have such as an engineer for Django. The conversational AI of orange. 2019, jingle has been helping our customers with their online shopping answers, and questions about customer service. And a technical troubleshooting, with experiences from production. And from my colleagues, I'd like to talk about continuous Improvement today. At first, we will go through the reasons why this topic is
particularly important for transfers, then how the process works and our family. The lessons we have learned will be shared as well. So let's get started. Why? This topic is particularly interesting because I like other such as the beauty of the natural way it Community, which lead us to the challenge of handling infinite spontaneous responses in certain contacts. So it would be difficult to plan for every eventuality if you for delivering to rail users, especially when it comes to things like contact to switch
tasks, execution, due to the predictive nature of a dialogue. Truck beds, projects. Learn by doing. It must be continuously, updated mentioned and he's bad with real-time arrival at information to stay up-to-date. So, a continuous learning cycle. List craft gifts at the big picture of how the process works. First of all, we collect users back and then we realize it to Maverick the effectiveness of so that we can create and update training data to generate a new model after changes, a mate is ideal to verify that the
Virgin is performing while and it's me he's already spectations. In the test is step. We will analyze over new model Tessa dataset to make sure that there are no regressions evaluation and should also be based on actual feedback. So it's important to automate the entire workflow, using a CI CD pipeline using a version control system to manage models and perform elastic Antoinette casting in the next step. Pre-deployment testing, will be done development environment before delivering Aid to rail users.
And then after deployment, we will be back in the first two phase again. From here. We can monitor Matrix to further that recite already improvements and the ring you recycle. Here we can see how did the process can be achieved, and I'd like to join your particular attention to analyzing and Improvement. The difficulty is that we have a huge amount of feedback the process and is unrealistic to read all conversations. So is imperative to have our macrovision with ninjacators which allows us to supervise the overall performance of
chatbots and Claire they show us what we should pay our tithe in Auburn Improvement efforts. The indicators can be General such as task, completion or client satisfaction. From our experience is also very beneficial to add some indicators, for the specific needs of every project, We need this analyzed to make the necessary improvements as efficiently as possible so that we can identify conversations to analyze EG tail for the next stage. So, you stands in the case of Father, you already troubleshooting chat but we want to understand why some customers
have given up during the last box to reset. So we can track this particular conversations to investigate very experienced. It could be due to incorrect detect a shadow of intent and Auntie or the explanation from the tripod wasn't scary enough for users to understand After completing assessments and analyzes, we have identified the weak spots. The next action to improve is to update workspace, based on the results, after the assessments for funding, in the analyst face, Any intense and to see or telling response that
wasn't correct. The Fix-It version should be added to training when appropriate. I would like to share some lessons learned if our teams. First Ali reading, real conversation is critical to improving the word frequently. Analyzing real conversations in detail is Highway roof, Monday. Secondly, they are as so many indicators is Tom's to marrying the chest by the performance. However, each but is how do you mirror your chat about success should directly color rate with your body? Use case is important to Define
over men, can PS2 better Parrot Eyes, especially for the power rate information when the touch, but has many weeks spots. The next one. It's not only a continuous attack but also a collaborative one which requires different skills as developers linguist dumb and expert, ux designers product managers and do Bob's still need to work together in this process. Last but not least, to boost productivity. This saves time on collecting transforming and analyzing information and collaborative
Buy this talk
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.