About the talk
Ever since launch, four years ago, the Autodesk Virtual Agent has been one of the cornerstones of our support strategy. However, as customer demands have evolved and internal stakeholder interest has grown, we have seen our original scope expand. After the initial novelty of the solution wears off, the continued adoption of a conversational AI solution depends on continuous improvement. I will share our approach to building a platform that prioritizes learning from customer input and supports personalized problem solving. I will discuss the evolution of our architecture over the last year, the technical decisions we made and the creation of a platform powered by natural language processing.
Presented by Technical Product Manager, Data Science at Autodesk Nikhil Mane at the 2021 Rasa Summit https://rasa.com/summit/.
Nikhil Mane is a Technical Product Manager at Autodesk leading a team tasked with creating a seamless blend of digital and human support across all Autodesk help channels. His work focuses on using Natural Language Processing and Conversational Interfaces to reduce customer effort while scaling support. Nikhil has a Master's degree from UC Berkeley's School of Information. In his spare time, Nikhil labels data.
- Learn more about Rasa: [https://rasa.com](https://www.youtube.com/redirect?even...)
- Rasa documentation: [http://rasa.com/docs](https://www.youtube.com/redirect?even...)
- Join the Rasa Community: [https://forum.rasa.com](https://www.youtube.com/redirect?even...)
- Twitter: [https://twitter.com/Rasa_HQ](https://www.youtube.com/redirect?even...)
- Facebook: [https://www.facebook.com/RasaHQ](https://www.youtube.com/redirect?even...)
- Linkedin: [https://www.linkedin.com/company/rasa](https://www.youtube.com/redirect?even...)
#conversationalai #aichatbot #nlp
Okay. So I hope everyone can see that I slide which should be there at the beginning of my presentation. I'm so I will be talking about the like, how we used to. But I also wanted to lay out a bit of our journey, just, I guess they're not that there's anything wrong with that but to actually keep the blue jeans into a platform. That's powered by natural language processing. My name is nikhil, I work at our desk as a technical product manager. I am part of the science team and
this is kind of the problem landscape that we saw in 2017-2018 a lot of cases. As you can see, the resolution times weren't great for our customers agents themselves, had to access a lot of systems to help resolve cases. And there was a cost of busy associated with it. So as you can see, customers for losing time, duplicate cases for being created and agent for being swamped. So, that's the problem that you're trying to address. But at the time I mean these are the
alternative solutions that existed. So you had decision trees which are good for issue triaging but the structure of a decision tree is often representative of the organization structure and that's not something that customers should be available. Then you have helped content which covers a lot of different problems and it's often like the go to solution but it's also dependent on search for Discovery. You have a human health and human health is nothing like the last line of defense that will get you the right answer, but it can cost in terms of time effort, And you also have
like we also have four rooms or Community where other users can help each other, but that is worth needed to let grow a community that will fit into the right Light phone. So specifically, I think, I wanted to talk a bit about the decision tree, I found this excellent diagram online, about on an abstract level, what any, any decision he looks like. I'm so you haven't decided by customer and your end goal is to end with a satisfied customer off and it goes through one of these situations. And as you can imagine not
every customer who visits a decision, tree ends up being satisfied. So the question is like on an average are you ending up with satisfied customers or would be satisfied customers? I think one, one specific example to highlight another issue. So this is Robinhood support and I don't know if everyone's aware of what is going on with the US markets by Robin Hood, has been under some scrutiny and the restrictions option into danger was not something that they always had. So this is I need to react to any increases
or spikes in volume and that's not sustainable, right? Currently let's say, you have ten different options in the drop-down. You can keep on adding a new auction every time a new issue, so I think is some of the limitations with the existing Health Solutions available. So what did we come up with a solution for sports agent? And that's the product I work on is a customer service option. That provides a blend of digital in human support across different channels and
provide instant access to products and services and skill support itself. So what does it look like? This is what we called classic TV show. The Classic is basically a monocle for the architecture. Is not something that I need to know. But it was a single webpage, a standalone application. We were focused on providing easy access to automated transactional use pieces and those are the ones that you can see highlighted which you could download. You could get an activation code. All of these would essentially
improvements and efficiencies diaper service to customers. Not to the chatbot recognition capabilities and decision-making based on dark and the ability to have a bit of a conversation with our customers. All of these. As you can see, we're both using IBM Watson or two custom engineering work. We also had a beetle used in to provide voice and video support. It's not something that we invest our effort in now, but that was another thing that he had tried
with the hope of trying to solve that problem all So you heard an encouraging start and I have some ethics from 2018 that you can look at. I think anyone who looks like this child would have a question about that and that's because we have like seasonal launches and he paid me. Is called the global launch. Glad you can expect that are more customers coming to us either for downloads are experiencing issues with those. And then we had some excellent Quality Meat from 2018 worth. Someone had a good experience with Ava,
but they were visible to other people. So I wanted to highlight some of the research that was carried out and then I'll talk a bit about how we move from this architecture and solution to what we have right now. So specific to watch Allegiance. These are the three main points. I think it should be shared later so you can go through them in debt but the main points that stood out to support the need for being context of our customers rather than they are already
facing issues, providing Solutions, and not just instructions and a combination of like the avatara and the track which was leading to a positive experience. The reason I talked earlier about decision tree was some research that he had also been comparing both of those approaches. And again, I think there are a lot of different points mentioned. The one thing that stood out to us, was that a conversational interface said yesterday, more friendly, comfortable engagement, and the power of context entry
was a real transforming efficient way to a specific questions and answer. They the same team we were uniquely poised to take better advantage Adidas in Goldsboro aligned with some customers record goals within the company, specifically focused around these areas and as it seemed like, I mentioned, our goal was also to allow customers to ask a question in their own words. We consider that to be natural for customers wanted to improve efficiency within
using a language processing and see if you wanted to increase understanding of customer and turned from the collected data. So then we come to the video of the application, which is what you'd be right now. If you weren't able to go to the store, so the customer said that experiencing issues. We wanted to build a platform that powered by a set of machine learning capabilities that are run on natural language, processing algorithms. And if he has and we wanted to create the platform that would be.
So I want to make the capabilities available in different help in two pieces across. Belasco. I mean, it's not directly related to the topic at hand, but it's an important one and that is I think related to learning from salary increase of durability in Orchard board and ultimately in the engineering infrastructure letter on, check So as a high-level plan to scale recognition, using multiple data, science models, or models that we ourselves as in the design team had created. But there's also informed that is done outside of Autodesk. And
in the general likes, our conversation from application of the station and Morgan, engineering, a specific that was important for us to bring in a level of flexibility to our existing Brahma condensation point of view. I wanted to continue supporting customers, but we also wanted to reach them on different devices and different environment. So customers were expected to come to Eva, they would have to visit the webpage specific. You wanted to create Bridget and Modular interactions. That other destinations at Autodesk could be
an intraocular lens on a technical level. We wanted to build for use by other digital Health. In this map is this is just an example of some of the things should be dead. The key points, your word to try and focus on possibility. So that's why we ended up using the yard because there was a focus internally on Rising sentence using and building for reusable. I already spoke a bit about separating out the application orchestration from conversation. So this was really important for us because I died point like I mentioned, he had a single monolithic. What face and he wanted to
study that out. So we could start multiple uses more efficiently. The expanding presence thing is I think more specific to Autodesk but we wanted to support existing use cases that for example, to download, you said I mentioned earlier, but in a new destinations like the address to account where customers were already experiencing certain issues with download and for the conversations that we were having birthday, engineering infrastructure that supported us one who season.
This is the platform that we ended up building and on a high-level, the capabilities are already mentioned to you. Along with behavioural data was used, as the next language in Port. Arthur station layer is very aggravated and then we created player, which essentially serves as a business logic engine and a high-level. These are the four types of solutions that a customer can get. They could get a cell service. Do I spell it using a reusable company known as the
contact app? I think this is I guess like a credibility slide. This is an extremely detailed Architects. Golf course it would be easier for us to build an orchestration layer that Services different models without the end developer knowing all the details about how they are implemented or how they should be called. The other thing to mention is using snow plow for our event instrumentation, and we've collected a large amount of data on the different behaviors that customers to power of teacher store. So we can provide
This is the architecture that we are looking right now. It is, I would say this is not something that we exposed to all of our customers. And you're taking a phased approach to that mostly, because we have like this existing architecture which already works. And we want to make sure that we shift slowly, so that we don't break up anything and it's not visible to customers. I don't know if you are a bit like I mentioned he wanted to encourage customers to Dallas more and be more descriptive. So he
expanded the bar into an input box, but we also wanted to make sure that transaction use cases are still available to customers. So we made those available as quickly and clicking on one of those can directly get there in two. We wanted to make sure that customers are eating agency under different solutions that are provided and also that they were able to escalate at any point. So what we do is be aggregated, set of solutions and that's why the lyrics called the next best action to provide a set of actions that we think are the best solution.
For example, of downloading is the best solution according to it, and it's also about that can immediately get the customer to a solution and solution or other support option. So how are you doing on the goals that we had set out in terms of customer effort reduction? This is a screenshot of the artist account and there's a specific URL Ender Dragon. If you are having issues with downloading a product, you can click on it and the account assistant which is another I guess, white label
version of The Avengers shows up. So that means she has contributed to about 5% Improvement in the score, a factor in that was also localization. It sells and collaboration with the accounting but it also like to benefiting you in work from home users. And this is I mean something that came up due to the covid pandemic and we needed for taxes to home, use licenses and we were able to do that with a more technical architecture. In terms of customer record production and also I think
additional benefits at the architecture and like using Raza has provided is that we were able to put together a different interface almost so we created a wizard and this is where like a reactant and comes in and the fact that we now decouple the orchestration with the conversation itself, to be able to create some improvements in an existing workflow by automating that about yourself. In terms of results on the PlayStation front, we've seen some positive impact specifically in
education support. So the blue line is Bad Evil was added. And as you can see, it's pending download. This was late last year. And it's something that I own at that time. The bahadur traffic is Spider-Man going on and leave. Now, expanded to include to allow more traffic to model, are you? So in terms of quality metrics, we also have a few models that died. We monitor our stories and photos to see if there are mentions or feedback related to how he was doing this, not only helps us understand
if. So, that is another baby are currently. Using this architecture to better understand what our customers and all of that stuff you have with the customers. So I mean I won't go to the slide but as you can imagine it sanitary to process we had on the new interface so that are certain things that have opened up after like we move to a more flexible class. A driver architecture is a conversational advertising or demand-generation use paste and this is something that
we're really excited about the opportunity to provide a more personalized conversations and product discovery. The reasons I think to use. Our kind of listed or do you want to focus on Celebrity maintainability? An observable tea? Using that allows us as an engineering team, a lot more control over the infrastructure that allows, for example, like the ICD and dad being kind of Defense in terms of new use cases. Looking at the mission is a mansion but we are also looking at in protocol and other improvements to the agent portal,
focus on virtualization and data-driven experimentation, and use natural language processing for basically powering their platform, so that we can continue to help our customers better. So, yep, that's it for me.
Buy this talk
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.