I am a tech lead and a principal software engineer at Oracle with around 15 years of experience in ecommerce platforms, search engines, distributed systems, artificial intelligence, and machine learning. NOTE: The views expressed in my posts are my own and do not represent those of my employer.View the profile
About the talk
In this session, Uday discusses how we can leverage Rasa to supercharge the command-line and graphical user interfaces. He will walk you through the design and sample implementation of these user interfaces with Rasa and Conversational AI at the core. By the end of the talk, maybe I will convince you (and myself) to call these "Instructional Dialog Interfaces".
Presented by Oracle/InfoQ Technologist, ML Practitioner, and Tech Lead Uday Tatiraju at the 2021 Rasa Summit (https://rasa.com/summit/).
- Learn more about Rasa: [https://rasa.com](https://www.youtube.com/redirect?even...)
- Rasa documentation: [http://rasa.com/docs](https://www.youtube.com/redirect?even...)
- Join the Rasa Community: [https://forum.rasa.com](https://www.youtube.com/redirect?even...)
- Twitter: [https://twitter.com/Rasa_HQ](https://www.youtube.com/redirect?even...)
- Facebook: [https://www.facebook.com/RasaHQ](https://www.youtube.com/redirect?even...)
- Linkedin: [https://www.linkedin.com/company/rasa](https://www.youtube.com/redirect?even...)
#conversationalai #aichatbot #nlp
Hey, y'all. My name is today. Raju. I am a tech lead and a principal software engineer at Oracle. My background and experience is an e-commerce platforms. Search engines information, retrieval machine learning and distributed systems. In addition, I am part of the AI advisory council at CompTIA computer stands for computing. Technology industry associations. So at County Route together with my fellow council members, I have strategize and Bill Technical Resources and guidelines related to Healing enterprise-grade. Artificial intelligence and machine learning Solutions.
refresher on conversational AI. I'm then going to talk about a brand-new read of user interfaces are in a rather supercharging existing user interface designs. I asked myself the question. What if we used Superchargers the well-known command line and graphical user interfaces with conversational AI? How would they look like? What would be the use of doing this? So I'm going to talk about these things and I will do a walk-through of building these supercharged interfaces using water as most of, you know what, size? A machine-learning based platform that allows us
to build conversational, AI systems and sewing the stock. You'll also see how one can integrate rasser with traditional sea, lies and broccoli and faces. And in the process of hopefully provide an enhanced user experience. So what is conversational AI? Let's take a look at some examples here and we are familiar with Alexa Siri and OK, Google. Write in fact, it is pretty evident that these digital voice assistants are becoming ubiquitous in households these days and interactions with these.
Assistants are examples of a i driven conversation. You may also be familiar with the things, like FAQ Bots. What I tend to call in football and other kinds of chatbots. Like, for instance, covid-19, symptom, checker, these are also examples of hydrogen conversations refers to the use of artificial intelligence more. So natural language processing with machine learning and the ability of the assistance, to naturally engage in conversations, naturally, engage and dye Locs with humans. And this could be done via voice.
Or 2x are could be a combination of both boys and texting faces? Now, A Brief History year of the first generation of assistance to Chad vought's were mostly rule-based. When is a first-generation to use a chat box were built in 04 years ago at least a couple of years ago and since these were rule-based they were kind of brutal. And as a result were unable to hold initial conversations with users which as we know can tend to be very messy and so it's no surprise that some of these chatbots fail to meet user
expectations. But advancements in natural language processing, the current the second generation of the assistants are much more capable in terms of understanding the nuances of natural-language, the context of the conversation and the ability to naturally, talk to humans and also perform actions on their behalf. So it's no surprise that these 2nd Gen Con. REI systems are being able to provide better customer support for sales or services to provide better citizen experience for instance, providing
factual an authoritative information regarding covid-19, the unfortunate and a macaron So now let's briefly, look at the underpinnings of conversation with a guy named Lee in a V and machine learning. When I think of natural language processing, I think of it as a field of linguistics computer science and artificial intelligence. It do used to be processing the extraction and it was illegal to be in Madison text. It could be a structured data as well, but basically all these things in a given human spoken language like English
or Samsung, And no doubt it's a it's an enormous cute things like speech recognition, speech synthesis machine, translation, natural language understanding, and language generation. These are all the various subdomain. So if and I repeat, but then they are two subsets under and repeat that. Keep coming up. When we talked about conversational AI his interview and an LG Natural language understanding and then you it deals with the system's reading comprehension abilities aspects. Like for instance, tax classification, you do for automatic analysis, self Nichols
blogs, news pieces and also sentiment analysis of the data. Extracting key pieces of information from both structured and unstructured data. Like, these are all things that fall under the interview. And as far as Energy natural language, generation is concerned deals with the ability of a system to Auto. Generate text based on context that topic or conversation an LG is still in its infancy. And I think we will continue to see more cutting-edge research in this field for sure. Amy. So all these latest
advancements in MLP and machine learning and deep neural networks in general. These are all making it possible for us to go better conversation. We had systems advancements. Write these include things. Like for instance, moving from the contacts free War, II rack and Club. Where do Marines do much more advanced, bidirectional contextual models models, the introduction of attention, based neural networks Transformer, base language models, the concept of transfer learning, and all of these are the advancements that I'm referring to. A quick
note on bird and GPT bidirectional and coder representation from Transformers part of, this is from Google has released in 2018. It is a Transformer based learning. It's also a pre-trained language model on a large car place. And it can also be fine to non-specific interviewed, aspirin since like, 5:00 showing it to do and the extraction and Achieve better, higher accuracy scores. So that's pert. Similarly, gpt2 generator 3 train transformer. Now, there's gptt as well, these are from openai
who is a large self-supervised Transformer based language model that can definitely be used for machine comprehension and next-generation, right? And I believe it was also trained on a large karpis 2DS. At was about eight million with pages and all these advancements. Use for the life purposes will help us create a conversational, AI, good conversation, we had systems Central a better user experience. Okay. Now let's see if we can leverage conversational AI to enhance, the usual command line and the well-known graphical user interfaces. Now I
believe we can supercharged these interfaces with conversational Ai. And what I mean by that is we can build and design user interfaces. Very conversational AI to the core like it's a cold engine. And before we dig deeper, I initially thought that we should call this new breed of interfaces as instructional dialogue. Interfaces IDI it did make sense to me to call them as instructional design interfaces. But now I'm not entirely convinced. If you think that the name makes sense, spread the word it's
not and I think that'll be like in case I am open two sessions from you anyways. So let's talk about how we can super showers these interfaces and how we go about doing that. So let's start with the command line, interface has we all know your lies. The these are the traditional interfaces for humans to interact with computers. There's always mostly Smalls 10th and haverstick syntax and believe it or not their kind of conversation in nature. Some Cialis for example do pose a series of questions to the user
and tracked using a terminal and the gather the information before executing a command France and see lies they do come up with help manual starts and you know in order to give you some hints on how to effectively use up a c l. I Now, what would it look like if you lie designs, he lies with conversational AI at the core. Of course, it's a matter of personal choice when it comes to using traditional, serialize traditional commands. Your eyes are very sexy and they definitely do the job really well. That is one of the eunuchs plus we write
a small into something right, but sometimes but he says he lies, they pretend to be complicated and done self using options and how we have structure. And so I believe we can provide better user experience. If your bills, the Allies powered by conversationally a silver example, Define feoli with connotation. Yeah, I would allow want to type something like find me. All Json files sitting on the sandbox created after the June 10th or find me all the files under the source folder that
contain the word n o p. so you can see doing still provides us with a few advantages, you know, y'all be staying in that enhances the user experience, since we can use natural text Definitely expressly onboarding process in terms of the user being able to effectively use this Eli and it also decreases the need for command line. Cheat sheets. I mean we call you use ma'am. I do use online cheat sheets. When I'm leaving with complicated, she lies like to control
Okay. Now that we know what it means to supercharge, a command line interface. Let me walk you through the steps that we can take in order to build these supercharged, he lies. So, here's one. Here's my solution at the high-level, we got to use llamo to define the seal. I am going to use Raza for the conversation. We are aspects of it. That's our origin of the ceiling. So we're going to leverage the rest apis / Raza and we can also enhances Eli engine to provide its own set of rest API
sitting on top of processor, does the rest apis. You know, it all depends on the needs of the CLA and then actually implementacija lie in a specific programming language based on the amulet that programming language could be Jaguar S-Type Okay. Actually, now is a good time to take a brief detour and discuss Raza as I mentioned before Raza is an, is an open-source machine learning powered framework, that lets you spell conversational, AI systems So one thing you can do with it as high as you can
describe the intense, the entities and troll, I training data, to build your own RC model car, given specific tasks buy and an intent. Actually in simple terms is something that a user is trying to help him accomplish or the intention behind the user. Ask where you're sitting at the job of the energy model is to classify given utterance into one of those Define intense that we tell rasa to identify now, entity extraction. He's the process of identifying key
words, that caddy additional information related to an intent. For example, and be at Ryan's final, Json files on the sandbox created. After June 10th, we can easily see that there are quite a few entities here. Banco, b e t folder sandbox. The other one could be the files. I have Jason and one more and it to you would be the date June 10th. So entities can be extracted using where is methodologies? Want to be a great questions. But, you know, we can do more sophisticated
techniques like named entity recognition, and things of that nature. Also, provide says a machine-learning part dialogue management system so we can actually build truly conversation AI systems, which is what I mean by that is you can enable multi turn context-aware dylox. Without always having to use hard-coded rules, but be able to use machine learning too Petty. Okay, so with that in mind, let's designer that has redesigned the you famous Cube controls your life. So
you might know this shoe controls, your lie is used to control and manager, kubernetes cluster extremely powerful. And also, as a result, fairly complex has multiple command sub commands options flags and blow bubbles and special. And so on, you know, I remember when I initially started, cool, using kubernetes and trucontrol, I actually rely on cheat sheets and help manuals to get things done. Actually, this is the initial phase of using a C L. I like all onboarding,
Okay. So we'll do the first step in already design of Q. Control is too. Like I said, one solution is to use this piano spec and enamel syntax straightforward here. As you can see we can Define the available commands, the commands, he'll text options and so on. Now, the reason I'm proposing the usage of a yamel file is that that'll make the implementation easier. It mean, you can pick the language of choice but still use a single Speck of fire to Define what you would actually do.
Like I say. You can do it in Jawas dotnet. Like I said it's pretty straightforward. You define you were available command, sub commands and things like that. and then, the next thing will be for us to use a rasa, NLP component, to convert a command sub commands to be defined in the previous step into intense. As you can see, the CLIA options can be defined as entities. Now I can't. No. This is a lot of specific piano file, it's called Domaine fire aware. And you can list the intent
names, empty names and things like that that are specific to your project to you or see you again Trey straightforward. Okay. Next, let's give us some examples of training needed for all the intense that we just listed. Do for each. Intent. We provide a few examples so that, you know, kamado can learn to identify the intense and any of these supposed to be to our CLA, I understand texts that supported by the training needed, a pretty straightforward some examples for each intent and
sometimes you'll also be able to say okay also identified this specific word as an entity, the current wants to something that I just find it. All right, so you've finally, we will try dresses on Sample conversations again, training data and Russell speed. These are called stories are also rules in a. So here we are. Providing simple, simple conversations that prints and say, hey, if the user's intent is identified as a show, config then she go to the action name action show can take
Similarly, if the user's intent is identified as current contacts than trigger the reaction named action, current context. Of course, I allowed us to provide more complex conversations. Even collect real user data so that we can better change model very you know the user gets distracted and ask for something else in the middle of conversation. Comes back and all the contacts wishing and things like that are those are all BB can actually use the stories to allow the machine learning model on conversations
and be better able to predict the sidewalk. All right, with the all this information, I think we can go ahead now and actually income industrial. I based on the llamo spect that we initially Define Hopefully soon be releasing a coal generator on GitHub to automate the process. And in terms of the implementation at a very high level of the workflow is pretty simple. Try the traditional route where the UCLA part 16 foot from the terminal. And if there are no parking issues,
great if you get the command, but if the sea lies, an able to parse the command are the options. Then we invoke, rasser obviously, you lied to Russell and see if I can understand that use a text and if if so great last I will send you the intensity and Steve's that is identified as to what the commands for command options are Again it obviously depends on the Raza model and your implementation. And So based on Lazarus output we can then we execute this man.
Let's take a look at the sea lion action. So let's see. Your were saying, you control, what is my default context to control didn't know that? Because we tried to converse with it in Nashville, text. But if we use our supercharged, see, like, which is here, just K, it was able to interact with rasa, rasa told us what the use of mint and it was able to exude Easter again here. One more example to control, show my config controller, not understand it. But our
supercharged feel I would be able to because it stalks Raza Raza tells us what to do, and it was able to perfectly mad show, the user of current. Pretty neat. Okay. Now let's go on our phone, you know, switch our Focus to graphical user interfaces just like with supercharging command line and faces. I believe we can provide an enhanced user experience by building graphical interface has bad conversation. I had court an intensive supercharging graphical interface has with conversational
actually, lets the user for instance, slice and dice a large dataset tabular data just by using natural text, again be able to expedite the user onboarding process. And I think it will allow us to ask or instruct the system to make ushe examples would be like, refreshing grass resignation and featuring of tables and things like that. So here's the screen capture of a trading Marketplace, you are that has this is an example of a supercharged Raptor using face it. So it has
conversationally score. So I Nation to providing, you are components like parents and drop rounds. They also provide the chat window that you can see on the left side of your, so the user can simply start asking to you. I what they want in this example, the user is interested in Bitcoin. So based on new user provider message in Facebook, so refreshed to show The Life Aquatic on. So you can see that the message to the user was. Got it look to the right. I'm not showing the Bitcoin ticker and some more relevant information for you. I think this user experience can be
either by integrating voice into the mix. So wise to text in past the first You Are So Into You There multiple ways to read enhance the user experience. Continuing along with our example here, as you can see the user asked a follow-up question to get the trending price. I noticed a DUI. The. Of the underlined conversational AI engine knows that the context even the context. Is Bitcoin, even though that does not specifically mention, but I use it, right? That use the user just said, tell me
interface provides users with real good information. So to recap talked about conversational, Ai and then talked about how we can supercharged, sea lies and graphical user interfaces by using the same conversation. I tend to call Sachin two faces, like I mentioned before instructional, I log into phases, I then watch you through the steps to build a CLA with us. I'd core 40 prototype and then I talked about how we can do. The same thing are similar to buy supercharging graphical user interfaces. Thank you for tuning in.
Buy this talk
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.