About the talk
Presented by Rasa Co-founder & CTO Alan Nichol at the 2021 Rasa Summit.
Alan is co-founder and CTO of Rasa, an open source company providing the tools required to build better, more resilient contextual assistants. Rasa has raised $40m from top venture firms including a16z and Accel. Prior to Rasa, Alan co-founded a productivity startup funded by Techstars. He holds a PhD in machine learning from the University of Cambridge.
#conversationalAI #opensource #aichatbot
- Learn more about Rasa: [https://rasa.com](https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqa292TWs4eUVTMWp1NTJuYlM5NWRPM3ZjUkhGd3xBQ3Jtc0trQldlM3ozV2szQjNBeDd0VE5HRVR1MmhRWnFNWjNqS2kxMGhJTk04Q21LWkZLWWNNWHJPSzRRXzFyUnFtaURYbkZFYWJLUkxXaFRyZTQtQXBsdG1VV2tCa0ducHpYQ3FHdVVoY0lZcUZ0TVY2eHhQdw&q=https%3A%2F%2Frasa.com)
- Rasa documentation: [http://rasa.com/docs](https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqa2Z5TkJraEdEcG9mS0hKelVHUjZoSzZGaFNFUXxBQ3Jtc0tuTTJxQ0MyZzVPdTR5R3cwbmF2aElfSF8wTW5IN0dUSG5ZbGVLRVpFYkNCX2ZoMklNYWJudXNsQnJpTEpULWNGYkRuaXdjNEhTZUdyMjd6cG9EeUJtRV9HUkZsZFdKcjliVExnQ2NONXpYV3JBNWsxdw&q=http%3A%2F%2Frasa.com%2Fdocs)
- Join the Rasa Community: [https://forum.rasa.com](https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbDRCcFpBLW96SUZ1Y1FJWXhHeGpISUdCMFh6d3xBQ3Jtc0tsNG1MRmVtVnY1d0N4a1pUYTJTa1hMRTRBcUxPdkVyaUZQZkc4RkIyenp1aHZDYTg0WnlHS0dOaktDR3pjRkpMS29JQ0NKTjg5M2dKckNEd3JQeVBkWGUxWEQzUkpybXBnNFJWYVA2X3ZqSTVoYlBSTQ&q=https%3A%2F%2Fforum.rasa.com)
- Twitter: [https://twitter.com/Rasa_HQ](https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbTNVNW00cXJUSkM5b2RUbVdZQnZrYnVma2s2d3xBQ3Jtc0ttcnBuN0ZfeU1vUG5VY3ZwbmJpcjRtN2hCNVNzcGkxMmV6T3JXTTh6cDdnSHVTYzN6NHFuUG9JV09xTUNUbFBrUHdIeXhBbFBJZkZHUnpNQVF1cFVqdVQwVE1veUgzZjRmYXVuUHd4QTNQdkdFWTBBdw&q=https%3A%2F%2Ftwitter.com%2FRasa_HQ)
- Facebook: [https://www.facebook.com/RasaHQ](https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqblJUSWhmOHFJWFpPZE9RY3BuQVpNOTlwRWdqUXxBQ3Jtc0ttMllOdjdfcE9zVU5lSjA2VjFSQ0c4eGRpVjdrXzBrbzVqcWZaOTdHcDl4T2tWX3RmNHdkQWtGa3BKc3diaE1adWo5SlFJNS1HLVBVSGlhWWFaSEJ4elZWdzB3MWZNdVBQZUNZUHhsYjNQOWVaTUIxOA&q=https%3A%2F%2Fwww.facebook.com%2FRasaHQ)
- Linkedin: [https://www.linkedin.com/company/rasa](https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqa2czR3UtUTJyNldHNjdZQ2pjRzlxZmM3UkZOUXxBQ3Jtc0tuTktyamZOZWhBX2lKZWotRml4THd4cGg3alBId1VLNWIzamlKd3M4SG9iRE5fSjRFM0FoYmtmWnctOHh3SE9vSE0xbXVxV2YwdTRDOGVvNUN1aTZJcFlhYk1wYUJxNmtZSDNWLTNjTWdwbnV4NTNZVQ&q=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Frasa)
This is the wrong with stomachs 2021. 19 year. Old's birthday party at the first talk kicking off this great summer and the topic will be scale and building conversationally is scale and a team. So I'm assuming if you here at the summit, you already know what it is. So just one slide by way of introduction, right? Brazos an open-source company rebuild, the infrastructure for conversationally. I have been around for a little over 4 years, I think. If you had to ask me another folks, for years
ago, it would have been obvious that none of the largest companies in the world across all sorts of different Industries, right? Healthcare insurance banking, Etc. I would all be using the same horizontal. Open-source infrastructure to be building mission-critical applications and I think that speaks to the strength of the community that we built up and how much we've been able to learn from each other over the past four years that we've made all of that happen. And so, what I'm talkin about today is Gail and it's a lot of what we are observing working and and what's not working,
when people are building applications at Enterprise scale, with a huge number of users, And so does one idea that I would love? For you to get into your head before and we start the rest of the talk and super simple idea. And it's very important one. So, yes, there, lots of unsolved problems in conversation and tell us if things are hard about it, and I think it's something we're, we're very vocal about it. And yes, they involve machine lighting, right? And I think building products and applications which have
machine lighting at their core is, is quite a new thing, but that doesn't mean that we should just throw away everything. We know about Gilda building, good products, and Billy, good software products. And so, one theme of this year's, can we just pour it over for me just coffee and bring with us and just make sure that we're doing them. And which things do we have to rethink? Before we can talk about all of that and what actually don't make sense, it doesn't make sense at the first thing, I think we have to do is agree on what we mean. When we talked about scale, I use in many
different contexts and I think people use the word scale to talk about all sorts of different issues. So let's get to the bottom of that first. App for singing version for right, if you have an assistance and it's doing its job, it's doing a great job at, you will naturally want a scale that to a larger user base, right? If you're helping users, you want to be probably the most expensive scale But of course, if you're helping users do something, you're going to need you to ask yourself over. What else can we help them do, right? We're going
to scale in the sense of the range of capabilities of you since we're building, the amount of things that we're going to help people do. And then the surface area of your assistant. Also tends to increase over time. So you might start by building an assistant that lives in a chat, widget on a website that might get ported over to experience. And then very often we see this showing up in the contact sensor ride so but now it's not just in chat but it's on top of that, you might be rolling it out in different countries were people speak different languages.
And so the number of ways and places that people can access your assistant also increases the complexity. And then the humans who are building these things, right? The did, the teams that build conversationally, I are very heterogeneous, right? We see that all the time at their developers, their data scientist. Their product owner is our conversation. Designers are all sorts of folks and don't have to work together. And then as they grow beyond that to Pizza signs and you'll have to coordinate work, right? And so you have these multiple
heterogeneous teams on his record, and that's another area in another dimension. And then finally course, as you roll out, now in a large organization, you're growing up multiple assistance. Those are often going to have some kind of shared functionality. You might have some things that all your systems have in common and you want to figure out how do we avoid everybody having to reinvent the wheel and do things over it and how to make sure that the shared pieces are consistent across all the building in our organization. And so
those are just some I'm sure it's not comprehensive but I think it's pretty good coverage of the things that people mean when they talk about scale and all the different dimensions in which complexity naturally increases as the assistance, that we build a game, some kind of success. How do we do that, right? How do we deal with all this crap and complexity without grinding to a halt? Right? How do we continue to move fast out of my continue to evolve the assistant that were building and ensuring that? We're solving more problems for more people.
And so I'll start with a bit of good news that we're not starting from scratch. So a lot of these things about even figured out different from building, any other kind of software product at scale and hopefully you're already doing many of these things, right? I mean you almost certainly using voice control, right? And you probably hopefully have a CI CD pipeline set up for talking about some kind of radical you things. I'm just talkin about the basics, the foundations of shipping products in Amarillo.
And most of these ideas. These are well-established ideas of how to ship Lively. Most of these translate very well into building a system as a most. Because there are definitely some that don't. And I think that's the meat of this. Talk is teasing her which things can we copy over and we just have to make sure that we do them and which things do we really have to rethink Andrea? So here's a very incomplete list but it covers. I think some topics that we see coming up again and again of things would feel like you're making Fast progress
in the early stages, been assistant and the things which actually scale to help us, you know, continue to associate with high velocity, in generating synthetic training. That we're learning from your conversation. Rather than using a black box tool that handles everything for us, we have customizable infrastructure that we can control. Rather than making edits and having changes go live, right away. We have changes that go live automatically after review and testing process. Rather than purely separating out code and
content reversion the code and content together to ensure that everything is perfectly reproducible. And rather than composing pieces that run time, we can post pizzas at Bell time. And so I'm going to talk about each of these things and how we see them play out. And what our view of these things is Now I'm the first one if you joined, you know, any training from Raza. If you've read or documentation, if you had any traction with us at all, you'll know this is something we advocate for very strongly that Your model should be trained on real Gotham, I don't think
that's it. Training. Motors on synthetic gas, has a bad idea for two reasons. One, you're asking your model to make predictions about real. Real Jews are messages which are messy, and noisy and difficult. And so if you're only training and secondly, if your training and testing on synthetic. You're going to trick yourself into thinking that you do have a good model and you're not even going to notice all the problems that you're my life, she has So rather than ask him yourself, okay, we don't have any training down to the start of this project. How do we know synthetically
generate some, how can we as quickly as possible? Get a feedback loop set up so that we know where annotating real messages and using those as the basis of our This be's, you know, if you're at Roswell Summit, probably don't have to convince you of this time but of course, when you're building that first prototype that first chat but you ever build, it's very nice to have a tool that just hides all the details from you. You don't have to worry about how the hell he works or anything like that. That's
that's what we want to fuss about that. You just want to prove out there and of course Very quickly. When you're getting two more serious application, you start to think, well, maybe actually we want to have some control. We want to take the Moto, that's best for us, write the best for our application and you want to be able to look under the hood and customize that, if you really Advanced, you might say, look we're going to trainerroad embedding. We're going to build our own really, really bespoke find my phone. The same time. Of course we have every six months or so there's a new
big language model. That's the new cool thing that you want to be able to keep up with all those developments. Or maybe there's a new language model, release the specifics of your domain or to a language that you're expanding into, if you want to be able to keep up with all those developments and use the latest things plug-in play architecture. And these two points are really about the foundation of building Reliable Software, right? And I want to spend one or two minutes talking about these pieces.
So there's a reason why all mission-critical software they can shift in the world. Now has tasked as in CI pipeline. Write the reason we write tasks, the reason we run task is to move faster because it means that when we want to build something new, we want to build your future. We can focus on building the new future rather than fixing some bugs and regression to be introduced to be before. It was really cute. The shipping high-gloss. The especially as you know, the complexity and sophistication of what you're building increases, it becomes more and more important and of
course they were very vocal about ensuring that you write stories around through test stories. You check any changes. You make don't break things, right? And I've said this before, but I believe very strongly if it doesn't support rising. And Then it's a prototyping tool. A mental rather than thinking of everything going live right away. You want to think about what we, how do we integrate this into our CI CD Pipeline? And of course, I mentioned conversation has been built by diverse teams and
it's not just about to empower others to contribute. That's how the conversation supposed to go. But they're making sure that those changes still go through your, make it back in to get or, you know, whatever version control system using. And they go through your CI CD pipeline that you know you're not breaking. Similarly, on the topic of Saint freezing, cold and content is not about what tool you use to edit. The response is typically sign people described his contact like the responses, but
it's not about what to use. It's about how you version and ensuring. So in the case of Brasa, you have a modal right now and that contains everything that contains a response and you want to try that, you have a single as a fact, which is a perfectly reproducible version of your assistant. So that if you have to you can do a roll back and so you can build instrumentation on your deployment and know what changed it, what time and those are really just table. Six things for shipping, a production quality
piece of software at a beauty Foundation. And then the final point is about dealing with the complexity of having multiple teams multiple assistance with shared functionality, right? So maybe you have two assistants that running your organization and they share like a login flow and symmetric use Answer. The question is, how do we make sure that those are shared and consistent? And without having to duplicate that were into an idea? We see lots of different people exploring and asking us about OK, Google. Maybe the shed functionality,
maybe we should deploy that as a separate service. Right now, that starts to look like I microservices architecture, but it's missing. The one thing that makes it possible to Build microservice architecture is, which is enforcing adherence to a very strict API. If you're letting users have talk to different switch conversation, without the one thing holding it together, that makes it possible to put off. Answer the way we think about, okay? How do you build reusable components? You
can do that and you can share those but you can buy them to build time, right? So that, when you train your model, you can check. Nobody else has introduced an intense clashes with yours. Nobody has, if you don't feel the flow of clashes with how your conversation should be going to end test and you can be robust and you know that everything still works rather than having a awesome. and so everything so far has just been about Complexity coming to you and scale coming to you and having to deal with it and keep up with him. But I'd like to see the
finish on a note of thinking about, what does the scale and able, and what can we do that? We couldn't do before. So something we talked about a lot at rasa is the five levels of AI systems and from the end-user perspective is that were asked to translate what they want into a language that we understand the language of, of our organization of their reality. And the way that they see and so, if we think about building something that scams, We want to build not just two or three or four AI systems but we want to build something that is greater than the sum
of its parts, right? Rather than shipping our org chart and imposing our are domain and our restrictions in our divvying up of the world on oranges. We can build a an interface, a conversational natural language interface between a large and complex organization and a huge collection of customers, think about the world differently and we can build that translation there and that means that we can meet customers where they are and speak their language. And so I think that's why we're all here at the at the summit, right? Everyone here. If there's a Common Thread between all these talks is
that everybody's thinking about, okay? What can we build in the future? What is the really great thing that we can build for our organization? What is something that a team when it's set up for scale? When it's your building and building at High Velocity in building in a way that they happy with? What can they achieve? And so I think that's something to keep in mind and to take away. Add to think about for the next couple days as you're thinking about them, as you taking in all the different talks is what can you build your organization? That's going to be really transform with it.
So that's the end of the talk I'm sure I'll have said some things that are controversial. So if you want to bring me a long email at about how I wrote about everything and I feel free to do so and I'll just finish my saying enjoy the summit and they're a lot of great speakers about a great time. And enjoy yourself asking questions, be kind to each other and enjoy. Thanks.
Buy this talk
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.