Always fast and fresh dashboards: Inside BigQuery BI Engine

Good morning, good afternoon, good evening, and thanks for joining the session on inside the Caribbean and joined by most of us mosquitoes. Research and we can talk about giving all of you want beef gravy. I engine is why we built it. What is it called? Value proposition. You talk about next week going to the internals of the engine and we spend most of the time in talking about, I'll be engine work. Indian will have some exciting demos to Showcase haupia engine and Inspire conference e and then we'll have Madeline from Tableau showcasing, a demo

of the engine with a blue and Mila. Kevin from Nucor talk about Alfie engine works. Doing VI on Big Data is quite challenging today has multiple around the house extract data manage complex PPL pipeline. Call Debbie, Eichner. Queen Edition management or Administration. For Obligations are used for extracting data from the screening will last you running Mighty jobs to request to ask you, who the end up looking at the dashboard, which has more customers are looking for, not only fast lane, practice dashboard, but also be looking at more real-time data so that they can get inside faster

and take action accordingly. Mafia organization has the skill globally. They want to share that information and insight not only to internal stakeholders but also the external partners and customers complaints department for their bi solution. You put some of the reasons why he wanted to build across. Starting from connecting to multiple data sources, weather is on can sources or soft as a service and points. You can bring him here after multiple sources in to pick where we should. We have provided, we provide food for this. Claudia float do vaccinations in in

Richmond on your streaming data for Claudia Fusion. Looks like personally TL, transmission beat up, Rock Sports Park in halusa work. Wanted Eliza Vickery, you can create a metadata layer, discover Dana. And then you can connect the Tito Vic way to multiple Vehicles. Before I first part of your Solutions, looker the studio and sheets. You also have connectors from Fox News including Tableau rvi microstrategy Analytics, Engine is really helpful is if it acts as an exploration layer with inventory and you'll be a key to all of it is to the other bi tools and use the

vision for OPI perform in traction on lifx in real time. Reply to compromise, reindeer freshness and see if you want to go to me, really fast when practice is start fresh. But if you want there to be fresh and later date. A new stainless, is it going to be wind in? We don't want to talk to my cycling session complete. You always want to give you a raise. It simplifies the architecture and it has smart feeling good. That's going to eat one. She subsequently engine is a column-oriented in memory execution engine

scale. Horizontally. As you have more requests coming in the engine can load balancer by making copies of the United memory. In those balances engines, works with Bitcoin streaming so as your streaming data in Levi's, automatic needs a fresh into the in memory care in terms of simplification of Architecture is built on top of a store. Does Muni fix cracked a little bit crazy automatically mudita between the stories to you and the memory care. Tell me some foods require you to create Cubes, but we do that automatically for you.

That produces complexity in architecture but what you're bringing to Market is the engine sequel ATI, which is integral to any of the other bi tools including looker, Tableau or Bi-Flex have no time for me to make any change on this site. And lastly R Us Smart, tuning built into behind. The engine has ended cash optimizations if you do automatically based on your data and the baby is a mission that he specify the location you want to lose weight just by the size and automatically be

able to start driving. So you can look at all of the beautifulest Mozart you to walk us to the Thanksgiving, I, my name is multiple Szymanski and I'm very excited about the inner workings of the texture of Big Red Sea when pieces on the left, that's why I declare play about some good company and between them Shuffle letter, which facilitates distributed processing. We should be a engine was just another future because it's not the ship right product. So it works on top of the same storage, same network, sing Shuffle there, and it's working on a loan.

Okay, well, if it's exactly similar to check shirt, what, what makes it possible for the engine to get such a dramatic performance improvements? And this is exactly what we talked about now. So we should be big pieces and all feel them contribute to the performance of the crater is a metal, Toyota time and compute. It should be. You are going to do now will pick each one of those components and talked how they engine makes this time. Go to jail. Okay, let's start was a disguise.

This one may seem to be the most obvious because we'll just put that in my mayonnaise. No descale older dating to it, but of course, it's more complex than that. We'll have to figure out what to keep in mind. Remind me to manager, this is really the most important pieces in the heart of the day. I enter. So what wonderful specific consent to my marrying, not the whole table, but more than that, with chunks of it and it works equally well what's partition tables and no partition tables. So, even if your table is not petition form, Taemin 2 kids dating, multiple years

back. But you only clear the last few days. I will figure it out next month and soul. Also, we talked about how the engine is always fast and totally stressed. So to be always fresh and still data is very tight integration with me, if knows exactly what time stop. Also, a manager most call to work with load balancer even if you kept small table, but with a lot of rain coming in, if my Schuster plated send data or machines, which is easy for us to do because between is distributed database. So this way, it will you smoke,

but all the credits will still have very fast response time because they're a little bit. You can sit in space a sophisticated system, what kind of control user cap over it. And this control comes in form of your reservations relations are very similar to Classic but the cuts. So you can understand how is he using the system based on user. And the orange one is, that was used and it takes effect immediately. And you know, when they see the first time that, you know, there is not enough to buy a little bit more.

So we're covered Cowboy jewelry stores component. Now, let's talk about compute with CPU needs that. We used to improve health talk about couple of them so what is the classic vehicle database has dealt with clamp Cole? Sometimes I try to anticipate is consumed by the time the next text next hour or so, but it's not the best one. Especially when we deal with Columbus data structure and was in memory of databases. Play some Peter became more popular instead of

the whole one, single column eight, like Susan volume and then send for so much more complex model, but you don't need to worry about it because we took Jennifer to put this new and time into VI engine. So that a lot of benefits for shipping before, it fills, the cash, what college is much better because you are very, very localized flu. And you know how second you Shield a CPU pipeline, much tighter advantage of modern CPU instruction Colt, SMG instructions, which enforces money values in the same and soul.

So many different and quotient, so support for a few encoding, personal storage system. But now that this has been many many more because it's much easier to change them. It works well when you have table with millions of rules, but that's only if you just seem to like me in this picture, only thing and the column was the name of this song. So when you have complex expression instead of confusion to the gym, to gain over each, one of those million values, in this case,

in all the result is swollen. But now, this is an example of one of the potatoes when we talked about my being a manager instead of doing, Dictionary version can be used in many other functions according to the example, on the right here, it works really well. When you have repeated many times to know what's the time again, instead of computing, the same expression, a * or computer and just safe and please keep the shoes out three times. Not, what's up? That's him. So different than Coatings and they can stack one on top of the other like you can use both diction

examples and many others. So again it's all automatic. So this is how we deal with CPU. Now, let's go and talk. How we deal with Metro, how can we eliminate Network, or did you So the wave I poach it is from the clay Planet perspective and then what say OK Google can improve it. Given the precarious Beer Engine was at State football, cakes and different scheduling polish chair and demanded. A couple of examples of what you can see. Here is a very simple addition player and the distributor plum in the classic between us

and then we'll send them all over the network to compute the final yourself. Okay. Whiz bi engine is the plan was bi insurance, so it's funny because it looks almost exactly the same. But now instead of doing this team to plan our schedule, each piece of data decide so much water can bundle them together. And safe day, do the slow congregation and then also emerged as a result. And now, only after that, you can send it to the disturbance execution. And the

little one is called broadcast Droid. It works very well. When you kept a big stable, would say, a fact table with a smaller one. Typical climb in between, it would be to take the small table and broadcast into all work as Ambrose goes over the net for the small table is small enough. They don't they just a placated all over the north, which one of the pieces of the part, able to focus a little bit more, but now I don't need to use Network. Oh, okay. So this kind of decision my manager can

do because those days. So if you can do this kind of a wimp of each one of them, and it's time to see this, you know, how it actually works. Yeah. Engine engelsk, I confirm. And this is an official demo, which was she related? When is the full moon, you have to accompany him to have a table, which is stable in this example in this demo. And what's US money and business users can finally push into work Christmas day. So what's a complex? Dartboard elements? So wishing

related symbol dashboard since 10 different class. OK. Google said people work with them so it's sad. You know, each one of those users changes dashboard Once every second that I wish I was dust boards. Introduction. God sends Clair 10 place every second since Mythic we're going to focus on in is this blue line which shows that? So put my sweater on GPS, Look and see the Plastics in Griswold, Wikipedia which each every second of the same in all days. It's Tony b. I n g, o.

Okay? Now let's go to Chicago system skill but would need to look into different matter, which is clearly a sponsor Matrix and the bottom of the kind of car to see because in all that, the canister, So deuced rewind ship isn't the best shows, the 50th percentile, which is medium, okay. And then, the next one is, it is a 99% that no matter how far they might be a 2000 que pasa in case, but only tiny bit also absolute times. Like, she was only 35 minutes seconds. 95 percentile is lower than 200 milliseconds and thanks to

9%. This is money. Money million rules table and Mm. Mm, fashion couture, go to unite to show to other demos, Thank you Marsha and I'll be able to scale. Watch more hired to provide higher conferencing initial preview of how Tableau and Victoria engine blocks together. What did I want to NY today at where my table? Pandora. First, I can pull out the number of record field to quickly in Road. Data contains lots of customer reviews. I can use Tableau to quickly aggregate the overall reviews for all of these products. Looking at the average review, I

can see one of our products have scored very high. And which ones having scores High? Tableau also makes it easy to quickly switch between various visualization types. Let's call this product reviews. Which of these products lives in various different categories. Apple makes it very easy for me to group these various categories together into something, we call a higher key. I'm going to create a new hierarchy, call product. And by doing so it makes it easy for me to drill up and down between various levels of granularity. After this to you, I'd like to get a

sense of the overall quantity of sales. I have for each one of my categories. I'd also like to add the context of the average rating as well. Every time I use aggregate rating here, I'm typically going to want to look at the average overall. So I'm going to set the default property to use out average. I also want to change the color scheme as well. I want to make sure that I don't miss any of these products that may have lower reviews as I go through my analysis. Now, I can see that there are certain product categories where I have a high

quantity of sales, but the aggregate rating isn't very high and that might be something I wanted us to gate. Let's call this product. Reviews on quality. I don't like to look at this data. Geographically stated that already includes State information so I can simply drag State out onto the view. And how long will stop that for me. Now, I can also add quantity as well, so we can see which states have higher quantity, sold. I'm going to add aggregate rating to the

view again. And I can see that generally the states where I have higher numbers of sales. I also typically have better product ratings. However, there are some problem areas that I may want to investigate. I can easily drill down to a lower level of detail. to figure out if there are specific cities or I have high quantity, but low ratings, Let's pump up the sides a little bit so it's easier to see. Call best product ratings and quantity by state. Finally, I want to pull together all of these visualisation Simon something we call the tableau.

Using a dashboard makes it easy to compare multiple visualizations, at the same time. It also gives me the ability to add interactivity between these areas worksheets. So, effectively creates an interactive application that lets me visual explore my data. I can look at some of our problem. Subcategories. And select a particular region to explore in more detail. Finally, I can pinpoint exact products that have low reviews. And so is that in just a few minutes, we

build an interactive dashboard on top of over 87 million rows of data. Thanks to bi engine. Now, I can publish this to Tableau server, a tablet online and share it with other people throughout my organization. Thanks my blinds. That was a great demophile, Tableau & Beyond the book together. Now, I want to use Kevin Maher, to the park manager, to talk about how Luca and the angels together. Hi, I'm having a product manager at liquor and I'll be showing you look at integration with big berry VI engine first.

Database, not injectors won't stay. That could you load a report or visualization? And liquor is being sent directly to your data warehouse? And as you can imagine having a very bad day to Warehouse in a very good experience in liquor. Wyman's Liquors, explorepage allows businesses to self-serve by exploring their data is powered by Lookers at modeling layer called. Look at me now, which provides an abstraction layer between your email and the business terms that your duties are familiar with.

I'll have to build a query. Here is blacked, imagine that measures and add filters to know the total revenue it ever made in our activity, I got to click on the total sales measure and hit rock. But I find it seems like I have generated Aquarius, very for me, and has sent that to very miss kpi engine and is giving me to answer the case, 9.6 million dollars. Filters, that I want to look at this constrained time frame. I can put on a filter. I can pick, you know, and it is easy points like option and I'll just pick a April

2020. So, starting on the 1st of April and ending on 1st of May, haven't done those interactions with these are in a bit again, looker, adapting, the people I want to run that people get sent to Be Crazy II engine. There is all supposed to be here. So, also, if I jump over to that big gray console and take a look at those queries and only a quarter of a second. I'll take a step further and maybe I want to create a visualization. Again hitting run. What type of cheese

equal? And resultant display hearing local user interface want to open up the visualization section of the case. Worker thinks that a line chart for the best. If my data was I like and I'll go ahead and save this. Have a nice little test dashboard, I added. if there are no papa open in my other job, Coronavirus dashboard to show off the engine and the bottom. I can see my new tile I actually added to my previous recording but forget it, this one. That's what the liquor Alta design to be heavily

interactive. So say that you know, her Facebook, I can do so like that, like Jen and all my other not going to see the day that they caused all of that, going to graphic profile. I also just filters Direct in the top down and look at this. Specifically intensity. San Francisco and locals, are offering a suggestion is on the underlying data. Things like that. Even the dashboard and see how the trending Justin's. That's it. Thank you very much. Thanksgiving in conclusion, right? Engine is available today. You can try today with data Studio 338

products available. Today engine sequel. API makes it possible to work to get the same expiration. Same benefits with other bi tools, including Google Tableau and that's coming. Sometime. End of this year will have a new supreme New Edition station to learn more about. You want to know about how big, what is the molding in other media, 2200 facts about Aquarius road map. And if you want to learn about how far is Moorland Road, map session,