Streaming and Real-Time Analytics - Level 200 (United States)

Hello, everyone, hope you're doing well. Welcome to 8 Summit online, and thank you for joining the session. If you're looking a way to stream and ingested, a DeWalt 20V Chargers station, just before you were going to be scarce, be a fundamental aspects on data to Main and what we can do from there or even setting up your own, did a post processing and talking about streaming. We will also cover on how we can just feed it to him in this session. I'm doing a proper. So, this is how we stopped at new

session to motivate you and to keep a practical example in real life, we're going to start with a few common business, implications for the other streaming by featuring use cases, from our customers to give you a comprehensive understanding on data streaming and analytics. We're going to swimming and principles of data through me, we're going to show you on how you can I collect data streaming and analytics system with help from a SS Let's see on how did the swimming wolf in real life fortnite mobile game. At that

age, resulting in the actual game. Currently in the market is set in a world where players can cooperate on various missions to fight back against a meteor storm or the time to be the last person standing in the game of Battle Royale emote fortnite players in less than a year. But fortnite is a free to play with its ripening coming inside. Ali from in-game microtransaction. Microtransaction means capturing the attention of the new content and continuous innovation. To operate this way, epic games needs. An up-to-the-minute understanding of game of satisfaction

guarantee their experience is one that keeps them that users engage. So how did you manage this kind of challenge? Sweating and Dentistry me. Epic games, collect millions of Records on a daily basis, tracking everything happening within the game. And this includes, I'll pay it incorrect. Or even their strategy, game story mode, and 14 petabytes of data and All Star team did electric powered by Amazon, S3 with growth of Jupiter, by super month. And if you think about this, it's a massive amount

of information which epic games decals to real time, using Amazon, Kinesis, and best analytics to detect any issues with in the game. So to summarize this an all Telemetry data streams which has two pipelines one for processing data in real-time and another for processing Star Theater in Vegas and just curious if you take away for next section, Why real-time data streaming and analytics is important for business. How is it relevant in a parody of forms

at lightning speed due to an explosive proof of real-time data sources from the faucet that can make decisions and take action to better their perform against their competitors. In other words, companies are getting fairly from data for creating their unique value proposition and the volume up. But how about Hello Kitty images of a time and to get the most money from the dealer, you must be processed at the very City in which it has created episode and see if you scratch different types of data

of experience and we need to start driving. So it's more intelligent real-time experience. They can't wait for dinner to be best process and was making decisions and taking action to reactivity will differentiate your business and to achieve a better customer experience. Organizations need to work with the first possible. For example, real-time QuikTrip in process to use this behavior and respond back almost in an instant recommendation to increase customer value and share of wallet. Let us show you a few times on how our customers use the data stream,

they can be used for messaging and they devilment of real-time analytics application down the road. I'll remember that if we show up earlier simply cannot meet the reaction requirements for businesses, afloat and solution. As a result, awesomest are coming up that curve to build higher value and lower latency experience. By providing microservices account action. Resource for all application events, making it easy for you to block and play new microservices and social events. Organizations looking to integrate applications and

downstream services with relational database, or capturing DB changes using data streams and using these changes to replicate the state of this, rather than carrying it directly with an increase in the amount of data being processed. In real-time by Machine learning and AI application. We need to ingest data as it generated rather than wait for hours or days from thousand of data sources, scale and 30 of sources is important. Aggregate filter and enrich the data in the store. And lastly, we make new data available for analytics and processing in seconds.

This point, we understand if you use cases of real-time data streaming and why it's so important for business? Next to me to lay the foundation for our understanding of a system music? It's if not, you cannot react in real time. The second you need a system that skills to support the injection needs of your business but also allows you to build your own applications on top of the data collected otherwise which s and arrows, your ability to react in real-time when it happened and how it happened.

Are the color of the datastream is a common law which is a record of events and producers and consumers are political system, specific. Of time. 24 hours, 7 days a week Sandra play. Parallel processing is not destroyed. When its width by Consumers Energy producers to relay the actual sequence of events to Consumers, then retweeted milliseconds and enabling you to achieve reactivity, reading your architecture. We mention the producers and consumers are fully to couple producer create data and white events. No data streams are

horizontally. Scalable meaning you can pop in your data stream across stories news, organizations, faced many challenges as they attempt to build out real-time data streaming capabilities and embark on generating real-time analytics difficult to set up how to achieve higher fertility. It's complex integrated into broader ecosystems, error-prone and complex, 2 minutes, overtime. Been enough of a reason for many companies to shy away from such projects,

it has been our core Focus over the last five years to build a solution that we use these challenges, see how I can help us to implement real-time streaming and Analytics. the eight solution is easy to set up and use has high possibilities and durability as the default be across three different regions, it's 40 minutes and Scandal which mean it reduce the complexity of managing the system of the time and scaling as demands increase, and also comes with seamless integration

services such as And finally, with a SS, you only pay for what you use, making the solution, very cost-effective in real time without having to deal with many, complications mentioned before work together and provide flexibility and options to tell her you streaming architecture to your specific use cases and scale on demand. Give me a history of Idaho's is a fast and simple way to stream data into did Alex or other and destination. Again as skilled with the ability to

execute transformation as required Kansas data analytics, allow you to build in the great and execute applications in SQL in Java for more context, use cases work together to enable customers to stream process and deliver data. In real time for you who plays Robert Acosta along side, table specific use cases. You can focus on building streaming application with the confidence that they would be cost-effective and able to scale. And finally he misses video streams allows customers to capture process and store media

streams for playback analytics, and machine learning and capturing of the real-time streaming data. And you can store it based on your processing requirements and process to tap into, you can send emails leaving to your other events, refund application. And finally move data to a persistently, Amazon Kinesis data streams is massively scalable and durable real-time data streaming service. It can cause you to capture gigabytes of data per second, from hundreds of thousands of

sources, such as websites, like streams, even streams financial transaction. Social media, feeds and location, tracking events. The data collected is available in milliseconds. Such as a sport and more with Amazon fire hose easy, easiest way and loose Trinity in to Amazon S3. And that's what you're using today. In a surface that automatically skills to match the truth of your leader and request nail ongoing Administration. You can also do batch compress transform an encrypted Data before loading it minimizing the amount of storage use at the destination and increasing security

Inside a mediator. I don't respond to your business and customers needs in real-time Amazon with other AWS Services data streaming, applications with Christopher and Recreations filter and more and better to customize it for you, specifically used skis. Let's see a demo on how we can use Amazon cleaning services to process real-time locks. In this will run through one of our solution example. Real time to Xtreme Mattress in real-time can provide insight into who visits your website where they come from and

what kind of content. If you get on Facebook accounts page views time, spent on side Channel, starting traffic, action, physical steak, and other Castle metrics and this inside, can help you react quickly to changing conditions and emerging situation, integration of Guinness services, and other services. Such as Amazon. S3, be an obvious solution is designed to provide as a framework for analyzing and fascia lising metrics. So once that you've deployed dissolution, you can focus more on adding new men. And gain Insight rather than managing the underlying

infrastructure. The Sorting stuff like this user access your website, you have a Nintendo DS, code on your web pages to send series of love to the beacon web service. The beacon web service will lock it for further processing and this Beacon web service, they hosted in easy to do with auto-scaling troop and Sir. Follow request from a LG and disputing web server will be our endpoint to receive the request. And wants to Beacon web service received the data. It will send the data to the King, has his data Firehouse not

here with him and started a fire in Amazon S3 bucket and in parallel sends the data to the keenest streams which will be relayed to an herbalist. Lambda function, a function rule safe to data in real time to Denham, repeat able to be stored up for two a week. We also have a real-time despots, which will fetch data from Lambda function. And this is a request payload, which will be sent from your web page web server or any kind of applications to Beacon web service. As you can see, on the example, we have pitch for Abel Klein ID and also what kind of

event. For example, this request was a picture event, which indicates a new user just access this page. Of course, you can add another metrics, but we'll keep it simple for this demo. On the other side, this is an example of data that has been processed by Genesis analytics. It snaps feel terrible as from the lock request and ask for more information. For example, IP user agent referral and the size of the request. Now, we understand the architectural structure of the data and how it goes around from one component to others. Let's see how it works. What we have on the

screen is an estimate page of dapitan web service. So we're going to hit this and point with a good stream. Payload now to simulate the real traffic, we're going to use a patent application to send bunch of traffic in 280, Sandpoint application. And it accepts few arguments such as the target. How many cost would like to make And Delay between the car. Now, to similar, to traffic to Beacon web Surfers. This app will try to run a match the events, such as click-based, use compulsion, and so on, it also has custom metrics several type of data. We have Tara, an

integer a string. And this application Wilson, all the data using spp. Headers with several different templates, with conscious of Randy Moss, Matrix with two different things on our page and by sending this data, it also means that the beacon will receive different era of a time and eventually we need to visualize all those information in out a spot. So that's that's it. I'm going to copy the URL of this weekend and I'm going to run this application to simulate twenty

thousand requests with 0.5 seconds away. As you can see that it's running now. And the first thing that I'd like to chat is where the fire hose archived data into Amazon S3. So, let's go to our Amazon S3 bucket. Nice. It creates a new folder 2020 and let's see what we have inside. It's either. It's all the data received from the beacon and here we have all the data start in different time for my dead father's first is to store the data into Amazon S3. What we are saying right now, and the second task is

to deliver to Amazon Kinesis analytics, so we can process the data, go to Amazon Kinesis, analytics passport This is our Kinesis analytics application test port and you can see, we have few information here, but let's go to the application details. That's what you will. See that a few sections Source real-time analytics and destination, which we will see in the application details pitch as well. In this case, we would like to inspect musical results. On the Genesis application

page. We have three types in the source. We can see that we have all the data from fire hose. Now, this is the Rotator and we need to do processing and if you see on the other side, we have sequels that man's to process. This road are not using secret that we've already paired with. Moving to do real-time analytics tab in this tablet, can also create and replace any application streams that will be useful if you wanted to send the data inside the streams into another pipeline. When you come take your application and put you map a streaming service to an implication stream that

is created. Did I continuously flows from the swimming Source into the in applications trim and in applications DreamWorks like a table that you can carry using SQL statements, but it is called a stream because it represents continuous data flow. So don't confuse it with other stream terminologies in application streams exists. Only in the context of an Amazon queen Isis data analytics application, and as implementing a pub sub messaging Pat a time. But for data streaming, 4in application streams. We have a few sample data as an output from our secret operations

that we have Define. No, next on the destination tab at, once that we have process, the data inside the stream, we can instruct this application to stream into our desired destination. Here, we can see all these of destination streams, and we can also connect or disconnect distance to Maxwell. And lastly, we have the Auto Trim, which is the key necessary streams, that will trigger a Lambda function to store in to Amazon dynamodb. Now we have all the data streams properly into kenosis surfaces. Next thing that we like to do is to check the information to demo inside

attitude. That that's happening real-time transport. This is the dashboard that we appeal to show all relevant information, we have Ethan's per hour. We have reached our accounts, usually agents and other stuff in this desperate, time to discuss, but will update the charts based on new data that they receive from triggering Lambda function. And now it's updated, and you can see that information flowing in. And we can see that it received more events and more visitors. Get there. Just so we understand the offer of framework on how to design their detriment for your system

and also to demonstrate different surfaces for different. As we mentioned that, the objective for Tina streaming is to gain Insight from Howard hughes's. We interact with our system and will it be much easier if we can just input or dinner? And let me introduce you to Amazon quickslide. Amazon Kwikset in a fast Clock Tower piece of intelligence service, that makes it easy to deliver in size to everyone in your organization. That's a quick slip easily create and publish interactive gas sports that include Michelin insides. And the

most important thing, quick status report can be accessed from any devices and embedded into your applications photos and websites making much easier to distribute in size for everyone. You don't need any service or software 2 minutes. I mean, 10 or deploy, your quicksight, you can start as small as having few uses and skill to thousands of users with no infrastructure management. Giving you peace of mind despite increasing data films theater, analytical complexities and users, Kelly national business. Intelligence with service

users are increasingly certain direction to not slow down quick sign. And it's also fully integrated with a SS so you can build an to an analytics and machine learning insights giving you a contextual relevance insights and not just charge. You can easily leverett's anabis Mission, Linen, cap abilities, and you like to use it with Machinery. Powered, anomaly detection, forecasting and customizable natural language narrative application. Are you own corporate branding to make easier since Management in no time?

What I'm going to do is to modify a bit or preface demo. So now we could visualize the data using Amazon quick side. And to do this, we need to get those data into our Amazon S3. Bucket and quick side will connect with out at 3 to fetch out later now. So that's the idea. And obviously, I need to modify. Our precious came up a bit. So let me know if you know princess demo. We only have one destination stream for Arkansas, analytics application. And now I've created another destination stream with Fargo's closest perfect fit, because once the data is processed with

Guinness analytics, what we want is actually started data into our Amazon S3. And if we go to the kenosis fire hose, we can see that rst destination, will be located at the same bucket, but with different prefix or kick. Now, I've been running the patent application to simulate the traffic in the background. And I celebrate getting together. So let's go to Amazon S3 bucket. And yes, we can see that the new father and 47th open one of the files. We can see that is on CSV format and we have all fields that we need call. So I'll next step is that we need to create our Amazon quick side

passport. It's really easy to import data into Amazon, Creekside passport, as quick side, has various integration with data sources. As you can, see you again Corona quick side with Salesforce s3at. Not quite sure if Twitter g r i get up and we even can upload my nosy or false. So I'm going to choose as tree Put some name for it and upload Kwikset manifest Falls which is essentially a descriptor to instruct quick sites to fetch data from specific S3 bucket and click on connect. And we have it. So that's quick. Edit and

preview as we need to change the headers of each column. We're going to change a few columns Heather so it makes it easier for us to visualize 8 on the creek side, passport? Okay. So what is Stan? We can refresh the data apps to import into spies and you can also schedule the refresh and 40, say, Maurice Creek and refreshed. Now, And we have all the data. Awesome. So that's click on create analysts and create a few simple charts to fertilize or data. It is

very straightforward official charts with quicksight, there are plenty chart Types on the left side and let's get one. We can threaten trop and will see the results immediately, like, for an example here, I'm craving a donut chart with metric type to see the composition of all metric tops that we have. We can also add another official and choose different type of fields. Now, if you want to change the fields, you can easily drag and drop from the filter on the left menu for an example, I'm changing the

failure from the metric type to your unit value, and it will change the chart as well. And once that we're done with structuring our desperate, we can puppies, eat and share with other uses within our organization. So that's a quick demo on Amazon quicksight and how to integrate with Amazon S3. Once that you have your data, it is fairly easy to integrate with quick side and you can get more inside please on the day that you have collected data services, can actually do beyond that. And this video

streaming Amazon Tina's video streams, make it easy to security stream video from connected devices to a labrus and we can process. If you do scripts for analytics, machine learning play back in order processing skills. All the infrastructure needed to ingest from millions of devices. It really starts and creeps and indexed video dealer in your streams and allows you to access your data too. Easy to use apis to play back video for lime and on-demand, viewing and

quickly. Build applications that take advantage of computer vision and video analytics to integration with Amazon rekognition video, and libraries for Michelin and Frameworks such as if he ingested a from s devices, smartphone, security, cameras, and other sources such as Thrones satellites and ask him. And we can easily build applications with real-time communication capabilities to integration with Amazon, rekognition video, and with machine learning

from books. We can also stream life and we coated media from your penis is feeder streams to your browser on mobile application. Using the key nieces video. Stream live streaming hls. In addition to Dad Amazon Kindle stream supposed to open source. Project website is for two-way. Real-time media streaming between web browsers, mobile application, and connected device with supposed to see you can use Simple, apis to build which applications like video chat with two-way communication between all your applications and connected device.

He is typically used for video chat and peer-to-peer media streaming video streams and where Party City. So, here we are at kinases video, stream transport channels on the left bar to start dealing with her to see if you do streams. And if you're looking to ingest streaming video from the devices, you might want to click on video, streams linstead. As you can see, I have created a sickening Channel called demo webrtc, and let's see what we have. The details best is actually structure to help us navigate through the process, we start with

connectors. Then, once we have built a connection, we can also test on media playback Field section as shown on the screen. And on the bottom side, we have few another tabs to help us getting Channel info and the monitoring. And then also time to leave setting to Define how long should the server pretended messages if they would not telephone? And there's also a nice demo. We can try. So, let's try to stammer. Invisible demo. We have a webcam as Master, which will stream the video and on the right side we have media playback and basically as a fewer you can also get a status

exception to help you check the connection from fewer. So that's not the webcam. And now, that started fewer. I can sit at the phone through the route without significant licensee and we can also hear the audio as well. I left one. And if you like to check the SDK, you can go back to the dashboard and find the link to start the flopping. In various platforms, including for web applications using JavaScript biochemistry, the stream opens a new opportunity for he wants to build a video of a solution that is quick demo. A

real-time video chat with our colleagues Schumann bass in India. This show you a few demos and here are some ideas for you to inspire what you can do with data to me. Locking Jackson you can use chemical services to ingest. Love data from many sources, including dose on the screen plus Amazon. For porch is used for a simple streaming processing, the data to elasticsearch and Athena, and is used for different set of curious as well as over larger time periods. Give me the data. Analytic compute Custom

Critical Matrix includes everything from efforts to Preston's house and Morty to bring a whole new level in sports. For example, by building Senses Into the equipment, you can collect data Rich information, such as speeds distance and speed to turn them into insights. You can use kleenexes data streams and fire hose to English terrier and to classify them and you slammed at the information learning models if you and your team want to learn more on how to build on a SS. You can get started learning with a free digital training

courses, we offer a variety of learning Parts by salvation and by roll to help as far as skills or prevent your team for AWS certification it'll blast at amazon.com training to learn more Thank you for joining us in this action and had some fun together with Genesis. Hopefully you session help you to understand how we can leverage a table surfaces to do data trimming and inspire you to enlist your ID and unlock many possibilities. Thanks again, I'm Johnny and see you next time.