Tuning Your Cloud: Improve Global Network Performance for Applications -Level 300 (United States)

Hi. My name is Richard Wade, and I'm a principal Cloud. Architect with Professional Services. Is a WIC in Singapore. One of my areas were interested, performance, engineering, and especially Network optimization. So, today I would like to talk about some of the things that you can do to improve the overall, the night with performance of your applications on AWS Solutions, but overall, it is designed for understanding systems working in order to understand how we can improve the performance of our applications. We need

to First understand some of the underlying mechanisms which influence things such as response time and speed of data transfer. He probably heard of the transmission control protocol TCP underwear that your applications. Use it for the reliable transfer data. Between your way, wi Services application, service on your customers, all uses in this session on how to say p functions at a low level and how we can engineer solutions to maximize its performance. We just understanding of TCP and aspects of networking, which directly affect her applications performance, will then look

at our AWS service architecture. Specifically how we can use a wosh services collection design, the right infrastructure to improve overall performance, finally with an understanding of TCP and having selected the right infrastructure to deliver our services will take a brief Deep dive into how we can tweak and tune verification service to get the highest three American System performance. Before we can understand how the TCP protocol affect your applications connections, we must first understand how we can categorize these connections and how they differ in their

requirements of the application server. Operating systems the network on a roll. Cloud architecture. Yes, this isn't a joke. One of the ways we can classify, JCP flows is by their relatives of the name. My price was given to those small flowers with transfer, a small number of fights may be in the tens or hundreds of kilobytes. The vast majority of connections on the internet are small and this means that they will so short-lived starting and stopping quickly in just a few seconds. Even though Hastings B2 allows us to Multiplex, I'll chat request onto a single Hastings, Peak

connection and TCP flow. We still have many web applications, which opened a very large number of TCP sockets in order to communicate to service on cloud platforms. This means that the single application If we think about show, Live Connections, Wichita transit, fare in a comparatively low, number of fights. We can imagine that they would benefit from Catching the established quickly and achieving maximum throughput in a very short period of time. We can also imagine that if I show connection is disrupted by slow networks, lossy networks and having to

slow down or we transmit that. When considering a large-scale, it results in a highly inefficient service anniversary Pro user experience and cold. Smoke connections, mice, a common term for Lord connections is elephants. The thing tonight is that while the majority of the number connections on the internet, a mice, the majority of the volume transferred is an elephant flows as you can imagine very long-lived and transfer, a large volume of device. This could be in the hundreds of megabytes or gigabytes or even terabytes. Search Lowe's to perform well, that require an efficient

network connection, which will give them a faccia of the available bandwidth on these Long Live Connections. And we will now look at how to City Works in order to understand why this is so that we understand what we can do to improve application performance. We'll take a brief Deep dive into how to see if he establishes. Connections accelerates the right to transfer a response to recover from pocket. Firstly, let's look at connections stock up or how long it takes for TCP to establish a network connection. You may already be aware that TCP protocol, provide, reliable delivery. What this means

is the average packet of data that you transmitted between a CCP client. And server is guaranteed to arrive because each packet has a sequence number which is acknowledged by the receiver. Once it arrives in order to establish a connection with the server TCP performs, what is commonly known as three-way handshake this consist of the sequence you can see on the screen, the same synagogue Firstly the client with cooking oil uses application or web browser, sends a TCP synchronized bucket. To the server this packet is acknowledged by the server and the

client technology is the final judgment is complete. The TCP connection moves into the established date or web browser. Can then proceed to send his application letter request such as a haystack, TV cast for remote, procedure called, the important thing here is between the client and the server. And this will also be very important to note the connection formance is the latency between the clients in the server's high. For example, in this case, we have 50 milliseconds one-way latency your 100 second round trip time. Then this three-way handshake can take some time

in the region of 150 to 200 milliseconds. This is before the application can start. Send his request and begin responding to these attractions. Can see how responsive application is very dependent on the speed taking place. And that for the round trip time between your uses on the application server. Now we can look at what happened between soba and the client. In this phase, TCP is trying to quickly discover the network paucity of viable on the path between the client in the server. It does this by

doubling the number of packets, it transmits. Each time it receives a successful acknowledgements for the ones that is already sent. Looking at the diagram, you can see that it starts off with one pocket when this is successful acknowledged pocket. And so on. TCP keep track of the number of packets. It is ending at any one time with a variable known as to send the congestion window or cwnd the key point to bear in mind here is that if you only need to transfer 100 kg to a 200 Visa connection

speed, this is a significant problem for mice but not for an elephant connection, which spends only a fraction of its time. Install Water for Elephants the steady-state performance is much more important. So what happens when CCP which has a capacity of the path between the use of a new server? Well, normally this results in one or more Pockets getting dropped or lost in transit which means that the receiver will only acknowledge packet sequence numbers to the point of loss. When

this happens to additional TCP will reduce its transmission rates by Harvey the conductor window or transmission, right, evidently for sure label, nice connections. This has a dramatic effect on the efficiency of the connection and to Longleaf connections as we will discuss. Now it can also have a big impact over the course of the connections. Life After experiencing a loss or timeout traditional TCP answers what is known as the congestion avoidance phase and is common during study State. This means that the TCP algorithm it now trying to avoid the packet loss of experience while rapidly

increasing its transmission rates in the start-up phase. It does this by incrementally expanding the congestion window, just adding one additional pocket to it. Each time it was a successful acknowledgements so we can see in the graph exponential growth and linear growth. During the congestion avoidance phase which takes place after packet loss on the connection has discovered the capacity of its network path. Again, different algorithms do things differently to measure. I want to be the only given internet path. However, if

we take traditional example, we can see how slow recovery time be from packet loss. If an elephant connection has to have its transmission rate and then slowly increment back to where it was so beaten gigabits-per-second. You can see how this can drastically affect the performance of an application over his lifetime. In fact, this graph was presented in the network for men session at re invent 2017. What date was collected to show how quickly the throughput of TCP connection drops as a percentage of the Lost increases. You can see that it's 0% loss. We are achieving 100%

of the possible through blood but it just 1% packet loss. We are 50% at 2% we're at 20% for months and so on this packet loss has to be sufficient. When combined with round-trip time, you can begin to understand how high latency. Then affects the time. It take to recover from loss. The main takeaway for this section is the TCP is important to your applications and the latency and packet loss or importance TCP with the vast majority of your applications. Using TCP is important to understand how did protocol fundamentally works and how the

responsiveness and performance of your applications is completely dependent upon it. High latency will affect how quickly you use as can connect to affrication how quickly they can send requests and actions to his user interface and how quickly they two objects can be returned to them. In the case of large transfers, there is the formula for the bandwidth, still a product, which describes precisely the maximum potential Freeport you can achieve using TCP between 7 and client, given a certain time. I'm available Network bandwidth and there is a link. So then explanation of this formula at the

foot of the slide developer or systems operator, there are three things that You can do to directly influence the performance of your card. Applications number one, you can design a cloud architecture which takes advantage of the features facilities and infrastructure available. This might be silly and regions close to your uses or it could be leveraging. Some of the managed Services AWS has to offer. Number two, when it comes to choosing between the detailed options available to you when design and construction field high platform, you can choose specific instance, types or

Edge services to increase the throughput or in certain cases, Place High Performance Edge compute. Closest, you'll use has finally number 3, if you fully understand your application, your network environments, and your user Behavior, you can begin to chew on your roof structure to suit, the application, the infrastructure, and your beautiful hide. With number one, Bruce Lee. Let's talk about how you can design a Nile to texture. To get your application closer to your uses using AWS activist has what do you use as a globally, distributed or reside within the given Regional

continent is the Amazon cloudfront content. Distribution network has 216 points of presence around the world. In these locations, we can position and deliver stuff. It comes in, I'll get it uses and even carry out intelligence functions at the edge using land or touch with regard to TCP. Let's see how hard from helps. Reduce the latency and improve the connection full minutes to. I use us. Let's consider an example where we have an origin server, which has a round trip, time of 150 million seconds for my uses assuming one and a half to two round trips to fully establish

a connection. It means that every new connection is in the region of $300 before the first application requests. It received by the server, similarly consider, how is that round trip time will affect the clogging of TCP start-up phase, and a full? How quickly connections will catch up to full speed. So how does a cloud print help with the situation? Well, in many cases can reduce the round-trip times. You'll uses to 30 milliseconds or less, which is a significant Improvement. This means that's an application request could be sent to a service within 6300

milliseconds without the cash. Generally speaking, the way that accountant delivery cash works is the static objects are pulled from the designated origin server. The first time they requested, so we can see here that the first you would have the same experience as if there was no cashing place, the request would be made to the designated. You are out Witcher. Rise of the cloudfront at Splash the cash into in that she's not content from the origin and returns it to the cash, which returns it to the user. However,

uses bringing the round-trip time for Content, responses down to 30 milliseconds, we can see in this diagram, how the first user experience, the full 30 +. 120 time, accountant is patch from the origin server. But the subsequent uses of just experienced a 30 second round trip July and requesting content directly from the cash. This results in a significantly more responsive service and much more rapid delivery of content. STC pecan. Firstly, establish

connections much more quickly and secondly, scale its condition window that has the added advantage of having issues in that weird path to the user which results in lower probability of congestion and packet loss overrule. This means that small application objects can be very efficiently and quickly delivered from the conference badge cash when it comes to larger objects, there's similar benefits due to the lower round-trip time, between the uses and the actual patient, the transmission, an acknowledgement mechanism, will function much more efficiently which results in the fastest

Android phase as well as fast as looking when in steady-state and therefore higher potential tree. But when we consider the theoretical maximum of the band Westlife. Reducing the latency between your applications. And New Uses is one part of the equation. The other part is giving her applications the necessary potential, throughput to require applications and connections between a web service and uses distributed across the internet. In many cases have more complex applications, which require high performance, TCP connections, between both local and remote

instances. With very high pockets II Freeport Firstly, dog attack should be aware that 9k jumbo frames are available within 50 sees and to be PCP is this means that each package but it puts down to the wire by your recent to insta. Tan. Bao Thanh nien K in size and I provided a link on the slide which will take you to the documentation on AWS to amazon.com which explains it more detail how to make use of this feature. The poops is at this discussion, we're interested firstly that you're able to enable jumbo frames. But also if we do need very high pockets report for

applications such as clustering and high performance, Computing infrastructure design options which can help deliver this report we need. Synergy Willis, hopefully head of the AWS Nitro System. Since it was announced that AWS reinvent in 2017. However, for those who want to wear what Nitro is, I'll give a brief overview in summary. The AWS Nitrous system is a current generation of a hardware and hypervisor virtualization platform for AWS ec2 services, and aims, to provide performance that is close to ruining your applications on batmetal

by optimizing. It does this using the very same KTM based had devised, a liar, and a dedicated purpose-built which combined storage, networking management, monitoring & Security station results, in three things that we are very interested in for engineering application performance. Improve to improve latency and improve packets per second, in these comparative measurements published by the AWS Enterprise strategy group, Nitro BAE Systems provide meaningful, improvements in performance across the board in

terms of 3 ft per second. Internet latency dance, 27 microseconds and capacity for over five million packets per second versus 1 million for the closest measure. Capacitor types for comparison, we have to teach three instances, show me the top left and standard see five instances. Shown in the top right of the Boston, highlighted in Orange. You can see the C5 end with n for Nitro and the relative performance metrics, with instance types capable of supporting of the 25.

Gigabit-per-second just ended and sustained fifty or a hundred degrees per second. But what if you want High Network performance in very close proximity to your uses. What if you need to deploy a workloads, which require low latency access to on-premise systems to do, local data processing, what you access, local date, storage Services apis, I'm tools today to sensors location spaces, an on-premises facilities, close to your uses applications systems and data on

the same. Not true hardware that you run in your age of us region-based ec2 instances. So it's just a C5. And I'm come be linked to AWS account in regions across the u.s. Euro southeast Asia. And I answered all deployed to The Outpost in the same console in apis, that you use today with standard AWS regions. In the section, we discussed how you can improve Network. Performance proposed, region-based workloads, such as those in clusters, will be PCS as well as those based on New Age services. So that way, we can take that same high

performance Computing, and networking infrastructure and care located with an on-premise applications data or uses in order to significantly, optimize the architecture. With this in mind, think about how a significant reduction in round trip time down to the single-digit milliseconds. Combined with optimized, networking Hardware can improve the responsiveness TCP Shreveport and transfer rights of your applications should not degree of performance be required. Finally, let's look at how we can use our knowledge of TCP performance to tune and optimize our Cloud infrastructure. In this final

section will use an example multi-region, ec2 deployment to highlight some of the characteristics of different TCP implementations and some of the colonel tuning options available which may or may not improve the performance of your applications. I say may not because it's very important to understand that there is no one recommended set of parameters which is guaranteed to improve altercation types or traffic happens. If it'll be that way for a reason which is to ensure stable balance level of performance, the

widest possible range of applications without causing problems for the civil rights of the application. Use it, with a network administrator has it. If you do choose to experiments into new systems, please do so safely in non production environments and in a controlled manner. With that in mind, let's look at experimental setup for these tests. I created to bpc's one in u.s. East one and one in AP. Southeast one in each PPC, I created a C5. N9ne Lodge instance, using standard Amazon Linux to. I am I and I paid the V PCS

in AP Southeast one. I have an Amazon page, make a note book, which I will use to analyze data written to an S3 bucket by the performance measurement. Software. Iperf3 is a commonly used tool to measure Network, performance, between to host, and it's popular in the network operations and see CP performance Community. We run it with him on his ass flag on the server side, and on the client side, we used to come on, shown on the slide. We provide the server IP address to the minus a flag. That's a traffics destination. In this case, we're going to transfer one gigabyte of data, and

I'm asking it to Output. A Json formatted log entry, every 200 milliseconds, with information about the TCP connection and data. Transfer, in progress, note that are on 220 milliseconds. This means we should get a log entry. Roughly every round trip time. So what are some key TCP values? Which we can easily chune in the Linux kernel. On this slide, you can see the second name, the default value provided in the Amazon Linux 2, a.m. I cuddle, and some chew and values, which I developed based on my other work with AWS customers. In this experiment will chewing the TCP socket Buffet values. In order

to increase the amount of memory assigned to things like the TCP socket office. This has the effect of taking the handbrake of PCP and allowing the congestion window to open up to size is beyond, what is permitted by the default settings, option, which lists, and alternative TCP congestion control, algorithms, the congestion control algorithms is the code, which tells TCP how to behave during start-up phase, as it tries to rapidly expand. Its congestion window and then which metrics and how to behave and its steady state. And congestion avoiding spaces is a default Linux

kernel to be all over them. You may also wish to experiment with a newer algorithm Uncle TC pbbr which can give better performance under certain conditions due to the limited time available for this experiment will only be tested with BBQ bacon and adjusting the colonel values for TCP socket buffers. I've also provided some references at the bottom of the slide which may be of interest. If you want to learn more about different tuning options for the Linux kernel, a reference, I've included the slide, which shows an example of how you make configure the experimental setting shown on

the previous slide. As I said before, please do be aware that. These things may not be suitable for your website or application. Please do so in a test environment, This side shows how to speak Arabic congestion window behaved, when transferring 1 GB of data from AP Southeast one, to u.s. East won the default settings, the transfer truck around 150 seconds. I was very stable. Give me some flat line across the lifespan of the connection. However, after tuning, it took around 90 seconds. This shows that we could expect a thirty to forty

percent form its Improvement in this particular scenario this Rose for around fifty five megabits per second to megabytes per second as a result of chewning. What we can also see from the graph that with the tune settings, the congestion window was much more active trying to expand it transmission, right? But responding to signals of congestion in the network. In practice, this means that you and Colonel were trying to be more aggressive in his transmission. And as a result, didn't come to packets

around 1,000 packets were lost by the tuna Capital mostly during the initial startup phase. 220 package lost with a default settings. Overall this experiment shows that we can use Simple TCP training techniques on our ec2 instances to improve the potential throughput of our applications at the possible. Expense of increased, give me Transmissions to Salon Lofts. In this case, we going to go to Improvement but it's worth reading around the subject and watching some of the relevant reinvent sessions such as the one

linked to Alia to get some ideas which might work for your particular environment in traffic profiles. Chicken food in this session, we've shown how you can improve the performance of your client applications by adopting architectures services and infrastructure options, which reduce latency and thereby, increase responsiveness and Freeport. We've also showing how you can feel that you and your infrastructure for stability or performance by using Colonel tuning techniques, thank you very much for your attention today and I hope you find the session useful and interesting.