Vinamrata Singal is a product manager on the Chrome Web Platform team. She works on Lighthouse, speed metrics, and Google’s overall performance narrative. Previously, she was a product manager on Google’s Next Billion User team, where she worked on building Datally and launching Google Station in Indonesia and Mexico. Vinamrata earned a Bachelor’s Degree in Computer Science from Stanford University.View the profile
Rick is an engineer and developer advocate focusing on web transparency. He maintains the HTTP Archive, a community-owned tool that tracks how the web is built, and the Chrome User Experience Report, a dataset of real user performance data from millions of sites. He is also the co-author of Using WebPageTest and a former web performance engineer at YouTube.View the profile
About the talk
Google has many tools to help developers, product managers, and marketers optimize their web experience. This session will demystify performance metrics, explain what tools should be used for which usecase, and deep dive into the Lighthouse and Chrome User Experience Report tools.
Good afternoon, everyone. Welcome. Thank you. My name is Venom a single. I'm a product manager on the Chrome web platform team. I work on Lighthouse feed metrics and a whole bunch of other Performance Tuning initiatives a Google developer programs engineer on the ecosystem team support in today. Rick and I are here to tell you a little bit about how you can use Lighthouse and the Chrome user experience report to guide the performance of your web application. But before we get started I wanted to tell you a little bit about myself so
you knew so you know who you're talking to so a fun fact about me when I was growing up. I've actually grown up in four different countries India Saudi Arabia New Zealand the United States and what was interesting as I was growing up across these countries was having access to information was very challenging either because it was restricted for me and my family or because no one around has had that information when I started learning how to use the web because they just made it so much easier to have access to that information. So that's why I really care about
making sure the web is available for everyone and it's a great experience for all but even as I was learning how to use the web. There are definitely some pain points without experience. So it's actually a picture of me as a kid. In Saudi Arabia on DSL internet if you remember those days learning how to use the internet and it was really really really slow. And what's interesting is that this loan S&S performance actually hasn't really gotta wait for a lot of users and it's Google we've seen that it has real implications for businesses as well. So we actually have been collecting a whole
bunch of data to show the impact that performance can have for businesses that we did with double click that shows that four sites that load in less than five seconds. They actually experienced a bounce rate that is 53% lower compared to sites that loaded more than five seconds. We have data from kissmetrics that suggests that Shoppers will have trouble with side performing 79% of them just don't come additionally for every second that there's a delay in Pismo time. There's a 7% drop in conversion rate that again for every second that your side is lower you can
7% of your users say that if you are a business that's online. You should care about the foreman and you might say to me like that. I'm right. I get it like I should invest in performance. But what do I do about it? What is the guidance where the tools? What are the metrics you come to the right place? And we hope that's answer these questions and many more to give you a little bit of overview what we talked about today. We're going to start off by giving you a little bit of information is to Performance metrics and then kind of walking through the overall tooling and guidance landscape.
Then Rick is going to take over and tell you a little bit about the current user experience report and then I'll wrap it up with Lighthouse. So let's get started with performance metrics. So when we talk about performance metrics what we're really talking about is collecting data and there's kind of two sources of data you want to collect the first is lab data, which is also known as much work in the lab when you're collecting my data. It's specially for from his data that you were collecting in a control lab like environment and this can be really useful for getting very granular
performance information of your sight II type of data is what we call steal data or metric from the wild or you might have also seen this reference dad ROM measurement or real user metric and idea is your collecting data about how real users are experiencing the performance of your site and this is really helpful in understanding the ground truth of the performance of your sight. Am I supposed to use one or the other but in reality you recommend that you use both of them together. So the first step is that you use feels that up to just like understand your users
understand the Baseline performance. Site where you can just coming from what devices are they on? What networks are there on that really helps? You understand how to calibrate your lab environment. What do you understand the Baseline and you know, whatever business are key performance indicators, you're trying to optimize for then you can set goals around, you know, the targets for whatever you want the performance to be. After you understand that you want to start looking at the lab data calibrated from what you saw in the field, you really drills down and understand what part of the
performance you need to improve as well as implement the optimization to make the site faster. What do you feed information the lab? You want to go back to the field to make sure that you're actually see the improvements in the lab in the wild and then also monitoring the performance of your site for regression. So now that we've spent a whole bunch of time talking about where the performance data comes from. You might be asking what data do I collect and when we think about the performance of a site for really trying to quantify is how does a real user experience the performance of your
site and there's kind of two buckets of those user experience visual metronome. So this is actually measures how fast is stuff getting painted on the screen and then there's the second the second bucket which is interactivity metrics and that's essentially measuring how quickly does your site becomes usable for your user and under these buckets. We have specific Metron individual individual metric section. We had metrics like first concert for pain and speed in that section. We have metrics like time to interact it really understand. What does symmetric mean? I think it's
helpful to visualize them in a timeline and let's go back to the concept of ultimately with performance. We're just trying to measure how to view all users experiences. Very far left you have first contact with pink. So what this means is when the first piece of content starts appearing on your screen could be text image or SVG edit. This is a really pivotal moment and that users. Because this is a point where users like, okay, the site is loading. It's like doing something and you have time to interactive what time for interactive basically for user means that everything is loaded they
can interact with any part of your page and it just is responsive. Is it worse and it's everywhere in the middle you have will be called Steed in that which is actually measures how quickly does the rest of your page load and expense report pages that load a lot of stuff earlier because that's what user it feels very very fast. And we've actually seen a lot of developers have success with his two kids of metric and it go to one of my favorite. So a lot of you might have heard of Pinterest, we actually spent a whole bunch of time with Pinterest helping them upgrade their mobile web experience
to a progressive web app and as a result of this they also wanted to improve the overall interactivity of their experience and they actually improve their time to interactive from 26 to 5.6 seconds. And as a result of this they saw a whole set of improvements across there. So they saw in overall 40% increase in the number of users who spent more than five minutes on their site. They saw a 44% increase in the amount of user-generated AD Revenue. They saw a 50% increase in the amount of AD click-through rate as well as a 60% increase in core engagement.
This is really awesome work, but what's interesting is I think we can still do a lot better to help developers understand interactive experience is only allowed metric. I think we can do a better job of helping developers understand. Interactivity in the wild, how are users first starting to actually understand the attraction for your experience, especially when they come to your app for the first time. I think in order to make this a little bit more concrete, I want to tell you a story. So imagine for a second that you're not
a developer you put your developer has down in your user. So don't pretend that you just heard about this new site. Let's say it's like Narada schoolstore.com and you decide to go in and on your phone and if you know, it's like loading and so you decide to start interacting with it and you click on a but but nothing happens if you keep clicking on this button, but nothing is happening raise your hand if you've ever had that happen to you. Cats like everyone up and keep your heads up if you were really frustrated by that experience. Are you
website had and this is exactly the type of user experience. We want to minimize with a metric called first Infidel introducing for the first time today delay is essentially measuring the latency of the user's first interaction with the page and look it's basically means it's on the time that a user interacts with your case. The time when that happens to the time when the code that you have to respond to that interaction actually run and focusing on the first user experience is really critical because if you saw a couple days ago a lot of users believe it's the first experience is
guidance. So this is kind of an overview of all the tools that we have how this is Google about sex performance and one of the most common questions we got is like Google like you if you put all these tools down but like, how do I navigate this? How do I use this? What school should I be using great question? And ironically we build a tool to help you understand it. But not ironically it's pretty awesome. So this is what we call the speech rules overview is essentially an infographic that helps you understand for which fuse caves weather could be, you know, I just want to build a
business case for performance. So my company invest in it to like I actually want to get really nitty-gritty into performance for all and all that. You say stuff in between help you understand. What will you should be using driving me crazy to check it out and you know, give it to me back about things that you think are missing and we're really looking forward to hearing more from you in a little bit about bullying and Metric. We thought it would be really helpful to buy vintage Tools in particular. Now who's going to tell you a little bit about the from user experience report?
Thank you. Show me talk about metrics from the field most people think of traditional real user measurement or Rum where you measure things like performance page load case reviews and conversion. This is great for understanding the user experience on your own site. But how do we understand the user experience on the web as a whole? How do we know if the weather is getting faster or more usable? How do we know if our site performance is considered normal? For that we need a different set of metrics from the field that represent
the web at Large. The Chrome ux report also known as products helps us understand the state of the web specifically. It's a public collection of what performance data from real Chrome users. Let's see how it works. Do you have a data set that covers the weather at large it enables analyses that were never before possible? For example, you can compare real laser performance across particular sites and you could see how you stack up against competitors in this example. We can see that competitive be is slightly faster than my website. This is a really good motivator for me to care about
wet performance because it's one thing to know how fast my sight is but it's another to see just how much faster or slower it is compared to a competition. Additionally, the Chromecast report is updated monthly so you can track your performance benchmarks overtime as I optimize the performance of my sight I can actually see those games in each iteration of a report. Or alternatively I could see that my competitors investing in performance and I really need to catch up. So what exactly does the Chromecast support measure it includes some of the visual metrics Venom rotten
mentioned earlier like 1st and 1st Continental tank. It also includes traditional Pageland metrics like Dawn content loaded and unloaded. When I'm exploring this data set my go to metric is always first contentful paint up to four metrics that most closely reflects the real user experience of feeling like the page is loading. It also includes useful dimensions for slicing the data like the form factor of the user's device desktop phone tablet. And the effective connection type in Mobile terms for G3 G2G and even offline.
This reports the effective speed not the advertised Network. For example, a user on slow. Wi-Fi would be considered 2G. Another dimension that really users requested was the ability to slice by country with it. You can see how do user experience varies by geography for example can help you understand how much slower your experiences in places farther away from your web servers. There are two ways the date is aggravated by URL and by origin. So how does it origin differ from a URL? It starts with the protocol either insecure or not. Skewer.
Subdomain for example ww.w mail or no subdomain at all. And of course their domain like google.com example.com, which itself include suffixes like.com. Co. Review what An Origin doesn't include is the path after the domain index. HTML product flash 123 performance from all of those pit pads are aggregated. There are approximately 4 million of these origins in the dataset. This is grown 400EX since it was first announced at the Chrome Dev Summit in October and it's still growing each month. Now it's worth mentioning that
there are way more than 4 million Origins on the web. So your website may or may not be included depending on spotfy larity. Okay, you're probably wondering how you start using it? The retina is available on Google bigquery, which is a data warehouse with SQL like syntax. You can write queries to get insights about specific origins or the web as a whole. In this query were calculating the monthly density or proportion of user experiences in which the first Continental paint is less than one second. It's broken down by two hypothetical competitors developers and chrome. Google.com.
The results are definitely informative, but it's kind of hard to understand just by looking at it. What we're trying to understand is which organ is faster and how are they trending? The Hope after that let's visualize the data we can integrate with two Google tools sheets and data Studio You'll notice on the bakery results page. There's a button to export to sheets that will automatically create a new sheet and populate it with the table of data. From there. It's easy to create a simple chart to visualize. The performance data is clear from this chart that developers. Google.com
has a higher density of First Continental pay experience is less than one second. Also notice that the performance fluctuates month-to-month, why would that be it's not just that the sites themselves are getting faster or slower. It could be user population changes as well. For example, the population of users on faster or slower connections could be fluctuating. You can also integrated with Google data Studio whether you can build fully customizable and interactive dashboards and get live updates from the raw date on Pick 3. So for example just asked for the showing one way, you could
monitor the distribution of performance metrics over time. But what if you don't want to work to get this data and you just want high-level results now, there are a few Google tools built on the Chrome to help you get inside quickly. The first is pagespeed insights, which was recently updated to include this really their performance data alongside. Its popular for format optimization recommendations. HP that site is built on the URL aggregated report data. This is more targeted performance data that you can get on big braids and here were looking at the homepage for Google Io. Play Spirits
lights also surface is the median first contentful paint and domcontentloaded metrics as well as an indicator to convey how fast or slow it is in relation to the rest of the web. And it turns out that the aisle webpage is actually considered flow. Pagespeed insights also shows the distribution so you can get a sense for how skewed user experience is really are. Mskcc that most first cousin for paint are fast, but 82% of Dom content loaded experiences are considered slow. Another Google tool built on the Chrome ux report is the speed scorecard.
This tool allows you to enter multiple websites and stack rank their performance. It also enables you to Target specific geographic regions and connection speeds to see how performance various. It also has something I really loved a waited for Elite Performance metrics with business metrics like Revenue. Estimate how much more Revenue could you earn if nothing else changed but site performance? You just use the slider to simulate performance improvements and it's predicted effect on Revenue. Again. This is another really great motivator to invest in performance and not just for
developers. But especially Business Leaders who think in dollars. Last I wanted to share a case study of how the Chrome ux report is being used outside of Google. Is a real user measurement product from Akamai a leading to the end provider. Acting like he's able to use the Chrome ux report to convince potential and pulse customers to care about real user measurement. For example, some companies don't have any rum tools at all and they may be relying on lab testing like webpagetest which only gives them a narrow view of their performance. How can I control these
companies there really is our performance in the Chrome ux report which may be vastly different from their performance in the lab. Best matches of surprised to learn that users experience your site many times slower than your lab test indicate. I can also looking to integrate this data with their temples product. Recalled it with metrics from the web as a whole if possible to see performance of competitors in mpulse for the first time a customer could see competitors real. He's our performance data alongside their own and by leveraging Form Factory Connection in geography Dimensions
site owners are now able to better understand how they Stack Up in specific markets. The trucks from here indicate how differently competing sites perform, especially on mobile. And this is exactly the kind of thing that makes the premiere export. So powerful understanding how your site performance fits into the larger picture of performance on the web as a whole. This is a small taste of the power of the Chrome you export data set and it's only getting better over time their new data sets being released every month. In fact, we just had the April dataset released this week.
We're also looking to add more user Centric performance metrics like first input delay mentioned earlier. I'm working with other partners to get this data integrated into more third-party tools. Headed back to Venom about lighthouse. Thank you, Rick. education class So more audience participation hear your hands, if you ever use my house before the lighthouse is a developer tools than you can run today on your website and it basically gives you personalized advice about what you're doing well and what you can do better across five different categories for Foreman
forgot to buy bathroom pwas accessibility best practices and search engine optimization for SEO today can red Lighthouse through four different ways. You can run it as a standalone Lighthouse Chrome extension. You can run it on the audit panel Chrome developer tools. You can run it as an npm module and this is the npm command to install it if you choose to What to take a step back and tell you a little bit about the journey that you know, he's kind of Taken on the on the product side give you a little bit of context of where we're going. So this is a screenshot of
Paul Irish. I was at chocolate on Lighthouse announcing the project at Chrome developers summit understand that there is a lot of advice kind of front end developers like everyday about like you think that you should be doing to create really good life experiences and really wanted it to be like the One Stop Shop that you have to go to school as comprehensive as the wheel actually launched over a hundred on it into the report including a whole new section focus on search engine optimization this year. Have you seen a lot of really great
adoptions in the community 2018 alone. We seen over 500,000 user running over 2 million plus on it, which is great to hear stories from developers all the time about all the stuff that they're having with Lighthouse someone to share you with you one of my favorite. So this is Milan Milan is a fashion e-commerce retailers based out of Haida and they actually use Lighthouse when they were making performance Improvement to their site as well as building a progressive web app Lighthouse pwa score 557 points to
110 Lighthouse performance for 5:35 points to a 75 and as a result of dressmaking these two Improvement. They were able to see a 10% increase in their mobile conversion rate as well as a 65% decrease in their Advance, right so pretty awesome stuff, but we also know that on the lighthouse team there still a lot we how to make the product even better for y'all to use So like a sunrise, this is like our top user request the first user request we get especially from people who are looking at the Lighthouse report for the first time. Can you
make this report easier to understand cuz it's like kind of cluttered and kind of confusing. That's one request. We did, you know, I was lighthouse but the performance metrics then scores change for brunch, right? Even if I'm running at the same way. Can you make Lighthouse more stable? And the third is you know, I love Lighthouse awesome, but I don't want to just run it on one side. I want to be able to run it on hundreds and thousands of sites. Can I conduct a full analysis? You think Lighthouse and today? I'm going to walk you through how you can do all three of those
things when you stuff that we've been working on get started with the first one about how to make a report easier to understand his concern pretty seriously and we actually did a tire usability study in the earlier part of this year results from that study the mobile web sandbox if you try using my house there but my personal favorite you feature of this you by which unfortunately you can't see from the screenshot is when you hover over all the metric there's like a little tooltip that comes up and if you
don't give you more information about the metric, so if you've never heard of life for CPU idle, you can kind of get a better understanding of what it is. And this new you why is actually launching with 3.0 where we're planning on releasing this week. You can get 3.0 today in p.m. It'll be the default way to run a lighthouse on the Chrome extension, which we'll be releasing later this week and it will be coming to decimals in from 68. If you want more information about everything else that's coming in 3.0. You can check out.. Over there. I'll be walking through
some more updates, but that. Has all the information. So let's move on to talking about the second concern that. And have Lighthouse which is around fee for form. So these are actually scores that I got for website that I will not name for their performance score with their site and it was just two consecutive runs one after the other problem that we've actually known for a very long time and theirs if we had a concerted effort since last year to work on it. So we're calling Clash of Clan turn in my opinion. It has the best logo out of all the logos that we've ever created. So you
understand what Lantern does that's different. I want to tell you a little bit about how Lighthouse computer performance metrics today is Computing the performance metrics of your sight. What it's doing is actually swaddling your page trying to still trying to emulate it as if it was a user on a mobile device on a 3-g network and then Computing the performance metrics accordingly what Lantern does is basically completely we change the way we do this instead it tries to estimate based on input to give it about what network and what device do users on to compute the performance metrics
itself and using this approach actually been able to see a 50% reduction in the amount of variability in the lighthouse performance for while maintaining accuracy with what you get with traditional Devil's rattling in addition. As a side phone has also reduce the amount of time. It takes to run and warm up Lighthouse by 50% for a lot of people So what question you might be asking is like how does like lanterns do this stuff? And I'm going to give you a very high broad level walkthrough of Whitehall Lantern does what it does. So the first thing that Lantern do we get to
as knowing the size of all the acids are required to transfer to be transferred in order to render your site Lantern content predict all the different performance metrics and on the bottom. You can see, you know, all the different things that need to occur in order to open the network connection are also taking into account and that's kind of very high level of color starting with 3.0 it so if you want to get started on understanding Lantern as well as using it, you can just get 3.0 with the way that I showed you earlier. Don't know what you're
talking about still Lighthouse and I'm happy to announce that we've been working on an API to the lighthouse API API that allows you to get Lighthouse results for anywhere else, and I'm really excited because I think I lost a couple of really important use cases that people have been talking to us about the person is it allows you to build dashboards? You can track not just your homepage or letting page but you can try all of the different type faces that you might have second is, you know, you can build tools on top of Lighthouse as well as Enterprise Lighthouse
into whatever continuous integration tests me that you might have in a final you can conduct large-scale data analysis using Lighthouse itself. This is an overview of what the ATI currently looks like. And for those of you who use lighthouse on the command line, you can see it's pretty similar to what the command line would give you except. We've made it a little bit more ergonomic for people to use. The current status of the API that is in private beta with currently been working with a couple of Partners and regenerate on a definition if you were interested or intrigued by the API, I'll
have more information later on in the talk about how you can get my listed for axes for planning on launching it as a free public pool service very very soon. So I want to take a little bit of time to tell you why I'm personally really excited about the API. What are the states that I was mentioning earlier, but you know, we have all these tools and we're trying to align or to link story together, but one of the problems that we've had in that if we don't really have a common infrastructure that all the tools can kind of rely upon in order to align the tools and I really think that
light has a p I guess it's one step closer because it's kind of creative pipeline that a bunch of other food can use in order to line all the tools together. And so I'm sure you'll be seeing a lot more chilling alignment initiatives in the future. So now you might be wondering is an external developer like what can Lighthouse do for me? So I want to tell you a little bit about what you were Partners did but the lighthouse API so the first one is mobile v logo, you can see over there a platform for retailers to build customer first shopping experiences through Progressive web apps and
accelerated mobile Pages some of their customers include Lancome living Pulitzer and car park and as far as platforms they have this product called the analytics dashboard, which is actually makes it easier for a Business Leaders at these retailers to monitor performance and customer engagement in order to make this process easier on Modern Performance in customer engagement. They actually integrated into the dashboard and this is what it looks like. So this really helps Business Leaders kind of get a better understanding of how the performance metrics the lighthouse is reporting can be
correlated to visit mattress. They might care about in this case is actually alive example of how one of them over by the client. Which is the leading British retailer actual Actually uses the data provided by the API to be able to correlate the scores with bounce rate, which is a metric that they care about and I'm really excited for this new skates, because I think it's really helped Business Leaders understand the return of investment that they're getting for putting work into performance and just makes it that much more obvious. So the second partner that I want to talk about it stood
up which is the website builder for what profession with an agency there currently over 12 million sites that are bills using two down and really wants to make sure that whatever sites are being built that they're fat and especially as it as you functionality to the platform and so they're actually using Lighthouse to add forgot to buy that functionality the platform, but now with the API the spot is becomes really easy because they're just able to integrate Lighthouse into their continuous integration pipeline. So as you can see to do their continuous integration tests, and as you
can see over here, they just added lighthouses one of the components and their if they're able to test specials for the developers so that you know, if it doesn't meet the threshold they failed to build and his Gates a build a build and they're able to kind of dig into the report to understand what artists are passing what on its are feeling. And again, I think this a lot of use case that we've heard from a lot of developers with Lighthouse, which is once you do all the work in the Time investment to make your side fast to make it a pwa then how do you keep it that way? Because you can
just get it braided into whatever continuous integration system you have any plans that wants to apis publicly launch new releases plug into the entire Jenkins community. So that check is it's that much easier to get started Lighthouse. I really want to thank you for all the work. We did it together. They're super awesome and I'm even so much more excited to see what you do with this API. So if you're intrigued by any of the demo that we just shared do you want to get an early access? They see I can sign up on this interest form over here. And a member of our team will be in touch with
you very very shortly. Recognize talked about a lot of stuff and I want to wrap it up with some key takeaways, which there's only one but me to take away. I hope that your dad's is that there's two kinds of performance data that we care about their data and field data. The lab data is essentially control C control really helpful for drilling down into performing specific performance information and getting granular and needy greedy is really useful if you want to understand the source of truth about the performance if you stay because it's
essentially data you collect in order to understand how a real users are experiencing the performance of your sight Lighthouse of the tools that can give you lab data and experience report that Vic talked about is it Is the tool that can give you feel data manager wouldn't be complete without any action items for you all to take four of them out. So this is a really good time to take out your phone and take pictures cuz I'm going to try to go to this quickly. So the first action item we have is about first input delay, which is our main interactivity metric in the wild. So if you're
interested or intrigued about improving a real users are arguing with each activity of your website, I heard you to check person for delay. There's a link to the doctor cuz it looks in the polyfill. Please give us feedback. The second is around the speed Schools overview. This is a tool that can really help you understand what tools you should be using when thinking about the performance of your site. Finally, we have the Chrome user experience before that Drake talks about if you want to understand how to Baseline your performance statistics do competitor analysis and all these
other really cool use cases. Check out Chrome user experience before it really cool tool always making a better have Lighthouse Lighthouse. You can run on your browser if you use Chrome and you can also look into using the lighthouse API for data using light house of scale hairs the early interest form. That's it. Thank you so much everyone for coming. I know it's like 5 p.m. On a Wednesday the really appreciate all your time. So we are going to stick around for a little bit if you have additional questions, but you can also find out that Stan boss.
Buy this talk
Access to all the recordings of the event
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.