Course  Startup School 2019
July 22, 2019, Mountain View, CA., USA
Course Startup School 2019
Video
Ilya Volodarsky - Analytics for Startups
Available
In cart
Free
Free
Free
Free
Free
Free
Add to favorites
41.48 K
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

I ruin my name is a lien on one of the co-founders of segments and I'm here to talk to you about how to set up analytics and then Linux Foundation to build your MVP and to measure these primary and secondary metrics. So there's going to be a little bit more of a tactical guy that round what tools are there in the Atlantic space in the marketing space. Which one should we actually be using? How do I set them up? Just before I get started a lot of you may not know what time it is. It's an analytics API we can send Alex data and then toggle on different tools. So we send the cool Market

position where we can actually see what tool startups are using and we can bring that information to you and secondly been around for about six years. And so we have information about a lot of different I start up says they grew and what tools they used so we can be sharing a lot of those today cool. So why talk about what we focus on analytics obviously primary and secondary metrics Drive the MVP and product Market process and you're using that to actually test product Market fit. You're also using it once you get out of product Market fit. Strip search to actually Focus the team.

So maybe there's going to be an acquisition issue in the company of preventing your growth or maybe the users of your getting aren't as good as engaged or maybe you're having some monetization issues. And so the final actually is a 14 function to understand your business and we're Founders should be actually spending their time and then finally all the way from, you know to three person team to a Google with 1 million employees. You actually using metrics to operate and drive teams. So eventually you have an engineering team. You have a marketing cement to Wood Goldie's in front

of the marketing team use an Elixir that okay. So today we're going to cover a few different things. So thirsty always start with a funnel when you're thinking about analytics. So that's the sequential series of steps used to go through to actually get value and then pay you as well. Then we can talk about collecting data analytics tools. Then we're going to talk about the top three metrics. So this will include primary and secondary magic tricks that work for most products and in a product Market fit methodology that you can apply on top of that. And I finally will make recommendations about

what tools are the best in the market right now that help the product-market fit Journey. Okay, so we started the final so we'll make an example funnel for Netflix which is a company in the world super familiar with any B2B product or b2c product actually has this type of funnel where you acquire user you engage your user over a. Of time that lupus called retention. And then finally you monetize the user and then metrics both primary and secondary are performance indicators on top of each stage in the final. So I on top of acquisition you can ask yourself how many net users did I

get this week versus last weekend with my growth rate there for engagement you take a cohort of user so from you know, Sunday to the following Monday you have you know, 16 people sign up and then you can track that cohort of users week-over-week and see what percentage of them are still using the product 4 Weeks Later, which is a good example of how to track retention and then we Commoditization, which is how much net new Revenue did I make this week versus last week and then you apply your own custom business funnel to this. So if your Netflix real familiar with a user sign up for

Netflix then day I play videos in a loop Netflix is obviously very sticky watching it a lot. And then finally when the trial Runs Out you do subscription upgraded to get access to more content. Okay. So how do you collect data once you have the spinal so there's analytics apis out there I'm using second is an example. You busy want to say user user one two, three in this case has done user sign-up event and they happen to be an organic user which means you're not invited by someone else then if your Netflix you might say that the user is video play it but eventually subscription

upgraded and so this is how you instrument you tracking and your mobile app or your or your web app, then you think about Advent properties. So imagine you're Netflix and you're holding one of these video plate events in your hands and you're wondering questions about it. So you know what, We are at the Caesar actually playing. How long is the video how far did the person get inside of the video? Right it quietly for holding a subscription upgraded event. You're going to want to drive monetization is a Northstar metric. So if your subscription business you want to send your monthly

recurring Revenue, if you're a transactional business, I can converse a retail you want to send the actual value of the transaction. Okay. So then you push this out into your web after mobile app, then you start seeing the day to come in you look at the bugger you see up at user sign-up is here. Everything looks good. You add your first Analytics tool will use amplitude as a good example here amplitude and makes them are pretty awesome analytics tools on the market right now. And then you start saying data flow inside of one of these analytics tools. So this is amplitude. You can

start seeing user sign ups growing as soon as you launch the real like mobile or web app. Okay. So now do you have an alarm set up it's time to focus on three different metrics. The first one is acquisition metric sign ups for week. It's really nice if your B2B business to cut this by the invite type, so you have organic user to signing up from coming to your website direct sign up and then some users are inviting other users are those are invite type right? So when you think about growth it's really important to go out there again accuser in that case, right another example python

properties are important the way you create this is you go to amplitude you say event segmentation report user sign-up next. Here's your graph. Right? So it's as easy as that just website some data gets the amplitude and then you can see the amount of organic users every week. And then if you're working on the acquisition step as a secondary metric you can Miss USA Today 218 users organically signed up in the last week, but by the end of the month, we want that to be a 300 and we're going to execute projects ab&c this month and they're going to watch this graph every single day on a TV

Sports in our office or apartment where we work and then we're going to see if our efforts are actually driving this right. So that's an idea of data-driven operation of a team. You said a metric and a goal and then you drive towards that everything. Okay. The second one is retention to go for it. So someone recently asked about retention what we'll talk about that right now. So I would retention you want to think about cohorts of users. So you want to say Monday to send a WhatsApp December 10th to 7th December 17th, 16 user signs up and then you look at those 16 users

as they use your product on a week 0 which is their sign up week week 1 to week to week three and week 4 and the general idea here is like you can convince your mom or your grandma to use your product once but even your mom or your grandma won't continue to come back and use your product overtime every single week rights as if you see users that are addicted that are coming back week-over-week. That's a really good sign of product Market fit. So this business can see that the December 10th cohort only 6.25% of those users are still around. Before that's a pretty low about ready. So you

probably want that to be somewhere between twenty or Thirty at least and so you can set a goal say I'm going to talk to a bunch of these Target users and try to figure out you know, why they're not getting value of the product and then make some changes as well actually pick. This is taken from one of Gustav slides. Pretty awesome. You think about what value your company is giving to your users to Airbnb give you value by letting you stay at different rental properties around the world, right? And they want you to do that at least one time of year. Otherwise, they consider you a turned

user equivalently Facebook give you value by letting you look at the news feed and that you connect with your friends and they want you to do that at least daily or monthly once and so when you think about product Market fit you basically have these two different curves that happen. So we have that cohort of 16 users. Sign up in one week and we tracked them overtime. And so what ends up happening is for products a dog product Market fit the end up tending to go to zero because people just don't care about the product to write and that's definition product Market fit. For this tools and

products to do a product Market that you'll see some kind of natural Plateau don't mind us access. It should be somewhere, you know in 20 or 30% at least. Okay. So how do you create this graph seems kind of complicated right? Luckily both mixpanel amplitude have really awesome reports for this so and amplitude it's a retention analysis report. You say users enter the cohort with user sign-up and then they returned with the video player to the subscription upgraded event. That's the value vent. And then you press next and outcomes this graph and then you could look at 4 week retention four

cohorts improve the product and watches You chords that strike the 4-week mark Do Better or Worse, right? And I chose you whether your changes we cover we can actually improving. Okay, finally of Revenue. So this is the primary metrics they want to be thinking about you will use for a subscription business the subscription upgraded event. You'll do a property some over you plan monthly recurring Revenue you press next and outcomes your weekly that your Revenue graph and then you could set monthly goals on top of this to make sure you're growing at the at the rate. So you want to

be okay finally found a team. That's really good to basic put all the stuff on the dashboard and then put this dashboard on a TV in your office is incredibly incredibly important. Basically, this is kind of the difference between the German team in latitude run team. A lot of Founders actually set up their analytics, but then don't look at them ever again because it's cookie can be painful right while data driven team will put it on a TV and talk about project. How can I get this project started changing the metrics that they're trying to drive and then just completely understand the

business every single day, right? And this kind of company instead of powdered Alexi scale to build better high performance companies because the next team Employees, they hire will also be looking at that same TV dashboard to be driven off the same metrics. So really important get a TV. Next. What you want to do is have some kind of social accountability around your metric. So if you have your friends, your parents are advised our investors package up how your business is doing into an email. This helps you synthesize what is actually happening and then send it out to to those advisers

and tell them where the business is struggling and what your plan is to fix it. This allows the advisors to quickly understand the business and then respond back with much more appropriate device. Cool, and now we're going to the start of Stack. So these are tools that we recommend that help this kind of tactical process of setting up these metrics. So I'm going to talk a little bit about that mvp business business workflow that. Michael talked about earlier. So initially, you're building an MVP segment built about seven different MVPs before we actually found segments and all those

failed and eventually we found we found one that worked in the process of actually building the BP is incredibly important. So once you have that little experiment built you want to enter private beta which basically just means getting like 10 20 30 customers to actually try this product and I'm having very direct lines of communication open with them. What segments does nowadays every new product we ship we open Slack Channels with each one of our customers and we have the product managers sit in those slack channels and talk to the customers for the products that don't get product Market

fit because we're just stopped responding and we're asking asking asking if they're not responding, right and for the products to do a product-market said the customers are Immediately being like ouu why don't you have this feature? This is broken. My I tried inviting mighty man is not work not working. So instead of you kind of pulling out of the customer to customer pulling at you. That's that's a good. Feeling a product-market fit at some point. The private beta is going. Well. You feel like people really care about this you understand your Target customer, then you want to get a larger

market segment to use it. That's the launch that we talked about earlier try to get there as quickly as possible. And then more users that you get to test product Market fit on and see if you feel product-market fit there. Then you can start scaling the company ride you hire salespeople, you start doing paid paid marketing and things like that. So different tools will guide you throughout the throughout this process. So as you're building an MVP in your about to give it to the first group of customers install Google analytics install amplitude. Google analytics will tell you who's coming from

the internet to your website and an amplitude will tell you which features of the using how engaged are they with that feature set unless you're able to Over the shoulders of all of your users 20% of the time analytics is the next best alternative for that. We also install live chat on the page. So either slack with your customers or if you can't do that then maybe have a live chat available in the beginning of segment customers would hang us, you know damn night and that's where we got the most valuable feedback from them. So just as many open channels of communication is possible next

ideal warehouse. This is something that we recommend used to be expensive is no longer expensive today. Basically, if you have a non technical co-founder on your team, but want to ask questions around the data and they'll always has the technical co-founder will have to provide the answers. So data warehouse kind of democratize the data not only for the non-technical co-founders, but for everyone else in the company that you hire after company dashboards, obviously, that's the last email in a push tools. So as soon as soon as user sign-up you want to send them

an email. I'll talk about that in a second and then and then I help desk so at some point you have Show me support tickets if you start feeling product-market fit and if they're all going to your Gmail at once and it will just get overwhelmed and not be able to answer them. So you want to have a shared inbox where multiple Founders can I can respond now me to go to a few different recipes of these different tools that we found really helpful and product Market fit. So the first one is improving product usability almost every product that's launched is unusable or highly unusual for

the first three months while you have the cake. So we see this with every single product no matter how much effort we put it into it ahead of time as soon as customers hit Italy start using it in ways that you just don't expect and so there's a school called. Sorry which helps you look at sessions of customers as they use your website. So I'll tell a quick story on this. This is a new feature in Sonos. I was just want to say I was products and we launched in the metrics look horrible. So like customers are coming in but they weren't actually completing it. They weren't they weren't using the

product and we thought I'd God we have to go back to the drawing board. Then one of the designers her team had this amazing idea. Let's look at the full story until we see this user going in about to start this creation workflow. They find this button. They clearly don't understand what the button does they get so frustrated. They just ask the patient and immediately all the metrics got better. And so that's why it's important to have this type of viewing either stand over your customer shoulders, which is great with the stripe co-founders did or get full

story, which is more skill the way to do that. Ok, call Vista 43-minute founder email. So when we launched a segment, we would wait about 43 minutes, and we would email the code email the customer and say hey, I'm Ilia. Thanks so much for signing up for Segment. Your next up here is to add a source to segment. If you have any questions at all, please email me or call me anytime. I'm available for you since we launched that email in 2013. We've had hundreds of Responses to it. It's the connection between you and the customer over email that if they get confused or respond to it. Will you can

use is a tool customer I owe its behavioral email to him which will say why every time user signs up wait, you know, 30 minutes 40 minutes 15 minutes, whatever and then automatically send them this continent you could template the first name of the company name and so forth based off of your analytics data so I can you dress me recommend and then finally offer democratizing data access, I would say this is more advanced. So this is after your MVP stage. You're feeling good about product-market fit. You might want install data warehouse Google bigquery and then mode analytics is a bi

tool that works on top of it. Just lets you ask questions on top of the rod data that you might not be able to do an amplitude of mixpanel. Just any kind of question. You can ask for SQL and then even a non-technical co-founders will eventually pick up sequel in and start asking these questions themselves. Okay, tell your mother we see with customers is trying to pick the perfect tool mixpanel the amplitude, you know bigquery redshift and spending way way way too long thinking about that. The truth of the matter is you shouldn't optimized for picking the right tool right

now both amplitude and expandable give you exactly the same result at your stage instead just get to the decision as quickly as possible, but set yourself up for change in the future. So this is an example of a diagram that shows a customer of segments that use different tools over a period of about three years. And so you can see that they use about a different tools between 2015 and 2017 and later hired someone or they decided that they're tools are no longer doing the job and they switch from one set of tools to another so best-in-class tools change over 2 years just be prepared for

Change and don't spend too much time trying to perfect your choice right now. Okay. This is my recommendation of what tools are the best for the MVP process. I'll walk you through them right now. So Google analytics is for understanding what users are coming to your website amplitude is for feature analytics. Google bigquery is to democratize data access with a data warehouse, which is the database of data mode is the tool used on top of that to ask questions on top of Bakery intercom is like a list of all your customers. They're really good CRM for early-stage folks. Lori is for improving

product usability and custom bios for email your customers. Once you get big enough you can start using Google ads Facebook ads to actually do the pay. So, I'll just jump straight to the slide when we were really young we when we were in 2011/2012 didn't have a lot of money and so we don't want to use all these different tools. And so we use Google spreadsheet for a CRM then use a sod every Trello because they were too expensive. So we use emails and the only things worth We're basically GitHub an AWS to host our product. Right? So today happy to see the second is

not free for early-stage startup. So for all you folks there's a I really like down here though that will send out afterwards the deals and then also segment went out and we did a bunch of deals with these customers so that if your earliest age you can have get all of these tools for free. So enjoy that and start using them to basic accelerate your product marketing process. Thank you. In terms of user base to make the numbers meaningful, I mean 10% of 100.

Central Standard Time versus one you're asking if you're starting with a very small base of users would real 3D need to be on to make to make rose meaningful. It'll be a 10,000 years before you. Yeah, I actually depends on the product red. So if you're an Enterprise B2B company, then you're closing deals for you know, 10-15 $20,000 and only a few of those customers can make I can give you enough Capital to hire more people to do more marketing fire alarm

for sale steam. If you are selling in a shirt and pants, then you probably need higher volume. So totally depends on if it wasn't going to company. I have a question about you not going to bother attention and touch and it shows like the chart of us by week for tracking your attention for my product is more like an Airbnb product its annual Maybe by ennio. Maybe even when our car if you will change Trend in history of Stanley buy it. So the question is what kind of attention. Do you want to look at? If I if you are renting

properties like an air B&B, that's really good question. I really don't care. Depend on the kind of company your building so I'm giving one presentation but there's hundreds of different companies that they need to apply the information slightly differently. So if your Airbnb you probably would expect people to at least come back to your app. Like how often do people travel right that they travel at once a month once a quarter, right? And so that's the that's the. Do you want it and make sure that they come back at? So it's not like a week is correct for everyone, but it's correct

the majority of a company. exactly Awesome. Alright. Well, thank you so much.

Cackle comments for the website

Buy this talk

Access to the talk “Ilya Volodarsky - Analytics for Startups”
Available
In cart
Free
Free
Free
Free
Free
Free

Access to all the recordings of the event

Get access to all videos “Course Startup School 2019”
Available
In cart
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic “Business”?

You might be interested in videos from this event

April 29 - 30, 2020
Online
14
157
b2c, covid-19, customer, ecommerce strategy, gmc, growth, marketing, martech, monetization, seo

Similar talks

Michael Seibel
CEO at Y Combinator
Available
In cart
Free
Free
Free
Free
Free
Free
Adora Cheung
Partner at Y Combinator
Available
In cart
Free
Free
Free
Free
Free
Free
Geoff Ralston
President at Y Combinator
Available
In cart
Free
Free
Free
Free
Free
Free

Buy this video

Video

Access to the talk “Ilya Volodarsky - Analytics for Startups”
Available
In cart
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
558 conferences
22059 speakers
8190 hours of content