Yigit leads the Architecture Components for Android, focusing on the developer experience and dreams of making app development easy. Previously, he was working on Data Binding and RecyclerView in the Android Toolkit Team. Prior to Google, he was the Android engineering manager at Path.com. He received his B.S. in Computer Engineering from Middle East Technical University / Turkey.View the profile
Chris Craik works on Architecture Components and performance in the Android Toolkit team.View the profile
About the talk
An introduction to the Paging Library in Android Arch Components. This session will go through local (with Room library) and remote storage (with Retrofit) examples, explain the design, and how it fits together with other architecture components.
Hey young Chris Craig from the Android framework team. My name is Eve. I also work and Android framework team. And today. We are going to talk about how to handle large list with Beijing and recycling. Ball is yours a very common thing in Android applications pretty much every single app has a list of something they want to display. So if you're using a Xavier wood Model A List like this. Single hair review Moto by your keep the list of items so that it survives configuration change your
butt. Of course, you have some server-side component by you pull the data from but usually beta is dynamic. It changes changes. You'll ask database to return your lie, they told list of things or a list of things so that you've been had those updates. For instance, if you pulled the updated version of two records from your database and you insert it and then room realizes there's a creep I'm serving the stable. Let me review of the result of the remodel from the remodel at update yourself with the correct
animations. This is cool. But there is actually a problem here. There's something we don't like to better understand the issue. Let's look at the interaction between the database under you. I never mentioned that they tell me this is going to review of the result it is going to reveal all of the results or if you're asked for the users or their by their last name and if you have 10,000 fusers, it's going to create ten thousand users. Why because you Seems like
8 to 10 items. Why would you create 10,000 items so we don't like it. We want to change this. So what are we looking for here? First? We like that school radius is so nice to tell the database. Give me a live date of list of users. We want the same convenience. If you wanted to handle multiple layers want to be able to bring data from the server put in the database and put in the yard and it should be very easy to implement. We want it to be fast. We don't want to do any unnecessary work. We don't want to do any big chunk of burgundy white red. It
should be efficient. We wanted to be life cycle are so if user is not looking at the screen, it shouldn't do any work last but not least. We wanted to be flexible. Everybody has their different apis different data structures or solution should work with all of these things. Know if you go back to our first example, how do we implement this list? If you don't use patient lies, they told police officers. But you're in your email. You hold a reference to eat and serve it to your UI. Your activity is
the school. See you may not heard about it. Let's focus on that one. Mr. There is a recyclable adapter that displays the list. So if you have the recycling of you on the right when you called the something in this method on that after we just displays it the nice thing is if you call the same song as this method with a different list is going to calculate the difference between these two lists on a background track an update with the correct information is available in recyclerview of 27 points
in the air if you cannot extend the adapter. Okay, let's go Victory example we have over there after we all observe the live data. Listen to listen to the Raptor. Soho by me to slice of adapter we need to give it a call back. It's us to function of this first one. It can check if two items are equal. I don't play another one checks whether their contents are equal or not so we can decide whether we need to re-buy those bees or not. What's your heart that you're on
buying? We were older methods you can call to get tighter function open the item in the list and do whatever you want with it. Super simple. Alright, so we see what that looks like with a live data of list, but you you also started out with this isn't good enough. So let's look at what this looks like when you're using a new paging Library. So first of all, the most important component of the pipe for the paging library is the page lists and this is a list that Lincoln is the listed or face, but it loads data in pages asynchronously. It's backed by a class called a data source that
provides data as the page Bliss needs it swapped it in. And updates the list. So this serves as a replacement for the list in all the examples that we done before so let's go ahead and start swapping it out. We have this new model. Let's swap out the list with a page list. And we go back to work. I don't know what we want to do here. If you want to ask our dial. Hey, can you give me a live data of a page Bliss but is actually an issue with this. So if we look over it what the data looks like on the database side, you could imagine paging this in
one way and you could imagine taking us in a different way. There's actually some interesting decision here. Are you showing really large items? Are you showing tiny items were you want to have a lot of data page? That one's well the page size really needs to be configurable to you know, serve every ass needs. So instead we can produce this data source class that backs of page lists, and that way the page lists can access it and say give me Autumn 6 through 12 and load those data directly into the page number cuz we say that we want multiple of them though. Remember that we want something
that's observable. So we just wrapped that in a factory every time that we need a new page list out of the database. We create a new data source. All right. So instead of the live data page lists what the user now provides us here is a data source Factory heat off of integer because we're using positions. Do we go back to ord our viewmodel here and Define? How are we going to live this day that will I get the data source Factory that we just declared in our use your towel and use this live page list Builder class from the library to create a live
data the minimum amount of data that you need to pass here is paid size and here we're passing 30. That's it. Now you have a live data on Page Plus. So now all of a sudden changes are done the view models updated. Let's go to the activity. So now we're using listed after before. Well, simply you change this into a page with adaptor adaptor is the exact same thing as listed after the pathan to handle. The loading of pages as continent is internally updating in those page list. So if we look at how we'd actually implement this list adapter, though.
We have to switch over to the new page list adapter and use the exact same just go back. We don't have to change any code. They're the only thing that changes is part of implementing this on the adapter side is that the user object becomes nullable and we'll get into why in just a little bit. So what's going on how you can do further configuration? Because we don't stop at just Pages size. So the Dakota we showed before of creating a life page list Builder. You can simply just passed a minimum page a page size. And what's the minimum amount of day that you need to pass but you can also
create a configuration object where you can declare more. So one thing that you might want to do is set an initial load size pants and what this does is it request that the initial load is larger to make that initial load avoid initial page stretches immediately after you've touched it. Another thing that you can changes prefetch Distance by default. This is the page size that you can configure this if you know that your data sources higher or lower latency. You can also control placeholders which are unimportant part of this Library. So let's let's start talking about those so
that you might have the expectation that you might have work. Typically you load the first page to recyclerview has access to the number of items that you've loaded and you see all you have a big scroll bar, right? The user can scroll through it. And once they get to the bottom, there's no more data. Once the page loads the scroll bar jumps and the user is able to scroll forward. So we also differently if you ask for placeholders in this is the default in the library, we will present your lips like this, you know, the true that
the scrollbar is a little bit smaller and that's because we're presenting the entire data set to the recyclerview immediately jump over scrolling as the user Scrolls down. You see that there are unloaded items and those represented as Knowles and the adapter these items as the data eventually load. Display and you'll get the nice animation for that. So let's talk about placeholders. They don't work everywhere. But we think they're really useful in a lot of cases. So what are the big positives of them is if the user control past what's loaded? The user doesn't have to hit a block at the very
end because you don't have any work today yet. The scrollbar looks correct instantly and also this is very important. You can use fast rollers very easily because you have the entire data set presented to the recyclerview so you can jump anywhere Another nice feature is that you don't have to implement a loading spinner at the bottom because the users can see the placeholder. They can see that there's like a gray icon instead of a user icon and know that that item is still loading. There are some downsides those plates holders. First of all, it's pretty important that your items. Stay the
same size. If you can't guess what the item height is going to be before you have content. For instance. If you have say a social media post an arbitrary amount of texts, the Crossfade animation doesn't look great. I just want your adapter has to be able to handle no items. So that's a little bit more code that you have to write to say if this item doesn't exist past the text you that kind of thing. And then your data source has to be able to count items now room count atoms out of the box. But if you're using something that is loading data from say the network you're back in may not be
able to provide a precise count. So we've been talking a lot about our of life date of this bus far but as he introduced at the beginning, there's whealton want to support our rxjava developers. So instead of producing a live day that you may want to produce a unobservable and we provide the class RX page list Builder to do exactly that. So you can change the return type here using RX page list Builder and then just specifically request an observable or a flowable out of it. And you get all the same behavior reviews all the same code that we were we were doing before with a
little bit of change to to switch to RX observation. Now let's go under the hood because he's shown how small of a code changes can be to switch the page in library. But how does that work? Underneath. Okay. So here on the left you have the repository that represents the data loading portion of your application. And then you on the right you have the view model, which is how it communicates to the UI. So inside the repository we want to build the stores for this live data. We want to produce something that will push updates to the UI. So when you call Life page list Builder. Build, we
create not only a live data that can be used to funnel information to the UI but the producer side as well. When someone starts observing that live data, we will create a new page list because that's the that's the that's the way that we start passing information down this Pipeline and to do that. We create a new data source out of the factory that we print that room is able to produce. That's a pass this page list over though. We don't want to send an empty list. If DUI can avoid it. We would really like to load data first. And so we do that on a background thread initialize that data and
create a page bust out of it and here you can see at the very beginning the first page looks that we produce has data only loaded at the very front. We can send that over to the UI thread and then if you remember our submit list call on page list adapter submit this to the adapter and the adapter can start immediately presenting those items know as the user Scrolls though. We might need a little more data. So the page lists internally will trigger a data load from its data stores. independent data directly to the page list now the Crossfade animation
occurs because we have signaled the recyclerview that these new items are updated again. But what happens when a constant update occurs, so if the database says, oh this table has been validated something has changed we can mediately stop loading from the data source Wheaton validate it. So let's look at what happens in order to start pushing those updates though from the other side. Tornadoes are things we can see all the database had an item added to it. That's why we invalided the previous data source, and at the same time the producer side is listening to the signal to know. I need to
produce a new page list. So we use the signal to create a new data source so that we can create a new page listings in Dover. And when we load the initial data for that though, we're careful to initialize that based upon the loading position that was single by the adapter. So the user has partially scroll down. So let's send her our load around where the user is. We send that over to the UI. And now we submit list again because these are two different lists and we really don't want to call notify dataset changed. What happens is that we
compute an asynchronous diff on a background thread just like with listed after as he mentioned earlier. Once that death is computed. We passed that Jeff result into the recyclerview and swap the new page list in immediately. You can a new item showing up a new animation for that and we only have to do the minimal amount of you. I work to bind and show the new item. So fundamentally under the hood. We do a lot of trickery to make this work, but from the outside to try to make it look as close as possible to a live data of a list because this is a really nice experience. It lets you keep
your UI. I really really simple and avoid all of the information about paging on that side and let you configure and construct your your your flow and one place from the repository. The page list adapter is able to handle both the new page lists as they flow in and also the internal updates of the page list as it loads. Okay, let's talk about data sources page list the list implementation that works with a data source, and we have different types of data sources. You can have a
positional data source an item cheat data source. Let's start with the positional data source, if you have an application like the contacts application on Android you have the day to locally but user may want to jump into arbitrary Jose shells positional data source is your best option. This is actually what room is behind the scene. So if you have a datastore, so you got it like that you expand dispositional datasource class and you specify the type of items but to better understand how this
data source behaves. Let's look at an example. So we have the UI here. We'll just do moths go to the bird's-eye view of you have a birthday to Source on the right back by a list the very first time you come to the recycling view page the smoke old this load initial methods on the data source pest start position at PACE size mattresses, usually larger than the pace size because you want to have more items at the beginning and whether placeholders are enabled or not in this example. We just sort
data source will return the data tell us start. Baseball bat will start displaying the beta but will also display the placeholders so that the size of the pages is equal to the total number of items in the data source. Bow user tester schooling as soon as usual stop scrolling Pages for the Allies that is going to run out of data is going to call this load range methods on the position data source presents. The start position of the first item is missing.
Other page size and get the new data if I made it to the list as usual. Jumps into Ann Arbor to the position where we do some type of the day that when this happens, it won't hold a date for Source load range position. We take me the position flight level recycling Rio and Paige so that we can display to the user. This is why positional data sources really good if user can jump into arbitrary positions. You never bought them. Play Say Something has happened in the database beta has been
invalidated. So we'll get the new data source of each state represent a snapshot of the data. So get this okay turning Pages for its initial page from this data source based on values. ER is in the previous one will bring this one as always on a background calculate the difference between these two lists and update the recycle area with the correct animations. Circle funnies the item kit data source Do you have some data like a list? If you look at the page out of this
you can't identify the items before this page by using the first item in the list on the next page by using the last item in the list. Basically every item can identify a page after or before it. If your data sources like that. You should implement the item keep data source course you provide the key type in this case without using names for strength and the item type in the list. Lecithin example the first time you come to the recycling view of the cold method, which is now because we don't load size
and whether placeholders are enabled or not. We'll just enabled in this example. There's no place holders. So start displaying it as soon as user stop scrolling. We will extract a key from the last night that we have and call the data source load after method to another page similarly as usual keep scrolling just exercised a key and then loads another page the same Ray. By this point I say the state. I was coming from the database similarly something has changed. So we lost the title source, meaning of
recreate the new patients for the status. Of course. We are going to extract a key from one of the items that visible in the recycling view closer to the top and load the page from Disney's data source. And we get that page. We understand these are faceless. We will calculate the deep and updated with the correct information. We don't pay the date on top anymore. So if you guys are trying to scroll up we need more data same thing. You just expect to keep from the first item call load before my
phone. The data source was get that page for pundits to the page list and user can't scroll outwards always be active. Turn off is the page case status. This is a very common way of paging especially on the server side apis. So your client sent the request is inclusive data also includes pointers for the next and previous kiss get up. So if you have a data source like that, you should implement the page keep data source specify the type of Yorkies that using pointers and item pipe. Okay, let's look at an example. First time
user Council DUI vehicle load in a show they give it the size of a ask for the place holders are enabled or not their usual disabled because your server doesn't give you an extra discount table easier of the next page key. Barbie Stop displaying it if user straws We need more content. So we're going to call the load after methods use the key that was returning the previous request to get the next page from the data source. This next page comes with a field called by Jason Paige K. So it's always tells okay. What is the next page in this direction is literally like a linked list of
pages. What's your take that? Eagles on Castroville a difference in this is how we have the email Deja view data source. But in the long division method, there is no key anymore. This is because I mentioned it's like a linked list. So if the previous list is invalid links in the list doesn't mean anything so you always need to log the very first page again at this late in the year. I this is usually not a problem in practice because you only do this if user. Swipe to refresh so they're already on top of the list. Okay.
Alright, so you talk through what we might see implementing a data source, but what happens when we want to load from multiple sources at once. So Gina Beast + Network solution can work a really nicely together when the database access a cash with a network. You can page from your back and that you have with all the benefits of the local tax. You can have a really nice offline support. You can resume quickly if the application has been killed and restarted and you can minimize Network traffic by taking advantage of data that's already on device. So
what's it look at? How does might look compared to what we were we were showing before with a single source of data? So how does that work fit into the system so that we could do this as we could basically say, well the network is my source of data when I'm connected in the database is my source when I'm not so if I'm connected then I spy ipage data from the network. And if I'm not I paid the data from the database the one problem there is that you don't have anything storing data in your database. Even when you're loading a network, but it's pretty easy as a side effect of loading from
the network. You can just store in the database and there you have paging from both. So we're switching model has the first problem that this connected state is really oversimplifying in reality individual request your servers can succeed and fail and a user that's connected like 20% Some of the packets go through that doesn't really fit really nicely into this model. The other big problem is that we're not using local data when it's present. So what's go about looking at a different way to do this? So what if instead we just
monitor the database and use that as our local single source of Truth? What we can do then is we can say well the only times I need to load data is when the database tells me that it's out of data. I can use that as a signal to load more data from the network storage into the database. And then I have my entire solution built. I just load data when I need you, but I can present only the database which makes things a lot simpler. So we get the benefits of consistent data presentation. We have a similarly simple process, but importantly this gracefully to create its own failures if
your if your user is in that 20% connected State, you can still produce you cancel present all of the data that you have locally and try and fetch or potentially retry when the network is around. So potentially you might say well just doesn't keep my data fresh and an easy way to work around that is so just say when everyone starts observing the stream of data, we we start anew fetch. We we see if we need to do to refresh our content and that's especially important. Usually when you have frequently updating data. So so in that proposed model here that we saw we
need an out of data signal from the database cuz the rest of that we basically already built in the first few slides. So when we have that signal we can't recover loads from the network directly into the database and the UI doesn't have to enter into any of that. So Patron built exactly the signal for exactly this reason and we call that the boundary called back. So let's go ahead and look at what that might look like. The first most important part of the the boundary called back that you that you implement is that you pass it to different that you'll want to provide it two different sources
of data the database and the network because that's its job. So the first the important call back that we have here is on item it in loaded in this signifies the last item the database has been loaded from the page list and if there's more from the network, it's time to load it. So the first thing that we do in response to this is that we over on a network thread we can request our service. Hey, give me more data. And in this particular case, we're using the item at the ends to feed which data we need more of because we're an item key case similar to the item key data source that you saw
before. Now is that request a successful? We simply jump over to the database thread insert that data that we loaded from the response into the database and we're basically done we've collected that signal that we needed. And now we've added Network to something. It was purely database another one of their trick that you need need to be aware of though is that it's possible when the database is being invalidated locally for multiple at end signals to trigger. So what you can do is you can just protect this with a simple Boolean to say if I'm
already loading, don't try And then we can reset that at the ends. So using his bike boundary called back is pretty simple. You can just add this back into your life page list Builder or to your Rx page list Builder, and then that gives you the database Plus network solution all isolated in that one call back. So what's talk about with a paging library is the paging Library provides paging from Database Network and importantly both as we just saw and I can load that data directly into recyclerview. It extends this live data of list this
observablelist pattern that we really liked because it keeps rui incredibly simple and it lets us contain all of the complex Logic on one side. It's configurable. We have configurable load size prefetch and placeholders and it integrates directly with room with lime data and with RX the paging 1.0 was just released. And so please give it a try. Take it for a spin. Hi, that was a lot of information. We know so where to go next station on the wall for Android, so we can just read
provide more details and more samples there. You can just check it out in the cold air Bay Area or you can try it online. We also have samples on guitar Barre Implement different data sources using Reddit API or different local database and you can compare and contrast the behavior of different data sources. You can also see how you can handle things are like error Street rice pilaf it there. Fuck if you've been sitting here for the last 30 minutes. I'm wondering whether these two guys never heard about cursor is Bill temp agent.
Rest assured we don't care about that right wrong. So we realize it is very unpredictable and inefficient if your cursor becomes large application internally, if you're using charger for lazy loading be very careful and pigeon together. It was this problem because we create much smaller queries not to rely on the page of behavior of cursor amazing blog post about this that you can Orchard. Lincoln, we will post the Seattle after the talk. It's amazing. But tldr Beijing is not the right way to the Beijing. We did it wrong to
be helpful. If he said this time did it wrong 10 years ago at least. Paging is part of a jetpack or new initiative to accelerate Android development. We had a great many Stalks at this Ajo. This was the last one. So, please if you were not able to attend those sessions check them on YouTube Everything Is recorded and caused a lot more about jetpack on our website and provide us feedback. Thank you.
Buy this talk
Access to all the recordings of the event
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.