About the talk
TensorFlow Federated (TFF) is an open-source framework for machine learning and other computations on decentralized data. TFF has been developed to facilitate open research and experimentation with Federated Learning (FL), an approach to machine learning where a shared global model is trained across many participating clients that keep their training data locally. By eliminating the need to collect data at a central location, yet still enabling each participant to benefit from the collective knowledge of everything in the network, FL lets you build intelligent applications that leverage insights from data that might be too costly, sensitive, or impractical to collect.
In this session, we explain the key concepts behind FL and TFF, how to set up a FL experiment and run it in a simulator, what the code looks like and how to extend it, and we briefly discuss options for future deployment to real devices.
Presented by: Krzysztof Ostrowski
My name is Chris. I'm leaving. I'm in Seattle and I'm here to tell you about learning and the classroom with you to support it. I got two parts of this talk first. About to learn how it works and why and then I'll switch to talk about the bathroom. I was good, but this is My Story begins with data Creek intelligence in better products from Dayton. Don't actually what we do is this costing them beside my right next to the mall so Supply. If it's so good why change that but actually in some applications
applications fast turnaround Communications customer that's about the computer. So it has its own decigram time. It's raining and doesn't communicate with so now there's nothing leave the device none of the other problems. I think applying just doesn't have an update that doesn't have an update on the receiving data from the seven-headed financial evolve further except that there's no problems here. Why does it matter if he has one concrete example? Show me lots of people stop using on your warrant. What happens are you think? Hey, it's a strong signal means of people but if you're not going to
make it not this but something you hook up, right? So this is what you want. What are the following is one way to do that here? We start with initial model provided by this one is not a luxury brand model and send it to the server. Independently owned and operated a client's husband is meant by data from her intern. So so we do get those benefits of scaling privacy. So let's look at this closely never left. The device right models does not retain store at end of the Klein models. It's simply ask them up and then throw them away.
Ask her how do you know what I don't see the final results of aggregation that do not have access to any other day in your pocket edition versus the final aggregate. You can still wonder how do you know that that doesn't cost anything? Is blonde clip this update and until bit of noise? So once they're finally going to get you noticed but there's still enough signal to make progress. If you have any more, since I'll be happy to discuss your feedback on the server as a nation weather for the next hour and then execute many thousands of that's how long it takes. I'm going to in this scenario
both Clans in several Capital Lighting continuity through this process as we move from one neuron to another state between and I'm in a practical applications clients. We want to give you the power source on the Wi-Fi network in idle so that the user is not negatively affected. And this is still slated on this system going to see you when you complete at night when everyone go to sleep in their phone is connected and beeps at lunch when everybody's talking into the phone while
eating so yeah, I'm going in so that means that I remove a participating Clans will change the Bluetooth connectivity drop out at any time where the participants Chosen and environmental factors when going through it including in a confirmation bias. But if that's important to remember is that that that that sell appliances. Okay, so in International Tractor Supply auction scenarios plants, are there many of them don't talk to each other. Everything is coming.
Keysha is the bottom like in this house in the in the house has them on to the animals and put them in the sense that you know, something later not indifferent to the effector in a result of memory does not guarantee wonder when they'll be back. So we finally Okay. So is it only for mobile devices you could to use fitted learning for things like in Oregon without wanting to learn something together or institutions that they work better when they're more clients. And so we are going to have to work on that or are you some more specialized by? What are contractures in several
applications including performance of a autocomplete model that I listen to the state of, Illinois? Alzheimer from before that I can actually learn new words that were not initially in the vocabulary and definitely want to use static differential privacy to make sure that the only thing you're learning is common things and do nothing Sensitive gets through. Okay. So the only Vice based diet is a high-quality or if it's a sad day for large if you can't pay someone to go and are
they able to they died because it's only right we can make people laugh heartily types of characters in so that's when I have to work harder to two additional signal in your application. How many volume extensions exist in this area? Xbox together and so that's why we've built so let's get to it. That is designed specifically for its also. I kind of computer programming language. It's not your face don't notice it but there's actually kind of a program that combines tensorflow and listen to the communication. Language of implemented in a number of
and provide everything you need and what you want to enable actions and so easy for you to express your ideas in NY the super Compact and you can see you in what you doing things. You can copy paste in for Canon modified so that includes the information but also it's still can't imagine but we'll have full intent examples of research reproduced with slim chicken hand and then you modify do whatever. Set an independent nation in factories is designed to be whatever
kind of resources you you might have lost it in the basement or something else in such a way that works on your Hardware. And so we wants to be able to take all the latest research and immediately. Is it in production? Hopefully that will happen in such a way that they want. Even though production we're not something we provided. Also, we've been discussing such a way. Whatever called you right you you're you're you're writing in TF2. I also wasn't together and make it work. Whatever you want using the
uniform. So, we we are investing our resources to make sure the product and Country Way. Espresso and I keep messing a new language. Why why didn't you language? Reason for that is that right this second thoughts and expected expected to be engineering the same except now you're in the diagram, and so obviously something that you should be able to programming language of socks with make it super easy to do that and messaging or taking and restarting checkpoints. Please try to use dolls. That's what I initially limitations for fraud. I would like
it was unreadable. It was very very difficult to work with. I hopefully see how and hugs an in-depth happened at it. You'll like it. Why stress portability? I didn't think about it until I started learning environment. We can't just look at the latest. So it may not be easy to see now. I'm so humble. You may want to just go ahead and deploy your mother into a life system that I'm a real devices made in drywall. So nothing gets affected and Anna and I did it right in this manner. So what kind of traditional boundary between the production versus research all this gets out
that the dog is constantly researching the Black Knight sat And number of visitors in theater. Like the fact that everything is taken platform-agnostic and everything is okay. So when you start The basis of very many of them days. They do all the painting 7 orcas place is computations and appliance that individual. This better Taste of the type of the endodermis and placement which is a density of the group of system participants. We got a little place McCall drive by and see if
I won't get into it, but we can also called anybody in this case. It's nothing about this and because there's just one example of it. Not as ghetto potatoes supposed you can think of it as a function of creating custom defense systems. A Mother's Son in inbox and you can even give you the functional type in this function takes clothing. Idiopathic Alcohol and Other surveillance programs. So let's fly. It goes like this you always start by defining types of things.
Next you're going to actually write the computation. And so she FF code is not python code. baci Express in Python and the same idea of what type of Not in the body of this python function the sensor readings parameter represents the BFF to the slice and dice. What is that? If a device on Samsung gets traced and we could we construct a little tense of computation and then it's kind of rice and tensorflow graph getting storm inside lights for behind programs are python. That's that's that's what it
means to get direction to present a presentation of accounting tff Okay and important because again, we want around those things on devices and so it did not come to be interpreted by regular by look at something so you larger clients. As we can think of this as you know, like a mobster beam apple juice and we provide the federal operator for the people to the Federated operator and introduced to find locations have the same except, you know, it's called so you start by defining
inputs as formal parameters. And so you see their temperatures. Yeah, and I need to buy this kind of this function again again the used it for the address to slice and dice those things. So you see the Chromecast That is implemented in Narragansett. I'm upset, you just use a slap the pipes on top of it to make sure that everything is locked on the time and go see if that likes things to be strongly typed into attack tactic for you. And that's it. That's the whole problem. And I think we have a version of this into thought I was
going. Okay. So now let's let's I'm going to show just a small example of of what we have described in a tutorial. Undead on the website And I'm going to park with just on their computation the presence of single round of federalist averaging just like what we have discussed at the very beginning of this presentation, Thirst is a Muslim. Okay. So the first thing we'll do is just before with broadcasting more than the learning right from the start. Now that's the clients got everything running late and there on slacker beta. Likewise
and finally you so so so the map function produces a set of plans. You can apply. But I don't do any kind of volume including stocks and bonds. So we have stuff expanding into it and make it more interesting. one, thing to do is wanting to integration by parts places in 2 to address virus cancer assistance broadcast apply and calling on the server before you broadcast and then use it for that does not function to decoding the clients after broadcast in so you can see how basically in a
few lines of coke give you what you want. But they decode and encode presently being implemented in town. Are you just call the operator to them today today arguments and to add some clicking noise? That's something that you would imagine how other modifications you can you can relax it. I even though I meant sometime BFF code is not python. You can call it in Python like a function and its in the Bronx in 540 Malaysian Darren return the numbers in Python.
I just how he drive. It's just kind of what you would expect and the full version of it is again, so you just called it sucks. When do you get data from simulation in that module? Which of these has allowed at the function when it called that you get a pair of patent a business objects know I mentioned before I was going to let you deal with individual plans on that. He's so this is things like inspecting with what's Lanka time in. My data said that's something that you can only do when orchestrating your simulation in
Python. You cannot do anything for you. Can you can look at me like what I just got to be just need the clients election, right? So he are you taking all the Clans are they speaking random sample of them doesn't my clients for this amount? And now I'm I called it the Entertainer object to the DMV today. This is an eagle. And wants to go into the computer before and it's very easy. Okay, if you don't want to you might use one of them. painting competitions
it's easier to use with Cara's if you have a cut off my hand and then these 1 yr. Koskey song here to take that model and construct computations that you can use for training and evaluation. And you just do me the same way I was before so the initialize crepes. And then the next computation the painting So to take the use of State before their time start at introducing you stayed after their own computer. State includes things like that before. We we can perform plant selection
and stimulates. You know, what is consistent Behavior? Yeah. So so it is very easy to explain everything. To the evolution computation. So the evil is a computation. I can just call and talk at you by metric socket. Okay. So yes this by default when it when it does involve computers like function on your machine in in in in in your process that I bought his ways to speed it up. That we provide the framework for X right now. There is one sort of a solution. If you want to run multiple simulations
that you can do online you create a lookalike. What time are you type in. If you want something more powerful nuts not long from now. Well, we'll have all inclusive solution Park. Running things on Google that you don't want to wait for that. You can actually build just going to stick it out yourself because all the components are basically there in in the morning space in the past tense of lie, like the cutest you can stack up together and we'll get our stuff that you can use to construct the
vs. Multi machine support for gpus and things like that ends ends. Now if you want to go beyond just running simulations, it is also possible. I'm for that but but the two that I can actually play plug in your physical devices into the simulation framework. So you can you can Implement a simple device or something and then you can you can fly. After working out into simulation framework American drama on your Kindle devices certainly doable for smoked floor scale experiments and
transform them into a form that has more amenable that I cook you something in particular kind of biking so that's just like systems. Makes me look like I'm pressuring, you know, if you need something that we haven't provided. This is intended to be an open framework and a community project. So by all means please continue to send out that people requesting and so I have nothing further to learning algorithms. You can copy or or or help us. Nutaku Botkins before actually planting season if you have any ideas.
But that's all I have to say. Thank you very much. So you're the question questions are when the client start learning and on their own data. And then do you have an average model on the server? Do you ever send the average model back to the clients for Life performance boost or do clients just been off on their own afterwards? And then the second question is how do you start the model? Do you use proxy data initially and how do you get trade with your models accuracy and things like
that? so for the first question In a system we have running production that the way it works and that's the scarf that's different from a client periodically come back to the server every time they automatically get that so you can arrange for this to happen. So you're kind of contributing as well as benefiting you think it's useful then, you know, it's certainly helps to play with it. At least you can get some idea of what model architecture you can never be sure because proxeneta is in on this so good and you never looked at the door but never did not for
sure how good are you might also choose not to write you can simply try a different mother architectures deploy them on devices in in Lancaster Mass. And I'm also it would be kind of running devices getting evaluated but not expecting anything other than the same time on different stops with public lands. Reproach. That applies to any kind of phone device data including when you have absolutely no idea where where to get translate. I like it might look
like that. Yeah, I'm Ghostbuster possible. First question I have is does the t f t f f Library does it integrate with TF light and the second question I have is since its language platform-agnostic. Are you able to use it and none Python? And you know it can I use it in the language? That's not python are not python. I think I kind of thing from the Spotify station is the latest stock show that has no relationship to python. So yeah, you could you could take it and you could executed in a completely different environment that has nothing to do with
this is so if you wear to run it on a different kind of time to the extent you can take this graph that's in converted for that other one time and that set up show itself doesn't integrate with your flight because BFF itself is that does not include a platform for on device exact you from the fact is more like a the best think of it it took you to think of it is more like a compiler framework and an environment. But if you could use it with your flight, so you could maybe
in that van that are light skin can absorb. Thank you. I am good. Thank you. I had a couple of questions. So our does the blind to the table model train until convergence convergence why so when the when the average model doesn't have access to the data, how do you measure the performance and how do you know it's good enough for now reply send it back to other plans. So the average model is on local server and then you don't have access to the data. What do you how do you measure the performance of the average model? And how do you know what when to deploy that Marcus
back? So I didn't know this crap for their that evaluation, but basically it's like the temperature. Now the Clans have the model and the latest they can evaluate the same idea. I just seen that her and another question was is there a way in started learning tensorflow where you can share part of the examples of clients have two clients have different labels resuming, but they have similar data. Is there a mechanism where you can say declined share most of the model but they have their own a couple of layers for them. Maybe the last layer of the network is a specific line, but not shared
across lines or is the entire design time will have to be shared across all the blinds. yeah, it's it's not a cup ability. We included at the moment, but Sounds like us about this something we could YouTube. Thanks. Hey, so one question that I had was that like when you kind of agree at all these models into a central server. It seems like. It seems like one of the problems of Federated learning solves is I guess Distributing computation. But when you
get to like a million people using the Google keyboard or a lot more actually it seems like either the server is going to have to reject some gradient computations or there's some hierarchical aggregation system where you kind of like Viagra gate the models of streamer, whatever. So I'm wondering if if if the second is true are there latency issues with gradients reaching the central model like after like by the time that the model has changed so much that it might corrupt it a little bit.
Computer turns on for a while. So what we say is what class under the server updates differences between that include a whole bunch of plants in which class have to have parties wedding invitation. So not all clients if you say have a million clients You could speak in a thousand clients samples and thirsty Thursday make an alliteration. You don't have to include others is that eventually must climb participate? So that most plants have a chance to influence the writing process at some point, but they don't have to
Buy this talk
Access to all the recordings of the event
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.