Senior Engineering Manager with experience in ML frameworks, low-level firmware for custom silicon hardware as well as system software. I derive strength from helping my team grow and deliver impact for our customers. I have recruited, trained and built teams of awesome engineers from scratch. I am humbled by the opportunity to support high-functioning teams by providing direction, context, leadership, coaching and mentorship.View the profile
About the talk
Abstract: PyTorch Mobile enables an end-to-end workflow from research and ML training to deployment on edge devices for iOS and Android. Running ML on edge devices is growing in importance as applications continue to demand lower latency and enable compelling real-time use cases, such as, audio/speech processing, computer vision, and computational photography.
Presenters: Gaurav Aggarwal, Software Engineering Manager , Facebook
Technical Level: Intermediate
Target Audience: Software Developer
Topics: #ArtificialIntelligence, #Mobile, Machine Learning, #ArmDevSummit
Type: Technical Session
Conference Track: AI in the Real World: From Development to Deployment
Air Date: 2020-10-07
Call everyone. My name is gaurav I am a member of the Pirates team and Facebook are working to democratize internal and external users. Today, I'll talk to you about my torch mobile and how that helps you to go from research to production, ask him. First, let me start with a brief introduction to pythons itself before going deeper into the edge Computing, and mobile portion, that would be the focus of our talk today. Open source project, for machine learning training, an infant.
It is very popular project with more than 1,500, contributors and 40,000 users on guitar. Many among researchers and Engineers around the world are part of this growing community. So what are some of the key design principles that have guided the architecture of Michael first support of eager execution to pythonic programming. Additionally, very efficient incident. Support for dynamic neural networks, that allows execution of models with controls those translation joint, or supports distributed learning
for computational expensive models, to support Hardware excavators, which helps achieve better work or what, or dollar and bring lingerie efficiency for data centers emphasizes. A single elegant, way to do things. This reduces cognitive and x-rays developer efficiency. My coach is old is focused on the development, the light and provides freedom, for researchers, to internet quickly and come up with the best models, the API clean, and easy to use offers and ever-increasing set of tools and library, is that a developer can use in their journey to take
them out of research and production. We are now nearing 1500 contributors on data which is almost 50% growth in the community are over here. The interesting thing for me is the breadth of the community members. We have my dinners in the project from Facebook, Microsoft. Google Amazon, Uber e. N u m. I n, many more people come together over the common passion for building the best platform to advance. Air is just so awesome for me. Okay, in terms of deployment, pythons can be deployed in many ways. Depending upon your production environment.
However, it will give you a good sense of where things are on the left. You will notice that I thought is deployed on data centers in the cloud on the right side. We showed the two different. Routes to take pytorch models to mobile Department. The most popular mechanism is via the pine torch mobile. Interpreter appropriate. Target markets such as CPU, GPU ESPN, interview presentation, There is an experiment of using the TV and computer conversion that produces code for CPU by utilizing arm elevator.
In the context of this mobile deployment, type of mobile is now the emmalyn friends platform of choice inside Facebook, empowers new, are you in Facebook? Family of apps for both IOS and Android Facebook, brother like Oculus VR, headset important, communication devices. Next, I would like to give an overview of the pinecones mobile Facebook. Many people ask us what is peikert mobile and if it is a separate framework on the core part of the quick answer is no
and love. We believe that we don't need to build a new framework to take your air models to add appointment. We develop the framework where we don't need to do model tomorrow. For doctors is part of this unified framework. That lets you quickly bring your ideas from research and production devices. So what can pythons mobile? As expected, we always hold a subset of python supporting entire python on a mobile device wouldn't be performing or even relevant for. That matter is built on top of
tosk, which is a large subset of Michael. We support all data types in control of portrait name to poop. You can continue to take off your morning as a program. Mini features applied directly to poor script even if we don't directly support all of my phone. Skip supports the use of most part and functions and mini python built-ins. So, how does it work in real life? You would train your Marlin by, and see if it worked. Skidmore can be loaded by code 510. See that there is no
which means we can also enable multi-threading that is ideal for production environments on Monday. Call compute Dawkins. I won't know do that. We don't support either mode on mobile production, Friendly, Ford Edge devices. Let's talk about a Mardi Gras on the mobile. Support multiple Hardware models on CPU libraries that are highly optimized for arm CPU. Operators have been optimized and there are many kernels in a very optimized Manor. Libraries, and is under development. We know that many recent SOC now include tsp and custom deep learning apps
to Android. N t, i and execution on DS, DSP via the vendor provided have included in Android. I don't support for iOS is very similar to that. We saw for Android support is via accident talking to me again for arm CPU and other CPU. We're also working actively to add support for Gigi on Apple devices. Be a metal which is very similar in principle to the bottom base design for man. Where is the next available tomorrow on the noodle engine by utilizing call ml framework. No, we just talk more about the workflow, to go from research
to production. The essential elements are shown in the indicative of high-level flu shot. Once you have a pipe or tomorrow, you can optionally quantize the money from Norbit position such as After that, you can trick or treats tomorrow. Optimization for mobile infants, you can then see you tomorrow to generate the modify this fight and then be loaded by the app on your mobile device, which we will discuss next. Oxygen model, quantization, and optimization steps.
And then you see the model to a modified shown in this simple code. Next you Loop this morning 5 on the mobile app and for this you really need a few lines of code to get going first, we need to add a dependency on white horse to win the Cleveland police fire mobile app. Android. You can use a job in Aquaphor lip torch, which allows you to continue to build your Android apps in Java and not worry about anything. You can use objektiv C plus plus to make calls into the C, plus, plus live
tour Champion. The code for Android and iOS looks pretty much the same. You get your future better you than create attention from the input picture of actor in the morning before work function and then extract the output better from the genetic cancer. okay, so now I just want to show you what populations are possible and what's the benefit of Sample optimizations can be applied to your Martin on the host before you save it for mobile usage. Can do a lot of heavy lifting for you. Infuse con
2D with Bachelor on today, it can transform Oxford, accident pack, use for tanning with quantity, remove Dropout at Central and Camelback. In addition, we recently, added support for channels last format. Deanna packing format is critical for colonization c as it impacts cash access pattern and utilization of CPU Aiza. If your data is not initially packed in nhwc, format text into NCH, W additional processing Cycles. There's more saving in the actual compute kernels, especially when using accident pack library
on CPU. These waiters optimization stools that we just talked about both be optimized for mobile devices, as well as China's last format and other improvements in the framework in multi-threading and using Peter, to actually reading amount of benefits on different models, someone run up to six times faster. Now compared to earlier flight was wonderful release. What is Asian nation is very useful for mobile devices. We're both students please and memory bandwidth order. The
premium leads to 4X reduction in memory usage, significant pytorch offers 320 mechanisms to quantize, model each method has its advantages given to we also post raining Dynamic, quantization where the deer exchange between us is still floating point and it suited for networks Sports training started. Quantization, we're both ways an activation or quantity 288 and works well for CNN. Which provides the best accuracy for all Model X. What station is now being used? Effectively on a wide variety of models on
Facebook. It's 2-2 to 7 x speed up in model execution. Of course, be space savings for Mary and Brandon is roughly 4X typically acceptable. We'll set up a useful technique for reducing the binary size of pipe torch libraries. The final size of a mobile app is crucial for mobile. Developers can use to selectively bred mechanism to exclude acquired by the house, can be used to prepare a list of operators that will ensure that only required to include it in the library. This can really help save hundreds of
going to buy some more from the final binary that runs on the X device. I just wanted to mention of recent exciting, use of pine orchard Mobile phone feature in 2018, This was a new immersive format for sharing pictures, with friends and family and a line on the New Orleans camera available. In New Zealand, smart hasn't been available only have a single rear facing camera. We recently used to do the Yard Machine learning techniques to produce 3D photos from virtually any standard to a picture. And the Deep learning
running on an IOS and Android phone in for the 3D structure of any image from a single to the photo that I can share with your friend. Has a 3D photo that is engaging and compelling. Let's talk about type of features and availability. Wonder 6 was released in July. This is a big day which means we are committed to taking all of these pictures or it was tabled version. We not committed to backwards compatibility and some soap or characteristics, might change going forward.
The next release for pike, watch 1.7 is expected in the next few weeks, and will not support for Jeep. You back in, for Android, into you back inside, I discussed earlier. We do expect you to be a stable release. Next year, will bring support for IOS as well as new tools and improved run time to make the binary size even smaller than it is today. I hope that God gives you an idea of current features and capabilities of pipe and how it can help you take a research into production
that can help you build compelling and engaging products for your end. Customers. I would like to leave you with a few additional quarters to help you in this journey. You can, please start with our main web page pipe. Northstar orgy, it has been recently updated as well as pointers to a couple of YouTube videos to help you take your models from research to production. Get off. Wait a few online courses available that provide the foundation and help you get started as a great intro to deep learning course waiter teaches
you how to get going on deep learning with python. Basseterre is a navigate website with lots of useful material and course in Savannah. I would also like, to point out that part, which is now available in Chinese, and there's a large community in Asia that utilizing the resources on this website and I asked you to check it out if you're interested. What is codeine as well? Feel free to engage with this large community in the APAC region, who stupid active
Community of North Florida stock. Please do check out the Pirates community and join the conversation or just listening, published research court, or just there to help users, or their problems. Please engage with us, tell us how you use pytorch. So you think it can benefit you what is missing. What would you like to see where therefore you please engage with the community reach out to us. I hope you found this. Talk to you soon and engaging. I would like to take the time to answer any questions and hear from you as to how you been using my Droid
Buy this talk
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.