Events Add an event Speakers Talks Collections
 
MLconf Online 2020
November 6, 2020, Online
MLconf Online 2020
Request Q&A
MLconf Online 2020
From the conference
MLconf Online 2020
Request Q&A
Video
MLconf Online 2020: Deep Learning Battle: TensorFlow 2 vs. PyTorch by Jon Krohn
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Add to favorites
811
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About the talk

This talk begins with a survey of the primary families of Deep Learning approaches: Convolutional Neural Networks, Recurrent Neural Networks, Generative Adversarial Networks, and Deep Reinforcement Learning. Via interactive demos, the meat of the talk will appraise the two leading Deep Learning libraries: TensorFlow and PyTorch. With respect to both model development and production deployment, the strengths and weaknesses of the two libraries will be covered -- with a particular focus on TensorFlow 2 release that formally integrates the easy-to-use, high-level Keras API into the library.

About speaker

Jon Krohn
Chief Data Scientist at untapt

Jon Krohn is Chief Data Scientist at the machine learning startup untapt. He presents an acclaimed series of tutorials published by Addison-Wesley, including Deep Learning with TensorFlow and Deep Learning for Natural Language Processing. Jon teaches his deep learning curriculum in-classroom at the New York City Data Science Academy and guest lectures at Columbia University. He holds a doctorate in neuroscience from the University of Oxford and, since 2010, has been publishing on machine learning in leading peer-reviewed journals, including Advances in Neural Information Processing Systems.

View the profile
Share

So yes, I will touch on deep learning techniques in general, but that is not the focus of today's talk. So the focus of today's talk is on tensorflow 2 versus 5 for the two. Most popular deep learning libraries are going to talk about the pros and cons, why you should consider using one or the other? You can get the slides for today from my website. Jean Crown. Calm talks. So when you go to that website, you will find a photo at the tops of what the world was like when I could present it conferences in person and then

today's slides you can just click and then you can download them in whatever format you want PDF PowerPoint. Whatever. So yes, yes. Yes, dries. Hello. What's going on? Dre's is on, he knows my dog oboe and he misses being oboe. Evidently from the chat. Well, this is an illustration of my dog over by the wonderful artist, ugly basses. So over the past few years, I've published 40 hours of videos and dozens of online training is on deep learning as well as lectured on deep learning at Columbia University. At the End

of This Book, deep learning Illustrated in November of last year across all about deep learning education that I do a single question is asked more than any of all of the others combined and that is so, Tensorflow or python. Given the popular demand. I couldn't resist dedicating my lecture to the topic today. Let's first do a couple of slides introducing deep learning for those among you who don't already know. It's so this is y'all know he works at Facebook alongside Igor. No doubt. They probably high-five every

day. When there isn't a pandemic. So he's at the University of Montreal and runs a startup called element. And who is a professor emeritus at the University of Toronto and he runs the Google brain team out of Canada, so, What do these three men have in common? They are often called the Godfathers of deep learning. And last year. They were awarded the touring prize which is the equivalent of the Nobel Prize in computer science for their work. On deep learning has

heard people are named, probably a lot of you can never be God are but I just loved it though stations of them and it's nice guys. That we have to thank them both for a lot of the deep learning Advanced. Let's start off by talking about a innovation of jet engines from 2012. So here is an architecture that Alex kraszewski. Jesus gave her and Jeff Hinton publish on, in 2012. It's named after the first offer on that paper towels kraszewski. So it's called Alex and Alex, next

is a movie called today a convolutional neural network architecture. You put in an image of one end, like an image of a cat and then at the other end. It is trained to predict a particular label. So it might say it is a ninety-nine percent chance. At each stage at each layer of this deep Learning Network. There are artificial neurons in these artificial neurons are designed to muslima next. The way that biological neurons biological brain cells work. And I've got here along the bottom snapshots of what the

artificial neurons at each layer or specialized to the text. So, here are each of the 96 artificial neurons in his first layer of the deep Learning Network. And what you can see is that each of these neurons is specialized to detect a straight line at a particular orientation. And that's it. So some of them to take a vertical line, someone to take horizontal line, others, detective, 45 degree angle, but overall the artificial neurons in his lair or specialized to detect straight lines in a particular orientation. The next layer can non-linearly recombine

information from the first layer. The representation from the first, So I have an example of 25 of the artificial neurons from this later and you can see that each one of them is specialized is a text if it's going to kind of the corner, then why the third layer, it can read them behind the representations of the second later, so we can have more complex curves and shapes. I've got 25 more artificial neurons displayed here. And by the fourth layer, these visual representations are very complex, very abstract including neurons that are specialized in a dog's face, a cat's face

to face and someone. So that's deep learning in a nutshell. That isn't the point of today's lecture, but I just wanted to make sure that we were all on the same page about what deep learning is before I dig into the main Jeep lending library. And in case it was already clear from Eagle to talk or if I'm just your experience and machine learning in general. You should already know that deep learning has been its transformative approach. Learning including starting really in the modern times from that house that architecture that I showed him the preceding slide. So in the

image that large-scale visual recognition competition, or let's, just call it a Isle HD receiver short. That's competition, was started in 2010 and you can see that in 2010 2011, 2012, the vast majority of the models entered into the competition or what I called, traditional machine learning approaches. I just use that as a term to mean on deep learning approaches. So traditional, MLB regression model support Vector machines, Brandon Flores and so on. So you're over here, from 2010 to 2011. We saw a on marginal Improvement in the top performing

model. The error rate goes down slightly. And then in 2012, if Alex had had not been entered into the competition, actually would have been worsening. I'm just in provement, you're over here, but said, Alex rychetsky, Jeff Hinton, and her team did, Andrew Alexander has a competition. And the error rate was reduced by a third year-over-year. Took notice. So across Academia across industry. All of a sudden, deep learning is something that people were interested in. And the subsequent years. All of the top-performing models, wear deep learning models and just one

really cool. Landmarks of his passing 2015, where he models overtook. A capacity honest, image recognition test. One last light on deep learning just to explain why it's so powerful why it was able to get those enormous gains over the traditional machine learning approaches. And that is that with the traditional machine learning approaches. We need to spend the vast majority of our time. Is an engineer working with those approaches. Coming up with features that we can extract edible raw data. So we, if we want to create

a face detection, algorithm. We need to become experts at the structure of space and white algorithms to be able to predict whether Mikey eyes above cheats for the darkness of the eyes, in the brightness, of the Chiefs, or the brightness of the nose, with the darkness of the odds, on the other side. So we write functions to be able to extract these creatures. Out of the raw data. He want to be a space expert. You're going to have to become an expert. Or if you want to build expert base models. You're going to have to become an expert in faces. If

you want to build a machine translation model, then you need to become a Linguistics expert in the vast. Majority of cases with this traditional machine learning approach. So you spent a huge amount of your time, becoming an expert in the particular type of data that you're working with, and extracting writing functions to extract features of the data. Spending mean or either time actually designing and training models with deep learning. This is turned on its head and you might spend a little or none of your time engineering pictures of the raw data.

Instead going to rely as we saw in Alex Ned on the layers of artificial neurons coming up with features. Increasingly abstract increasingly complex features that builds on simple Peters. In the first layer in order to efficiently map. Whatever our input is, in this case pixels to whatever our output is, in this case labels like cat or dog and what have you? So this deep learning approach the future Engineering 10, fully automatic lot of cases. And instead we spend our time designing

and training is deep learning models, which can be a bit more labor-intensive. It is almost always more computational intensive in it. A lot of cases, notwithstanding things are transfer of learning some other, specialized cases, likely involves larger data sets. Then you might have needed for the traditional machine learning punch you. Design, state-of-the-art were approaching state-of-the-art face detection, algorithms, or approaching state-of-the-art machine translation algorithms without becoming an

expert in face detection or Linguistics respectively. So it empowers all of us and Answer. Now today, received this morning everywhere in Machine Vision, like unlocking your phone, any voice recognition algorithm today is going to use deep learning about the serial networks are really cool technique for being able to take a sample data set and then generate new images in the styles that David's head. So here are two made up, celebrities these people don't exist. But again, drew them based on the data set of

my resolutions. Let me photos that I was trained on complex sequential decision making when applied into something so we can define a lot of board games video games and cars other applications where we have a series of sequential decisions at the reinforcement learning task. You can insert a deep learning algorithm into the model that we used to solve that reinforcement learning problem and you can end up with things like the like Play the board game, go and defeat the world's best. Go players. Despite this being a

highly competition, we intensive and what players were considered to be an intuitive game. That's deep learning in a nutshell. Given how powerful and transformative need learning is hundreds of software. Libraries have sprung up in recent years to make designing testing and deploy deep learning models easy. So sheer or is a snapshot of the most popular libraries in the morning today. So you can see tensorflow and red as the clearly and then for a long time we have to

Charis Library here in gold which was, you know, it's still listed a very popular but it's been overtaken. Harris has been overtaken and even tensorflow in terms of Google search frequency. Could be overtaken soon by will be having green beer, which is ice works. So, these three libraries, those are really the only three that you need to be aware of today. I'm probably going to insult some people in the audience with something with a comment like that. But these three, libraries, tensorflow. Carrots, and pytorch, are so much more wildly popular than

any of the other libraries out there, you know, in fourth and fifth place in terms of popularity or probably ask nest and Microsoft e n, c, k library and as he just need to be left in the dust mite gives me a GTA a little boost. I suspect it is mostly used by Microsoft employees. So kind of high-level chore. I have more sides coming up with details on these libraries. The Chesterfield library was open source by Google. You can see there was a huge amount of popularity when they had a marketing Splash when they first open source it and made it available publicly directions. Something called

an internal project that they had help in disbelief project and is the most popular library in production systems today? However, tensorflow, particularly for its first few years was relatively challenging to use. And so, people came along Francois Chalet in particular, came along with me Karis Library, which was originally designed for another Library called fiano. But as Diono developers left, the University of Montreal where they were working on the T on a project and got scooped up into higher-paying rules at

Google in Montreal. They began to look at the cameras. People began to allow their high-level API for Designing deep learning models to to work on top of tensorflow as well. And now today, the library's were very close together. So the Charis API can be used for lots of different lower-level, deep learning competition, libraries under the covers. However, It is primarily used with tensorflow to the extent that the tensor flow to release that happened in a lease. 2019. It packages up here is as a module with the tensor flow and recommends

Terrace as the primary tool to use to build your model. Particularly, if you're just getting started in deep learning, so that's a no flow in Charis. They are bedfellows. This is based on the torch Library, which I'll talk about a little bit more but it's Facebook developer, almost single-handedly ported that over original torch Library into pytorch into Python and it's a lawn mower pythonic. It's easier to use relative for tensorflow. And so it's really taking off, people love it. So it was just a little bit more data on on intensive, Lone Pine Forge. Now, that I've kind

of introduce to you the temp in Charis, or you can even think of them as the same in a lot of cases. Not exactly the same, but they used to get there. So often you kind of think of them as the same. So how popular are these two libraries in terms of job postings in the US? Will you can see the tensor flow hasn't replied, torches close behind. So, you know, regardless of the Metro region if I told his less popular, but approaching the popularity of tensorflow. I'm still on average across all of these Metro areas in the US. You can see that is

featured at 60% the rate of tensorflow with West Coast locations. Tending to perform Northeast or Midwest turn locations. But either way in a 40 to 60% at 3:40 up to 80% of droplets exemption pytorch vs tensorflow. So in a bit more detail on these leading deep lending libraries, you'll notice first that I have a colon here on Cafe, which used to be a very, very popular Jeep. Lending library. I venture to say that is more popular today. Still then, cntk War. Except I

wasn't able to show it for you on that Google search results because people many orders of magnitude more people search for coffee, then they do deep learning library. So we kind of just load up that hole shark. Sol Cafe? And I said in tensorflow, they all have built-in python implementations. They are all they can all be called from python. George and it was originally, written was primarily designed to be called from luau, which is a little bit of an obscure programming language a little bit like JavaScript. However, that is

the primary way that you would access the torch Library. And an accident has a flow, have a lot of other languages that are supported tensorflow. For example, has big projects supporting pythonmc originally designed in C plus plus before we put it over to Python and there's unofficial support in these libraries that I have in italics. So one of the major differences between his library for a long time with the programming style, so up until the tensor flow to released late last year,

tensorflow was in the symbolic programming style like half a is and this is a, it's a clever approach to programming but it requires you to have many steps before you can even flow data through algorithm and trainer model. So, you had to step where you design, what your model should look like, without putting any data. When you have a second step, where you take that design and you allocated optimally across however many servers. However, many CPUs, Howard many Graphics processing units. You have and,

and then finally, after you've done all that, you can start to actually flow information through your model and train it. So clever approach, a more difficult approach to use. And so in recent years, the imperative programming style approach has really taken off and that's what is a big party. That's a big part of what to approach. Now, Tempest, lose popularity. So this imperative programming a style is what your most used to do. If your in a jupyter notebook and you create a variable, you assign a value that variable and then you

added to another variable and you get the results in real-time. That's the imperative programming approach. It doesn't require you to set up all the elements of the operation that you'd like to perform allocate those elements across your devices and then finally added value to them in allow the operation to happen. It's much easier is more interactive. And a lot of people would say, I certainly would. And so has caught on to that. Instant answer for 2.00 has

been imperative by default that you can convert it into the simple, like the old symbolic mode if you would like to. Now, all these libraries allow you to paralyze your data across, many GB. Use the last train to happen more quickly, obviously, but parallelizing and all of them except allow you to even have different parts of your model, swept across different CPU. All of these libraries provide you with access to large propane bottles that you can find to buy fine-tune by transfer money, for some specific tasks that you have often told model zoos

and easy to build models in. It is it is there are really other way to go. It is by default a high-level easy interface for Designing Depot. Nemo. Tensor flow has the clearest Library which of course. I've already mentioned on proceeding slides and then I talked about the pie porch. It makes it easy to a designer purse models. Hi, John, what's up? Your time is up, but you know, take a couple more minutes and if there's questions, we should be taking it now.

So please answer your questions into the chat box and we can take care of those and if you want to wrap it up, John, that would be great. Wonderful, any chance I can take a few extra minutes since I started played. Cuz there's actually no session right? After this, the next session I believe in this track starts at 11:35. So that's why I'm around. So I'll wrap up the talk in the next few minutes. And then I'll hang around any audience members that would like, you through the brake. I'd

be happy to answer your questions then. So, yeah, so much better for tensorflow. Much better for production deployments, which is really just the argument that I would make is that you you the only dude, wanting libraries. You need to know, or pytorch or tensorflow, right word to summarize it. In a single line. You can think of it as numpy. It is super easy to use. If you're used to numerical operations in Python, accept it, it's optimized for gpus. So, I events and also for automatic differentiation which you need in order to train a machine learning

models. Tensorflow. On the other hand, was boarded, python from C plus, plus more difficult to use that as much longer stacked races, that are difficult to understand. Where's when you get in there in pie, torch, very easy to understand and will be resolved very quickly. Play Schwartz allows for Dynamic Auto differentiation. So the size of your computational graph of you be planning model can change with each round of training, with each batch of data that you throw that it where is tensorflow historically and today, even in Pensacola to assume your model ecstatic. Meaning that

all of your data within your output, have to be the same size. I already talked with the Charis API for tensorflow. There is also a fast on a iapi on top of which I think over simplifies the process very little insight into what's happening under the covers. As I mentioned, debugging is easier than torch, you know, in contrast tensorflow is more widely adopted today, particularly in in production systems in terms of additional software additional available for these packages. The library has the torch script

just in time compilation, which allows pytorch, even though it's now a Miss imperative, programming style. It allows it. It allows you to compile your code after you designed it in imperative style and allocated efficiently across all the devices that you have this tensor Flow To, You have the same kind of thing, allowing that imperative to symbolic conversion on Temple. Serving has lots of other library is associated with it as a food serving for I'm allowing you to train with the ploy. Your typical model easily cross on the server, is the Dow JS library

for the remodel right into someone's browser, the lights library for lying to have it on a Blu energy to buy soya, milk and pizza bites like a phone or car. And the libraries that allow you to have your data preprocessing and all have been right there in the library. So, overall is more enjoyable for model design and suppose that are for production deployment. The next couple hours. I'll just give you a few examples of a code in his to libraries contrasting them, and you can see it. No. Use the fire symbol

for hydrogen water for blowing. So, right from the get-go, you can see that even simple things like creating a Cancer and this case is Vector tensor. One dimensional array of numbers in pi. Torsion tensor flow. It's about equally equally if it's the same level of difficulty to create that, but looking at the output much cleaner numpy like output in Pine Forest, where is intensive? So you get this very long. I put a lot of the information isn't information. You're going to need regularly and the information that you might most be

looking for his kind of included there. I was in a list for more examples of simple tense of structures. You can check on my intro to linear algebra notebook at this GitHub repo with mine and Nations repo. In terms of automatic differentiation, which allows us to calculate partial derivatives in machine learning model, in order to figure out how to adjust the parameters in the model to optimize them for some tasks. It is much much easier in passwords. So if we want to do this

simple operation for this equation, y equals x squared calculate the derivative of y with respect to X and then finds a derivatives for some particular value of x in this case 510. And we get that answer and either library but imply Forge. It's much easier. It feels really straightforward. In tensorflow. We need to wrap our operations within a statement and it's quite intuitive. You can see more examples of this in the calculus notebook that I have m.

I n o, r Foundation repo. In terms of Designing a neural network here. Both libraries are effectively identical and so you can see examples for my deep net and pytorch note notebook. Like he's in intensive low notebook which are both available in my password to get a repository. And then, finally forfeiting models in tensorflow, assume you're using. The Charis model is a single line of code in fight or to get as many more lines of code. And so, this means that tensorflow caressed for the novice D20 practitioner is a little bit. Easier is actually a lot easier, particularly on this

set. But once you get comfortable with deep learning, the pie Forks Library allows you to have a lot more control that allows you to have Dynamic size rafts that are changing with each batch of data and so on. And I come over the years of using people any libraries to really love this pytorch approach to training models. All right. So, ultimately at the beginning of this talk, I asked, should you pick tensorflow or my torch? And the answer is both. So, if you understand, it's very easy to understand the other,

if you're using people only professionally, you will run into both all the time. Anyway, you get a more granular perspective on model, shaming pipe, 1 in pipe. Watch, which makes it a little more complex and keris, but also more flexible, is more fun and easier to the bug because the Stacked races are surfboard tensorflow. As a stronger, open-source ecosystem for distributor training and more flexible model options. So I do highly recommend both languages. Can you translate between them using me? Open neural network exchange or Onyx

Library? So, if you're a brand new and he's learning, I'd recommend starting with a tense of blow Library, particularly. It's Kara's modules. It scares module, and then moved towards once you're comfortable with the tentacle Cara's approach. And then you can be creating your models, having a lot of fun doing that in jupyter, notebooks and use the Onyx library to Trello, perhaps for your production and you can take advantage of all the surveying Dolce s light. If your brand new deep learning

approach, you can do. Exactly that. So I have a special code here. You can access it a junk, run.com. / ml 20, all caps on the ML and then use the code. Ml contouring, check out that are 40% off my books as well as other books by my publisher Pearson in the machine learning space and 60% off my videos. So please just stay in touch later where I push out any new talks, any updates might get up repositories, have lots of content coming out on deep learning but also the foundations of machine learning

linear algebra calculus, probability statistics. Lots of big horse. Lunches. If you're here today, I'd love to connect with you on LinkedIn to get to put a career and a space to you know, your name in it. I absolutely love being hectored of what he feels on LinkedIn. I do have a YouTube channel where publish lots of free videos on machine learning into putting a particular, and I treated it as well. But I feel like I've now just talked way too much and so I will turn it over to see but he's very patiently waiting.

Cackle comments for the website

Buy this talk

Access to the talk “MLconf Online 2020: Deep Learning Battle: TensorFlow 2 vs. PyTorch by Jon Krohn”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Ticket

Get access to all videos “MLconf Online 2020”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic “Artificial Intelligence and Machine Learning”?

You might be interested in videos from this event

February 4 - 5, 2021
Online
26
106
ai, application, bot, chatbot, conversation, data, design, healthcare, ml

Similar talks

Vinay Prabhu
Chief Scientist at UnifyID
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Igor Markov
Research Scientist at Facebook
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
John Whaley
Founder at UnifyID
+ 3 speakers
Christoforos Kachris
Founder and CEO at InAccel
+ 3 speakers
Anmol Suag
Data Scientist at Blueshift
+ 3 speakers
Martin Isaksson
CEO at PerceptiLabs
+ 3 speakers
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Buy this video

Video
Access to the talk “MLconf Online 2020: Deep Learning Battle: TensorFlow 2 vs. PyTorch by Jon Krohn”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
949 conferences
37757 speakers
14408 hours of content