International JavaScript Conference 2019
October 23 2019, Munich, Germany
International JavaScript Conference 2019
Video
Introduction to machine learning in Node.js | Tamar Stern
Available
In cart
Free
Free
Free
Free
Free
Free
Add to favorites
1.64 K
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About speaker



I am a software manager and architect.
In the past, I managed a development group in Edgeverve Systems and was an architect of a transaction engine. I also led a team of performance experts at NCR Retail, was a solution architect in Personetics, and I had a start up of my own in the social network field.

I have a decade of software engineering experience in various technologies: Server side, big data, mobile, web technologies, and security.

I am currently focusing on Node.js, and have a deep knowledge of Node.js server architecture and Node.js performance optimizations.

About the talk

Node.js is one of the most popular frameworks for writing server side applications now, and machine learning is rapidly gaining popularity. More and more problems are being solved by using machine learning tools. The use of machine learning solutions is spreading, and it is not limited to researchers. Software developers are also starting to gain ML knowledge and skills. In this talk, I will give you an intro to machine learning in Node.js. We will go over some machine learning fundamentals, the useful libraries that you can work with when writing an ML server in Node.js, and architecture tips about how you should design your server according to the nature of machine learning systems combined with Node.js capabilities. After this session, you will have a good basis to get into the area of Machine Learning and how to write and design your server.


Share

Hi everybody, and my name is Tamar and welcome to my session. Today. We're going to talk about machine learning 00:04 and no JS and it's going to be an introduction to machine learning and how to combine it and you are no dress code 00:13 mainly architecture wise and facing the problems that nodejs rises to machine 00:22 learning implementations. So, you know before we're going to Deep dive into the technical part, I have to introduce myself part of the dod. 00:32 So my name is Tamar software engineer. I managed are development teams stuff for architect. 00:42

And currently I work in Palo Alto networks Palo Alto networks is the largest Service cybersecurity Company in the world, 00:51 and I'm working girl outfit. They asked for the last four years my interest in nodejs is actually started 01:00 when I found in my own startup, you know, the application that rewrote was a 01:10 social network for food lovers. Then I rode my entire server in nodejs and at that point, I really fell in love with a 01:19 technology and started to like really explore it and become become an expert in that technology and you can all just 01:28

follow me on Twitter. That's my Twitter handle and the bottom of the screen and bold some other things about myself. 01:38 I Lazi finish my maternity leave. That's your little thing in the side of the of the slide is 10 months years old 01:48 right now. I have three kids and Toto and I'm also I'm also a professor. Violin player. I studied in the 01:58 Music Academy once and Elsa have a degree in computer science course and I'm also a community leader in the JavaScript Israel 02:07

Community. We have two lines of meet up and Israel attacks me go Community manager. So we have really cool Daphne whole staffs if you happen to be 02:17 around there is in Israel. There's a meeting of JavaScript Israel course, you're all invited after doing with the sir getting 02:26 done with introduction. Let's talk about what we can do with machine learning and for that I have I 02:35 have well prepared in advance. I really really small down low. So I would like us to classify some 02:45

image. So what in this image below? Yeah, that's a panda bear of course, but actually I have a server here 02:55 behind the scene. I mean the angular Clan is not really the point here. I have a server behind a sandwich. I'm going to ask what what's in this image? 03:05 And yeah, my server still need. Hey, that's a giant panda or find out or if I find a bear or I don't know. What is the last word but I don't know 03:13 why I guess it took us from some kind of a kind of species. But yeah, it sounds like really cool predictions. Let's move on. So, what is this? 03:23

That's the rabbit. Yeah, but when I'm asking my server what it is it tell me that's an air and air it's actually a female rabbit and that's an 03:34 interesting because it's recognizing is not as a rabbit is a female rabbit, which is something I can do. So, it's very cool. Now that's 03:44 a really sweet forward ice cream. So I'm going to ask my server what it is. I've got to tell me. Hey, this is an ice cream cool. 03:54 I've got an image number for and that is Amazing flowers and also my server identifies that 04:03

correctly vase of flowers are emergent number 5 bunch of balloons and asking my server. Hey what it is it tell me this is a 04:13 balloon it actually like really I don't know smart or something like that. So that's example is kind of cool because I'm going to ask my 04:23 server what it is. So that is a butterfly but my server is identifying the exact be shielded butterfly cuz that word here. 04:32 Sorry how to invest more my CSS, but that's like the butterfly species. Okay. 04:42

That is cool. Last thing is I am going to ask my server what it is is tell me how that's the valley 04:50 and that image is well those are cookies, but I'm going to ask my server what it is. It will tell me 04:59 if you tell me that this picture was taken actually from a bakery shop. I mean, so it actually like making more assumptions 05:09 then why I'm told him to do so that is really really cool. And these are some of them limitations if we can do 05:19 with machine learning. So what is machine learning exactly will actually has several building blocks. Okay. So the first 05:28

building block is having a model which was a model is implementing an algorithm. And this session we are going to mainly, 05:38 you know, use the algorithm has a black box. We're not going to you know, right machine learning algorithm or train them because this 05:48 is something we can take month has of courses, but we have a model this model is implementing an algorithm. 05:58 Okay, and the nest thing that you should do is to give your model a test date. OK the model is studying 06:07

your test data. We're giving him a data sets your data sets and it's steady the dataset but what it steady it's it's dry. Recognize 06:17 patterns in your data. Okay. So according to that these patterns the models making some assumptions 06:27 and according to this at the end of model needs to be able to perform a task with no instructions according to the bathrooms. It's 06:35 all in you know, the test data sets cool machine learning applications. I have here Spotify Discovery least I 06:45

have like an Amazon or other e-commerce. The product recommendations are based on machine learning email spam 06:55 filters are also based on machine learning even when you're getting into YouTube. You're like logging into YouTube and see what's your to recommend 07:05 you to see that's also machine learning. But before I don't know we can just see something so many patients. I would like us to get 07:14 into some fundamental. In order that's all of us would speak in the same language. So we're going to do it like a really really really 07:24

quick course in machine learning. So the first about the first thing that we're going to talk about is the model itself and 07:34 we need to of course. Well, these are actually just the four phases that are needed to build a machine 07:43 learning model. First of all, we're defining the motto and which algorithm is going to implement and then we are fitting the model meaning that we're 07:53 going to select some things that we want to predict. I will give the example of house prices. Let's say that I have a data set of house prices and I 08:02

would like to predict how much my house would cost. Okay, so I need to predict it according to something but say that I'm going to predict it 08:12 according to the number of rooms. The house has the size of the house. The type of the house whether that's a private house or an apartment. 08:21 Let's say that another thing that I'm interested in is how what is the distance of the house from the city center? So those 08:29 aren't the features OK which I'm going to see using every machine learning algorithm has a prediction Target and features. Okay, so 08:39

predicting it just what it sounds like we're giving after, you know, the model style patterns in the data according to the feature that it needs to 08:49 examine. So we're giving him udata and Dad's the model is predicting for example for house prices and models is going to 08:57 predict a price for a new house that it never seen before according to the parameters that if 09:07 needs to examine it and at the end we have something which is called or evaluation. That means last day that the real price. 09:16

The house is $100,000. But my motto has predicted $150,000. Okay, there is an error 09:26 over here of $50,000. And that's a lot and we do some kind of an evaluation process in 09:36 order to understand how accurate exactly is our medel now, let's see an example of a data set. So that is actually 09:46 at 8% which is Famous let's say and now you know in basic machine learning algorithm that is a screenshot 09:55 from a really big CSV of Melbourne house prices, which is used to do which is used 10:05

as like a dataset for a lot of simple training machine learning algorithm. Why do we have in this dataset? You see first of all, 10:15 we have the price and that is you know, mainly the most that the most thing that you know, everybody else is trying to predict and interesting thing. 10:25 That's we have here at which suburb this house is in how many rooms does house has what is the type of the house? What is the distance from the city 10:34 center who's selling the house? So that is like a really an image from a real date if that's called in Melbourne Melbourne housing. You can look at it 10:44

online. And now let's try to focus on choosing on which model to train. So I am going to 10:54 display to a model call decision tree. Why decision tree decision tree is very simple machine learning model. 11:03 I would say that it is the simplest that we have. Okay. I think it's really really easier to understand because it is based on the signatories, 11:13 which I hope that at least most of you had encounter decision tree and you are Development Career and it is the basic form are 11:23

complicated machine learning algorithm such as random forest, for example, but the basic running Force resemble is is a simple decision 11:32 tree and we're going to do just understand the fundamentals with this example of Melbourne house pricing with this one Jake's 11:42 ESV. So what is a decision tree? That's a really really really simple decision 3. I think it's 11:52 very straightforward. I'm going to ask whether my house is more than four bedrooms. If the house has more than four bedrooms that I would say. Yeah, 12:02

the price is above $150,000 and if the price is below at and if the house has lasted for a bedroom that I'm 12:10 going to predict that the price is below $150,000. Of course, it's not it's Kathy getting much complicated. So I'm here is like a 12:20 little bit more sophisticated decision tree with two levels. And so here I'm checking in the route. I'm checking whether my house have four bedrooms. 12:29 And then in the second layer on checking whether my house is like this size is smaller than 100 square feet or bigger 12:39

than 100 square feet. So Is that there are several leave so imagine my house like arriving through the roof and 12:49 then you know it just going down and then it gets to like one leaf and that's what my algorithm is going to return. 12:59 So, of course, it's like a really simple example imagine thousand the rules being created a huge decision tree, which is 13:09 classifying your data into a lot of groups every leaf here in the day to sat at every leaf in the decision tree would represent 13:18

a group. Okay. So what the algorithm is doing at the end is trying to understand to which group your house fit and I'm 13:27 going to keep that it's returned to fry. So that is a decision tree. It's real it's real bass. Just imagine, you know, I don't have enough place in my 13:37 slide. Just imagine like a huge decision tree with a lot of layers and a lot of roofing. So the statue is we're 13:46 going to fit the prediction data. Okay, what does it mean as we sat? We have to take features. Okay, if we're predicting house price, 13:56

for example, I wanna I need to predict how much do houses cost according to the number of rooms. The house has 14:05 less a number of rooms at the time from the house the distance from the city center SoDo zombie my features and then I would 14:15 select one call him to predict to call him to predict is the prediction Target and that's in most of the cases would be marked as Y 14:25 and we're going to predict the pattern we're going to find patterns in the database of X features and according to that. We are going to predict why 14:34

I would like to show you some code. Doing that. Okay. So let's see here. 14:44 Actually, I have the image classification demo. So I was just open. I 14:54 will just open the that's it. I was just open the damn. Well that 15:03 I have that I have made in advance. So I want us to look at what we have here. First. Dad is an Express server. 15:13 I hope that all of you is no developers are familiar with Express Library special Express library for writing servers. And 15:23 well this you know, this is a really difficult Express server simple with only one route called predict. 15:32

So after that, let's go to the model and let's look at the interesting part. By the way that the 15:42 folder structure is like I have this main files the server. That's the model which gives me I like the models MP3 itself. And do those are the 15:52 controllers. I would try to make it bigger. Okay. Okay. So here we are starting to see some this 16:01 sound machine learning code. So first of all, I'd like to talk about this this Library call decision free. I would like 16:11

to say that first of all and JavaScript and and knowed we don't have a lot of I don't know we don't have like the richness of libraries in machine 16:21 learning is not that big so I found out library and well that library was not maintained for 4 years, but it gave me like really surprising for 16:31 4 minutes. Okay, though, so it was really able to load the decision Tree Pass. I'm sorry too loud the Melbourne house pricing 16:41 very very fast. So and I don't think that we have very CSV to Json. It's just going to 16:50

help me convert my CSV into a Json array. So let's Where to look at here. So first of all, yeah, I 16:59 have to cease be here by the way. It's not very interesting just to CSV very big one and I'm here. I'm just 17:09 telling I'm telling my fire serial to CSV to buy specific columns as numbers and here is my date 17:19 and that light by the way everybody is seeing the code or I need to make it even more bigger or that is fine. Okay. So here's my day to set 17:29 and okay here. I'm doing another thing. I'm still to ring filtering places where my prediction Target 17:39

which is the price is empty. Okay. So why am I doing that? So handling empty columns 17:48 is a hard problem in machine learning and learning. And we're not going to get into it right now. There are ways to solve it. I'm not going to get 17:58 into it. But just you know for the basics, I'm removing the empty lines where the prediction Target is empty here. Okay. I'm just 18:08 going to tell my motto that I'm going to predict price according to the features in the future is going to be room distance and type 18:18

and here I'm going to get a new instance in of my Modell's at this Constructor surprisingly. The training is happening. Okay. 18:28 So the training is happening here inside. So am I that's why this Constructor is expensive. So my motto is 18:38 looking at the data and searching for patterns and in predict. I'm giving in you dated. Okay, so I was just a kid 18:48 up. and Why do I have low right? I have the image classification 18:57 server. Just listening at the same for it. So I would try to take it up again. Right now cool 19:07

debugger attached and I have here last thing I would like to show you. I have a small client that I wrote. It's 19:17 not very you know, what it's doing is just sending some requests with Express promise, but what's important here for you to see that 19:26 here is how am I test data? Looks like I'm going to stand is the rate one after another using promise all and 19:35 just I would like to understand what which prices my server is going to give me. 19:44 Okay, I would just go out here. I think it has to be larger. Okay. 19:52

And here I'm going to do in node Klein. Jazz. And for the first how did you had me about 1 billion and $600,000 for the 20:05 second house is gave me. Close a million dollars and that is the prediction for the third house. Okay. 20:15 So that's like the first machine learning code that we've done a node nice and let's continue now we're talkin about Eric's valuation. 20:25 Okay Valley writing errors. So why were you doing here last day that my testing data have 1,000 live? Okay, so 20:35

I'm not say that's for for each of them. The absolute value of the error is like 50000. Why does an absolute value? Let's say 20:45 they at the house price is less a $50,000 but my model has predicted $100,000 or the 20:55 $50,000 difference. So and you know, it can be a negative integer or a negative double or something. So we're taking absolute value 21:04 in order to measure the air and dividing it in the number of deaths object. And then we are getting the error. 21:14 We're going to look at some code. Which is doing that exactly and 21:22

that's my server. I would just open up a new one which of course I wrote in advance to that. 21:31 So that is going to be here. Okay, that's also working on 21:44 the Melbourne house pricing and okay. So here's the model again. We're doing here again 21:53 far sings the CSV putting it in memory removing the empty, you know price call him and then we have the class 22:02 name that we have the features that we have the new instance and that's the predict. Okay. And another thing that our motto is giving us I'm not 22:12

doing it myself. I could have ride a code which this which you know, calculate the error and understanding everything 22:22 and you know when divided it but well decision tree is giving me that functionality out of the box because it's really 22:32 it's really important. So what it's doing its decision to reevaluate and its needs to get an array of JavaScript. JavaScript Json 22:41 object. Okay and the controller here I would have two functions. I would have predict I am of course. I'm requiring 22:51

my motto. I would have predicted and predictive seen earlier and I would have value. And well my client itself. 23:01 So that is a test data and here I would just put the desde to whoever didn't see that that state I just look like that. 23:10 Json array with objects and I would just you know, put the test date out here under header. Okay 23:20 again using request request promise and I'm going to try to understand why does my head hurt when my arrow is going to be? 23:30 And if we're going to run that code, I know it's tax needs to be there that if we're going to run that code. 23:39

Then you're going to be going to see something interesting. 23:52 Well, yeah, the are evaluations for this data set is 0 I just read engineered it to be like that then my mother would you like super 24:03 accurate predictions now, actually, I just chose my test data wisely. But yeah, this model is not so accurate, but when it did, 24:13 so the real price of each one of them is is like exactly the prediction and that is why they are evaluation is 0 okay. 24:22 So after we spoke about basic principles, let's talk about popular know Jeff machine learning libraries 24:32

first library that I guess you have all heard about is Demeter flow and 24:42 tensorflow actually what it does most of the ugliest algorithms are working with mattresses and vectors. 24:51 Okay, so it looks for a fatter and is now Forces and vectors next 3 libraries are neural networks libraries when 25:00 they were allowed to Rose algorithm. We have rain dress, which is the most which is the most at maintained and famous Library out of these three 25:10 neuroleptic net worth and see nothing Jazz. Give us some more advanced capabilities in your left works. And the last one, Jas 25:19

was written by Sanford. Now, let's go back to our image classification demo, which was written 25:29 tensorflow. But before that just flow itself. It's an open-source. 25:38 It has like a sad a really nice models. You can train them yourselves. And what is also good about it that you have a lot 25:47 of models which were already trained. I mean, for example for image classification, I'm not going to train the mall on myself. 25:57 I'm not going to feed them all of a date data. I have a pretty five model which I can use and it is really cool. He has also like for you to find 26:07

model than you can use while taking a video in your computer camera and try to you know, try to look for stuff 26:16 in the video image which is received and you're just getting all of this out of the Box, which is cool. 26:26 Tell the base unit here is 10 sour and everything's are has some several properties. Like what is the rank of this stands over? For example, 26:36 just imagine like multidimensional arrays in a lot of mattresses. That's at the on how everything looks like. So, how are you doing 26:45

image classification with tensorflow and nodejs so I would just go back to 26:55 The image classification server that we have seen in the beginning. That was so widely detected what's in 27:06 a lot of images. And that's look at the tender flow here code here. 27:16 So, okay. So why do we have here that is again? Really really simple Express server Renee 27:27 node. Okay. It has one line of code, which is called classified. Okay. 27:37 And that route that Rob that he has with classify. Well here it switches. That's a server that is just getting an image 27:47

name looking at in the fastest in itself and then doing classified that is not the interesting part of the 27:57 code. Let's not get the interesting part and how we're working with counterflow and node. 28:06 So first according to tutorial you have to take to libraries you have to require their flaws and we have to require 28:16 down there full nude. Okay. So both of them are needed so it sounds are flow itself. It just needed as a global variable, but the other 28:25 phone I will give you other nice capabilities. Well converting let's a JPEG images stray two 28:35

towns are sensors themselves, which is cool. You don't have to do it in your room. By the way, when I rode the demo at the 28:45 beginning, I just your own converted the image itself into atoms are arranged a witch was like 100 lines of code. And then I would just like to wait a 28:55 minute 10:30 F note is giving me that functionality. I don't need to do that. Then makes my code reader really short. So what we're doing here. 29:05 Well, first of all, okay. Another thing we are using that library that library Mobile that is also its anger flow model and that is a model 29:14

which is doing the image classification and it is already trained. Okay, so that is the model itself and I saw a lot of intimidation 29:24 of it and client-side and server-side and it is it's very very easy to use. What you do need to do is just to 29:34 load it because it has to like, you know yet all that rain data pattern from some file that it is saving in the file 29:44 system. And after you have finished loading it then that is a classify itself and 29:54 after reading the file synchronously, I know I can read it asynchronously and that's very important, but I'm just doing because jpeg. I mean 30:04

my 100 light of code or summarize into one line of Dakota J bag and then we're doing classify and if I'm going to take 30:13 out this wait Okay, the server is up and 30:23 here also have a client with just sounds a lot of stuff. So if I'm going to run them. The Climb that I have 30:32 Is well just getting me like the panda image of the rabbit that we've seen it's getting me the prediction. So after we've seen some 30:47 code and tensorflow really simple code in Sumner flow back to go to code and brain JS, which is the library that gives 30:56

us neural networks algorithm. Why is well, it's use a GPU acceleration. What is GPU acceleration GPU 31:06 acceleration is like combining them GPU graphical unique with a CPU unit just like really high-level explanation that 31:16 makes that gives us better performance. It is really fast and easy to learn its implementing neural networks. What is neural networks? This is 31:26 a sad of algorithms which are modeled after the human brain and there at the end recognizing patterns. Okay. That's what 31:36

our algorithm Machine learning algorithms are doing recognizing patterns. What is in urine? So then 31:45 you're on the real urine. The biological urine is is it unit in our nervous system? 31:55 Okay, and that's like the main unit in or near system. Sorry. My microphone is falling down. Give me a second. 32:04 wait Hope it will be fine. And you're in is the main unit and our nerve system. 32:15 Okay, which what in urine is doing is just imagine like in our brains are like a really 32:25

really big Network on yours. And it's one of them is sending a signal to the other one. Okay, and when you 32:34 are a signal gets an input if we do is turn out okay and this app which is getting as a signal to the other neurons in our 32:44 you know when your system so we are using artificial neurons. Okay, we're not using real 32:54 brain and nervous system and artificial urine actually an artificial neural is just a mathematical function, 33:03 okay? And this mathematical function is getting an input and produce turn out. Okay, there's a lot 33:12

of mathematical functions that the New Orleans are using but just imagine of net of artificial in artificial objects in each one of them 33:22 implements and mathematical function. Okay, I'll be on my tablet. If we have this image of a starfish there. Now, let's say that this 33:31 new one is just transferring here. I have blue here. I have five arms and stuff. So it's getting to the end of the network and that's how it 33:41 is actually can allocate that. Earn a small. I said I have to you. 33:51

Something else that I would like to say is that you can see that combining the machine learning algorithm. 34:01 Sorry is can be quite easy because at the end after you understand the fundamentals it just it just like it's a 34:12 black box that you can use in order to do. cool stuff So what I'm going to do here 34:22 is I am I have the brain JS Library here and I'm going to try to understand where this sentence 34:31 was tweeted by Donald Trump or not. Okay. So, how are we going to do that? So I have your several 7 34:41

and if it was tweeted by Donald Trump, I am giving it know it's going to receive that outfit and if it was not tweeted my 34:51 tall frap is it's going to return to me fake from okay, for example, I'm not human and I don't speak English that 35:01 was tweeted by Donald Trump at some point. And if I drop soap on the ground is the ground is cleanest or is the soap dirty. That 35:11 was not tweeted by Donald Trump. Okay. So what miles worth of them is doing So first of all, of course, we are 35:20

requiring the brain JSM algorithm. I'm sorry the library we need to require it and I have every variable of train net 35:30 and I'm okay. I'm just exiting the neural network instructor here 1/2 process training day that will 35:40 what's process Training Day Day's doing what I'm doing here. I'm just doing encoding to each to each character that I have 35:50 and according to the encoding we are going to detect pattern and the sentence itself. Okay. So here I'm going to do train that to function 35:59

and that's like the Finish training and here is a function which Give this and you input and according to the new and foot. I'm going 36:09 to give us the result whether that was actually tweeted by Donald Trump or not. And that just open it and see whether it's 36:19 working. It would be working. So the phrase that we are examining here is when you're waiting for 36:29 when you're waiting for a waiter aren't you the waiter and that was not tweeted by Donald Trump that after talking about all of this. I 36:38

would like to talk to you about another problem that we have. I want you to pay attention to something which is interesting. 36:48 Have you noticed that the train and the execute down sometime async await our promises in their 36:58 code or call back? Have you noticed that? Nope. Okay. So here we are getting 37:08 to the next problem that we have with working with machine learning and notes. So 37:17 yeah, did you know there's a deal Griffin was cpu-intensive and synchronous. Okay. So right now I'm going to ask you 37:27

whether JavaScript is that go to programming language for machine learning and the answer is 37:37 what's its maybe I would say it's maybe closer to know but I would say maybe because we have several issues that we need 37:46 to adjust. Okay. So why are the main problem? So first of all the machine learning algorithm are cpu-intensive. Okay, 37:56 and most of them are synchronous, I mean whoever wrote them is not really not really understand the JavaScript internals. Okay, except 38:05

after float, which is the Which is like that the lights but beside beside of it, there are cpu-intensive 38:15 and there are synchronous manipulation. And this is not like a 38:25 ray array management is not really officiants in JavaScript in total. And the last thing that I want to say is that the 38:35 richness of Library isn't is not compared to a safe item? Okay that I want us to get into that cpu-intensive 38:45 problem and having said that let's talk a little about no GIF architecture and I'm not shelves out what's 38:54

happening when any request is coming through our server well wedding venue request is coming I call back is being registered 39:04 and to the event and then you know that the set of call back of those requests is getting into the event. Luke the Evangelist Q 39:13 itself, so we have any method which is running on single thread with one process and then we have a worker thread pool with a constant amount of 39:23 thread. So what the events will be doing is offloading I operations and also cpu-intensive operations to the worker thread noodles 39:32

wide is doing that because the Lord is working in a pattern called non-blocking IO. Okay, if you're working with other 39:42 languages by Java or python or stuff like that. These are working as blocking I'll make sense because if I am 39:52 example doing a database query my thread is blocked and waiting for the results and it makes sense because usually I want to do something without 40:02 result. Okay, but here the events of itself because it's using call back and the mechanism is saying okay while I'm 40:11

waiting for my date of is Corey my nuts with wood in the meanwhile handle other work that is generated from 40:21 other request. So if you're doing a CPU intensive operation or synchronous operation inside, even though you're going to block it that is not only 40:30 for machine learning but in general if your leg doing read files think like a lazy lead in my coat samples you're blocking de Bambu. 40:40 Yeah, we already discussed as it cpu-intensive operation will definitely block leave but what is better than showing you actually 40:52

let it happen and showing you some performance differences in order for you to believe me that I'm telling the truth. So 41:01 let's again open something that I have. Prepared in advance. So this is a demo. 41:11 of cpu-intensive operation Here I have like really really do single server. 41:21 I'm using the crypto that Marie and what I'm doing here is just cashing fast forward. So this Express server 41:31 is implementing Gwen route and here there is a password which I give and I am using cash password. Hash password 41:41

is just using the crypto API and is hatching eggs with a salt Okay, I'm using this algorithm, 41:51 you know crypto algorithm are really really cpu-intensive as a man to do a lot of work and that is asynchronous. Okay, 42:00 that server code is exactly the same but the difference here and that I would work with a call back and I'm using 42:10 the AC call the scripture where the promise here and here, you know, the operation itself would be 42:20 offloaded to the workers Red Bull. Okay, so I was just go to the command line. And I would 42:30

try to make it even bigger. So everybody would be able to see. Okay, so that 42:40 start with that synchronous. Server, I'm taking it up. It is up. Okay, and now I'm taking this down as well. 42:50 And I'm going to use a tool called autocannon. Okay? I think Adam is a really nice 43:00 library of mpm, which is helping me in order to measure stress. I'm going to do hear something very simple. 43:09 Just generate again request with 10 concurrent connections. Parallelly, so let's hear enter 43:19

and let's wait for that to end. 43:29 Okay. So when were measuring performance just like give you the the notion we want to look at the average. OK the 43:39 average request how much the average the average requested stuck and that religion of the 97th percentile and the 99th percentile. Okay. So here 43:49 it mean 99% of the request work faster than this and 1% white and 1% of the request or slower than this and 43:59 here we have approximately two seconds. That's the 50th percentile and the average is here is about one second that statistic is how many requests 44:09

per second are server can handle and that is on average is close to 9 44:19 request for a second now. I would like to take this server down and I would like to take out the ace and Krypto server. 44:28 So here that work safe and chronically LS do the test again and Let's see, which results were gone again. 44:39 Okay. Black stars compared with an average here when I'm working a sequel to sleep. My server was able to handle 44:52 17 requests per second when I'm working to fall asleep myself server was able to handle 9 request for a second 45:02

almost double. Okay now I took it as their measurements then working 45:11 a sequin sequin see the average of the request is about 500 milliseconds like half a second when I'm 45:21 working synchronously. The average request is one second. That's also double. Okay. So do anything since doing 45:31 cpu-intensive operation synchronously can really really really harm your server performance. Okay. So 45:41 try not to do that not related to machine learning but also related to file reading and I don't know anything else that you 45:51

want to do. So how to deal with that. Okay so we can for can other process. 46:00 Okay. Now that is interesting since node 10.5 there is a new IP. I called worker threads. Okay, and it can be used for 46:10 to solve that problem. It is running threads and known and each worker thread have its own Odin since its own old engine 46:19 but it is still one process and it's only been so pleased worker thread has his own. So that's an API on Hue API. That's really nice to 46:29 consider. And the last thing with that where we will briefly talked about is scaling up with q's. Okay, so child versus I 46:39

hope that all of you are familiar with child processes. Okay, so she'll process model you're forking another another 46:49 processing there is like a messaging and messaging system between the master process of a child process that help them communicate 46:58 and in Dallas. Cases for example for machine learning code and for them criptico that I showed you really really 47:07 prefer not to use it and actually prefer his delegate. Just don't do it. Okay y Okay, one more working with 47:17

Chad processes. Usually we met a process to a CPU. Okay, like our know this is map to one 47:27 of the CPU in our machine. So we met a process to a CPU and usually the implementation of that is having example in master process with if I have 47:37 857 slave processes in each. One of them is Matthew a CPU and it's working. Well if I'm working in a server, okay, which is 47:46 serving Y100 request for a second. Imagine that I would have 100 new processes 47:56 open in one second, which would call my instance my machine to crash. Okay. So just that something that you should not 48:06

do working with work or bad as a sad. They're available in New Or No torsion. It's experimental 48:15 until version 12K conversion of 12 and has became not experimental. And yeah, and they 48:25 will do to execute JavaScript in parallel. And as I said one process multiple thread an event look for each thread 48:35 and a JavaScript engine free read. It's it's a really nice API communication between the world which we worker threads are with 48:45 messaging and Lexie how machine learning code is fitting into work or said so 48:54

I'm not sure that we have is time to run is even or just like show you the code itself and now, you know, everything is going to be available on 49:03 Twitter at the end so you would be able to run it. Okay, so, okay. That's the machine learning 49:13 model that we have seen with the Donald Trump sentences and coding and everything. So 49:23 that's like the main file that I have here. I'm requiring workers thread and I need worker main thread and Perrin Port. Okay and 49:32

what I'm doing I'm doing a new worker. It's it's really really resembles working with child processes. I'm giving it a bath to a 49:42 file. Okay, and I'm giving it work or data and I'm trying to catch on message again and that's how they're working and worker itself 49:52 what it's doing. It's just executing the machine learning itself and all the training is 50:01 done in the later even in an earlier face. So that's how workers that's cold is doing a scan. Be 50:11 nice to solve this problem by the best solution that we have is scaling up with you. So if some of you are familiar with 50:21

components like rabbitmq, for example, or if you're working at gcp Google pubsub. So what you're doing is left side of my server is going to send a 50:31 message to q and so I have consumers which I listening to messages. Okay, 50:41 and if I have an orchestration layer like kubernetes, for example, then I could say hey we're Nazis map a consumer that I have 50:50 to a CPU and the cloud for example, and if I have a lot of messages in the queue just add more consumers automatically according to the leg is called 51:00

the leg in the queue. So why is that the best way to do it because When you're working with worker threads you are 51:10 limited to the hardware. That is currently. That your instance has okay. So you're 51:20 limited to that if you're working with rabbitmq and a Docker and kubernetes, then you would be able to 51:30 scale your consumers and do much more. Okay, so that is like for CBR and that's the rub for machine learning. I think that would be the best way to 51:40 work. I think I took all of my time. Do you have any questions? 51:49

I need to set an alarm. the time that possible you can be done 52:04 with the library's phone number 12. Yeah. So for example brandy brandy s is using GPU you don't have to do anything in order for it to use 52:14 your future building as most of the libraries have GPU built-in. You have to check whether each library has GPU or not. 52:24 other question Yeah. Yeah. 52:35 Ricky Museum together so for each consumer use worker threads and you know dockerize using kubernetes and working with a cute. 52:46

Yeah. It's possible. Any other question? Okay, so thank you very much. 52:56 That's my Twitter handle if you wish to get the demos on the slides through going to be uploaded to their here. So if you have any questions, I'm 53:06 available. Thank you very much and so fast so you will be first in the queue for lunch at there's a lot of food outside right now. 53:15 I know that's a hike. 53:25

Cackle comments for the website

Buy this talk

Access to the talk «Introduction to machine learning in Node.js | Tamar Stern»
Available
In cart
Free
Free
Free
Free
Free
Free

Video

Get access to all videos “International JavaScript Conference 2019”
Available
In cart
Free
Free
Free
Free
Free
Free
Ticket

Similar talks

Krzysztof Piechowicz
Senior Software Engineer at adesso AG
Available
In cart
Free
Free
Free
Free
Free
Free
Christoffer Noring
Senior Cloud Developer Advocate at Microsoft
Available
In cart
Free
Free
Free
Free
Free
Free

Buy this video

Video

Access to the talk 'Introduction to machine learning in Node.js | Tamar Stern'
Available
In cart
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
439 conferences
17532 speakers
6299 hours of content