About the talk
Modern recommender systems require diverse data processing and feature engineering at tremendous scale and usually employ heavy and complex deep learning models that require expensive GPU clusters to shorten the training time. Techniques for developing an effective, performant recommendation system remains a challenge for most data scientists and engineers.
In this session, we first survey the recommendation system landscape. Then, we walk through challenges endemic to building a recommendation system with a focus on the cost prohibitive nature of training a new system from scratch. We then propose an end-to-end solution that makes it much more cost, resource and time efficient to develop a recommendation system. This solution will include practical ways to optimize parallel data processing based on Spark and hyperparameter optimization and model selection with SigOpt. We will then apply this method on commodity CPU clusters to demonstrate how this combination of tooling boosts the pipeline efficiency for typical end-to-end recommendation systems like DLRM and DIEN. Finally, we’ll conclude the talk with a discussion around the future of this process and where further research would be valuable.
Jian Zhang is a senior software engineering manager at Intel, recently he and his team primarily focused on implementing and optimizing end to end AI solutions on distributed CPU cluster, democratizing AI models to improve scalability & usability on commodity hardware. He has 10+ years of experience in software development and optimization, ranging from open source virtualization systems like Xen & KVM, distributed storage systems Swift and Ceph, to big data and AI systems. Jian holds a master degree in Computer Science and Engineering.View the profile
Hi Aaron, welcome to the Seahawks Summit. Welcome to my session driving into my name is Jim Tom. I'm from in her eye and now she's going to text you and Let's take a look on the agenda today. So we will talk a little bit about what his recommendation system. What is the look at why we need to enter in a argument, which are we trying to work with? What we have accomplished so far? So what is a recommendation system? Recommendation system is the software that I listen to the interest and a preference of individual customers
to make recommendations. Accounting. Wait. What about I should buy and what is the best place for my family? There with different item, too easy for me to upload Genesis to see you over the past few years is costly model. You comments else can be translated into a different direction. Bookstore, for example, of recommendation, link on amazon.com, use it for you. The personalized, the recommendations on the user profile for age and relaxing. The second type music library, to record information from many users
to provide a pretty, like, a tell me. What is the most popular among? My peers is flax and analyzing information, likely calls the similarities between What can I consume? What is a what I have liked? It is based on a profile direct items similar to which a given item meatless Mondays on the items. Knowledge-based recommendation. Explicit explicit about the item apartment. You the property, the recommendation. What species am I need my grandpa? When you trying to buy a car
you may have a specific requested on the car, the non-existence of a problem. The last one is a hybrid. Where is a combination of input and the composition of a different? Because most most recommendations today? Now you the hair braid across the property to Beauty and the Beast and other approaches. Wheelies, let's take a look under recommendation system architecture, a typical recommendation system, can be divided into two different today. The first day
we stopped to use the narrow, you can use that processing processing or baskets, for planting tomatoes and trying to make the motor for an hour more often. Is it pretty challenging evolve? Recall writing and rewriting the recall, try to generate a small item, might be interested in a file. All the products are characteristics between spacing need to be extremely fast. Is the most critical part of the mall in Centereach. The first and foremost, one amount will
run into separate and Protestant. Atlantis. The future in urine will create a numerous issues, need to be crated and Engineering technologist like that. And at least a little creature on in Trabuco sitting in the complex models, with me, even with all these different two parties, you need to repeat it is, too because I didn't know, you can get a single. With all released, a feature of the recommendation system in Banning and memory power. Let's take a look around. What is the what? And what we can do
to make everyone but secretly training, Marcos socket 60 faster. We want to make you feel about to put gas in the car, which is better. Do the data platform is the way you need to make the easier and simpler as Pastor to Call Me. Darlin computer in my car, Computing infrastructure that provides which means we want. We want to let you know, so expensive. We need to reduce ain't a bad area, like the automatic switching all-time hour to improve the Explorer and it's me while we want to see lots of the best to the
most beautiful models and we need to be safe and least better. Not. So, where are you? You want to? Assassin tomatoes and models and Arkham release. Our goal here is pretty straightforward. We want to bring entering the existing throw tomatoes with green cases where you are seeking for the highest score and Thalia Elementary. Unblocking me wanting deep knowledge of how to get to the Comcast modem from entering perspective and reduce the time to. And then we trying to identify the critical of the meeting functions and
improved nutrition. Rustica, hello, several examples. We have been tried and the performance P. We have seen this here. We threw a 100 times. 17 times speed on a 9.45 speed for the armoire 2020 respectively. Reviews of the MLB leader reduce the community communication faster. We are inviting an insult. Wheelies, I will give a freaking charging over several different interview. We actually Call using spark comparing the last picture and the picture last picture down by the original
grande2go. We develop the project call, DirecTV recommendation system. They processing a hamster. The motor will be down the price per night. Also, providing. You need to use a p. I do not need to worry about how to run Sparkle also doing. Bark, like a spear into a joint strategy. Like the England, Skylight UDF functions and the also leveraging the interactive Master, an extra package, falling Pro stock performance easily into a form. So here we want. So we can
do the day trip from the they processing. The centralized Authority directly. It also. So we also need the number of a picture plaque, local catching to improve performance and also support data from the library to buy some pasta with several will ever income pension Road in powder, will not release the compliment in the past or we can we can do anything with last dispatch directly without him. Menus. Play Chaka. On the training on, Lisa's it out, wide and deep moral example.
They ensure optimal the principal. Oh, we apply the open champion champion Edition, which day was 1.9, speed are trying to reduce the size. And also you need to leave it at one point. One possible. Are we all going to stop? Because the the seams go with even more than I do with miss you all so much. Out of time by 58 time. Just tell me how do you replace the previous two single Friday. Are we needed to how we need to figure out how to change the optimizer to new framework and scallops together.
So we need to evaluate the how can we Larry to see doctor to do the monkey bars easier to check around the trainings? And thus you have a m and an improper fraction form. What are we trying to do here in the warm to Larry Cigar Lounge? Where does the doctor with DirecTV? And what we have. Which is a Generation ship Model, A M S pro automatic? The priest Walks Like A FD generation Department arrived. C. Penney, filed, last year, national average, cigar pricing at Belletetes it
wrong through locking, gas, cap from Chicago, to the mattress and a report of the of the reasons, then we can go to group outside of the golf course with the rack BP and it works pretty smoothly for Wii. As a single project and the wheelies plastic roll of ground beef. I'll provide Emmanuel foreman and it is Armand. Like the natural cleaner sizes, getting Pizza, 51, my staff directly, if they have some kind of inside knowledge, always have their own different shirt,
preference. Cleaning process and antibodies can be reduced. Imagine that. Previously completed the process to remind me about speeches, like mathematical Madison, Magic's record. 38. What's the apartment is an accident right now? On the apartment. I'm a few Valium. I'm in Catawissa previous menu of my tomatoes menu, shooting of my receipts into all that. Now, this will limit the entry barrier position in the best value, all the best. Camera reviews, like 3-4 20 + 1.07.
Not only bad. She coughed also provided a easy to use and understand you. I will help you to come here tomorrow. You'll explain the Renaud which is a ground hatching. Many centimeters is the mountain, vertical axis graph with magic and hermit has the horizontal axis is audited by the perimeter, important as the most important information will be limited in your rent. Make sure you able to see animals from eating from each region of my lease. Will she go after all, we can come up with hire, a maid
North menu. The last part is we want to take a close look at how she got the chips that we tried so far. The performance before we in your body to Features. We actually noticed, likely Columbus regression with a single magic out right now. That would have been sold from the left picture of the menu of Mazda models, the best time with no matter if we do two acts of aggression and we talk with each other, in bring up on this back and not for the reason it would compete. Competing matchups. Like,
we might have Optimum values for different from each other and trying to maximize the Motor Performance and the meaning of the train time. Reduce an extra minute to get out of the bath tub models. Meanwhile, let's play. I will be sitting up in a tree in Tamarac. Elementary School. Elephant, take a look on the graph. This is an explanation of how much magic sanametrix rockhold houses from the West Indies single match tomorrow and current. How many people are increasing, amounts house fly
out and we added a message to Raul Malo. Time to 3:40 a.m. To ask Harmony to increase the human wheels off A Time, release the key, where she got the house. How to wrap a child? Antoinette assistant off ml improved, access ability. So tonight it is processing like a model. And how come you give me a real chat and make it easier to see all the models were from being a know-it-all.
Buy this talk
Buy this video
Our other topics
With ConferenceCast.tv, you get access to our library of the world's best conference talks.