Events Add an event Speakers Talks Collections
 
MLconf Online 2020
November 6, 2020, Online
MLconf Online 2020
Request Q&A
MLconf Online 2020
From the conference
MLconf Online 2020
Request Q&A
Video
A Novel Approach of Bench marking Recommendation Systems
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Add to favorites
44
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About the talk

Recommendation systems are growing progressively popular due to their ability to offer a personalized experience in a unique way. Leading them to become a very useful tool for different domains from commercially to research. The algorithms used in recommendation systems perform differently depending on the domain and task. To determine what to use, the designer must choose between a set of candidate approaches. Within this process, it must decide which properties of the application should be focused on when making the choice. As recommendation systems have different properties that affect user experience from accuracy, scalability, robustness, and etc.. This causes us to evaluate recommendation systems in many, often incomparable ways. Within this paper, we will discuss an optimal way of evaluating recommendation systems based on different properties and application scenarios. Based on our research, we will propose a novel evaluation concept for recommendation systems based on ideas from research and industry to ensure quality when deployed to the user. The goal is to ensure the recommendation system is significant and goes beyond the conventional accuracy criteria.

About speaker

Vanessa Klotzman
PhD Student at UC Irvine

Vanessa Ilana Klotzman is a first year PhD student at the University of California, Irvine in Software Engineering. Vanessa comes from industry experience from working in Software Development and Data Science. She obtained her Bachelor of Science in Applied Mathematics and a Minor in Computer Science from California State University, Northridge. She continued at Northridge to receive a Master of Science in Software Engineering with Distinction. Her doctoral research focuses on machine learning ,but specifically using machine learning to enhance the software development life cycle. In her M.S she focused on utilizing recommendation systems to improve the four year graduation rate in the Cal State System. Apart from spending countless hours of studying and doing research, she is a Pilates guru, loves fitness, loves to cook, devotee to coffee, travel, life long learner, arts and crafts, and spend time with friends and family

View the profile
Share

So basically, today we're going to talk about a novel approach to benchmarking recommendation system, as I've usually we use different properties for evaluating recommendation systems and different metrics and classification or accuracy or or coverage. But the goal is to ensure, we have quality in a recommendation system and it goes beyond the conventional accuracy criteria. So basically what we're going to do Spurs, starting off with the gist of things. Basically, a recommendation system were supporting users and fighting items. They would be interested in. Think about using

machine learning to help users discover new products or services. Like every time you shop online, a recommendation system is guiding you to product me. What you might want to purchase think of recommendation systems like your personalized salesman. They know you basing your life preferences and much more. So basically the motivation of This research is on because we can think about like the Netflix we can think about the Netflix price and then also Netflix price and also to just think about maybe like your typical Netflix

or Amazon or Amazon that sometimes that maybe it doesn't really know that maybe we need to be using more criteria to to evaluate how successful your recommend, you like your recommendations or not, or if they will satisfy the use of the user. Automotive base. So the motivation basically of This research is that despite despite commercially and academic research, a systematic evaluation model. Is missing which we need a systematic evaluation model that considers all aspects of requirement and data can sometimes not always be great. And sometimes it's

too cold for you. There's two spots by their information needs in a collection and sometimes there's disadvantages and advantages to each machine learning algorithm. So maybe we could take into consideration of evaluating recommendation systems between functional and nonfunctional requirements and business requirements. So basically what I'm proposing is maybe that we could be using a maybe like kind of a benchmarking technique and use different metrics to body weight, the quality of our a recommendation

or recommendations system functional and nonfunctional requirements. End-user end-user requirement. So basically with the business with a business model, if you took a business model into consideration with the business model, it can help us say maybe we could evaluate the effectiveness of a recommendation system based on company revenue or how can we assess if the company is generating enough Revenue? Maybe we could do this through our click-through rates, or adoption in conversion, or

username gate, user engagement or how, how do you say our house, the sales distribution? I can sum up as we want to improve the customer experience and information gathered from the recommendation from a business app. So it can be an asset to the user. We don't want to persuade. Teaser teaser system, so maybe we could use me metrics like this from a business perspective. So then what's so then we could also then take into consideration. The next expect is a user. Requirement is

user requirements. As with user requirements, we want to take this into consideration as having strict user requirements will help persuade the user. And as based on personalization the recommendations made Patti user to keep using the product. So like maybe like a knowledge-based recommendation system, can help explicit user info and item assortment of presents and based on the knowledge system. Patient. As unlike other approaches. It does not depend to a large bodies of Scott data, even though recommendation systems are the heart of the most

straightforward way, may not be the best selling long ways of similar products Customer Loyalty. So maybe about the fact that Miss of recommendation systems with user requirements. We can use me be Advanced and non-traditional Tech from Deep learning social learning tensor factorization, which could be a step forward. Acquirement embedded, simple methods, like adding mass at, like adding Mac product recommendations or purkett, purchase confirmation for collecting data from abandoned shopping carts or what, customers are buying now and sharing other, customers views and

performance. Improved customer experience, which is the key to all value in a business. So another, so on top of user requirements, a balling, the effectiveness of two value of a recommendation system is maybe we need to look at the technical aspect. so, I'm sorry about that. Sorry. So we need to look at the technical aspect and sometimes with the technical aspect as it will, it'll give us the big picture and specify the technical parameters of the system. We can't improve after it. All right, if we can provide

accurate recommendations, The Rock system is invaluable. So poor quality does not help the business in any way. As a real life. We must have been a consideration technical requirements. So we could be like a functional requirements together from system, scalability robots. Dissonant adopt ability. How to scale buildings. Can the system growing capacity to meet the rising demand for services after we adopt use lead to an increase of resources like memory? We do not want

slowness in the system and rocks to be returned in a timely manner or data. Constrain television shows, television shows, missing be a missing a season missing a season or Missing sorry but missing a season or recommendations return faster on iOS versus Android or robustness. How well did the system respond with errors to ask if there's errors as at the end of the day, the business value of the technology is not examine, and they still looking at Tech, like, Amazon Netflix or Spotify, even though accuracy main fluid

and rubbing you indirectly there exists. No soloway to evaluate dimensions of a user of user requirements and business models whose cases are ranking and classification and challenges. Like the Netflix flies to use explicit ratings to profile users. So I'm basically in general like Technical and Technical requirements on construction should be together. Also, True Value Edition methods for recommendation, systems is offline and online recommendations systems are often evaluated offline, but it off

like it's easier to deploy evaluations are easier and reproducible. The currently, basically what we're missing, what were missing and currently we have like the metrics. We have can basically, use the requirements aren't really considered and sometimes non-functional requirements are not considered have to give us a warning. If it does not reach satisfaction to the user, as at the end of the day, at the end of the day, we want recommendations. Is that reached out at auction as functional. And function of a non functional requirement

should be evaluated together as we can't provide accurate recommendation, recommendation systems, not valuable. So, thanks, maybe we should take into consideration, is why not reactivity. As with reactivity. We can provide recommendations and real-time and the froeschle depends on the style. You, or we can do scalability. It provides quality recommendations independent data during initialization using computational Resources with data size, and it serves

large amounts of parallel recommendation. Request in real time without significant degree in service or adaptability. Should we be able to reach him reach and change of user preferences? Or. We're back next to me handle the parcel, missing or Krug data. No matter what states were in as generally speaking are mutually exclusive if you want to. And if you want to optimize during recommendation system. We should have a combination of metrics, being evaluated

together when the recommendation system deployed as there's different criteria for each, each type of recommendation system depending on the industry. Okay, so here are just some of the metrics. Here's some of the metrics. Maybe we could start as we could take into consideration for our recommendation system if you want extender message. Okay. And curly existing that tricks that we have ranged from classification predictive coverage confidence and learning rate and really basically on some of the criteria that we use for recommendation systems. Now,

it's hard for these metrics to cover maybe functional requirement of a non functional requirement or maybe use a requirement if it's user satisfaction with the system me from u. I r u x and even if the scores are good for this recommendation system if we're trying to evaluate. Unfortunately, the state-of-the-art mattress that we have from diversity novelty Serendipity and user satisfaction or difficult to measure offline. so basically, Basically, when I'm proposing is that on, would it be possible to have a system?

A system recommendation system that uses is functional and nonfunctional requirements to stand. Evaluation aspects of a real-world recommendation system. So maybe can we evaluate a recommendation on top of accuracy and precision and recall to be in Valley with the recommendation system on how? Well does me the user in how? Well does the user like the user interface? Or maybe can you look at it from a business aspect like me from click-through-rate or how much how many times a day does that use a return to the actually

can't do that. But like four. Or like, if you know or user or maybe we could be looking, right? User retention, if maybe we can look at more than just like a theoretical ask, if then those theoretical numbers when we get, from all those to Tuscola valuation, as we want, as we want to be able, we want to do. I'm making your system effective productive, and having increased uptake, so users can continue to return because recommend think of recommendation system like a personalized salesman. They know, they know,

they know you and since they know you, they want something know you, you'll probably keep coming back because they give you a good suggestion. And if they don't give you some good suggestions, then why would you want to come back and recommendations systems for the economy? They beheld, booster customer retention in different Industries. Thank you so much. Thank you so much for hearing my talk today. I am so sorry for the technical difficulties. But if you have

any questions, please feel free to reach out here. Some of the sources that I utilized and I will be uploading everything online. So people can access. Thank you so much and have a wonderful day.

Cackle comments for the website

Buy this talk

Access to the talk “A Novel Approach of Bench marking Recommendation Systems”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Ticket

Get access to all videos “MLconf Online 2020”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic “Artificial Intelligence and Machine Learning”?

You might be interested in videos from this event

February 4 - 5, 2021
Online
26
104
ai, application, bot, chatbot, conversation, data, design, healthcare, ml

Similar talks

Igor Markov
Research Scientist at Facebook
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Meghana Ravikumar
Machine Learning Engineer at SigOpt
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Jake Shermeyer
Research Scientist at IQT CosmiQ Works
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Buy this video

Video
Access to the talk “A Novel Approach of Bench marking Recommendation Systems”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
949 conferences
37757 speakers
14408 hours of content