Tech Talk: Efficient hiring suggestions with a recommendation tool
Let's picture a job marketplace eager to increase its competitiveness in the market.
The aim is to offer users items that correspond to their individual preferences. The solution must be built so that it is possible to propose the items that will most interest the users. However, it must also be possible for an item to find different users who might be interested in this item.
This can be achieved by building a recommendation system they could integrate into their platform. This Tech Talk article explains what recommendation systems are, how they benefit companies, and the different approaches you can take when implementing a recommendation system.
What are Recommendation Systems?
Recommendation systems are machine learning models integrated into platforms to provide AI-powered recommendations to users.
Recommendation systems are a hot topic in data science, with so many big-name companies like Netflix, Spotify, and Amazon making them a core part of their product. In an age where AI-powered recommendations are becoming commonplace, it's important to know how they could benefit your company and how to build them.
Recommender systems can be implemented on many different platforms and in many different industries. The benefits they bring to a company can differ based on how they're used and what the goals are. Here are just a few of the ways they can benefit a company:
Deliver relevant content to users/customers - this increases the time people spend on your website.
Convert shoppers to customers because they are more likely to get recommended a product they need.
Increase revenue from customers purchasing more products.
Retain customers - customers happy with the products recommended to them are more likely to return for future purchases.
We will discuss the three most popular recommendation systems to give you an idea of what they are and where their possible shortfalls are so that you can better decide which would benefit your company. These are:
Content-based filtering
Collaborative-based filtering
Hybrid recommendation systems
Content-Based Filtering
The aim of content-based recommendation systems is to make recommendations between an item and a user based on similarity. The items that have the most overlapping features with the user’s preferences and/or characteristics are recommended to that user.
This method does not take into account any information on other users' preferences. However, it can incorporate direct and indirect information on the items and the users to determine their similarity better.
An essential component of a content-based recommendation system is the information we have about the user's interests - the focus is the user. Without this, the model would not be able to give decent recommendations.
Usually, users would be required to fill out a profile when they first sign up to the platform, where they can specify some of their interests. For example, Spotify requires new users to pick out their favorite artists and genres of music so that this can then be used to provide recommendations about similar genres or from similar artists.
Content-based filtering can help overcome the 'cold-start problem' when we don't have sufficient historical data on the user's interactions on the platforms. However, recommendations are specific to the user and for a very small subset of the items that may be offered.
Several metrics can be used to calculate the similarity between users and items, such as:
Euclidian distance
Manhattan distance
Jaccard distance
Cosine distance/similarity
One of the most popular metrics in this list is the cosine similarity. This method calculates the similarity between two vectors (the user and the item, for example) using the cosine of the angle between them. This means that lower angles between two vectors have higher cosine values and, thus, are very similar to each other.
Collaborative-Based Filtering
Collaborative-based filtering overcomes some of the shortfalls of content-based filtering. If we have information on prior interactions between the items and the users, this can be used to build a recommendation system that provides a wider variety of recommendations.
For example, if user A purchases a product and user A is similar to user B, then that product will be recommended to user B.
Similar users are grouped, and the model considers their interactions with the items to make recommendations to other users. Users can explicitly rate an item to show that they liked or disliked it, or the system can infer this information just based on the users' interaction with the item - if they click on it or add it to their cart, for example.
However, an important disadvantage of this approach is that it needs information on these prior interactions, and it needs quite a lot of it. In other words, it requires a 'warm start.' This type of data may not always be available, especially for younger platforms that don't have a large user base or many prior interactions to go on. In addition, if new items are added to the platform, they won't get recommended by the system until users interact with them or rate them.
There are two methods for building collaborative-filtering models: memory-based and model-based.
Memory-based methods calculate the similarity metric and then make recommendations based on an algorithm like k-nearest neighbors.
Model-based methods use dimensionality reduction algorithms to compress the user-item matrix. This method is especially useful when the matrix is sparse - in other words, users give a rating to only a small number of items, so most of the entries in this matrix would be blank. A common approach is to use the singular value decomposition (SVD) algorithm.
Hybrid Recommendation Systems
This approach combines content-based and collaborative-based filtering into a single recommendation system. This way, we can combine the best of both worlds and get better recommendations by overcoming the shortfalls of each approach.
We can build a hybrid model by creating content- and collaborative-based systems separately. Then, we can combine each of their scores using a linear combination with weights that we can specify.
For example, if there isn't enough data on the prior interactions of the users, we can give more weight to the content-based system. Once more data becomes available in the future, we can adjust these weights.
Closing Remarks
This article introduced the types of recommendation systems that have become so popular in the last few years. While content-based and collaborative-based filtering do come with some disadvantages, they can be incorporated into a hybrid system that offers a powerful approach to providing AI-powered recommendations in your platform.
There are two final points that you should consider as you build these systems:
Recommendation systems must be retrained regularly to stay relevant. How often you do this depends on the growth of your platform and how much new data comes in.
The speed and performance of the algorithm are very important when integrating it into your platform. Optimizing the code and the infrastructure used for this system is a crucial step that should be taken into account before and during the project.
To learn more about recommendation systems, Google has a free course that covers the topic very well. Additionally, check out the RecSys 2022 conference for the latest research and new ways of building recommendation systems (for a summarised version of the conference, Eugene Yan wrote an excellent recap of a few of the papers presented).
