Recombee Research Laboratory (RecombeeLab)

Recombee develops a recommender system in the cloud. Using a vast amount of different kinds of data (user behaviour and preferences, information about content and products), Recommbee recommends interesting content tailored to fit individual users. Small and medium-sized companies around the world use the system (currently in 27 countries). Thanks to the system they don’t need to build their own data science teams and are still able to offer their customers personalization.

Recommbee’s aim is “good” personalization, and as a team of creative people we are most proud of use-cases that bring real added value to users, from recommendations on similar classified listings in second-hand shops to recommended multimedia content. On the other hand, we practically do not deal with ads personalization, which very rarely brings users any added value at all.

The system’s core relies on sophisticated machine learning and artificial intelligence algorithms that process the available data in many different ways, with the one aim: to use them to create useful recommendations for users.

What we do

In the laboratory, we mainly focus on research into recommendation algorithms and the exploration of different research avenues in this field. Our great advantage compared to similar laboratories at universities around the world is the contact with real users. Thanks to the fact that the Recombee system has hundreds of product integrations for millions of users around the world, we have something that most researchers in the field don’t have: real and extensive industrial datasets and the possibility to test algorithm prototypes in real operation using A/B testing.

Members of the laboratory

doc. Ing. Pavel Kordík, Ph.D.

Equipment

Although the laboratory itself might look rather ordinary at first sight, its most important facilities are located outside the lab – in data centres from which Recombee rents high-performance hardware. As part of this collaboration, students get high-performance hardware for computationally intensive algorithms (for course projects, bachelor and diploma theses). Servers with graphics card GeForce® GTX 1080 can be used for deep learning and servers with 40 CPU cores or 256–512 RAM can be used for algorithms with high demands on traditional models of computation.

Publications

Comparing Offline and Online Evaluation Results of Recommender Systems

Authors

Kordík, P.; Řehořek, T.; Bíža, O.; Bartyzal, R.; Podsztavek, O.; Povalyev, I.P.

Year

2018

Published

REVEAL RecSyS 2018 workshop proceedings. New York: ACM, 2018.

Type

Proceedings paper

Departments

Department of Applied Mathematics
Faculty of Information Technology

Annotation

Recommender systems are usually trained and evaluated on historical data. Offline evaluation is, however, tricky and offline performance can be an inaccurate predictor of the online performance measured in production due to several reasons. In this paper, we experiment with two offline evaluation strategies and show that even a reasonable and popular strategy can produce results that are not just biased, but also in direct conflict with the true performance obtained in the online evaluation. We investigate offline policy evaluation techniques adapted from reinforcement learning and explain why such techniques fail to produce an unbiased estimate of the online performance in the “watch next” scenario of a large-scale movie recommender system. Finally, we introduce a new evaluation technique based on Jaccard Index and show that it correlates with the online performance.

A tour of the lab

Where to find us

Recombee Research Laboratory
Department of Applied Mathematics
Faculty of Information Technology
Czech Technical University in Prague

Room TH:A-1354 (Building A, 13th floor)
Thákurova 7
Prague 6 – Dejvice
160 00