Get the insights from our inaugural Recommender Systems Meetup in Berlin
At Bibblio Events we recently organized our first-ever RecSys Berlin Meetup, hosted by the team at the German Tech Entrepreneurship Center. The data science get-together took place at their offices at WeWork Potsdamer Platz.
Read on to find out more about the work they've been doing to improve rec's at their companies and grab the link to Gugu's presentation slides too.
Feeling the Love for Recommender Systems
"Based on gender, age, location, sexual preference and a few hobbies we could already identify retention rate and lifetime value to 90% accuracy - and that's back in 2013." - Lele Canfora
Lele is an entrepreneur, speaker and project management consultant for block-chain, artificial intelligence, data analytics and business intelligence. He kicked off the evening by speaking about his experience at LOVOO, a company building a dating app a year before Tinder took off in 2014.
Lele's experience at LOVOO exposed him to the true power of machine learning, and not just on the user side (where people would be matched), but also in the form of the internal system put in place to predict the revenue growth of the company. Lele remembers being impressed by the power of systems utilizing "statistical libraries and primitive Scikit-learn".
He took what he learned to start his own company Deckard, which focused on predictive analytics for project management. Their system would ingest information from the software development team of a client (e.g. from Jira), crunch it and calculate size of tickets for sprint planning and suggest names to work on them, and when, too. Unfortunately, the company didn't last as it proved really hard to fit into the workflow set-ups of companies.
Now, Lele focuses on enabling connections between enterprises, startups, investors and engineers at his new company Beyond Machine. His goal is also to increase public understanding and discussion of the implications of machine intelligence. To get the audience thinking more about that space, he brought on Rupert Steffner, founder and CEO of wunder.ai, and one of the people he works with.
Rupert's company offers a B2B service, optimizing the eCommerce experience his clients offer their users. Part of their service includes a product recommender. The most popular use cases are what he described as 'Next Best Offering' and 'Most Similar Item'; the latter being the item you show when end-users are searching for an item that is out of stock.
In his short talk he discussed the most common challenges and biases associated with this recommender case study. This included well known issues such as the sparsity problem, but also stability bias where your systems always expect the same behavior of a user over time. As Rupert pointed out, humans are unstable creatures that can change their minds at any time. That's why every year we're spending billions on marketing as a society, he added. Next up was the relevance bias. This describes how many companies mistakenly treat the output of nearest neighbour algorithms as optimized personal preferences.
Another well-know bias, which plays a huge part in recommendation experience, is the position or presentation bias. You can't choose what you cannot see. This bias would play a key role in the evening's last presentation about restaurant discovery too. See below for more:
The Recipe of Delicious Recommendations
“Well known and popular restaurants receive a substantial number of orders — customers get what they expect and naturally give great ratings. Some customer groups however do not want popular restaurants in their recommendations list.” - Gugu Ncube
The last presenter of the evening was Gugu Ncube, Director of Search and Discovery at Delivery Hero. This online food ordering and delivery marketplace maintains a network with more than 150,000 restaurant partners. Delivery Hero is active in more than 40 countries.
The company currently handles 1 million orders a day. Gugu kicked off by elaborating the goal of recommendations in the food context: enabling customers to discover new restaurants.
Gugu mentioned that, after trying out different algorithms, the content-based algorithms worked the best for the recommendation and discovery of restaurants. His team created a model which captures passive preferences of customers based on past orders on individual ingredient level, and then ranks the restaurant by the rating it receives for orders that include dishes with those ingredients.
Other key factors influencing the recommendations are recency of orders at a particular restaurant, cuisine, time of day and budget. Below is the content algorithm in detail:
Gugu outlined the architecture and tech stack of their recommender system. He concluded with a captivating discussion on the biggest challenges his team is facing when building great food recommendations:
Data sparsity, also mentioned earlier by Rupert, is one of the main challenges. Customers often repeat orders at the same restaurant - they have very few touch points. As a result, user-based collaborative filtering models produced mostly popular and somewhat random restaurant recommendation. This motivated them to research into content-based algorithms. Early success gave way to other new challenges: inconsistently labelled restaurants meta data. The meta data of dishes such as ingredients and cuisine is typically labeled by restaurant owners. Some restaurant owners may omit characteristics, which they view as insignificant.
“The cuisine of a restaurant may be labelled simply as "Asian". In detail, the restaurant’s focus can be Japanese, Chinese or Singaporean. In certain cases, ingredients are simply not labelled at all. Knowing that a dish is a hamburger is not specific enough for recommendations. What is actually in the Hamburger? Is is beef, chicken or tofu ? We began to extract additional meta data from the menu items in order to create better models.”
Popular restaurants and how how to rank them present another challenge. Popular restaurants top quality metrics from ratings to order volumes and positive reviews. This is very popular chains, who already have a large customer base offline:
"We discovered a phenomena concerning well-known and popular chains, that most of the customers have eaten from offline too. So when they order from the chain online, they will get the food they expect. Thus the customer will always rate the restaurant disproportionally higher than say a previously unknown restaurant."
The challenge is that, despite being highly rated, some customers will simply never order from those chains. An algorithm which only relies on ratings is liable to suggest popular fast food chains only.
Presentation bias affects measured metrics more that expected. The top positions, especially the first position, gets the most clicks and typically has the highest conversion. Hungry customers cannot be expected to be patiently and carefully inspect a list of 200 restaurants fully before making a choice on where to order. The top 10 listed items account to almost 60% of the orders. Deciding the position in which to show each restaurant based on customer relevance is task which is continuously optimized.
The another challenge is latest to diversity of food and cuisines depending on a particular country. Through the order volumes and frequency, Gugu found out that Italian restaurants are particularly popular in Germany. This means that Italian restaurants typically get a lot of orders — which however does not mean they automatically better than other restaurants. Restaurants offering other cuisines also need a fair chance to be displayed when browsing for a restaurant nearby:
“We are working on a/b-tests, in which we show restaurants in alternative sorting orders. In this way we’re trying to counterbalance the existing biases. It’s about giving the restaurants which are normally not shown a chance to be seen by the customer.”
I'm grateful we were able to host our inaugural Berlin event at such an cool and welcoming venue, and it was fun chatting to everyone over drinks and pizza. If you want to join us at the next Berlin meetup on recommender systems, then join the group here (147 members already)! Our second meetup will be sometime in September. Also, we’re always on the lookout for speakers to wow our future meetups, so if you know, (or are) somebody who’d like to share a story — big or small — then please contact me.