Ingredient-Based Menu Planning Search Engine
Introduction
Have you ever wondered what you can cook using the ingredients you have in your pantry and fridge? Have you ever started preparing a particular dish and failed to complete it as you realized you were missing the key spice, condiment, or vegetable listed in the recipe? Did you ever get frustrated about not being able to come up with a dinner plan for the evening? You are not alone! For many people, the problem of cooking the “right food” does not start with how to prepare the food itself; it begins with deciding on the menu. And, sadly, not all of us are culinary savvy to mentally create and design exquisite restaurant-caliber masterpieces the minute we lay our eyes on diverse ingredients.
Food recipe search engines have become popular in the last decade, as food connoisseurs, gourmands, expert chefs, and amateur cooks alike have taken over the Internet to offer their specialized recipes. However, traditional recipe search methods depend on the exact input of a dish name in order to achieve accuracy and relevance in search results. To accommodate a search process that is based on the existing ingredients, we build a search engine for recipes with pre-specified ingredients. Moreover, the user is also given the choice of including time and calorie constraints while searching.
Data
To collect recipes for our model, we used https://www.food.com/. We selected this website because it had a good variety and amount of dish recipes (over 500,000 dishes created by users). We wrote a script using the BeautifulSoup package to crawl the recipe data from the website, where we stored the title of the dish, the time it takes to prepare it, the calories it has, the ingredients, and the instructions for the dish. All attributes were stored as strings in a .json file.
The final result was 51,600 entries of recipe data. Here is one example of the data where the instruction is partially shown:
“name”: “Low-Fat Berry Blue Frozen Dessert”,
“time”: “24hrs 45mins”,
“calories”: “170.9”,
“ingredients”: “4 cups blueberries, fresh or frozen. 1⁄4 cup granulated sugar. 1 cup vanilla yogurt, 1% or nonfat. 1 tablespoon lemon juice. “,
“instructions”: “Toss 2 cups of berries with sugar. Let stand for 45 minutes, stirring occasionally. Transfer berry-sugar mixture to food processor. …”
The scraping code is provided below:
from bs4 import BeautifulSoup as bsoup
import requests
import json
import re#first_id = 38
#last_id = 541115begin_id = 38
end_id = 55737count_to_checkpoint = 2000 # ============================================================================recipe = {}count = 0
name_begin = begin_idfor id in range(begin_id, end_id + 1):url = 'https://www.food.com/recipe/' + str(id)
strhtml = requests.get(url)
soup = bsoup(strhtml.text, 'lxml')
data_title = soup.select("#__layout > div > div.gk-site-wrapper > div > div > div.recipe-layout__title > div > h1")
if not data_title:
continue
data_time = soup.select('#__layout > div > div.gk-site-wrapper > div > div > div.recipe-layout__content-left.recipe-layout__truncated-element > div.recipe-layout__facts > div > div:nth-child(1) > div.recipe-facts__details.recipe-facts__time > span:nth-child(2)')
data_ingredients = soup.select('#__layout > div > div.gk-site-wrapper > div > div > div.recipe-layout__content-left.recipe-layout__truncated-element > div.recipe-layout__ingredients > div.recipe-ingredients > ul > li > div')
data_calories = soup.select('#__layout > div > div.gk-site-wrapper > div > div > div.recipe-layout__content-left.recipe-layout__truncated-element > div.recipe-layout__ingredients > div.recipe-ingredients > div.recipe-ingredients__nutrition > div > div > section > div:nth-child(1) > p.recipe-nutrition__item.bold')
data_instructions = soup.select('#__layout > div > div.gk-site-wrapper > div > div > div.recipe-layout__content-right.recipe-layout__truncated-element > div.recipe-layout__directions > div > ul > li')ingredients = ""
for item in data_ingredients:
ingredients += re.sub("\s{2,}", " ", item.get_text() + ' ')[:-1] + ". "
instructions = ""
for item in data_instructions:
instructions += item.get_text()
result = {"name" : data_title[0].get_text(),
"time" : data_time[0].get_text(),
"calories" : re.sub("\s{2,}", "", data_calories[0].get_text())[10:],
"ingredients" : ingredients,
"instructions" : instructions,
"url" : url
}recipe[id] = result
count += 1
if count % count_to_checkpoint == 0 or id == end_id:
with open("recipe_{}_{}.json".format(str(name_begin), str(id)), 'w') as outfile:
json.dump(recipe, outfile, indent = 4)
recipe = {}
name_begin = id + 1
For a given query, we break it down into 4 parts: ingredients, optional dish name, optional time constraint, and optional calories constraint. For instance, the query “show me recipes using onion and beef, 10 hours, less than 400 cal” will have [‘onion,’ ‘beef’] as ingredients, no dish name, 600 minutes as time constraint, and 400 calories as calories constraint.
Get Ayaz Mammadov’s stories in your inbox
Join Medium for free to get updates from this writer.
For a given recipe, the relevance score is calculated as the following:
- if the dish name is in the recipe name, + 2;
- for each ingredient in the recipe’s ingredient list, +1;
- if there is a time constraint and the recipe meets it, +1;
- if there is a calorie constraint and the recipe meets it, +1.
The table below summarizes the 10 queries and the corresponding relevance scores for each query and each recipe:
Methods
The first step we need to take before starting to work on the recipes is to convert the json files into csv files.
def dict_to_csv(recipes, filename = "recipes.csv"):
df = pd.DataFrame(columns = ['name', 'time', 'calories', 'ingredients', 'instructions', 'url'])
count = 0
for id in recipes:
df = df.append(recipes[id], ignore_index=True)
count += 1
if count % 1000 == 0:
print(count)df.to_csv(filename)
Query processing
As the first step in query processing, we extract the time and calories constraints from the query using a rule-based approach. Once extracted, the constraints do not affect the retrieval. As such, we remove the corresponding words in the query, as well as some stop words.
def extract_time_calories(query):
time = 99999999
calories = 99999999
time_adjs = ['fast', 'quick']
calorie_adjs = ['low', 'diet', 'healthy']
s = nltk.word_tokenize(query.lower())
for i in range(len(s)):
if s[i] in time_adjs:
time = 30
s[i] = ''
elif s[i] in calorie_adjs:
calories = 300
s[i] = ''# did not consider inputs like "half an hour" "two and a half hours"
time_mins = ['min', 'mins', 'minutes', 'minute']
time_hrs = ['hour', 'hours', 'hr', 'hrs']
calorie_words = ['cal', 'cals', 'calorie', 'calories']
for i in range(len(s)):
if s[i].isnumeric():
n = int(s[i])
for time_min in time_mins:
if time_min == s[i+1]:
if time == 99999999:
time = n
else:
time += n
s[i] = ''
s[i+1] = ''
break
for time_hr in time_hrs:
if time_hr == s[i+1]:
if time == 99999999:
time = n * 60
else:
time += n * 60
s[i] = ''
s[i+1] = ''
break
for calorie_word in calorie_words:
if calorie_word == s[i+1]:
calories = n
s[i] = ''
s[i+1] = ''
breakstop_words = set(nltk.corpus.stopwords.words('english'))
stop_words.update(['show', 'me', 'within', 'less', 'calories', 'calorie', 'time', 'recipes', 'recipe'])
s = [word for word in s if word != '' and word not in stop_words]
return s, time, calories
We then move on to score each recipe.
Recipe scoring
We run the new query against dish names and dish ingredients separately using two different BM25 rankers. We sum the two scores with two different weights based on the constraint criteria.
def get_model(picklefile = 'data.pkl'):if path.exists(picklefile):
return pickle.load(open(picklefile, 'rb'))
else:
with open("recipes.json", encoding='UTF-8', errors='ignore') as inputFile:
recipes = json.load(inputFile)
ids = []
names = []
ingredients = []
times = []
calories = []
for item in recipes:
ids.append(item)
names.append(recipes[item]['name'].lower())
ingredients.append(recipes[item]['ingredients'].lower())
times.append(time_to_min(recipes[item]['time']))
calories.append(float(recipes[item]['calories']))
times = np.array(times)
calories = np.array(calories)
tokenized_names = [doc.split(" ") for doc in names]
tokenized_ingredients = [doc.split(" ") for doc in ingredients]bm25_name = BM25Okapi(tokenized_names)
bm25_ingredients = BM25Okapi(tokenized_ingredients)
pickle.dump((ids, names, ingredients, times, calories, recipes, bm25_name, bm25_ingredients), open(picklefile,'wb'))
return (ids, names, ingredients, times, calories, recipes, bm25_name, bm25_ingredients)
Constraints
Once we have scores for each recipe, we then apply the constraints by reducing the scores for those which do not satisfy the constraints by some amount.
There are two ways to use the constraints: a hard constraint — any recipe using more time or having more calories than the limit is not considered, and a soft constraint — recipes satisfying the constraints have a bonus score.
The implemented weighting scheme for hard constraint is:
1 * name_score + 1 * ingredients_score
The implemented weighting scheme for soft constraint is:
1 * name_score + 1 * ingredients_score
+ 5 if satisfy time_limit + 5 if satisfy calories_limit.
def hard_req_top(query, name_weight = 1, ingredients_weight = 1, n = 10):
tokenized_query, time_limit, calories_limit = extract_time_calories(query)
name_scores = np.array(bm25_name.get_scores(tokenized_query))
ingredients_scores = np.array(bm25_ingredients.get_scores(tokenized_query))
combined_scores = name_weight * name_scores + ingredients_weight * ingredients_scores
combined_scores[times > time_limit] = 0
combined_scores[calories > calories_limit] = 0
combined_scores_sorted_id = combined_scores.argsort()[-n:][::-1]
return combined_scores_sorted_id
def soft_req_top(query, name_weight = 1, ingredients_weight = 1, time_weight = 5, calories_weight = 5, n = 10):
tokenized_query, time_limit, calories_limit = extract_time_calories(query)
name_scores = np.array(bm25_name.get_scores(tokenized_query))
ingredients_scores = np.array(bm25_ingredients.get_scores(tokenized_query))
combined_scores = name_weight * name_scores + ingredients_weight * ingredients_scores
combined_scores[times <= time_limit] += time_weight
combined_scores[calories <= calories_limit] += calories_weight
combined_scores_sorted_id = combined_scores.argsort()[-n:][::-1]
return combined_scores_sorted_idResults
The four figures below show the NDCG@20 results for the two baseline models and our hard and soft constraint approaches. First, let’s check the baseline models. For a given query, the random retrieval randomly picks 20 recipes as the output, where the BM25 method calculates the BM25 score for each recipe and picks the top 20 as the output. Overall, BM25 retrieval exhibits a better performance than random retrieval. Figures 1 and 2 show the NDCG @ 20 for random retrieval and BM25 for the 10 testing queries:
Our proposed model outperformed the baseline models because it considers the dish name and ingredients separately, rather than combining the two and performing retrieval.
def testing_retrieval(filename = "doc_score.csv"):
queries = ["recipe for chicken and dumplings",
"broccoli salad recipe less than 100 calories",
"beans, pasta, casserole",
"apple, banana",
"what to cook with ground beef in 1 hour",
"onion, beef, 10 hours, 400 cal",
"food recipes bread eggs",
"eggs recipe in 10 min",
"show me a recipe using beef, pork, and chicken",
"recipe for white sauce"]
df = pd.read_csv(filename)
for i in range(10): # 10 queries
col_name = 'score_' + str(i) # load the relevance score for query i
scores = df[col_name].tolist()
hard_ids = hard_req_top(queries[i], n = 20)
soft_ids = soft_req_top(queries[i], n = 20)
hard_dcg = scores[hard_ids[0]]
soft_dcg = scores[soft_ids[0]]
for j in range(1, 20):
hard_dcg += scores[hard_ids[j]] / math.log(j + 1, 2)
soft_dcg += scores[soft_ids[j]] / math.log(j + 1, 2)
scores.sort(reverse=True) # sort the original scores to get idealdcg_ideal = scores[0]
for j in range(1, 20):
dcg_ideal += scores[j] / math.log(j + 1, 2) # Ideal DCG
print("query {} ndcg @ 20 for hard requirements: ".format(i+1), round(hard_dcg / dcg_ideal, 3))
We attribute our model's better performance to the inclusion of constraints, which the baseline models do not consider. Figures 3 and 4 show the NDCG @ 20 for hard constraint and soft constraint approaches for the 10 testing queries:
Overall, our approach showed promising results in the evaluation stage. However, it is crucial to understand the limitations behind the current model. First, the weights of calculating scores in our model are not fine-tuned. Our model could thus, potentially achieve better results. The NDCG@20 results for both hard and soft constraint models are nearly identical due to the lack of tuning. With further fine-tuning, we can potentially have two models that perform uniquely. Another cause of concern is that our model considers each word in the query separately. If a potential user searches for the combination of words, such as ‘sour cherries,’ it will be treated as ‘sour’ and ‘cherries’ separately.
What’s next?
Despite the promising results we have achieved in our early exploration of the ingredient-based menu building search engine, there are potential extensions to our approach that could increase the utility and accuracy of the engine.
- We wanted to include cuisine type as a searching criterion. We were unable to find relevant data to pursue this track. The necessary features were also lacking in the source file — food.com. There are existing models online that take in ingredients and output cuisine type. As an extension of this model, we could utilize the existing cuisine classification models to create the cuisine label for each recipe.
- Part of our initial plan that we were unable to implement was the inclusion of additional searching criteria and constraints, such as the difficulty of preparation and dietary restrictions and allergies.
- Additional data would also prove helpful in increasing accuracy. If we wanted to improve on the project, we could scrape more recipes than we already have.
