From API to My Own Server: Building a Faster Food Search

Creating a Clean Nutrition Database and Testing Meilisearch Locally

⌛Why Open Food Facts Didn’t Work for Me

I was using the Open Food Facts API for the food search feature, but it was painfully slow. The app already supports multiple languages, and when I typed in English it kind of worked, but searching in other languages took forever — and even after waiting, a lot of results were missing calories or macro information. It felt more like the app was broken than just slow. I only kept using it because I was testing, but honestly, with so many good apps out there, no one would use something this laggy.

⌛Looking for a Faster Way

I asked AI how other apps stay fast without storing huge amounts of data locally. The answer was to rent a server and use Meilisearch. Hearing that costs could increase with more users scared me a bit, but if I didn’t try this, the app I’ve been building would just end up useless. So I decided to go for it. Today’s note is about what I’m learning while switching to a rented server and Meilisearch.

⌛What an API Really Means

API means pulling data from someone else’s database. That also means depending on the condition of someone else’s server. If that server is busy or overloaded, my app becomes slow too. I realized I had zero control over that part.

⌛Renting a Server vs Using an API

What I didn’t understand at first was this: APIs also run on someone else’s server, and renting a server is also using someone else’s computer, so why would there be such a speed difference? The explanation was that renting a server gives me my own space where I can install the exact program and database I want. With public APIs, requests from developers all over the world are mixed together, but on my server only my app is using the resources. That alone makes a huge difference. Services like AWS, Google Cloud, Oracle Cloud, or Hetzner all work this way.

⌛Meilisearch and Why It’s Fast

Meilisearch is a high-speed search engine that uses indexing. The way it was explained made sense: normal search is like flipping through a book page by page to find the word “apple.” Meilisearch is more like having a prepared index that says “apple — page 1, banana — page 5,” so it jumps straight to the result instead of scanning everything. That’s why the response feels instant.

⌛Getting Ready to Learn the Real Workflow

Now that I roughly understand what all this is, I want to learn the actual working steps.
First, I need to set up my own server environment. That means renting a server. I really wanted to use Oracle Cloud because it’s basically free, but everyone says getting a slot there is almost luck-based. I’m seriously broke right now, but I decided to save money on something else and just try Hetzner instead.

If this were the very beginning of making the app, I would’ve stubbornly tried to win the Oracle lottery no matter what. But I’ve already wasted so much energy on random struggles that now I feel it’s better to pay a little and stay sane, then use that energy to upgrade the app or study more. Still, it hurts. I even held back from Netflix and Spotify to save money, and now I have to spend it on this. I really want to get rich soon so I can stop worrying about every single payment.

⌛Installing Meilisearch

Step two is simple in theory: after renting the server, install Meilisearch on it. That part sounds straightforward compared to what comes next.

⌛The Hardest Part — Building My Own Database

Everyone says step three is the real challenge: collecting and organizing the data for Meilisearch. And this is where I already ran into problems. When I tested my app yesterday using the Open Food Facts API, English results were fine but Korean didn’t work at all. Some foods only had calorie info with no macros, so I thought my app was broken.

I realized that when building my own DB, I should only include items that have complete information. For English, German, French, and Spanish users, Open Food Facts apparently has pretty good coverage, so I can probably rely on it. But for Korean and Japanese users, there isn’t much data there, so I’ll need to find other sources. I still don’t really know how to do that yet — I’ll need to study the whole process more carefully later.

⌛Connecting the App to My Server

Step four will be changing the current code so that instead of calling the Open Food Facts API, it calls my own server.

There were also a few important tips:

• Before installing Meilisearch on the server, install it on my own computer and test everything locally first.
• Don’t just dump the entire Open Food Facts dataset into Meilisearch because the size is huge and could increase costs. I need to filter smartly.

⌛How to Put Data into Meilisearch

The basic plan for building the data:

1. For German, English, French, and Spanish users, go to Open Food Facts and download the full data dump from there.
2. For Korea, use the Public Data Portal.
3. For Japan, get data from the Japanese Ministry of Education food composition database.

That’s the rough roadmap for now.

⌛Cleaning and Converting the Data with Python

Each source will probably have totally different file formats and structures. I’ll need to use Python to extract only what I need — food name, calories, protein, fat, carbs — and convert everything to match my app. Units also need to be unified, like converting everything to a 100g standard. After cleaning, the goal is to generate a single JSON file that fits perfectly with my app.

⌛Sending Data to Meilisearch

Once the JSON is ready, I send it to Meilisearch. Then Meilisearch creates its own index based on that data so searches become fast and efficient. That part almost feels like magic compared to how the API worked.

⌛Keeping the Database Small and Clean

As the DB grows, the size will explode, so I need to extract only what’s necessary and remove duplicates carefully. Even if Coca-Cola exists in English, Korean, and Japanese separately, the barcode is the same, so those should be stored as one item using the barcode as the key.
Then I can tell Meilisearch to search specific language fields — Korean queries should look at the Korean field, Spanish at the Spanish field, and so on.

⌛More Detailed Plan

1. First filter only items that have full information — name, calories, protein, fat, carbs — and also bring serving_size and unit data.
2. Split the data into two groups:

• Items with barcodes
• Items without barcodes
• Barcode items will go into an integrated dataset.
  Non-barcode items will be stored by language, with a country code added to the ID.

3. Before unifying everything to “per 100g,” check how my app currently handles nutrition units, just in case I need to adjust the logic.

4. When creating the JSON, prepare fields for all six languages in advance. Even if some languages are empty for now, it will be easier to fill them later when new data appears.
Also add a main_language field to record where the data originally came from. Future me will definitely appreciate that.

⌛Downloading the Raw Data

I went to the Open Food Facts website and found a section at the very bottom called “data, API and SDKs.” Inside there was a page to download the full dataset. I did the same on the Korean and Japanese sites.
The OFF file was huge and downloading insanely slowly, so while waiting I downloaded the Korean and Japanese ones first. I can’t read Japanese, so I opened the Korean file to check the structure — and it was surprisingly clean and well organized.

Get Lia’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The formats were all different:

• Korean data → Excel
• Japanese data → CSV
• OFF data → JSON

I created a folder called Food_db_cleaner and put the downloaded Korean and Japanese files inside it. Then in VS Code I made a Python file called cleaner.py.
At first I was only going to extract calories and macros, but I decided to include sugar too. These days people seem to care about sugar a lot.

⌛Understanding the Python Code

import pandas as pd
import os

• pandas is a library that makes handling Excel-like tables really easy.
• os is for talking with the operating system, like handling file names and paths.

for index, row in df_kr.iterrows():

df means DataFrame, basically a table. iterrows() goes through the table line by line and pulls one row at a time.

if pd.isna(row['에너지(kcal)']) or row['에너지(kcal)'] == '-':
    continue

If the calorie value is empty (NaN) or just “-”, skip that item completely.

if len(row) < 10 or pd.isna(row[3]):
    continue

For the Japanese file, the food name was in the 4th column (index 3). If the name is empty or the row has almost no information, skip it.

ensure_ascii=False, indent=2

ensure_ascii=False is important so Korean and Japanese characters stay readable instead of turning into weird Unicode codes.
indent=2 just makes the JSON pretty and easier to read.

⌛First Run and the Usual Chaos

The goal was to run this Python script and generate clean JSON files from the Korean and Japanese data.
I installed the necessary tools:

pip install pandas openpyxl

But when I ran the script, it said the packages weren’t installed. Turns out Python was installed in multiple places on my computer and everything was tangled.
So I tried:

python -m pip install pandas openpyx

This time it installed under the Python I was actually using.

Then I noticed in VS Code the Python interpreter shown in the bottom right. I clicked it and a list of Python versions appeared at the top. I selected the one labeled global, ran again — and finally the script started.

⌛Fixing Column Name Hell

Next problem: the column names in the Excel file didn’t exactly match the code, so errors kept appearing. Every tiny mismatch caused another crash.

I added this:

df_kr.columns = [c.replace('\n', '').replace(' ', '') for c in df_kr.columns]

There were hidden line breaks and spaces in the headers, so I removed them all.
Then another error came because some column names were numbers instead of strings.

df_kr.columns = [str(c).replace('\n', '').replace(' ', '') for c in df_kr.columns]

Converting everything to strings first solved it.

⌛Finally Success

After all that, the script finally worked and created the JSON file.
I was so happy I actually took a photo of the screen. It’s all in Korean, but still — first real step toward my own food database.

⌛My First Real Database

Now I finally have this precious file containing both Korean and Japanese data. Time to face the real monster — Open Food Facts.
First I had to move the huge file from the Downloads folder to my project folder. I learned that copy-paste actually creates a brand new file, while cut-paste only changes the file location reference, so it’s much faster. I used cut-paste.

For extracting the archive, I installed Bandizip because people said it decompresses faster. There was an option called “auto extract,” which chooses the right method based on the file extension, so I just trusted that.

While doing this I noticed something weird — the file wasn’t .json but .jsonl. I had no idea what that was. After looking it up, I learned that JSON wraps everything in one big {}, but JSONL stores each item on a separate line, so it can be read line by line. That actually made sense for such a massive dataset.

⌛Checking the Giant File

After extraction the file was so big I couldn’t even open it normally. I created a small script called checker.py to read just the first five lines where all fields — language, name, calories, protein, fat, carbs, sugar — were present.
It opened perfectly. Huge relief.

Things I re-learned:

if __name__ == "__main__":

I studied this for certifications long ago but had forgotten — it means “run only if this file is executed directly, not when imported.”

if count % 10000 == 0:

Do something only every 10,000 lines. Super useful for huge files.

⌛Creating a New Script for OFF

Since the Korean and Japanese DB was already clean, I felt it was better not to rerun the old cleaner. Instead I made a new file off_adder.py to extract only useful data from OFF and merge it with the existing DB.

The Korean+Japanese DB had about 29,000 items. I thought OFF would shrink a lot after filtering only entries with full nutrition info, but the original was over 3 million records, so even after filtering it was still massive.

I got worried about storage costs and asked around. Apparently even 3 million records with name, calories, macros, and sugar are roughly equal to only 40–80 phone photos. That made me relax a bit about server bills.

In the end I got 2.51 million food items. Seeing that number felt unreal.

⌛Testing Meilisearch Locally

Next step was to test everything with Meilisearch on my own computer. I installed Meilisearch into the project folder and followed the instructions AI gave me — typing the startup command with the key and password in the terminal.

It showed a local address, so I opened it in the browser and logged in. Of course the DB was still empty.

I created upload.py to send my JSON file to Meilisearch.
At first it failed with an error, so I reinstalled the Meilisearch Python package and tried again. Still didn’t work. Then I realized Meilisearch must stay running in one terminal while I execute the Python script in another.

I pressed the plus button, opened a second terminal, kept Meilisearch running in the first, and ran the script in the second. Finally — it worked.

⌛More Data Cleaning Reality

But then more problems appeared. Some records were basically empty shells, and Meilisearch couldn’t read them properly. After filtering to only really clean entries, everything worked.

I definitely need to refine the cleaning process more carefully. I tested by loading 20 random items and the structure looked good.

One thing I noticed: many entries only had barcodes without proper names. That means I should add barcode search in addition to name search. Maybe even a “suggest a name” feature for users could be useful later.

⌛What’s Next

Now that local Meilisearch testing works, the next step is to move this setup to the actual rented server and connect it with the app.