I’ve recently been learning the song 但願人長久 (May we live long) sung by 鄧麗君 (Teresa Teng), later covered by 王菲 (Faye Wong). What’s special about this song is that the lyrics are directly taken from an ancient Song dynasty poem, 水調歌頭 (Shui Diao Ge Tou) by the poet 蘇軾 (Su Shi).

This song means a lot to me. In order to explain why, I need to talk a little bit about ancient Chinese poetry, the birth of modern Chinese pop music, the actual poem itself, and my grandmother. So buckle up your seatbelts, we’re in for a ride.

Every language has its foundational texts that every learner of that language needs to know. For example, in English, everyone had to learn Shakespeare in school. Shakespeare is, for English speakers, a common cultural reference point — every English learner knows at least Romeo and Juliet, and can quote Hamlet (“to be or not to be, that is the question”). In Chinese, I argue that the foundation texts are poems from the Tang (唐) and Song (宋) dynasty. You can stop anyone on the street in China and ask them to recite you a Chinese poem, and they will at least be able to recite one, if not multiple. Every Chinese learner, from age 5 till 18, had to learn to write and recite these poems in school.

Growing up in Hong Kong, I did the same. But it was only after years of learning these poems did I learn that these poems weren’t poems, they were lyrics to songs. In the time when they were written, these words used to be sung, with music. The reason why we know them as poems today is because there was no documentation for the music, and so we have no idea how those songs sounded. That felt so special to me — something that had been a part of my life for years, that I found maybe a little boring, was actually a big mystery.

Which is what makes this song, sung by 鄧麗君 (Teresa Teng), so special. 鄧麗君 (Teresa Teng) was the Elvis Presley of China — like Elvis, she was the first ever mega pop star in China, loved internationally by people not just in Taiwan and China, but also in Hong Kong, Japan, Malaysia, Indonesia, and even the US. Her music was so popular that, during communist times in China when non-government approved music and movies were banned, her cassettes were being sold on the black market for the equivalent of a month’s rent. Imagine paying almost 2000 euros for a CD! She was that good.

In the 80s, 鄧麗君 (Teresa Teng) came out with an album called 淡淡幽情, in which every song was an ancient Chinese poem. They took these poems, got modern composers to write the melody, and 鄧麗君 (Teresa Teng) to perform them. And one of these songs was one of my favorite poems: 水調歌頭 (Shui Diao Ge Tou) by 蘇軾 (Su Shi). It was such a cool concept — looking at the song credits, you can see it written: music by [a modern composer], performance by 鄧麗君 (Teresa Teng), lyrics by 蘇軾 (Su Shi). It’s like seeing a song where the credits say: music by Ed Sheeran, performance by Taylor Swift, lyrics by William Shakespeare. Pretty fucking cool, eh?

The poem itself, 水調歌頭 (Shui Diao Ge Tou), is one of my favorite poems. It’s essentially about longing. It’s about missing the people who are far away from you — whether they are on the other side of the world, or have passed away and left this world altogether. My favorite lines come at the end of the poem: 人有悲歡離合，月有陰晴圓缺，此事古難全。但願人長久，千里共嬋娟。I especially like the first phrases, which explain that all the ups and downs we experience as humans are just like the different phases of the moon — a natural phenomenon that has existed since the beginning of time. I often remember this phrase when I’m going through something difficult. Often, at those times, it’s easy to start asking questions like, “why is this happening to me?” Or, “why is the world like this?” But this phrase reminds me that suffering is not a sign that something is wrong with my life, it’s actually a sign that your life is unfolding exactly as it should, like the phases of the moon.

In 2016, my grandma (on my mom’s side) was suddenly diagnosed with late stage cancer, with a few months left to live. She was living in Toronto, and I had just moved to the Netherlands at the time. Being a very practical family, my mom told me that there was no need to fly back to Toronto to see her for one last time — I just needed to come back for the funeral. So instead of flying back, I wrote her a letter. I don’t exactly remember what I wrote (how I wish I had kept a copy!), but I do remember quoting the last lines of that poem. I told her that, while I was sad and shocked by her diagnosis, those lines gave me peace. The final line was also beautiful — it means, “may we live long, and witness the same moon a thousand miles apart”.

My grandma passes away a few months later, and I went to Toronto for the funeral. Shortly after, I met up with my aunt, who was responsible for cleaning up my grandma’s room and distributing her possessions. My grandma had left me a beautiful broach — a butterfly made of jade and gold. My aunt also told me that my grandma loved the poem so much that she had pinned my letter on her pinboard next to her makeup mirror, so she could see it everyday.

That’s the thing — my grandma and I weren’t that close. I grew up in Hong Kong, she lived in Toronto, and besides, she wasn’t a particularly emotionally expressive person. But because this poem was a common reference point, something she also learned in school, it allowed us to create a connection. It gave both of us strength. It still gives me strength.

I ended up getting a small tattoo on my arm of the phases of the moon. To remember this poem, and of course, to remember my grandmother.

May we live long, and witness the same moon a thousand miles apart.

但願人長久，千里共嬋娟。

Lately, I’ve been trying to get more serious about my personal finances. I wanted to understand where my money was actually going — not just how much I was spending each month, but what I was spending it on.

My bank’s app gives me a monthly overview of my income and expenses, but it stops there. It doesn’t break my spending down into categories like groceries, rent, or eating out. So while I knew I spent, say, €2,000 last month, I had no idea what that money really went toward.

Around the same time, I heard about the ChatGPT API. It lets you automate tasks by sending text prompts to ChatGPT programmatically. That gave me an idea: what if I could use it to categorize each line of my bank statement?

Here’s the basic idea:

• For each row in my bank statement, I extract the description field. This usually gives a clue about the expense.
• For example, a row might have something like:
  "BEA, Apple Pay Albert Heijn 1852,PAS033 NR:ZBQ2MP, 03.10.24/16:16 DELFT". That’s clearly a grocery run at Albert Heijn, a supermarket chain in the Netherlands.
• I send that description to ChatGPT via the API and ask it to guess what kind of expense it is — groceries, transport, rent, etc.
• Then I take the answer and store it back into the same row, effectively labeling each transaction with a category.

Yes, I know there are personal finance apps out there that do this automatically when you connect your bank account. But I wanted an excuse to play around with the OpenAI API. Sometimes a girl just wants to have fun, ok?

One quick note before we dive deeper: the API isn’t free, but it’s affordable. Categorizing about 400 rows of bank statements cost me around 30 cents.

Walking through the code

The script is split into three main parts. If you want to dive into the full code, it’s up on my GitHub.

1. PdfStatementExtractor()
   My credit card statements are only available as PDFs, so this part extracts the data and converts it into a .csv format I can work with.
2. BankStatementPreprocessor()
   This step combines my bank and credit card .csvs into one file and does some basic cleanup. It also auto-labels recurring transfers like rent, savings, and investments.
3. BankStatementCategoriser()
   This is where ChatGPT comes in. It takes the description for each transaction, sends it to the API, and gets back a guessed category like groceries, transport, or eating out.

Part 1: Extracting Data from PDF Credit Card Statements

The first part of my script is the PdfStatementExtractor(). Since my credit card statements are only available as PDF files (not CSV), I needed a way to turn them into structured data I could actually use. This class does exactly that—by asking ChatGPT to extract the tables from the PDFs.

Here’s a quick look at what happens when I run this part of the script:

def run(self): 
  self.load_openai_client() 
  self.deleteOldCsv() 
  self.createAssistant()
  self.getFileNames() 
  for file_name in self.file_names: 
      self.messageAssistant(file_name)
      self.runAssistant()
      self.recordOutput() 

The first step is to set up the OpenAI client and create a custom assistant. OpenAI makes this really simple — just about 15 lines of code.

def load_openai_client(self):
    load_dotenv()
    api_key = os.getenv("OPENAI_API_KEY")
    self.client = openai.OpenAI(api_key=api_key)

def createAssistant(self): 
    self.assistant = self.client.beta.assistants.create(
        name="PDF Extractor",
        instructions=(
            "You are a helpful assistant that extracts tables from credit card statement PDFs "
            "and returns them in CSV format."
        ),
        model="gpt-4o", 
        tools=[{"type": "file_search"}],
    )

Once the assistant is ready, I upload each PDF and send it a prompt asking it to extract all transactions into CSV format. I give specific instructions to make the output as clean and machine-readable as possible — CSV with headers, fields in double quotes, correct date format, and amounts as numbers.

def messageAssistant(self, file_name): 
    print(f'processing {file_name}...')
    message_file = self.client.files.create(
        file=open(file_name, "rb"), purpose="assistants"
    )

    self.thread = self.client.beta.threads.create(
        messages=[
            {
                "role": "user",
                "content": (
                    "This is a credit card statement. Extract all transactions as a table with columns: "
                    "['date', 'description', 'amount'].\n\n"
                    "Change the transactiondate values to DD/MM/YY format. "
                    "Change the amount values into numbers (floats) – with 'bij' as positive and 'af' as negative. "
                    "Return the transactions in plain CSV format, including the header row. Enclose all fields in double quotes. "
                    "Do not add explanations or formatting, just output the CSV text."
                ),
                "attachments": [
                    { "file_id": message_file.id, "tools": [{"type": "file_search"}] }
                ],
            }
        ]
    )

This way, I can extract structured data from messy PDF files using just a few lines of Python and the ChatGPT API. It’s surprisingly effective, and honestly kind of magical the first time you see it work.

Part 2: Cleaning and Combining the Data

The second part of the script is the BankStatementPreprocessor(). This step is pretty straightforward, so I won’t go too deep into it.

Basically, it takes the cleaned .csv files from both my bank and credit card statements, tidies them up a bit more, and merges them into a single file. Here’s the core of what happens:

def run(self): 
    self.preprocess_bank_statements() 
    self.preprocess_creditcard_statements()
    self.combine_statements()

During this step, I also label a few recurring transactions — like rent or savings — that I already know and don’t need to send to ChatGPT. With everything cleaned and combined, I end up with one consistent dataset ready for categorization.

Part 3: Categorizing with ChatGPT

This is the main part of the script — the BankStatementCategoriser(). It takes the combined bank statement data and figures out what each transaction was for. Structurally, it works a lot like the PdfStatementExtractor()—it loads the OpenAI client, creates a custom assistant, and then processes the data in batches.

Here’s the basic flow:

def run(self): 
    self.load_openai_client() 
    self.delete_old_csv()
    self.create_assistant()
    for i in range(0, len(self.df), self.batch_size):
        self.get_descriptions(i)
        self.process_categorised_descriptions(i)

The assistant is given clear instructions to categorize Dutch bank transaction descriptions into expense categories. I also asked it to ignore metadata like account numbers and just focus on the meaning.

Here’s the exact prompt I used:

You are a helpful assistant that categorizes Dutch bank transaction descriptions into English-language expense categories. 
Descriptions may include Dutch terms, abbreviations (e.g., 'AH' = Albert Heijn), and SEPA/IBAN metadata. 
Focus on the meaning and ignore account numbers or reference codes. 
Only respond with a numbered list of categories, one per line.

Categorize the following 20 transactions into one of these categories:
- Rent
- Utilities 
- Transport
- Groceries
- Restaurants & Cafés
- Shopping
- Books & Education 
- Subscriptions
- Insurance
- Health & Hygiene 
- Travel
- Taxes 
- Other

Return exactly 20 numbered lines, one per description, in the following format:

1. <Category>  
2. <Category>  
...  
20. <Category>

This makes it super easy to map each result back to the original transactions. I process them in batches (e.g. 20 at a time), store the results, and boom — my spreadsheet now has a column with labeled expense categories.

Conclusion

And there you have it!

With just a bit of Python and the ChatGPT API, I built a tool that categorizes my bank transactions — saving me time and helping me understand my spending habits better. The real power of the API is that it lets you automate tasks you’d normally do manually in ChatGPT, and for just a small cost. For example, processing around 400 transactions only cost me about 30 cents.

The power of this API is that it allows you to automate tasks for ChatGPT for a small fee. This would allow you to automate any existing tasks you already do manually on the website. For example, I’m now using the same API setup to help with a literature review — having ChatGPT read paper abstracts and sort them into topics. The results have been good, so stay tuned for that.

I also know people who are using the API for things like lightweight computer vision — uploading images and asking ChatGPT to guess what’s in them (like whether there’s a bike in the photo). Sure, building your own model is more powerful and customizable, but if you want something quick and low-effort, the ChatGPT API is a great place to start.

And if anything in this article confused you, just copy-paste the code into ChatGPT and ask it to explain. It’ll probably do a better job than I can.

Best of luck :)

If you ask anyone who’s done a PhD, they’ll tell you: the hardest part isn’t the research. It’s not the coding, the reading, or the equations. The real challenge is everything around the work — your mindset, your relationships, how you manage your time, energy, and expectations. That’s what makes or breaks a PhD. This article is about the things I learned the hard way, in the 4.5 years I was a PhD at TU Delft: seven mindsets that helped me not just survive, but (sometimes) enjoy the ride.

In my opinion, the key to a fulfilling PhD comes down to three things: be selfish, lazy, and pessimistic. Okay, okay, not really. It’s actually: be assertive, effective, and realistic. But if you’d told baby-PhD-me that, I’d have heard it as selfish, lazy, and pessimistic anyway. You live, you learn.

Here’s what’s in the article — feel free to skip to what you need:

Part 1: Be selfish

• Hell yeah or no
• Drop some eggs

Part 2: Be lazy

• Minimum viable product
• Start with the end in mind

Part 3: Be pessimistic

• The terrible first draft
• Valley of shit
• You’re not at the top, you’re at the bottom

Let’s get into it.

Part 1: Be selfish (or assertive)

Hell yeah or no

At the start of my PhD, I said yes to everything. Organizing workshops? Sure. Helping with someone else’s project? Absolutely. I was eager to learn and didn’t want to miss out. My guiding question was: “Will this bring value to my PhD?” And the answer was always yes — everything seemed useful in some vague, future-proofing way.

The problem? I ended up overloaded with obligations that had little to do with my actual research. By the time truly good opportunities came along, I had no time or headspace left.

Eventually, I learned to ask a different question: “Is this essential to completing my PhD?” Or more bluntly: “If I didn’t do this, would I still graduate?” That simple shift saved me a lot of time.

This mindset goes against how I was raised. My parents, who grew up where opportunities were rare, always encouraged me to take every chance. But in academia — especially at a place like TU Delft, where I did my PhD — the problem isn’t scarcity, it’s overload. If you say yes to everything, you risk saying no to the things that really matter.

So now, unless it’s a hell yeah, it’s a no.

Drop some eggs

Early in my PhD, I became known as the person who got things done. I jumped on every task in our EU project, and before long, I was buried in responsibilities. When the pandemic hit, the stress became overwhelming. I asked for help — multiple times — but no one stepped in. Everyone was too busy.

Eventually, my PhD supervisor gave me some excellent advice: “Tanya, if you want time to do your PhD, you’ll need to drop some eggs.”

When people are too overloaded to help, the only way out is to stop doing the things you shouldn’t be doing. Especially tasks outside your job description.

One of two things will happen: either someone else will pick it up, or no one will notice — and then it didn’t matter anyway. Both are good outcomes.

Part 2: Be lazy (or effective)

Minimum viable product

The idea of a minimum viable product comes from the world of software development. I’ve heard it’s a bit out of fashion now, but I still think it’s incredibly useful — especially for navigating a PhD.

At its core, the concept is simple: create the most basic version of what you’re trying to build, just enough to test whether it’s going in the right direction. In software, that might be an app with only one working feature. In a PhD, it might be a rough paper outline or a quick prototype analysis.

There are two parts I find especially helpful: the minimum viable part, and the product part.

Let’s start with minimum viable. This means doing just enough to move forward — not less, but definitely not more. A minimum viable paper might be an outline with bullet points under each section. A minimum viable analysis could be a quick regression run on messy, incomplete data. A good rule of thumb: can it be done in a week? Then it’s probably minimal enough. Always aim for the smallest version that still achieves your goal. If your supervisor says to test 10–20% of your samples, test 10%. If lit reviews in your field typically include 80 papers, read 80 — not 120. The goal isn’t to cut corners; it’s to avoid digging tunnels.

Then there’s the product part. PhD work can feel invisible, especially early on. You can spend weeks reading without having anything to show for it, which makes it hard to get useful feedback. Supervisors, who are often too busy to keep track of your thought process, need something tangible to respond to.

That’s why I try to end every task with a product, no matter how small. When you read papers, build a summary table. When you’re brainstorming ideas, sketch a mind map. When you meet with your supervisor, bring something they can react to — even if it’s rough.

And yes, sometimes the fear of creating a product is that it might be terrible. Spoiler alert: it probably will be. But that’s the point. You can’t get useful feedback until you’ve made something imperfect. More on that in Part 3.

Start with the end in mind

Back in my architecture school days, I managed something rare: a (relatively) healthy schedule. I didn’t pull all-nighters, and I actually left the studio on time. If you’ve studied architecture, you know that’s borderline miraculous.

What helped was a simple mindset shift. I realized I wasn’t designing a real building — I was creating a representation of one. My end product wasn’t a finished design, but a set of drawings and models to pin up on a 2m x 2m wall. One month before the final review, I laid out exactly what would go on that wall. Every drawing, every model. From then on, every decision I made was guided by one question: Will this show up on my wall? If not, I let it go.

The same principle applies to academic research. You’re not building a perfect model or prototype — you’re writing a paper about one. These are very different goals. That’s why I recommend starting with a rough paper outline at the beginning of every project, not at the end: title, abstract, and bullet points for each section — intro, theory, methods, expected results, and discussion.

Once you have that, you can sanity-check your tasks. Ask yourself: Where will this work end up in my paper? Is this just a passing reference for the intro? Then don’t spend a whole day on it. But if it’s central to your analysis or discussion, go deeper. Can’t find where it fits in your paper? Let it go.

Part 3: be pessimistic (or realistic)

The terrible first draft

I wrote my first academic paper about 1.5 years into my PhD, deep in the pandemic. At that point, I felt behind, unproductive, and under pressure to produce something amazing to make up for lost time. Unsurprisingly, I couldn’t start. I had results, but they didn’t feel strong enough to justify a paper — especially one that had been 18 months in the making.

The only way I could break the block was to lower the bar. Way down. I told myself, “Today, I’m going to write a terrible first draft. It’ll be shitty. I’ll send it to a shitty journal. No one will read it. But at least I’ll have written something.”

That mindset shift gave me the freedom to start. I did write the terrible draft. My supervisors gave feedback. The paper got better. Eventually, I published it — in a mediocre journal — but it’s now my most cited paper. (Admittedly, it’s also had the most time to collect citations.)

Sometimes the way forward is to assume the worst — so you can just get started.

The valley of shit

At some point in your PhD, you’ll enter the Valley of Shit. It’s that low point where your research feels meaningless, your motivation disappears, and your methods seem full of holes. You dread work, doubt yourself constantly, and scroll through LinkedIn wondering how everyone else is publishing, winning grants, and thriving while you’re… not.

Here’s the thing: every academic — yes, every one — has been through it. The PhD student next to you, the tenured professor down the hall, even that person whose paper you’ve been citing all year. It’s a rite of passage. (Seriously, it’s really a thing, Google it. Even ChatGPT knows that damn valley is real — I checked.)

The worst part of the valley isn’t the self-doubt. It’s the shame. So let go of that. You’re already feeling awful — don’t add isolation on top of it.

For me, it hit hard between my first and second year. Around that time, I was part of a research group where we took turns chairing weekly meetings. When it was my turn, I totally forgot until someone reminded me days before. I panicked — I was already feeling stuck and burnt out, and now I had to prep a presentation?

In the end, I did something different. I emailed the group and said, honestly, I was deep in the valley of shit and needed advice. Before the meeting, people reached out — some I barely knew — offering to chat and sharing their own stories. At the meeting, professors I’d only seen as distant and accomplished opened up about their own struggles. People stayed behind afterward just to talk. I was blown away by their kindness.

So if you’re in the valley right now: you’re not alone. Not even close. Tell someone. You’ll be surprised how many people have been there — and how generous they’ll be when you let them know you’re there too.

You’re not a the top, you’re at the bottom

A PhD isn’t the crowning achievement of your career as a student — it’s the beginning of your career as an academic.

Early on, it’s easy to think of the PhD as the final feather in your cap. You’ve done school, your bachelor’s, your master’s, and now the PhD is the cherry on top. But thinking this way isn’t helpful. If you see yourself at the top, you’ll expect yourself to know everything, and set impossibly high standards.

It’s better to think of your PhD as the bottom rung of a new ladder. You’re not at the top — you’re at the bottom. The people you’re sharing your office with — postdocs, assistant and associate professors, full professors — they’re all more experienced than you.

For me at least, this was very helpful to know. Because it means no one expects you to know everything. Everyone was where you are now, and they remember how clueless they were back then. Don’t worry. You’re expected to fail, to ask stupid questions, to go down endless dead ends. This is your first time doing research professionally. No one gets it right on the first try.

Hope this helped!

So there you have it — how to have a fulfilling PhD experience: be selfish, lazy, and pessimistic. Or, in slightly more academic terms: be assertive, effective, and realistic.

Good luck out there. You’ve got this (even if it doesn’t feel like it yet). :)

A few months ago, I went to see an exhibition of Ai Wei Wei, a famous Chinese artist. I was surprised to find that some of his artworks were made of Lego. Like many (uninformed) viewers of modern art, I thought to myself, “hey, I could do that at home…”

Walking around the exhibition, I wondered what I could make out of Lego — a portrait, a painting? In the end, I decided to make a map of Hong Kong, the city I was born and raised in. Making the Lego map of Hong Kong took four steps:

1. Gathering and processing satellite data of Hong Kong
2. Converting the satellite data into information to build the Lego map
3. Buying the Lego bricks and optimizing costs by picking the right vendors
4. Building the Lego map

To keep things simple, I only show snippets of the Python code written for this project. To see all the code, check out the GitHub repository here.

Gathering and processing satellite data of Hong Kong

In order to make a map of Hong Kong, we need to gather and process the required satellite data. The map uses two data sources: the bedrock elevation of the region, and locations of high population density. From these two sources, I created a map with different layers, where each layer represents a certain element (e.g. sea, land, or built area). In the paragraphs below, I will walk you through the steps I took.

First, we import the required libraries. Two important libraries are ee, which stands for Earth Engine and allows us to download satellite data from Google Earth Engine; and geemap, which provides an interactive mapping interface for visualizing Earth Engine datasets and analyses.

import numpy as np
import osmnx as ox
import json
import math
import matplotlib.pyplot as plt
import geemap.foliumap as geemap
import ee
ee.Authenticate()
ee.Initialize()

Then, we get the data needed for the map: the bedrock elevation, which shows the topography of the region (both above and below the sea); and land cover data. Both datasets are downloaded as Earth Engine Images, and then cropped to only show data for Hong Kong.

hk_bbox = ee.Geometry.Rectangle([xmin_n, ymin_n, xmax_n, ymax_n])
bedrock = ee.Image('projects/tanyatsui/assets/seabed_grid').clip(hk_bbox)
bedrock = bedrock.reproject(crs=bedrock.projection(), scale=600)
land_cover = ee.Image("MODIS/051/MCD12Q1/2001_01_01").select('Land_Cover_Type_1').clip(hk_bbox)

The bedrock elevation was downloaded from gebco.net, which allows users to download bedrock elevation data (also called “bathymetry data”) from anywhere on earth.

The land cover data was downloaded from the Google Earth Engine data catalog, which contains a wide range of open source satellite data, much of it with worldwide coverage.

Then, from the datasets we downloaded, we create the different layers of the map: sea, land, and built area. As an example, the code snippet below shows how the bedrockdataset is processed to make the land elevation layers. First, we crop the bedrockdata so that it only shows points that are above sea level (elevation >= 0), and within the Hong Kong political border. This newly cropped data, hk, is then separated into 4 layers according to the elevation, essentially creating a contour map.

# land elevation 
mask_land = bedrock.gt(0)
mask_combined = mask_land.And(mask_border)
land = bedrock.mask(mask_land)
hk = bedrock.mask(mask_combined)
hk_elevation = hk.lte(200).multiply(1)
hk_elevation = hk_elevation.add(hk.gt(200).And(hk.lte(400)).multiply(2))
hk_elevation = hk_elevation.add(hk.gt(400).And(hk.lte(600)).multiply(3))
hk_elevation = hk_elevation.add(hk.gt(600).multiply(4))

Then, we take all the layers needed for the map and save them as numpy arrays.

arrays_dict = {}
for name, eeImage_list in layers_dict.items(): 
    array = get_array(eeImage_list[0], eeImage_list[1])
    arrays_dict[name] = array
    np.save(f'data/{name}.npy', array)

Here’s the resulting map, with blue representing the sea, green representing the land, and orange representing densely built area.

Converting satellite data into Lego data

In order to turn the pixellated satellite map into instructions for building a Lego map, we need to calculate, for each layer, which types of Lego bricks (in terms of color and size) should be placed where. I decided that I would use four types of Lego bricks to build the map: 4x2, 2x2, 2x1, and 1x1 plates. To do this on each map layer, we iterate through the layer one pixel at a time, trying to fit the Lego bricks, starting with the largest brick type (4x2) first.

def _try_fitting_brick(self, i, j):
    for brick_hw in [[2,4], [2,2], [2,1]]:
        temp = self.arr[i:i+brick_hw[0], j:j+brick_hw[1]]
        if temp.shape == tuple(brick_hw) and np.all(temp == 1): 
            self._place_brick(i, j, brick_hw)
            break

This results in a summary of all the Lego bricks needed for each map layer, sorted by color and size.

Then, we print out instructions for how each layer will be assembled.

Buying Lego: web scraping and cost optimization

Now that we have calculated how many Lego bricks are needed for the map, it’s finally time to buy them. At first, I was planning to buy the Lego bricks from the Lego website, but it was too expensive. So I settled on a second-hand Lego brick marketplace, bricklink.

However, buying second hand Lego bricks was not as easy as it seemed. The site had hundreds of vendors, each offering different Lego bricks at different prices and amounts, and no single vendor offered all the brick types I needed. Each vendor charged a delivery fee, so it was important to have my order covered by as little vendors as possible, at the lowest possible price.

Trying to find the right combination of vendors by manually clicking through the site was impossible, so I wrote a Python script that scraped the bricklink website for information about vendors, and used an optimization algorithm to pick the combination of vendors that would result in the lowest total price.

To do this, we use selenium to gather information for each type of Lego brick we need. This results in a csv brick_vendors.csv, which provides, for each Lego brick, an overview of vendors and their offer prices.

def get_vendors(row): 
    # Setup browser and define constants
    browser = webdriver.Chrome(service=Service('chromedriver.exe'))
    brick_types = ['1x1_3024', '1x2_3023', '2x4_3020', '2x2_3022']

    vendors_list = [] 
    for brick_type in brick_types:
        # Extract and construct details for URL
        brick_number, brick_name = brick_type.split('_')
        url = construct_url(https://medium.com/row.color_code,%20brick_number,%20brick_name,%20row[brick_type])
        
        # Get vendors and append to list
        browser.get(url)
        time.sleep(1)
        vendors = get_vendors_from_soup(BeautifulSoup(browser.page_source), row[brick_type], browser, row.color_code, brick_number, brick_name)
        vendors_list.extend(vendors)

    browser.quit()
    return pd.DataFrame(vendors_list)

# Apply function to each row and concatenate results
results_df = pd.concat(shopping_list.apply(get_vendors, axis=1).tolist()).reset_index(drop=True)
results_df.to_csv('results/brick_vendors.csv')

Then, to find the best combination of vendors, we use the “greedy set cover” algorithm. This is a method for solving the set cover problem, where the goal is to cover all elements of a universal set using the smallest number of subsets from a given collection of subsets. In our case, the method allows us to find the smallest number of vendors to cover all the Lego bricks we need.

def greedy_set_cover(universe, subsets):
    """ Find a family of subsets that covers the universal set """
    universe = set(universe)
    covered = set()
    cover = []

    # While not all elements are covered
    while covered != universe:
        # Choose the subset that adds the most uncovered elements
        subset = max(subsets, key=lambda s: len(s - covered))
        cover.append(subset)
        covered |= subset

    return cover

Building the map!!!

Now that we’ve processed the satellite data, counted the number of Lego bricks needed, found the optimal number of vendors, it’s finally time to assemble the map! If you’re still reading this article, I compliment you for your patience :)

I glued twelve 32x32 dark blue base plates onto a plywood board with epoxy, mounted it on the wall of my living room, and started assembling.

And voila!! Here is the final product in all it’s glory.

And a bonus photo: here’s me and my high school friends putting a little red flag to show the location of our high school.

Whenever I meet someone new, and they find out that I’m doing a PhD, they are always surprised by how happy I am when I talk about work. The typical stereotype of a PhD is someone who is stressed and overworked, juggling a hundred tasks at the same time, and generally having a miserable time. While I’ve also had lot of stressful and miserable times (my entire first year would be a good example), I would say I enjoyed most of my PhD.

Now that I’m nearing the end of my PhD, I’d like to share four tools that anyone can adopt, that doubled my productivity, and helped make my experience more enjoyable. For each tool, I will explain in detail how I use these tools in my daily life.

Final note before I start: I haven’t been sponsored by anyone to promote these tools. There are alternative companies providing similar tools, but I haven’t tried them out yet, and they are probably just as good. Rather than promoting a specific tool, I’m aiming to promote a few types of tool that are handy for PhD research.

Here are the four tools I’d like to share: The Pomodoro Technique, Mural, Roam Research, and ChatGPT.

The Pomodoro Technique

For me, this was probably the tool that saved me the most time over the long term, and allowed me to enjoy my work the most. If you haven’t heard of this technique, it is a time management method that involves breaking work into focused intervals (typically 25 minutes) separated by short breaks.

Here’s what a typical Pomodoro cycle might look like.

• Think of what you want to achieve in the next 30 minutes. If it’s difficult for you to estimate the amount of work you can do in thirty minutes, try thinking of three 10-minute tasks, or two 15-minute tasks.
• Set a timer for 30 minutes, and work on the task you just set for yourself. I use the online tool tomato-timer.com.
• When timer goes off, stop whatever you are doing, and ask yourself if you have completed your task. Then decide what you’re going to do for the next 30 minutes
• Take a 10 minute break
• Repeat the whole process again

I use this google sheet designed by the company Ultraworking as a template for going through this technique. It looks a bit complicated, but I highly recommend it. Took maybe 30 minutes for me to figure out how it works, but I have been using this template every working day for the past three years, so it really paid off.

I love this technique because it forces you to set and achieve very short term goals, which is very satisfying in the long term. The technique forces you to take breaks, which allows you to take a step back, evaluate what you’re doing, and rest. This also prevents you from getting stuck and a certain task for hours on end.

Mural

Mural is a virtual whiteboard for brainstorming, planing, and analyzing using sticky notes, shapes, and templates. It’s generally marketed as a collaborative brainstorming tool, but I found it useful for solo PhD work as well.

I tend to use one mural board for each research project, using the same board from start to finish. Here’s an example of the mural board for my fourth paper on spatial optimization of circular hubs.

In the beginning of the project, I use it as a brainstorming tool. I add screenshots of papers, formulas, diagrams, and links in order to create an overview of existing literature on the topic. I comment on what I gathered using post-it notes, try to find research gaps, and present all this to supervisors and co-authors.

Then, during the research process, I screenshot preliminary results (such as charts, maps, tables…etc.) and comment on them using post-it notes, highlight findings, and present the board to supervisors / co-authors. This allows me a keep track of all the changes and assumptions I’ve made throughout the different stages of the research.

At the end of the process, I organize my results into a rough ‘story’ that can be translated into a paper.

I like this tool because it allows you to put everything associated with a project onto a single page, which allows you to keep track of every step you took throughout the research process, and prevents you from losing any interim ideas / results. The link to the mural board can also be shared with co-authors, which makes it really to update and collaborate with them. I also found it really handy for teaching a seminar for Masters students — they were guided to create they own research design using sticky notes and a framework I made.

Roam Research

Roam Research is a note-taking tool that allows you to create notes (called pages) that link to other notes, which allows you to create a database of networked pages. I use Roam for two things: writing, and as my “second brain”.

I use Roam to write journal articles and online articles like this one that you’re reading now. Whenever start a new article, I create a new “page” for it, write the whole article on that page as full sentences, then paste them onto a google doc / word document.

I find Roam really handy, because each new line on a page is a bullet point. It’s also possible to nest bullet points under each other, and to collapse bullet points at any level. Since I was already writing in bullet points (on Microsoft Word) before discovering Roam, this feature was very useful for me, especially the collapsible points.

Here’s what my writing process looks like in Roam. First, I write the title of each section as a bullet point. Then, for each section, I write key ideas as bullet points, rearrange points to form a coherent story.

Then, I turn these points into full sentences.

As you can see, one handy tool for Roam is the word-count, which allows you to could the number of words nested in each bullet point. This also gives me a quick overview of how many words I’ve written so far.

I also use Roam as my “second brain”, which is a system devised by Tiago Forte for organizing information, knowledge, and ideas. I won’t fully explain the concept here, but I highly recommend reading his book “Building a Second Brain” or watching his YouTube videos. Essentially, I use Roam as a database for all (and I mean ALL) the information I need in my professional and personal life, so that I don’t have to remember this information with my own brain (hence the name “second brain”). Here’s what parts of my “second brain” looks like.

Here’s the “Projects” page, which gives an overview of all the current projects I actively have in my life right now, both professional and personal.

Each project also has its own page, filled with all the information I might need for the project, including handy links, contacts, deadlines, and ideas. For example, here’s my page for my fourth journal article, which includes handy links, a debugging log for the optimization code, and the entire written contents of the article.

What I like about Roam is that it allows me to collect all (and I mean ALL) the information I need in one place, giving me an overview of all the projects I’m currently juggling in my life. I started using Roam as my “second brain” in the beginning of my third year, when I had accidentally signed myself up for too many projects. Every night when I was falling asleep, I found myself jolting awake multiple times, suddenly remembering that had forgotten to do a certain task. I would then frantically send myself an email, hoping that I would remember to do it the next day.

After a month or so of this, I finally got sick from the stress. I stumbled upon a YouTube video on “building a second brain”, and used the rest of my sick days create a “second brain” system for myself on Roam. Having a grasp of all the projects I’m dealing with seriously reduced my stress. I spent significantly less time on administrative work and putting out fires, and more time on work I actually enjoyed — research.

ChatGPT

Any article covering handy tools for PhD (or for anyone, really) would be incomplete without ChatGPT. ChatGPT is a version of OpenAI’s GPT model fine-tuned for conversational interactions and text-based tasks. In other words, it’s a quite intelligent chat bot that can help you with any text-based tasks. While there are countless uses for ChatGPT, the three main ways I use the tool for my PhD are: coding, literature review, and writing.

I use ChatGPT as a helper for when I’m coding. Not only is it handy for writing code from scratch, it can also debug code, and teach me how to code things I have no knowledge about.

Last week, for example, I wanted to turn my Jupyter Notebook into a very simple web app, but had no idea how to do so, because I have zero experience in web development. ChatGPT gave me some options for Python libraries, showed how they worked using code snippets, and debugged the code together with me. I went from having no idea how to make a web app to making a very simple one in half a day.

I also use ChatGPT as a first step for doing a literature review. Let’s say I want to get a general understanding of a new topic — agent based modelling. I usually ask GPT to write about the topic in the style of an academic literature review, including references. This gives me a quick overview of the topic, as well as some first papers to dig into, if needed. I’ve tested GPT with topics I’m very familiar with (circular economy, circular cities), and it does manage to cover most of the existing literature and key references. One important note is that I always use GPT-4, the newer paid model, rather than GPT-3.5. This is because GPT-3.5 comes up with references that look really interesting, but are completely fake!

Finally, I use ChatGPT to write letters, emails, reports, and proposals. A recent example is that I had to write a cover letter that went together with a journal article submission. I simply pasted in the chat:

• the requirements for the cover letter from the journal website,
• the abstract of my journal article, and
• explained that I wanted to write a cover letter to a specific journal, as well as a few key points I definitely wanted to include.

The resulting letter was almost good enough, and was ready for submission only after a few tweaks. I use the same technique for writing cover letters to job applications (especially the ones I don’t really care about).

I don’t actually use ChatGPT for writing content that I think is important, so I tend to write my journal and online articles by myself. This is because I think writing helps me think and clarify my ideas, something I don’t want to delegate to ChatGPT. And honestly, I prefer my writing style of ChatGPT’s.

Hope you found these useful!

So there you are, four tools that helped me through my PhD. I hope you found it useful! If you have any handy tools, I’d love to hear about them in the comments.

Recently, I found out about ChatGPT, an ai chat bot developed by OpenAI that is capable of responding to users in a remarkably human manner. Not only can chatGPT give responses using text, you can ask it to write code as well. In fact, an acquaintance recently told me that she used chatGPT to do some of her homework for her Masters in urban data science, both for writing essays and writing code.

As someone who spends most of her working hours coding, having a chat bot write code for me sounds both incredibly exciting (think of all the hours I could save!) and a little bit scary (does this mean I’ll be out of a job in a few years?). I decided to try out the coding capabilities of chat GPT, specifically for spatial data analysis with Python. This article shows some first results, and my first thoughts.

For this article, I asked Chat GPT to do a typical task I have to do often: turn a polygon (could be the shape of a country, city, or municipality) into a grid of cells where statistics could be easily aggregated.

Before we start, what is Chat GPT, actually? I asked this question to the chatbot itself, and this is its response:

GPT (short for “Generative Pre-training Transformer”) is a type of language model developed by OpenAI. It is a neural network trained to generate human-like text by predicting the next word in a sequence given a large dataset of examples. Chat GPT refers to the use of this model in chat applications, where it can be used to generate responses to user messages in real time. It is designed to be able to carry on a conversation by understanding the context of the conversation and generating appropriate responses.

Let’s start!

First, I asked Chat GPT, “make a geodataframe grid within a polygon”, and this was its response:

import geopandas as gpd

# create a polygon
polygon = gpd.GeoDataFrame({'geometry': gpd.GeoSeries([Polygon([(0,0), (0,1), (1,1), (1,0)])])})
# create a grid of geodataframe within the polygon
grid = gpd.gridify(polygon, cell_size=0.1, geometry='Polygon')
# plot the grid
grid.plot()

I was quite impressed with the response, especially its suggestion to use gpd.gridify(), a geopandas attribute I’ve never heard of before. However, when I tried to run the code, I found out that the module geopandashas no attribute gridify. In other words, the chat bot suggested for me to use a tool that looks really handy, but doesn’t actually exist.

There is, however, another python module called gridify, so perhaps the chatbot got confused by that, see link here. It also forgot to import the shapely attribute Polygon before using it, but that was just a minor issue.

The interesting thing about chat GPT is that you can have a continuous conversation with it, and it will remember your previous interactions. So I asked a follow-up question to try to get a working piece of code. So I asked, “I tried this code but got the error “module ‘geopandas’ has no attribute ‘gridify’”. can you fix this for me?”, and this was its response:

While the bot claimed that shapely has the attribute gridify, I found out that wasn’t true when I ran the code. shapely doesn’t have the attribute gridify either.

So I asked another follow-up question, “I tried the code but got the error message “No module named ‘shapely.gridify’”. Any other options?”

The bot then proceeded to confidently tell me that gridify IS part of the shapely library, but has been removed in newer versions — that’s also not true as far as I know.

The code it wrote produced … a square loop, which is far from what I was looking for. Honestly I found it pretty hilarious.

Second attempt: it worked!

My first conversation with chat GPT clearly wasn’t going very well, so I decided to start again, and ask my question more clearly. I asked it, “python code for generating a grid of cells as a geodataframe”, and here is its response:

import geopandas as gpd
from shapely.geometry import Polygon

# Define the bounds of the grid
min_x, min_y = 0, 0
max_x, max_y = 10, 10

# Define the size of the cells in the grid
cell_width, cell_height = 1, 1

# Define the projection to use
crs = "EPSG:4326"  # WGS84

# Initialize an empty list to store the grid cells
grid_cells = []

# Iterate over the bounds of the grid and create a Polygon object for each cell
for x in range(min_x, max_x, cell_width):
    for y in range(min_y, max_y, cell_height):
        # Define the coordinates of the cell
        cell_coords = [(x, y), (x+cell_width, y), (x+cell_width, y+cell_height), (x, y+cell_height)]
        
        # Create the Polygon object
        cell = Polygon(cell_coords)
        
        # Add the Polygon to the list
        grid_cells.append(cell)

# Create the GeoDataFrame
gdf = gpd.GeoDataFrame(geometry=grid_cells, crs=crs)

# Visualize the grid
gdf.plot()

…and it works!! The code above managed to create a grid of cells as a geodataframe, which could be cropped to fit onto a specific area (e.g. a country, municipality, or city). I could probably ask chatGPT to do this next step as well, but unfortunately the bot is currently unavailable because too many people are using it — something that’s been happening more and more often lately.

So chat GPT is indeed capable of writing simple code in Python for spatial data science, as long as it is given a well written and clear question.

Final thoughts

This brief experiment with chatGPT highlights two interesting things about the chatbot, and perhaps AI in general.

Firstly, we can see that AI can generate artifacts that may look legitimate at first glace, but turns out to be wrong under further investigation. A common example is images of faces generated by AI, where the faces may look realistic at first glance, but turn out to have strange anomalies when looked at more closely.

When writing our code, chat GPT did the something similar by making up a geopandas module called gpd.gridify() — a module that seems to make sense, but doesn’t actually exist.

Secondly, the purpose of chatGPT (and perhaps other AI bots) is not to present accurate information, but rather what looks like accurate information. According to the FAQ page of chatGPT, the model is trained to mimic human speech as much as possible, and not to provide us with the truth.

This is why it so confidently claims that gridify is part of the geopandas library, and when proven wrong, claims that gridify is part of shapely library instead. Neither of these statements are true, but perhaps that is not the point of chatGPT.

What’s next?

I’m definitely very excited about the capabilities of chatGPT, and will be using it to work on other more difficult coding tasks in the future.

I also found that it is quite handy at generating ideas — I asked for ideas for data science projects for sustainability + spatial data science, and while it didn’t produce ideas that were novel to me, it provided quite a good overview of the sustainability / spatial data science world, which I can imagine could be quite handy for someone new to the field.

Here’s chatGPT’s response when I asked for project ideas:

• Sustainable transportation planning: Develop a tool that helps planners optimize public transportation networks to reduce greenhouse gas emissions and improve accessibility. This could involve analyzing patterns of travel demand, integrating with public transportation data sources, and implementing algorithms for network design and routing.
• Sustainable land use planning: Develop a tool that helps planners optimize land use patterns to minimize environmental impacts and enhance sustainability. This could involve analyzing patterns of land use and land cover, integrating with land use data sources, and implementing algorithms for land use allocation and optimization.
• Environmental impact assessment: Develop a tool that helps organizations assess the environmental impacts of their operations or projects. This could involve integrating with data sources on environmental indicators, such as air and water quality, and implementing algorithms for impact assessment and prediction.
• Renewable energy site selection: Develop a tool that helps organizations identify suitable sites for renewable energy generation, such as solar or wind farms. This could involve analyzing patterns of solar insolation, wind speed, and other relevant factors, and integrating with data sources on land use and topography.
• Sustainable supply chain optimization: Develop a tool that helps organizations optimize their supply chains to minimize environmental impacts and enhance sustainability. This could involve analyzing patterns of material and energy flows, integrating with supply chain data sources, and implementing algorithms for optimization and risk assessment.

ChatGPT is open for everyone to use for now on chat.openai.com. To use it, all you need is an internet browser. Feel free to try it for yourself, and let me know what you find :)

Acknowledgements

I’d like to thank my friends Alexey Ilyshkin and Alexis Oh for sharing their thoughts on chatGPT and AI with me. Our coversations were integrated into the ‘final thoughts’ section of this article.

Introduction

Last week, the Netherlands was hit with an unprecedented heat wave where temperatures reached as high as 40 degrees Celsius. When I was about to get on my bike to cycle home in the blistering heat, I thought to myself, “I wish Google Maps could give me directions to cycle home in the shade. In fact, I’d probably be willing to cycle an extra 10 minutes just to stay in the shade.”

I got home, started coding, and here is the result. It’s by no means a functioning tool, but it’s a prototype that can calculate the shortest shaded cycling route from one point to another in the Dutch city I live in — Delft.

Let’s start!

Before we dive in, let’s import some handy packages: geopandas and shapely are for handling geo-spatial data, pandas is for handling non-spatial data, matplotlib is for plotting maps, and osmnx is for handling street network data.

# import modules
import geopandas as gpd
import matplotlib.pyplot as plt
from shapely.geometry import Point
import osmnx as ox
import pandas as pd

Data

To make this work, we need two pieces of information: where the bike paths are, and where the trees are. Bike paths can be found on Open Street Map, a free editable geographic database of the world. Tree data can be found in the “Bomen in Nederland” (“Trees in the Netherlands”) dataset created by the Dutch Institute for Public Health and Environment (Rijksinstituut voor Volksgezondheid en Milieu, RIVM).

Here, we read the data on trees and bike paths in Delft. The tree data is a grid covering the Netherlands with cell sizes of 10 x 10 m, with each cell showing the percentage of area that’s covered by trees (for more info, see here). The grid was then converted to points (instead of grid cells) and saved as trees_delft.shp, and read as treesGdfin this code. The bike path data comes in two parts: bikeEdges, the bike paths; and bikeNodes, the intersections on the bike paths.

# read data on trees and bike paths
treesGdf = gpd.read_file('data/trees_delft.shp')
bikeGraph = ox.io.load_graphml('data/bikePaths.graphml')
bikeGdfs = ox.utils_graph.graph_to_gdfs(bikeGraph)
bikeNodes = bikeGdfs[0]
bikeEdges = bikeGdfs[1]

Then, we estimate where the trees’ shadows could be. We assume that locations with more than 50% tree coverage would generate enough shadow to cool down its surroundings, and that this cooling effect can benefit areas within 20m of the trees. This is represented as treePct and bufferM. These numbers are completely arbitrary, but can be further refined by experts.

# input
treePct = 50 # % tree cover to be considered as shadows
bufferM = 20 # m for buffer to create shadow

# find shadows created by trees
trees = treesGdf[treesGdf.trees > treePct]
treesBuffer = trees.geometry.buffer(bufferM).unary_union
shadows = treesBuffer # multipolygon
shadowsGdf = gpd.GeoDataFrame(treesBuffer).rename(columns={0:'geometry'})
shadowsPlot = shadowsGdf.copy()
shadowsPlot = shadowsPlot.set_crs('EPSG:28992').to_crs('EPSG:4326')

Then, we map out the data we’ve created so far: the trees, the bike paths, and the shadows.

# plot 
fig, ax = plt.subplots(figsize=(15,15))
trees.to_crs('EPSG:4326').plot(ax=ax, markersize=0.1)
shadowsPlot.plot(ax=ax, fc='#b2e8a0', alpha=0.5, zorder=1)
bikeEdges.plot(ax=ax, color='grey', linewidth=0.5)
ax.axis('off')
plt.show()

Analysis

The next step is to find the bike paths that are shaded. In other words, to find the bike paths that touch the shadows that we have created.

# prepare elements for finding shaded bike paths 
bikeEdges_a = bikeEdges.to_crs('EPSG:28992').reset_index()
bikeNodes_a = bikeNodes.reset_index().to_crs('EPSG:28992')
shadows_cut = ox.utils_geo._quadrat_cut_geometry(shadows, quadrat_width=1000)
sindex = bikeEdges_a.sindex # build the r-tree index

# find shadedEdges and shadedNodes
points_within_geometry = pd.DataFrame()
for poly in shadows_cut.geoms:
    possible_matches_index = list(sindex.intersection(poly.bounds))
    possible_matches = bikeEdges_a.iloc[possible_matches_index]
    precise_matches = possible_matches[possible_matches.intersects(poly)]
    points_within_geometry = pd.concat([points_within_geometry, precise_matches])
    
# output: bike paths and nodes that are shaded 
shadedEdges = points_within_geometry
shadedNodeIds = list(shadedEdges.u) + list(shadedEdges.v)
shadedNodes = bikeNodes_a[bikeNodes_a.osmid.isin(shadedNodeIds)]

# plot
fig, ax = plt.subplots(figsize=(15,15))
trees.to_crs('EPSG:4326').plot(ax=ax, markersize=0.1)
shadowsPlot.plot(ax=ax, fc='#b2e8a0', alpha=0.5, zorder=1)
bikeEdges.plot(ax=ax, color='grey', linewidth=0.8)
shadedEdges.to_crs('EPSG:4326').plot(ax=ax, color='green', linewidth=0.5)
ax.axis('off')
plt.show()

The two maps below show the original bike path on the left, and the shaded bike paths on the right. On the original bike paths map (left), it is possible to get from any point to any other point on the map by following the paths. However, on the shaded bike paths map (right), this isn’t the case — it is not possible for any point to reach any other point, because the network is disconnected. This is a problem for us, because a street network used for navigation needs to have every point connected to every other point.

When a network is disconnected in a way that doesn’t allow all points to be connected to each other, like in the shaded bike paths, it is called a weakly connected graph. In order for us to use these shaded bike paths for a navigation tool, we need turn the weakly connected network into a strongly connected network, where every point is reachable to every other point. For more information on strongly connected graphs, see video here.

# covert bike path gdfs to graph
shadedNodes_g = shadedNodes.to_crs('EPSG:4326').set_index('osmid')
shadedEdges_g = shadedEdges.to_crs('EPSG:4326').set_index(['u', 'v', 'key'])
G_shaded = ox.utils_graph.graph_from_gdfs(shadedNodes_g, shadedEdges_g, graph_attrs=None)

# impute missing edge speeds and calculate edge travel times with the speed module
G_shaded = ox.speed.add_edge_speeds(G_shaded)
G_shaded = ox.speed.add_edge_travel_times(G_shaded)

# convert to strongly connected graph
Gs = ox.utils_graph.get_largest_component(G_shaded, strongly=True)

Now that we have a strongly connected shaded bike path network, we can input two addresses and calculate the shortest travel path between them.

# input
origAddress = 'insert address here' # address omitted for privacy
destAddress = 'Prometheusplein 1, 2628 ZC Delft'

# geocode orgin and destination address 
from geopy.geocoders import Nominatim
geolocator = Nominatim(user_agent="shadyStreets")
origLoc = geolocator.geocode(origAddress)
destLoc = geolocator.geocode(destAddress)

# find origin and destination nodes on bike path
orig = ox.distance.nearest_nodes(Gs, X=origLoc.longitude, Y=origLoc.latitude)
dest = ox.distance.nearest_nodes(Gs, X=destLoc.longitude, Y=destLoc.latitude)

# output: shortest shaded route
route_shaded = ox.shortest_path(Gs, orig, dest, weight="travel_time")

Finally, we can map out the shortest shaded travel route, and compare this to the normal (un-shaded) travel route.

# input
plotNormal = True

# calculate normal (unshaded) path
if plotNormal == True: 
    # calculate normal path route
    bikeGraph = ox.speed.add_edge_speeds(bikeGraph)
    bikeGraph = ox.speed.add_edge_travel_times(bikeGraph)

# get the nearest network nodes to two lat/lng points with the distance module
    orig = ox.distance.nearest_nodes(bikeGraph, X=origLoc.longitude, Y=origLoc.latitude)
    dest = ox.distance.nearest_nodes(bikeGraph, X=destLoc.longitude, Y=destLoc.latitude)
    route_normal = ox.shortest_path(bikeGraph, orig, dest, weight="travel_time")

# plot routes
fig, ax = plt.subplots(figsize=(15, 15))
ax.axis('off')
trees.to_crs('EPSG:4326').plot(ax=ax, markersize=0.1)
bikeEdges.plot(ax=ax, color='grey', linewidth=0.5)
shadowsPlot.plot(ax=ax, fc='#b2e8a0', alpha=0.5, zorder=1)
if plotNormal == True: 
    ox.plot.plot_graph_routes(bikeGraph, [route_shaded, route_normal], route_colors=['g', 'r'], ax=ax)
else: 
    ox.plot_graph_route(Gs, route_shaded, node_size=0, ax=ax, route_color='b')
plt.show()

And there you have it — a basic tool that helps you navigate your bike in the shade.

Further work

There’s a lot to add and improve with this basic tool. We could refine the selection process for the shaded streets by changing the threshold of tree coverage, the shadow buffer of the trees, and more importantly, ranking the streets by how shaded they are, rather than just deleting streets that are not shaded.

In fact, I found a much more sophisticated tool by research scientist and software developer Keith Ma called Parasol, which uses high-resolution elevation data to simulate sunshine and constructs routes that keep users in the sun or shade, whichever they prefer.

Another follow up was suggested by my girlfriend Genn: you could use this tool to find the streets in Delft that are in the most need for trees, by selecting streets that are both frequently accessed (or with high centrality) and don’t have trees nearby.

Documentation

All the required data and code can be found on https://github.com/TanyaTsui/shadyStreets

Generating solar energy will be a major part of our transition to an environmentally sustainable future. For me, one interesting thing about generating solar energy is that it takes up a lot of space — you can see this at solar farms, where seemingly endless rows solar panels are laid out on a vast piece of land. Knowing this, we can say that the amount of solar energy we can produce depends on the amount of land that’s available.

This brings up an interesting question: how much land would it take to generate enough energy to run an entire country? In this article, I’m going to answer this question using the Netherlands as an example.

Before I dive in, a disclaimer need to be made: I’m by no means an energy expert, so what I’m about to show you shouldn’t be taken too seriously. What I’ve done is a quick and dirty spatial analysis that can give us a very rough answer, based on many assumptions. If you see any mistakes or would like to suggest ways to improve the analysis, please let me (and other readers) know in the comments.

Getting the data

In order to answer my question, I needed two pieces of data: the total amount of energy used by the Netherlands, and a map of the potential amount of solar energy that could be produced by the country.

First, I looked up the total amount of energy used by the Netherlands, which, according to the Dutch Central Bureau of Statistics, was ~877 billion kilowatt hours (kWh) per year. For your reference, a typical washing machine will use 2.3 kWh per load.

Then, I looked up maps that showed the amount of solar energy that could potentially be generated in the Netherlands. The websites solar GIS and global solar atlas provide a lot of solar energy related geo-data (maps) for free. The data I used was the Global Irradiation for Optimally Tilted Surfaces (GTI) map, which shows, in each pixel on the map, how much solar energy could potentially be produced per square meter (kWh/m²).

Diving into the analysis

Now that we have the data, it’s time to dive in and start coding!

First, we import the relevant python modules and read our required data.

Now, let’s a take a look at the data we have, which is the map of Global Irradiation for Optimally Tilted Surfaces (GTI).

As seen in the table above, our data has three columns:

• geometry, which records the location of each pixel on the map
• gti, which shows, for each map pixel, the amount of solar energy that could potentially be produced per square meter per year (kWh/m²)
• kWhPerYear, which shows, for each map pixel, the total amount of energy that could be produced per year (kWh)

The same information can be shown as a pixelated map, as seen below — the more yellow a location is on the map, the more solar energy could potentially be produced there. We can see in the map that the highest solar energy potential is on the east and south east of the country, on the coast.

Now that we understand the solar energy potential of the country, it’s time to map out the land needed to generate enough solar energy to run the Netherlands.

As mentioned before, the total annual energy consumption of the Netherlands is around 877 billion kWh. What the code below is doing, is it iterates through the energy potential map of the Netherlands, pixel by pixel; from the pixel with the highest solar potential to the pixel with the lowest solar potential. Each time the code gets to the next pixel, it records its energy potential (kWh), and adds it to the energy potential of all the previous pixels it has iterated through. As soon as the total energy potential reaches 877 billion, the code stops, and makes a record of all the pixels it has already iterated through.

And that’s it! With the code above, we can now identify the pixels on the map that would be required to generate enough solar energy to run the entire country. We can plot these pixels in a map, using the code below.

The map shows, in red, the land required to generate enough solar energy to power the whole country, which adds up to around 12% of the total land. Most of this land would be on the coast, on the west side of the country. I was quite surprised by the amount of land required, as it’s a lot less than I expected, especially given the fact that the Netherlands is rainy and cloudy most of the time.

Disclaimers

While the map I’ve created gives us a rough idea of the amount of land required to generate enough solar energy to run the whole country, the result is based on many, many, assumptions.

The biggest assumption is that everyone living, working, and growing food in the red areas on the map are willing to move out, to make way for the biggest solar farm the Netherlands has ever seen — highly unlikely.

There are also other assumptions that are not related to land. For example, I haven’t taken into account the inefficiencies of a solar park at such a large scale, or the fact that solar energy is less consistent than, say, fossil fuels, or the amount of critical materials that would be needed to make all these solar panels.

Final words

While this spatial analysis was pretty quick and dirty, the results do roughly match a more in-depth study done by Dutch energy consultants, which found that, to meet solar energy’s envisioned share in the 2050 energy targets, 10 percent of Dutch agricultural land needs to be repurposed as solar fields.

In this article, I’ve shown that it takes a surprisingly small amount of land to run the country on solar energy. This suggests that the barriers to a renewable energy transition isn’t the lack of land (or sunlight), but other factors, such as the difficulty of changing land functions, the availability of critical materials, and the scaling up of renewable energy technologies.

Documentation

All data and code for this project can be found on https://github.com/TanyaTsui/nl_solar

Introduction

Recently, I watched a snippet of a Jordan Peterson lecture on the Pareto’s Principle, which states that for many outcomes, roughly 80% of consequences come from 20% of causes.

It’s a rule of thumb I’ve found to be accurate in many parts of my life. I often notice that, for example, ~20% of people in a group project do ~80% of the work; or ~20% of students in a classroom contribute to ~80% of the questions. While the numbers 80/20 are by no means accurate, I can intuitively understand that, in many situations, a minority of people own/create/contribute to the majority of available stuff.

In the video, Dr. Peterson explains that you can also see this in a game of Monopoly: if the game goes on for long enough, one player ends up with all the money, while all the other players end up with none. This ‘winner takes all’ outcome is, according to the professor, an inevitable consequence of multiple trades that are conducted randomly.

In fact, you could simulate the process with a random trading game. Imagine you invite 10 people to play a game, and in the beginning of the game, each player is given 10 dollars. Then, for multiple rounds, the players get paired up and toss a coin to decide who wins or loses, allowing the winner to take a dollar from the loser. When a player runs out of money, they are eliminated from the game.

If enough rounds are played, Peterson claims that the outcome will be similar to monopoly: one player will have all the money, and the others will all have none.

My ears perked up when he talked about simulation. I thought, “it can’t be too difficult to recreate this simulation with python!” And so that’s what I created— a python code that simulates, records, and plots out a random multi-round trading game with 10 players. The simulation starts off with 10 players with 10 points each, and continues running until there is only one player left in the game. The performance of the 10 players over the multiple rounds are then plotted in a line graph.

The code is shown and explained below in the following paragraphs, so feel free to copy or change it as you please. Comments or suggestions are also very welcome!

Walking through the python code

Let’s start by importing the required modules.

Here, we create 10 players and give them 10 points each. The information about the players is stored in a dictionary called players, and the dataframe scoreboard will be used to record the number of points each players will have in each round.

We can now write what happens in each round of the simulation. First, the players are randomly paired together.

For each pair of players, we determine who wins with random.randint(), and allow the winner to take a point from the loser. The new scores for each player are then stored in the dataframe scoreboard.

After each round, we remove the players who don’t have any points left, and check if there’s only one player left in the game. When enough rounds have been played and only one player remains, the simulation stops.

Finally, we cleanup the scoreboard and display the performance of the players over time in a plotly line chart, which you can see in the next section of this article.

Results

I tried running the simulation a few times, and here are what the results looked like.

The number of rounds it took for one winner to emerge varied across different simulations, from 1000 to 5000 rounds. In each simulation, the story is roughly the same — most of the players are knocked out early on in the game (~500 rounds), leaving two or three players to battle it out until the end.

What does it all mean?

So there you have it — a simulation of how “the winner takes it all” in a random game of chance.

So, do the line charts somehow explain or justify the inequality we see in the world around us? Yes and no.

These charts are by no means a complete representation of the real world, or of the real life ‘games’ we play, like starting a company or developing career. The simulation didn’t take into account factors like our tendency to share and cooperate, our desire to help those who have less than us, as well as our instinctive dislike for those who take without giving back.

On the other hand, this simple simulation is also a powerful example of how deeply inequality is embedded in reality. No matter how fair a system is for distributing resources, there will always be a inevitable tendency for resources to accumulate in the hands of a few lucky players.

What do you think?