It’s hard to deny that Large Language Models (LLMs) are making a profound impact on various industries and applications, revolutionizing the way we work and interact. Even though the initial hype around ChatGPT has calmed down since its release about six months ago (in November 2022), its influence remains significant. It seems that autoregressive LLMs will continue to be a part of our lives in the near future, and it is worth developing skills to interact with them, both as a developer and a user.

As Chip Huyen stated in her blog post, it is relatively easy to achieve something impressive with LLMs, but it is quite challenging to build something production-ready considering the limitations and potential issues that LLMs currently have. However, while the research and engineering community is actively working to address these challenges, it is worth acknowledging the fact that individuals can already benefit greatly from LLMs, at least using them as personal assistants for everyday non-critical tasks or as collaborators for brainstorming.

In my previous article, I discussed the best practices of prompt engineering, providing insights to help you develop local LLMs-based applications. In this post, I will share a set of techniques that enable you to utilize models such as ChatGPT for effective code debugging and accelerated learning of programming. We will also take a look at the example prompts for writing and explaining code. These techniques will be valuable not only when interacting with ChatGPT but also when seeking assistance from your colleagues or even tackling programming challenges independently.

This article is primarily targeted toward beginners, so I tried to provide illustrative examples and explanations. I hope these techniques will assist you in understanding and troubleshooting code more efficiently.

General Framework for Code Debugging

In fact, ChatGPT has not made significant changes to the debugging process. The great thing is, now you can easily connect with a virtual colleague without worrying about being a bother or feeling hesitant to ask stupid questions! But the techniques that we will consider exist as long as software engineering exists, and therefore will be useful not only when interacting with LLMs, but also for a better understanding of the process and more effective interaction with coworkers.

To find a bug in your code, you only need two essential steps (there’s three actually):

1. Isolate the bug and demonstrate it with the minimum amount of code;
2. Make an assumption about your error and test it;
3. Iterate with more assumptions until you find a fix.

While you can start using ChatGPT right away, it’s actually a better idea to begin by reproducing the error for a few reasons. First of all, it might be challenging to include all the related points and explain exactly what you’re trying to achieve within the context of the language model. Secondly, it will allow you to gain a better understanding of the issue and possibly find the error yourself. Let’s see.

By the way, in this post I am using the vanilla version of ChatGPT (GPT-3.5), but for coding tasks, GPT-4 is typically more proficient.

Step 1: Isolate and Reproduce the Problem with the Minimum Amount of Code

The first step is to reproduce the problem. As we know, the majority of issues can still be resolved with the classic “turn it off and on again”. It’s possible that you might have become tangled up with the code execution order in Jupyter Notebook.

If possible (and it typically is), it’s recommended to write new code that throws the same error and keep it as simple as possible.

Let’s consider the example of a TypeError: ‘int' object is not iterable, which occurs when you try to iterate over some_integer instead of using the range(some_integer) construct.

Bad example: a function calls another function that then invokes a method of a class. At first glance, it may require some time to determine where the actual computation occurs, despite this being a relatively simple example. Similarly, for models, it becomes more challenging to locate the relevant information among unrelated details.

Better example: get rid of the class by moving the functionality of do_some_work() function (which is causing the error) directly into the function we are calling.

Besides the fact that we still do a terrible job with the variable naming conventions (remember, variable names should be descriptive and meaningful!), this code is still easier to debug and understand.

Even better example: we can get rid of some_function() as well.

Overall, we have shortened the code by more than half. Compare how much easier it is to find a bug in it.

In the context of pandas, for example, this principle can mean not using the original dataframe. Let’s consider a situation where we want to calculate the average salary for each position using our data and encounter a KeyError. Here is a bad example:

First, we can’t be sure that the dataframe contains the data provided in the comments. Indeed, we only need two columns from it, and if we create a similar mini version, it will be much easier to understand that we simply misspelled the salary column (Salary vs salary).

By the way, ChatGPT is quite good at generating dummy data, so it can be helpful here too!

There are countless types of errors, and it’s impossible to list them all, of course. Overall, try to modify the code in such a way that it produces the same error as you encountered, but make it as easy as possible to understand quickly.

Due to the so-called “rubber duck debugging,” this step often helps you understand the cause of the problem on your own, without seeking external assistance. For example, if your mini-code doesn’t generate the same error, you’re already halfway toward finding a solution. However, even if it does, it’s still a positive outcome. :)

Step 2: Make an Assumption, Test It, and Iterate

If you still can’t find a way to fix the error, it’s worth seeking assistance. But it’s helpful to have your own assumptions about what could be wrong.

Find the Exact Line

First, find the expression and the exact line of code that are causing the problem. You probably already know that it is inside the last line of the mini-code you wrote earlier.

Remember that Python traceback displays the error message at the bottom and the corresponding executed code at the top with internal functions’ calls in between.

This can be quite easy for straightforward bugs, but it can be more challenging when dealing with logical errors that don’t generate any error messages but result in unexpected outputs due to logical mistakes. In such cases, it’s helpful to observe the values step by step with a debugger or simple print() statements and define the line of code that doesn’t align with your expectations.

If the error is caused by a complex expression, such as df.groupby(‘Occupation’)[‘Address’].apply(lambda x: ‘, ‘.join(x)), you can first break it into parts and explore the output step by step, e.g. first run df.groupby(‘Occupation’), then df.groupby(‘Occupation’)[‘Address’], and so on.

Think about Common Reasons

After that, consider some common reasons for the error:

• Could it be that the necessary library is not installed or installed with the wrong version?
• Maybe there’s a simple typo or syntax error somewhere?
• Might it be that the error is related to data types, e.g. you sum a string and a number?
• And so forth.

Ask ChatGPT

If nothing comes to mind, it’s time to ask ChatGPT for assistance. Simple issues can often be resolved by simply pasting your code and asking what’s wrong. However, for more complex problems, you may need to provide additional relevant information. For instance, if you’re encountering a system error, it might be helpful to include the version of Python you’re using. Overall, try to always include the error message and describe what you’re trying to achieve. You may have to try several wordings, so don’t be afraid to experiment.

Situations can vary greatly, so let’s move on to some examples. First, let’s take a look at the pandas KeyError we encountered earlier.

Sample prompt:

This Python code results in <error> in <this line>.

```
Insert your code here
```

Tell me how to debug the code to solve the given error.

Example:

This Python code results in KeyError: 'Column not found: salary' 
in the last line.

```
import pandas as pd

df = pd.DataFrame({
    'Occupation': ['Engineer', 'Doctor', 'Engineer'],
    'Salary': [56056, 61304, 86850],
})

average_salary_per_occupation = df.groupby('Occupation')['salary'].mean()
```

Tell me how to debug the code to solve the given error.

Looks good! Let’s see a more challenging example with a logical error.

Sample prompt:

This Python code <does this, but I want it to do this>.

```
Insert your code here
```

Tell me how to fix the code to solve the problem.

Example:

This Python code trains a random forest classifier on the Iris dataset.
As far as I know, the dataset is relatively simple and I expect 
the classifier to make the perfect predictions,
however, I am getting about 95% accuracy even with 100 trees in the forest.
Are these results reasonable for this situation?

```
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split, GridSearchCV

# load the Iris dataset
iris = load_iris()
X = iris.data
y = iris.target

# split the data
X_test, X_train, y_test, y_train = train_test_split(
    X, y, test_size=0.2, random_state=42)
X_train, X_val, y_train, y_val = train_test_split(
    X_train, y_train, test_size=0.2, random_state=42)

param_grid = {
    'n_estimators': [3, 7, 15, 25, 50, 100]
}

# create a random forest classifier and 
# use grid search to find the best number of trees
rf_classifier = RandomForestClassifier()
grid_search = GridSearchCV(estimator=rf_classifier, 
                           param_grid=param_grid, cv=5)
grid_search.fit(X_train, y_train)

# get the best estimator from grid search and test it
best_rf = grid_search.best_estimator_
y_pred = best_rf.predict(X_test)
test_accuracy = accuracy_score(y_test, y_pred)
print("Test accuracy:", test_accuracy)

# get the best number of trees from the grid search
best_n_estimators = grid_search.best_params_['n_estimators']
print("Best number of trees:", best_n_estimators)
```

Tell me how to fix the code to solve the problem if there is a problem.

The model was able to find the hidden problem and fix it. Great!

Since ChatGPT remembers your previous messages, the possibilities opening here are endless. You can ask it to explain some concepts you’re having trouble understanding, suggest alternative solutions, translate code from one language to another, and so on.

Moreover, since ChatGPT can understand the code, it can also write it.

Use ChatGPT to Write and Explain Code

In this section, we will explore a couple of tricks that can be used when coding with ChatGPT. But first, I think it is important to remember that before the emergence of ChatGPT, Google was the primary tool for software developers.

And I think it’s important to not forget how to use Google, for various reasons. In the end, with Google you can probably do everything you can do with ChatGPT (in coding settings), it just might be slower. Although, when it comes to specific tasks like creating a diagonal matrix with numpy, I would likely do it faster using Google.

I think the fitting analogy is the process of learning a foreign language (although one can say that you are doing exactly that by learning a programming language :). Using Google is translating individual words using a vocabulary while utilizing ChatGPT is akin to translating entire sentences and paragraphs of text using online translators. While ChatGPT can be incredibly powerful, you may face challenges in identifying hidden errors or understanding certain code blocks, especially as a beginner.

Given the immense capabilities of ChatGPT, it can be employed in various ways. I advise you to use it for generating short code snippets — acting as a dictionary for idioms or stable expressions. Making sure you have a solid understanding of the generated code is essential because it can help prevent future troubles and complications.

So, use ChatGPT, but remember to google things as well!
This article offers some valuable tips on effective googling for software developers.

Write

One technique that is helpful in these situations is called role prompting. Instead of simply asking the model to generate code, you can ask it to write code by being a junior developer, for instance. By assuming this role, the model is more likely to produce code comprehensible to beginners and avoid overly complex constructs.

Sample prompt:

Act like a junior Python developer.
Write code with comments explaining what is going on.
When providing the solution, ensure the output is well-formatted
and the code is well-documented.
Include example usage and explanations.

<Describe your problem here>

Example:

Act like a junior Python developer.
Write code with comments explaining what is going on.
When providing the solution, ensure the output is well-formatted
and the code is well-documented.
Include example usage and explanations.

Write a program to find common values between two numpy arrays.
It is a function that takes two numpy arrays as inputs and
output a numpy array.

In this example, ChatGPT did a pretty good job following all of our instructions, including examples and detailed explanations.

Don’t forget to compare the code with your initial assumptions and draw conclusions that will allow you not to make such a request in the future.

In general, remember that LLMs have no prior knowledge about you or the specific problem you’re facing, so the more information you provide, the better output you’ll get:

• Describe your task;
• Define code structure: e.g. it is full script, class, or function;
• Specify inputs and outputs: e.g. the function takes two integer arguments and outputs float;
• Mention tools/libraries you want to use, such as numpy or pandas, as well as a programming language;
• If feasible, add your suggestions on what the solution might look like: e.g. I suggest using the pandas.DataFrame.groupby function to calculate the average salary by position.

Optimize

If we previously asked the model to behave like a junior developer, we want it to be stronger in programming for tasks such as code optimization.

By the way, you can use the same role prompting trick for debugging as well.

Sample prompt:

Act like an experienced Python developer mentoring a junior developer.
Provide explanations and comments about the concepts that can be
hard for beginners. Use Python best practices and write docstrings.

Rewrite the above Python code and optimize it.

```
Insert your code here
```

Example:

Act like an experienced Python developer mentoring a junior developer.
Provide explanations and comments about the concepts that can be
hard for beginners. Use Python best practices and write docstrings.

Rewrite the above Python code and optimize it.

```
def unique_list(l):
    # Get a list of distinct elements from the given list

    x = []
    for a in l:
        if a not in x:
            x.append(a)
    return x
```

Here, to extract unique elements from the list, ChatGPT suggests using the set datatype (that does not allow duplicate values by definition), instead of a for loop. A good choice, since it’s basically a one-line solution.

Explain

We can use the same notation requesting comments and code explanations. Often this tends to output redundant comments, like import numpy as np # importing numpy library. However, this can still be useful at the start of your journey, and help the model to express its internal thoughts as discussed in the Chain-of-Thought reasoning section in my previous article.

Sample prompt:

Act like an experienced Python developer mentoring a junior developer.
Provide explanations and comments about the concepts that can be
hard for beginners. 

Explain and comment the following code.

```
Insert your code here
```

Example:

Act like an experienced Python developer mentoring a junior developer.
Provide explanations and comments about the concepts that can be
hard for beginners. 

Explain and comment the following code.

```
def func(n):
    trow = [1]
    y = [0]
    for x in range(max(n, 0)):
        print(trow)
        trow = [l + r for l, r in zip(trow + y, y + trow)]
    return n >= 1

func(6)
```

Here, the model was able to identify the task behind this code: generate rows of Pascal’s triangle. Good job!

While there are numerous useful applications for ChatGPT you can come up with, it’s important to note that it may not always be able to solve every problem you encounter. It’s worth discussing this aspect further.

Few Notes about Potential Pitfalls

While ChatGPT can be very useful and perform various tasks, it is important to keep in mind the potential drawbacks which can be tricky.

As Autoregressive LLM, ChatGPT is not Deterministic …

ChatGPT is an example of a Large Language Model, and current Large Language Models are autoregressive, meaning that they are trained to predict the next token in a sequence. The output of a model is a probability distribution across all possible tokens, and we sample the final text from this distribution token by token. As a result, the sampling process is non-deterministic, meaning that you can have different outputs for the same input due to probabilistic reasons.

To illustrate this, we can imagine the sampling process as a tree. Here the initial sentence is shown in blue, selected tokens are shown in green, and non-selected ones are represented in red (without their further evolution). The probabilities are picked randomly just for illustration reasons.

The input sequence is: “My name is” and ChatGPT completes it with “ChatGPT, pleased to assist you!”. This is an example from my previous article, where I discuss the basics of LLMs.

… and That’s Why ChatGPT can be Wrong

In practice, this means that you can get suboptimal output just because you got some rare unlikely tokens at the start. As a result, you may need several runs of the same input to examine the different outputs and select the most suitable one or even combine different parts from different outputs.

Additionally, it’s important to highlight that modern language models, especially GPT-4, possess impressive self-correction capabilities. If the generated code contains errors, you can simply return it and indicate that it is not functioning correctly. GPT-4 is adept at debugging its own code and offering relevant suggestions. You’ll often be able to obtain the correct code after a couple of iterations.

Overconfidence

Despite the fact that LLMs can sometimes provide incorrect outputs, they have been trained to prioritize accuracy. This can make their output appear very convincing, even when it is incorrect. As a result, identifying hidden errors can be challenging because the model often cannot explicitly say “I need more information”, although ongoing research is actively exploring ways to address this limitation.

In this sense, using ChatGPT for generating small snippets of code to solve specific tasks, as I mentioned earlier when referring to Googling, can be safer. It is essential to ensure that you have a solid understanding of the code you receive which allow you to navigate potential pitfalls effectively.

Conclusion

In this article, we have explored methods that can assist you in debugging, not only with ChatGPT but also with yourself.

By isolating the problem and rewriting it in minimal lines of code, you are likely to gain insights into the underlying issue. Or let ChatGPT help you by providing complete information about what is happening, making assumptions, and experimenting.

You can also utilize ChatGPT for tasks such as writing, optimizing, or explaining code, as we discussed in the example of role prompting. The other code-related applications are endless and include creating dummy data, writing tests, generating documentation, and more.

But remember the limitations of LLMs, as they can introduce hidden problems. Due to their autoregressive nature, LLMs can be wrong while appearing confident, which may require asking more questions or running multiple iterations to select the best output.

Wishing you the best of luck on your learning journey!

Resources

Check out this article for a broader guide of making ChatGPT your personal coding mentor.

Here are my other articles about LLMs that may be useful to you. I have already covered:

• Estimating the Scale of Large Language Models: what are LLMs, how are they trained, and how much data and compute do they need;
• Best Practices for Prompt Engineering: how to apply prompt engineering techniques to interact with LLMs effectively and how to build local LLM-based applications with OpenAI API and Streamlit.

You can be also interested in:

• Free Learn Prompting course to gain a deeper understanding of prompting and various techniques associated with it;
• Recently released short courses by DeepLearning.AI to build applications with OpenAI API.

Thank you for reading!

• I hope these materials were useful to you. Follow me on Medium to get more articles like this.
• If you have any questions or comments, I will be glad to get any feedback. Ask me in the comments, or connect via LinkedIn or Twitter.
• To support me as a writer and to get access to thousands of other Medium articles, get Medium membership using my referral link (no extra charge for you).

Using ChatGPT for Efficient Debugging was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.

Prompt engineering refers to the process of creating instructions called prompts for Large Language Models (LLMs), such as OpenAI’s ChatGPT. With the immense potential of LLMs to solve a wide range of tasks, leveraging prompt engineering can empower us to save significant time and facilitate the development of impressive applications. It holds the key to unleashing the full capabilities of these huge models, transforming how we interact and benefit from them.

In this article, I tried to summarize the best practices of prompt engineering to help you build LLM-based applications faster. While the field is developing very rapidly, the following “time-tested” :) techniques tend to work well and allow you to achieve fantastic results. In particular, we will cover:

• The concept of iterative prompt development, using separators and structural output;
• Chain-of-Thoughts reasoning;
• Few-shot learning.

Together with intuitive explanations, I’ll share both hands-on examples and resources for future investigation.

Then we’ll explore how you can build a simple LLM-based application for local use using OpenAI API for free. We will use Python to describe the logic and Streamlit library to build the web interface.

Let’s start!

Best Practices for Prompt Engineering

In this article, I will interact with ChatGPT through both the web interface and API. The gpt-3.5-turbo model I’ll use is the one behind ChatGPT, so you can experiment with your prompts right in the browser.

One important thing to note here is that ChatGPT is not only LLM; as you probably know, it’s also a SFT (Supervised Fine-Tuning) model that was further finetuned with Reinforcement Learning from Human Feedback (RLHF). While many developers currently utilize OpenAI’s models for experimental projects and personal exploration, there are other models that may be more appropriate for deployment in production settings within large corporations due to privacy and other reasons.

If you want to know why base models (such as GPT-3, Chinchilla, LLaMA) do not function in the same way as fine-tuned and RLHF-trained assistants (e.g., ChatGPT, Koala, Alpaca), there is a talk by Andrej Karpathy about training and using GPT-like models. I highly recommend checking it out for a deeper understanding, and it is just 40 minutes long. For a summary, take a look at this Twitter thread.

Now let’s dive into best practices for prompting instruction-tuned LLMs!

Iterative Prompt Development

Just as any machine learning model is built through an iterative process, effective prompts are also constructed through a similar iterative approach. Even the most talented developer may not create the perfect prompt on their first attempt, so be prepared for the reality that you may need dozens of attempts to achieve the desired goal.

It’s always better to understand things through examples. Let’s start building a system to extract information from a job description. The sample job description we’ll be using in the examples is the following Machine Learning Engineer job posting from LinkedIn.

The initial prompt can be as simple as asking the model to extract specific information. Additionally, I’ll use delimiters (you can learn more about it in the ChatGPT Prompt Engineering for Developers course I will mention later). While it’s unlikely that a local application would be susceptible to prompt injection attacks, it’s just good practice.

Hmm, that’s not very helpful. We need to be more specific about what we want from the model. Let’s ask it to extract the job title, company name, key skills required, and a summarized job description.

Remember that this is just an example, and you can design your prompts to extract as much information as you want: degree, years of experience required, location, etc.

Looking better! To make the output more compact and concise, let’s ask the model to output skills as a list and a more brief summary of the job description.

It’s a significant improvement from our initial attempt, isn’t it? And we accomplish this in just two iterations! So, don’t lose hope when things don’t go smoothly initially. Keep guiding the model, experiment, and you’ll certainly achieve success.

Asking for a Structural Output

The second point I would like to discuss is asking the model to output results in some expected structural format. While it may not be as critical for interacting with LLM through the web interface (e.g. as we do with ChatGPT), it becomes extremely useful for LLM-based applications since the process of parsing the results is much easier.

One common practice is to use formats like JSON or XML and define specific keys to organize output data. Let’s modify our prompt to show the model expected JSON structure.

Such output is much easier to parse and process in the following logic of your application.

It’s worth saying a few words about the development of this direction. Some tools are aiming to strictly fit the model’s output into a given format, which can be extremely useful for some tasks. Just look at the examples below!

One of the possible applications is the generation of a large amount of content in a specific format (e.g. game characters information using guidance).

Languages like LMQL bring a programming-like approach to prompting language models. As these tools continue to evolve and improve, they have the potential to revolutionize how we interact with LLMs, resulting in more accurate and structured responses.

Chain-of-Thought Reasoning

Chain-of-Thought (CoT) reasoning was discovered to be very helpful for tasks that require… well, reasoning. So if you have the opportunity to solve the task by breaking it into multiple simpler steps that can be a great approach for LLM.

Take a look at the example from the original paper. By splitting the problem into smaller steps and providing explicit instructions, we can assist the model in producing correct outputs.

Interestingly, it emerges later that appending a straightforward and magic “let’s think step by step” at the end of a prompt can improve results — this technique is known as zero-shot CoT. So, compose prompts that allow the model to “think out loud” since it does not have any other ability to express thoughts other than generate tokens.

The best zero-shot CoT prompt so far is “Let’s work this out in a step by step way to be sure we have the right answer”.

More sophisticated approaches to solving even more complex tasks are now being actively developed. While they significantly outperform in some scenarios, their practical usage remains somewhat limited. I will mention two such techniques: self-consistency and the Tree of Thoughts.

The authors of the self-consistency paper offered the following approach. Instead of just relying on the initial model output, they suggested sampling multiple times and aggregating the results through majority voting. By relying on both intuition and the success of ensembles in classical machine learning, this technique enhances the model’s robustness.

You can also apply self-consistency without implementing the aggregation step. For tasks with short outputs ask the model to suggest several options and choose the best one.

Tree of Thoughts (ToT) takes this concept a stride further. It puts forward the idea of applying tree-search algorithms for the model’s “reasoning thoughts”, essentially backtracking when it stumbles upon poor assumptions.

If you are interested, check out Yannic Kilcher’s video with a ToT paper review.

For our particular scenario, utilizing Chain-of-Thought reasoning is not necessary, yet we can prompt the model to tackle the summarization task in two phases. Initially, it can condense the entire job description, and then summarize the derived summary with a focus on job responsibilities.

In this particular example, the results did not show significant changes, but this approach works very well for most tasks.

Few-shot Learning

The last technique we will cover is called few-shot learning, also known as in-context learning. It’s as simple as incorporating several examples into your prompt to provide the model with a clearer picture of your task.

These examples should not only be relevant to your task but also diverse to encapsulate the variety in your data. “Labeling” data for few-shot learning might be a bit more challenging when you’re using CoT, particularly if your pipeline has many steps or your inputs are long. However, typically, the results make it worth the effort. Also, keep in mind that labeling a few examples is far less expensive than labeling an entire training/testing set as in traditional ML model development.

If we add an example to our prompt, it will understand the requirements even better. For instance, if we demonstrate that we’d prefer the final summary in bullet-point format, the model will mirror our template.

This prompt is quite overwhelming, but don’t be afraid: it is just a previous prompt (v5) and one labeled example with another job description in the For example: 'input description' -> 'output JSON' format.

Summarizing Best Practices

To summarize the best practices for prompt engineering, consider the following:

• Don’t be afraid to experiment. Try different approaches and iterate gradually, correcting the model and taking small steps at a time;
• Use separators in input (e.g. <>) and ask for a structured output (e.g. JSON);
• Provide a list of actions to complete the task. Whenever feasible, offer the model a set of actions and let it output its “internal thoughts”;
• In case of short outputs ask for multiple suggestions;
• Provide examples. If possible, show the model several diverse examples that represent your data with the desired output.

I would say that this framework offers a sufficient basis for automating a wide range of day-to-day tasks, like information extraction, summarization, text generation such as emails, etc. However, in a production environment, it is still possible to further optimize models by fine-tuning them on specific datasets to further enhance performance. Additionally, there is rapid development in the plugins and agents, but that’s a whole different story altogether.

Prompt Engineering Course by DeepLearning.AI and OpenAI

Along with the earlier-mentioned talk by Andrej Karpathy, this blog post draws its inspiration from the ChatGPT Prompt Engineering for Developers course by DeepLearning.AI and OpenAI. It’s absolutely free, takes just a couple of hours to complete, and, my personal favorite, it enables you to experiment with the OpenAI API without even signing up!

That’s a great playground for experimenting, so definitely check it out.

Building the LLM-based Application with OpenAI API and Streamlit

Wow, we covered quite a lot of information! Now, let’s move forward and start building the application using the knowledge we have gained.

Generating OpenAI Key

To get started, you’ll need to register an OpenAI account and create your API key. OpenAI currently offers $5 of free credit for 3 months to every individual. Follow the introduction to the OpenAI API page to register your account and generate your API key.

Once you have a key, create an OPENAI_API_KEY environment variable to access it in the code with os.getenv('OPENAI_API_KEY').

Estimating the Costs with Tokenizer Playground

At this stage, you might be curious about how much you can do with just a free trial and what options are available after the initial three months. It’s a pretty good question to ask, especially when you consider that LLMs cost millions of dollars!

Of course, these millions are about training. It turns out that the inference requests are quite affordable. While GPT-4 may be perceived as expensive (although the price is likely to decrease), gpt-3.5-turbo (the model behind default ChatGPT) is still sufficient for the majority of tasks. In fact, OpenAI has done an incredible engineering job, given how inexpensive and fast these models are now, considering their original size in billions of parameters.

The gpt-3.5-turbo model comes at a cost of $0.002 per 1,000 tokens.

But how much is it? Let’s see. First, we need to know what is a token. In simple terms, a token refers to a part of a word. In the context of the English language, you can expect around 14 tokens for every 10 words.

To get a more accurate estimation of the number of tokens for your specific task and prompt, the best approach is to give it a try! Luckily, OpenAI provides a tokenizer playground that can help you with this.

Side note: Tokenization for Different Languages

Due to the widespread use of English on the Internet, this language benefits from the most optimal tokenization. As highlighted in the “All languages are not tokenized equal” blog post, tokenization is not a uniform process across languages, and certain languages may require a greater number of tokens for representation. Keep this in mind if you want to build an application that involves prompts in multiple languages, e.g. for translation.

To illustrate this point, let’s take a look at the tokenization of pangrams in different languages. In this toy example, English required 9 tokens, French — 12, Bulgarian — 59, Japanese — 72, and Russian — 73.

Cost vs Performance

As you may have noticed, prompts can become quite lengthy, especially when incorporating examples. By increasing the length of the prompt, we potentially enhance the quality, but the cost grows at the same time as we use more tokens.

Our latest prompt (v6) consists of approximately 1.5k tokens.

Considering that the output length is typically the same range as the input length, we can estimate an average of around 3k tokens per request (input tokens + output tokens). By multiplying this number by the initial cost, we find that each request is about $0.006 or 0.6 cents, which is quite affordable.

Even if we consider a slightly higher cost of 1 cent per request (equivalent to roughly 5k tokens), you would still be able to make 100 requests for just $1. Additionally, OpenAI offers the flexibility to set both soft and hard limits. With soft limits, you receive notifications when you approach your defined limit, while hard limits restrict you from exceeding the specified threshold.

For local use of your LLM application, you can comfortably configure a hard limit of $1 per month, ensuring that you remain within budget while enjoying the benefits of the model.

Streamlit App Template

Now, let’s build a web interface to interact with the model programmatically eliminating the need to manually copy prompts each time. We will do this with Streamlit.

Streamlit is a Python library that allows you to create simple web interfaces without the need for HTML, CSS, and JavaScript. It is beginner-friendly and enables the creation of browser-based applications using minimal Python knowledge. Let’s now create a simple template for our LLM-based application.

Firstly, we need the logic that will handle the communication with the OpenAI API. In the example below, I consider generate_prompt()function to be defined and return the prompt for a given input text (e.g. similar to what you saw before).

And that’s it! Know more about different parameters in OpenAI’s documentation, but things work well just out of the box.

Having this code, we can design a simple web app. We need a field to enter some text, a button to process it, and a couple of output widgets. I prefer to have access to both the full model prompt and output for debugging and exploring reasons.

The code for the entire application will look something like this and can be found in this GitHub repository. I have added a placeholder function called toy_ask_chatgpt() since sharing the OpenAI key is not a good idea. Currently, this application simply copies the prompt into the output.

Without defining functions and placeholders, it is only about 50 lines of code!

And thanks to a recent update in Streamlit it now allows embed it right in this article! So you should be able to see it right below.

Now you see how easy it is. If you wish, you can deploy your app with Streamlit Cloud. But be careful, since every request costs you money if you put your API key there!

Conclusion

In this blog post, I listed several best practices for prompt engineering. We discussed iterative prompt development, the use of separators, requesting structural output, Chain-of-Thought reasoning, and few-shot learning. I also provided you with a template to build a simple web app using Streamlit in under 100 lines of code. Now, it’s your turn to come up with an exciting project idea and turn it into reality!

It’s truly amazing how modern tools allow us to create complex applications in just a few hours. Even without extensive programming knowledge, proficiency in Python, or a deep understanding of machine learning, you can quickly build something useful and automate some tasks.

Don’t hesitate to ask me questions if you’re a beginner and want to create a similar project. I’ll be more than happy to assist you and respond as soon as possible. Best of luck with your projects!

Resources

Here are my other articles about LLMs that may be useful to you. I have already covered:

• Estimating the Scale of Large Language Models: what are LLMs, how are they trained, and how much data and compute do they need;
• Using ChatGPT for Debugging: how to use LLMs for debugging and code generation.

You can be also interested in:

• Free Learn Prompting course to gain a deeper understanding of prompting and various techniques associated with it;
• Recently released short courses by DeepLearning.AI to build applications with OpenAI API

Thank you for reading!

• I hope these materials were useful to you. Follow me on Medium to get more articles like this.
• If you have any questions or comments, I will be glad to get any feedback. Ask me in the comments, or connect via LinkedIn or Twitter.
• To support me as a writer and to get access to thousands of other Medium articles, get Medium membership using my referral link (no extra charge for you).

Summarising Best Practices for Prompt Engineering was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.

Thanks to Regan Yue, you can read the Chinese version of this article at mp.weixin.qq.com, juejin.cn, segmentfault.com and xie.infoq.cn!

In the recent past, machine learning was considered a complex, niche technology that only a select few could comprehend. However, as ML applications become more powerful, the public’s interest has surged, leading to a vast amount of content surrounding Artificial Intelligence. The culmination of this happened in November 2022, when we saw ChatGPT, and continued in March 2023 with the release of GPT-4, when even the most skeptical person was surprised at what modern neural networks can do.

While some of this content is undoubtedly valuable, a significant portion of it perpetuates fear and misinformation, such as the spread of the idea that robots will replace all human jobs or discovering secret ways to make huge sums of money on neural networks. As a result, it has become increasingly important to dispel misconceptions about machine learning and large language models and provide informative content to help people understand these technologies better.

This article aims to discuss the crucial aspect of modern machine learning that is often overlooked or misunderstood — the cost of training large language models. At the same time, we will briefly take a look at what LLM is and some possible techniques to optimize its inference. By providing comprehensive examples, I hope to convince you that these technologies do not come out of the air. By getting an idea about the scale of the data and the underlying calculations you will better understand these powerful tools.

Mostly, I will rely on the recent LLaMA paper by Meta AI because of its clarity in the sense of the amount of data and compute the team used to train these models. The post will be divided into the following sections:

1. First, we’ll briefly look at what modern LLMs are;
2. Then, we discuss how much it costs to train such models;
3. In the end, we briefly consider some popular techniques to optimize language models for inference.

Stay tuned as we delve deeper into the world of large language models and you will see that everything is very simple and very complicated at the same time.

Introduction to Large Language Models

Before we explore the costs associated with training Large Language Models (LLMs), let’s first briefly define what a language model is.

In simple terms, a language model is a type of machine learning algorithm designed to understand or generate human language. Recently, exactly generative models have become more and more popular — the GPT model family developed by OpenAI: ChatGPT, GPT-4, etc (stands for Generative Pre-trained Transformer, honoring the Transformer architecture on which it is based).

Less popular, but still important examples include GPT-3 (175B), BLOOM (176B), Gopher (280B), Chinchilla (70B), and LLaMA (65B), where B refers to billions of parameters, although many of these models also have smaller versions.

Nothing is known about the number of ChatGPT and especially GPT-4 parameters, but it looks like these are about the same numbers.

These models are “trained” using vast amounts of text data, enabling them to learn the complex patterns and structures of natural language. However, the task they solve during training is very simple: they just predict the next word (or token) in a sequence.

You may have heard such a model called autoregressive, which means it uses its past outputs as input for future predictions and generate output step by step. This can be seen, among other things, in the example of ChatGPT:

You can notice that the model generates the answer gradually and in chunks that are sometimes less than one word. These chunks are called tokens and they are very useful in NLP, although not so important for us now.

At each time step, the model concatenates the previous output to the current input and keeps generating. It does so until it reaches the special End of Sequence (EOS) token. Omitting the prompt task and taking words as tokens for simplicity, the process can be illustrated as follows.

This simple mechanism together with a huge amount of data (more than any person could read in several lifetimes) allows the model to generate coherent and contextually appropriate text, mimicking human-like writing.

Note, that here we are talking about generative models only. Why if there are other model families?

The reason is quite simple — the text generation task is one of the most difficult to solve and at the same time one of the most impressive. ChatGPT gained 1 million users in just 5 days — faster than any other application before, and continues in the same spirit.

So-called encoders (BERT model family) can be much less exciting, but they can also solve various problems with human-level performance and help you with tasks like text classification or Named Entity Recognition (NER).

I will not provide particular examples of what LLMs can do — the Internet is already full of them. The best way to get an idea is to try ChatGPT yourself, but you can also find plenty of exciting resources like the Awesome ChatGPT prompts repo. Despite their impressive capabilities, current large language models have some limitations. The most popular and significant of them include:

1. Bias and staticity: Since LLMs are trained on data from various sources, they inadvertently learn and reproduce biases present in those sources. They are also static in the sense that they cannot adapt to new data or update their knowledge in real time without re-training.
2. Comprehension and disinformation: Although LLMs can generate human-like text, they may not always fully understand the context of the input. Also, the autoregressive way of generating output text does not prohibit the model from generating lies or nonsense.
3. Resource-intensive: Training LLMs requires substantial computing resources, which translates to high costs and energy consumption. This factor can limit the accessibility of LLMs for smaller organizations or individual researchers.

These and other drawbacks are active topics for the research community. It is worthwhile to note that the field is growing so fast that it is impossible to predict what limitations will be overcome in just a few months — but without a doubt, new ones will arise.

One possible example is the fact that earlier models simply grew in the number of parameters, but now it is considered that it is better to train smaller models for a longer time and give them more data. This reduces the model size and the cost of the model’s further use during inference.

In that way, the LLaMA release freed the hands of enthusiasts and these models were already run locally on computers, Raspberry Pi, and even phones!

Having a big picture of what LLM is, let’s move on to the main section of this article — estimating the cost of training large language models.

Estimating the Cost of Machine Learning Models in General and LLMs in Particular

To estimate the cost of training large language models, it is essential to consider three key factors that any machine learning algorithm consists of:

• Data,
• Compute resources, and
• Architecture (or the algorithm itself).

Let’s delve deeper into each of these aspects to better understand their impact on training costs.

Data

LLMs require massive amounts of data to learn the patterns and structures of natural language. Estimating the cost of data can be challenging since companies often use data accumulated over time through their business operations together with open-sourced datasets.

Additionally, data needs to be cleaned, labeled, organized, and stored efficiently, considering the scale of LLMs. Data management and processing costs can add up quickly, especially when factoring in the infrastructure, tools, and data engineers required for these tasks.

To make a particular example, it is known that LLaMA used a training dataset containing 1.4 trillion tokens with a total size of 4.6 terabytes!

Smaller models (7B and 13B) were trained on 1T tokens, while larger ones (33B and 65B) used the full dataset of 1.4T tokens.

I think now you understand that no one is overstating when calling these datasets huge and why it wasn’t technically possible ten years ago. But things are even more interesting with computing resources.

Compute

The actual training process accounts for a significant portion of the LLM budget. Training large language models is resource-intensive and is done on powerful Graphics Processing Units (GPUs), due to significant parallel processing capabilities. NVIDIA releases new GPUs every year, the cost of which hits hundreds of thousands of dollars.

The cost of cloud computing services for training these models can be huge and reach several million dollars, especially considering iterating through various configurations.

Returning to the LLaMA paper, the authors report that they train the biggest 65B model for 21 days on two thousand GPUs with 80 GB of RAM each.

NVIDIA A100 GPU authors used is a popular choice for modern neural network training. Google Could Platform offers such GPUs for $3.93 per hour.

So let’s do some quick calculations:

2048 GPUs x $3.93 GPU per hour x 24 hours x 21 days =

4.05 million dollars

Four million dollars is a budget that not every researcher can afford, huh? And it is a single run! To give you another example, this article estimates the cost of training GPT-3, and the authors got 355 GPU-years and 4.6 million dollars.

You may have heard that “neural networks train very quickly on GPU”, but no one says relative to what.

They’re really training fast taking into account the enormous amount of calculations, and without these GPUs, they would have been training for decades. So yeah, 21 days is pretty fast for LLMs.

Architecture (and Infrastructure)

The development of state-of-the-art LLMs also depends on the work of skilled researchers and engineers to develop the architecture and configure the training process properly. The architecture is the foundation of the model, dictating how it learns and generates text.

Expertise in various computer science areas is required for designing, implementing, and controlling these architectures. Engineers and researchers responsible for publishing and delivering cutting-edge results can command salaries reaching hundreds of thousands of dollars. It is worth noting that the skill set required for LLM development may differ significantly from the skill set of a “classic” machine learning engineer.

I think now you do not doubt that training LLMs is a very hard and resource-intensive engineering problem.

Now let’s briefly discuss some methods for making the process of LLM inference more efficient and cost-effective.

Optimizing Language Models for Inference

Do we actually need optimization?

Inference refers to the process of using a trained language model to generate predictions or responses, usually as an API or web service. Given the resource-intensive nature of LLMs, it is essential to optimize them for efficient inference.

For example, GPT-3 model has 175 billion parameters, which is 700 GB of float32 numbers. Approximately the same amount of memory will be taken up by activations, and remember that we are talking about RAM.

To serve predictions without any optimization technique, we will need 16 A100 GPUs with 80 GB of memory each!

Several popular techniques can help reduce memory requirements and model latency, including model parallelism, quantization, and others.

Model Parallelism

Parallelism is a technique that distributes the computation of a single model across multiple GPUs and can be used both during training and inference.

Splitting the model’s layers or parameters across multiple devices can dramatically improve the overall inference speed and is very often used in practice.

Quantization

Quantization involves reducing the precision of the model’s numerical values (such as weights). By converting floating-point numbers to lower-precision integers, quantization can result in significant memory savings and faster computation without a substantial loss in model performance.

The simple idea that arises quite quickly is to use float16 numbers instead of float32 and reduce the amount of memory by half. It turns out that it is possible to convert model weights even to int8 almost without accuracy loss due to the fact that they are located close to each other on the number line.

Other Techniques

Finding ways to optimize LLMs is an active area of research, and other techniques include:

• Knowledge distillation — training a smaller student model to mimic the behavior of a larger teacher;
• Parameter pruning — removing redundant or less important parameters from the model to reduce its size and computational requirements;
• And using frameworks like ORT (ONNX Runtime) to optimize calculation graphs with techniques like operator fusion and constant folding.

Overall, optimizing large language models for inference is a critical aspect of their deployment. By applying various optimization techniques, developers can ensure that their LLMs are not only powerful and accurate but also cost-effective and scalable.

Why did OpenAI Open Access to ChatGPT?

After all the above, one might wonder why OpenAI decided to open access to ChatGPT, given the high costs associated with training and inference. While we cannot be certain of the company’s exact motivations, we can analyze the benefits and potential strategic reasons behind this decision.

First and foremost, OpenAI has gained significant popularity by making state-of-the-art LLMs more accessible to the broader public (see AI revolution is more of a UX revolution). By demonstrating the practical applications of large language models, the company has captured the attention of investors, customers, and the tech community at large. Moreover, this allowed OpenAI to collect huge amounts of feedback and data to improve their models.

Secondly, OpenAI’s mission revolves around the creation and advancement of AI. By opening access to ChatGPT, the company is arguably moving closer to fulfilling its mission and preparing society for unavoidable changes. Providing access to powerful AI tools encourages innovation, driving the field of AI research forward. This progress can lead to the development of more efficient models, more extensive applications, and novel solutions to various challenges. It’s worth noting here that the architecture of ChatGPT and GPT-4 is closed, but that’s another discussion.

While the costs associated with training and maintaining large language models are undoubtedly significant, the benefits and strategic advantages that come with opening access to these tools can outweigh the expenses for some organizations. In the case of OpenAI, opening access to ChatGPT has not only increased their popularity and proved to be a leader in the AI field, but also allowed them to collect more data to train more powerful models. This strategy has allowed them to advance their mission and contribute (in some sense) to the broader development of AI and LLM technologies.

Conclusion

As we have seen, the cost of training large language models is influenced by various factors, including not only expensive computing resources but also big data management and the expertise required to develop cutting-edge architectures.

Modern LLMs have billions of parameters, are trained on trillions of tokens, and cost millions of dollars.

I hope you now better understand the scale of training and inferencing large language models, as well as their limitations and pitfalls.

The field of NLP has been experiencing its ImageNet moment for several years, and now it’s the turn of generative models. The widespread application and adoption of generative language models have the potential to revolutionize various industries and aspects of our lives. While it is difficult to predict exactly how these changes will unfold, we can be certain that LLMs will have some impact on the world.

Personally, I like the recent tendency of training “smarter”, not just “larger” models. By exploring more elegant ways to develop and deploy LLMs, we can push the boundaries of AI and NLP, opening the door to innovative solutions and a brighter future for the field.

Resources

Here are my other articles about LLMs that may be useful to you. I have already covered:

• Best Practices for Prompt Engineering: how to apply prompt engineering techniques to interact with LLMs effectively and how to build local LLM-based applications with OpenAI API and Streamlit;
• Using ChatGPT for Debugging: how to use LLMs for debugging and code generation.

If you become interested in LLMs and want to learn more about them, here are some resources that can help you with that:

• Illustrated Transformer is a great introduction to Transformer architecture which started the NLP big bang by Jay Alammar;
• How GPT-3 Works — Visualizations and Animations is a visualization of an autoregressive decoding process by the same author;
• GPT in 60 Lines of NumPy is a great article by Jay Mody where the author builds his own simple GPT;
• Let’s build GPT: from scratch, in code, spelled out is an awesome video by famous Andrej Karpathy, who worked on Tesla Autopilot as Sr. Director of AI at Tesla;
• To get a more in-depth idea of the field, check out the Awesome-LLM GitHub repo for a more detailed list of resources. Take a look at Chinchilla and LLaMA as one of the most influential papers of recent times.

Thank you for reading!

• I hope these materials were useful to you. Follow me on Medium to get more articles like this.
• If you have any questions or comments, I will be glad to get any feedback. Ask me in the comments, or connect via LinkedIn or Twitter.
• To support me as a writer and to get access to thousands of other Medium articles, get Medium membership using my referral link (no extra charge for you).

Behind the Millions: Estimating the Scale of Large Language Models was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.

I started blogging on Medium and publishing articles on Towards Data Science about one year ago. In this article, I would like to summarize my writer’s achievements so far and tell you how much money I have earned through Medium Partner Program during this time.

You might consider Medium as your data science/machine learning blog platform, or perhaps you are thinking about monetizing your writing talent, I hope my story will be interesting for you.

TL;DR

Respecting your time, the final number is about 830 USD without any investments. On average, this is about $70 per month.

For those who stay, we will explore not only earnings. I will show you detailed statistics for my articles and followers, and we will find out how to estimate the earnings of any Medium article! Using machine learning, of course. :)

Before We Start

My blogging journey actually starts on September 2021, not January 2022. So why I am reporting to you from January 2022 to January 2023?

As you probably know, the Medium Partner Program requires you to have at least one hundred followers to earn money from your articles.

Note: if you wish to earn money using the Medium Partner Progam, first carefully read its terms and conditions, including the list of countries that support it.

These 5 months (from September 2021 to January 2022) actually took me to gain my first 100 followers. The reads my article collected during this time are thus non-monetized (free).

I also didn't invest any money to promote my articles and my Medium profile. All I did was publishing articles on LinkedIn and Twitter, as well as on a couple of public data science communities, like this or this. There are two main reasons why I haven’t put any money into my writing yet:

• Curiosity— to understand how far I can go from scratch;
• Unnecessity— I don’t actually need this, at least for now.

Don’t get me wrong, but I am not primarily a writer and I am not as productive as one of them. I am just sharing my projects and thoughts that I think can be useful for other people. One day my position may change, but for now, it is what it is.

Initial Dataset

During these 17 months (from September 2021 to January 2023), I have written 24 articles. This is 1.4 articles per month on average, although the distribution is not so uniform.

Using the Medium platform, I collect the following article data for analysis:

To avoid displaying the full title of the article on the plots, which would look inconvenient, I came up with shortcodes. If you are interested in the characteristics of a certain article, you can get its full name from the table below and find it in my profile.

In addition to the titles, the dataset includes the following columns:

• date — article publication date;
• category — the article category: cheat sheet, tutorial, review, or project;
• n_views — number of users who started reading the article (thousands);
• external_views — the percentage of views from non-Medium sources (social media, search results, etc.);
• n_reads — number of users who reached the end of the article (thousands);
• n_fans — number of unique users who clapped for the article (can be interpreted as the number of likes);
• member_reading_time — the total amount of time Medium members spent reading the article (minutes);
• earnings — the total amount of money the story earned from the moment of publication to today (US dollars).

It will be more accurate to say that earnings are collected from the moment of metering your story to today. Since some of my reads are free (for the time when I didn’t have a hundred followers), we cannot observe a direct correlation between earnings and, for example, the number of views. But we’ll talk about this a little later.

By the way, after gaining access to the partner program, of course, you can meter those stories that were written before receiving it.

Let’s finally build a couple of visualizations.

Values per Article

Number of Views and Reads

First, let’s look at the views and reads. Here and further, I sort articles by a certain value and calculate how many of the most successful articles take 80% and 95% of the value.

For views and reads, these numbers are 10 and 17 respectively (out of 24). The dashed line on the plot represents a simple average.

As you can see, the spread between the maximum and minimum values is very large (about x100 times!) — 48k views and 6k reads for the overfitting principles vs only 300 views and about 100 reads for the movie color analysis.

The same information can be shown more compactly using a single stacked bar plot:

Number of Fans, Member Reading Time, and Article Earnings

As we will see later, the number of fans and member reading time are better correlated to final earnings than views and reads.

80 and 95 percent of fans were collected by 12 and 19 articles. For the member reading time and earnings, these numbers are 11 and 17 respectively — that is, about the same as for views/reads.

The least popular article for these indicators is the same — movie color analysis with 3 fans and 26 minutes of member reading time. But the favorite has changed — now it is the top ML resources with 105 fans (x35) and 63 hours of member reading time (x150).

As well as for member reading time, 80 and 95 percent of earnings were collected by 11 and 17 articles. The most and least popular articles remain the same: $233.82 for top ML resources and $1.36 for movie color analysis with an average value of about $35 per article.

Values per Category

To dive in a little deeper, I decided to cluster the articles into categories, using the following:

• Cheat sheet — 6 articles: a short summary on a specific topic, which can be used to refresh your knowledge or as a little learning path;
• Tutorial — 7 articles: articles where I explain some concepts in my own words, usually (but not always) reinforcing them with code. Also, here I’m just telling things— e.g. about my resume writing experience or interesting machine learning applications;
• Review — 3 articles: my opinions, which can be useful to others, usually about learning resources: courses and exams;
• Project — 8 articles: various project descriptions. They require the most time due to hatching an idea and writing more code and documentation.

Let’s look at the same indicators, grouping the data by category. I will report the average because the sum does not make much sense: the number of articles in the various categories is different. The dashed line indicates the average among the four categories. Under the average values graph, I also bring a violin plot to get an idea of the distribution form.

Number of Views and Reads

Tutorials have the highest number of views on average, and projects have the least.

The situation remains the same for reads, but the value spread (deviation from the mean) here is much smaller. We can say that on average, readers read my articles about the same way.

Number of Fans, Member Reading Time, and Article Earnings

The ranking of categories in relation to the number of fans, reading time, and earnings is the same: tutorial (the most successful), cheat sheet, review, and project (the least successful).

The least popular category gets only 15 fans, 6 hours of reading time, and $15 on average, while the most popular one gets 40 fans (x2.5), 20 hours of reading time (x3), and $60 (x4).

It turns out that by all the values, the Tutorial is the most popular category, and the Project is the most unpopular. Meanwhile, projects require much more time for creating and describing.

If we imagine such indicator as the ratio of time spent to expected profit tutorials will clearly win, and not only because they are easier to get ideas and write (although not always). Given the same time, you can just write more of them, and some will definitely attract the attention of your audience.

I do not attempt to determine which topics are most profitable to write on — I am not writing for this. After all, I have already seen that my projects are not so popular: there is no need to plot any charts for this. But an approach that maximizes profit seems to minimize creativity — write a lot of short articles on various topics and some will clearly bring you profit.

Important disclaimer

Indeed, the tutorials show the best averages, but look at how stretched their distribution is. This is because the three most successful articles (top ML resources, overfitting and graphs) are tutorials. At the same time, they are far of the average values and can even be considered as outliers or anomailes.

It’s a toy analysis, but estimating a distribution based on 10 data points is clearly not a good idea.

Predicting Earnings

What if we train the model to predict earnings based on the values discussed above? After all, we have the dataset. Yes, pretty small, extremely small, but we can at least see what happens. Thus, we will also be able to visually monitor the degree of correlation between the article's stats and the final earnings.

Let’s take a linear regression, assuming that the relationship will be simple and linear. I will also use only one value at a time and remove the bias suggesting that there is no initial “boost”. Thus we get the simplest equation from the first chapter of any machine learning book: y=a*x. In our case, y is earnings, x is feature (n_views, n_fans, etc.) and a is the only learnable parameter.

Filtering Data

The first thing to do is to correct the data. As I said earlier, some reads were free, and for the purity of the experiment, we need to minimize their impact. I will use those articles that began to earn instantly, that is, articles that were written after receiving the partner program in early January 2022.

I use the entire dataset for training and do not put aside the test set because:

• there is too little data and a test set containing only several data points will give a biased estimate, and more importantly
• we can evaluate the relationship visually since we use only one variable.

Don’t forget that splitting the test set and putting it aside at the very beginning is the first thing you should do in a real task.

Predicting Earnings Based on Views/Reads

It turns out that the number of views and reads is not a good feature for predicting the earnings of an article. Although it may seem that the Root Mean Square Error is not so bad, it is visually clear that we are far from a simple relationship and there is a lot of noise in the data.

This can be explained by the fact that most of the views on my articles (about 75–80% on average) are from external sources (social media, Internet search, etc.) and thus most likely from unregistered users.

Predicting Earnings Based on Fans/Member Reading Time

Intuitively, member reading time — the time Medium members spent reading the article — is the best feature for predicting the article’s earnings.

And it really is! Based on my data, I can expect 6 cents for every minute of Medium member reading, which is $3.6 per hour.

Obviously, this is the best possible feature, but it is private and available only to the author. Can we use some public article characteristics to evaluate earnings? It turns out — yes, and this is the number of fans.

The trained model says that we can expect $2.2 for every fan. Look at the chart — it’s almost as accurate as the previous one! The intuition can be explained by at least two facts:

• First: only registered users can be fans;
• And second: if a person clapped, then he liked the article, which means he spent some time reading it. If it’s a Medium member, then you’ve earned some part of his membership.

The interesting fact is that the number of fans is publicly available and now you know how to estimate the earnings of any Medium article, provided that it has been metered since the beginning of the publication. Just multiply the number of fans by 2.2 and get approximate earnings in USD!

Note that the number of claps and fans are different things. The number of fans is the number of unique users who have clapped for an article, regardless of the number of claps.

In the example below, Igor clapped 50 times but increased the number of fans just by one. But it looks like you really enjoyed the article, Igor!

Well. Do not forget that the final earnings consist not only of the article's earnings but also of your paid subscribers — referred members. Let’s take a look at them and at subscribers' stats in general.

Followers Monthly Growth

Here are my followers' stats for these 17 months.

Basically, on average they grow by 30 followers per month, that is, one person subscribes to me per day. Also on average, I get 2 email subscribers in a month and about 0.6 referred members, although the latter have been unsubscribing recently. Each of the referred members brings me about 2 USD per month. Thanks!

In total, I earned $302 on paid subscribers, which is $18 per month on average.

Total Number of Earnings

$880 article earnings + $300 followers earnings = $1180 in total.

But how much did I actually get?

Here is the screenshot of my Stripe account payments:

The sum of all Stripe payments is 830.42 USD, which is about 1080 CAD.

This is the final number of my earnings so far — for 17 months of my writing career, 5 of which were unmonetized.

Top 10 of My Most Popular Articles

Analyzing the indicators of articles, I constantly came across the same titles among the best. And although 80% of the result was achieved not by 20% of the articles, there are clearly favorites.

Here are the top ten of my most successful articles (with fans and earnings in brackets).

Projects

• Automatic Video Editing using Python (31, $30) and Speech Recognition with Timestamps (17, $30)
• How to create a real-time Face Detector (43, $33)

Reviews

• What you need to know before taking the Machine Learning course by Stanford (38, $38)

Cheat sheets

• TensorFlow/Keras cheat sheet (23, $54)
• Unsupervised Learning algorithms cheat sheet (21, $61)
• Clustering cheat sheet (21, $68)

Tutorials

• Overfitting and Underfitting Principles (82, $50)
• Graphs with Python: Overview and Best Libraries (47, $94)
• Top 7 Machine Learning resources I wish I knew earlier (105, $234)

Conclusion

Here’s what I’ve got: 24 articles, 200k views, 12 days of member reading time, 500 followers, and $800 earnings. It’s not a lot, but it’s not a little. As they say, it ain’t much, but it’s honest work.

I would like to take the opportunity to say thank you to all my readers and followers. All these successes would not have been possible without you.

I could have done better. But I don’t want to force myself to release an article a week, copy other people’s posts, or talk about things that can be found in a few seconds once again. I will write about what I think is interesting and useful for others, and I am honest with myself and with you.

In the end, this article is about my writing journey, and not about anyone else. But I can share this data with you and I believe it is necessary to do so. Maybe if you start writing next week in a year you will have five hundred subscribers. Maybe twenty hundred. Maybe just one. But it will be your subscribers.

Keep learning!

Thank you for reading!

• I hope these materials were useful to you. Follow me on Medium to get more articles like this.
• If you have any questions or comments, I will be glad to get any feedback. Ask me in the comments, or connect via LinkedIn or Twitter.
• To support me as a writer and to get access to thousands of other Medium articles, get Medium membership using my referral link (no extra charge for you).

Thanks to Regan Yue, you can read the Chinese version of this article at mp.weixin.qq.com, juejin.cn, segmentfault.com and xie.infoq.cn!

The most effective way to understand new machine learning architecture (as well as any new technology in general) is to implement it from scratch. This is the best approach that helps you understand the implementation down to the smallest details, although it is very complex, time-consuming, and sometimes just impossible. For example, if you do not have similar computing resources or data you will not be able to make sure that there is no hidden bug in your solution.

However, there is a much easier way — to count the number of parameters. It’s not much harder than just reading the paper, but allows you to dig quite deep and check that you fully understand the building blocks of the new architecture (Transformer Encoder and Decoder blocks in our case).

You can think about this with the following diagram, which presents three ways to understand a new ML architecture —the size of the circle represents the level of understanding.

In this article, we will take a look at the famous Transformer architecture and consider how to calculate the number of parameters in PyTorch TransformerEncoderLayer and TransformerDecoderLayer classes. Thus, we will make sure that there are no mysteries left for us about what this architecture consists of.

TL;DR

All formulas are summarized in the Conclusions section. You are welcome to take a look at them right now.

I present not only exact formulas but also their less accurate approximate versions, which will allow you to quickly estimate the number of parameters in any Transformer-based model.

Transformer Architecture

The famous Transformer architecture was presented in the breathtaking “Attention Is All You Need” paper in 2017 and became the de-facto standard in the majority of Natural Language Processing and Computer Vision tasks because of its ability to effectively capture long-range dependencies.

Now, in early 2023, diffusion is gaining extreme popularity, mainly due to text-to-image generative models. Maybe, soon they will become the new state-of-the-art in various tasks, as it was with Transformers vs LSTMs and CNNs. But let’s take a look at Transformers first…

My article is not an attempt to explain Transformer architecture since there are enough articles that do it very well. It just might allow you to look at it from a different side or clarify some details if you haven’t fully figured it out yet. So if you are seeking more resources to learn about this architecture, I’m referring you to some of them; otherwise, you can just keep reading.

Resources to know more about Transformer

If you are looking for a more detailed Transformer architecture overview, take a look at these materials (note that there are plenty of other recourses on the Internet, I just personally like these):

• First of all, the official paper. It may be not the best way for the first time, but it is not as complex as it seems. You can try Explainpaper to help you read this or other papers (it is an AI-based tool that can explain the text you highlight).
• Great Illustrated Transformer article by Jay Alammar. If you do not enjoy the reading, watch the YouTube video by the same author.
• Awesome Tensor2Tensor talk by Lukasz Kaiser from Google Brain.
• If you want to go straight to practice and use various Transformer models to build real applications, check the Hugging Face course.

Original Transformer

To begin with, let’s remember Transformer basics.

The architecture of the Transformer consists of two components: the encoder (on the left) and the decoder (on the right). The encoder takes a sequence of input tokens and produces a sequence of hidden states, while the decoder takes this sequence of hidden states and produces a sequence of output tokens.

Both the encoder and decoder consist of a stack of identical layers. For the encoder, this layer includes multi-head attention (1 — here, and later numbers refer to the image below) and a feed-forward neural network (2) with some layer normalizations (3) and skip connections.

The decoder is similar to the encoder, but in addition to the first multi-head attention (4) (which is masked for machine translation task so the decoder doesn’t cheat by looking at the future tokens) and a feedforward network (5), it also has a second multi-head attention mechanism (6). It allows the decoder to use the context provided by the encoder when generating output. As the encoder, the decoder also has some layer normalization (7) and skip connections components.

I will not consider the input embedding layer with positional encoding and final output layer (linear + softmax) as Transformer components, focusing only on Encoder and Decoder blocks. I do so since these components are specific to the task and embedding approach, while both Encoder and Decoder stacks formed the basis of many other architectures later.

Examples of such architectures include BERT-based models for the Encoder (BERT, RoBERTa, ALBERT, DeBERTa, etc.), GPT-based models for the Decoder (GPT, GPT-2, GPT-3, ChatGPT), and models built on the full Encoder-Decoder framework (T5, BART, and others).

Although we counted seven components in this architecture, we can see that there are only three unique ones:

1. Multi-head attention;
2. Feed-forward network;
3. Layer normalization.

Together they form the basis of a Transformer. Let’s look at them in more detail!

Transformer Building Blocks

Let’s consider the internal structure of each block and how many parameters it requires. In this section, we will also start using PyTorch to validate our calculations.

To check the number of parameters of a certain model block, I will use the following one-line function:

Before we start, pay attention to the fact that all blocks are standardized and used with skip connections. This means that the shape (more precisely, its last number, since the batch size and the number of tokens may vary) of all inputs and outputs must be the same. For the original paper, this number (d_model) is 512.

Multi-Head Attention

A famous attention mechanism is a key to Transformer architecture. But putting aside all motivations and technical details it is just a few matrix multiplications.

After calculating attention for every head, we concatenate all heads together and pass it through a linear layer (W_O matrix). In turn, each head is scaled dot-product attention with three separate matrix multiplications for the query, key, and value (W_Q, W_K, and W_V matrices respectively). These three matrices are different for each head, which is why subscript i is present.

The shape of the final linear layer (W_O) is d_model to d_model. The shape of the rest three matrices (W_Q, W_K, and W_V) is the same: d_model to d_qkv.

Note that d_qkv in the image above is denoted as d_k or d_v in the original paper. I just find this name more intuitive, because although these matrices may have different shapes, it is almost always the same.

Also, note that d_qkv = d_model / num_heads (h in the paper). That’s why d_model must be divisible by num_heads: to ensure correct concatenation later.

You can test yourself by checking the shapes in all the intermediate phases in the picture above (the correct ones are indicated at the bottom right).

As a result, we need three smaller matrices for each head and one large final matrix. How many parameters do we need (do not forget biases)?

I hope it’s not too tedious — I tried to make the deduction as clear as possible. Don’t worry! The future formulas will be much smaller.

The approximate number of parameters is such because we can neglect 4*d_model compared to 4*d_model^2. Let’s test ourselves using PyTorch.

The numbers match, meaning we are good!

Feed-forward Network

The feed-forward network in Transformer consists of two fully connected layers with a ReLU activation function in between. The network is built in such a way that its internal part is more expressive than input and output (which, as we remember, must be the same).

In general case, it is MLP(d_model, d_ff) -> ReLU -> MLP(d_ff, d_model), and for original paper d_ff = 2048.

A little visualization never hurts.

The calculation of parameters is quite easy, the main thing, again, is not to get entangled by biases.

We can describe such a simple network and check the number of its parameters using the following code (note that official PyTorch implementation also uses dropout, which we will see later in Encoder/Decoder code. But as we know, the dropout layer does not have trainable parameters, so I omit it here for simplicity):

The numbers match again, and only one component remains.

Layer Normalization

The last building block of the Transformer architecture is layer normalization. Long story short, it is just an intelligent (i.e. learnable) way of normalization with scaling, that improves the stability of the training process.

The trainable parameters here are two vectors gamma and beta, each of which has a d_model dimension.

Let’s check our assumptions with code.

Good! In approximate calculations, this number can be neglected, since layer normalization has dramatically fewer parameters than feed-forward network or multi-head attention block (despite the fact that this module occurs several times).

Derive Complete Formulas

Now we have everything to count the parameters for the entire Encoder/Decoder block!

Encoder and Decoder in PyTorch

Let’s remember that the Encoder consists of an attention block, feed-forward net, and two layer normalizations.

We can verify that all components are in place by looking inside the PyTorch code. Here multi-head attention is indicated in red (on the left), the feed-forward network in blue and layer normalizations in green (screenshot of the Python console in PyCharm).

As noted above, this implementation includes dropout in the feed-forward net. Now we can see the dropout layers related to layer normalizations as well.

The decoder, in turn, consists of two attention blocks, a feed-forward net, and three layer normalizations.

Let’s look at PyTorch again (the colors are the same).

Final Formula

After making sure, we can write the following function to calculate the number of parameters. In fact, these are just three lines of code that can even be combined into one. The rest of the function is a docstring for clarification.

Now it’s time to test it.

The exact formulas are correct, meaning we have correctly identified all building blocks and deconstructed them into their components. Interestingly, since we ignored relatively small values (thousands compared to millions) in the approximate formulas, the error is only about 0.2% compared to the exact results! But there is a way to make these formulas even simpler.

The approximate number of parameters for the attention block is 4*d_model^2. It sounds pretty simple, considering that d_model is an important hyperparameter. But for the feed-forward network, we need to know d_ff, since the formula is 2*d_model*d_ff.

d_ff is a separate hyperparameter that you have to memorize in the formula, so let’s think about how to get rid of it. In fact, as we saw above, d_ff = 2048 when d_model = 512, so d_ff = 4*d_model.

For many Transformer models, such an assumption will make sense, greatly simplifying the formula, and still giving you an estimate of the approximate number of parameters. After all, no one wants to know the precise amount, it’s just useful to understand whether this number is in the hundreds of thousands or tens of millions.

To get an understanding of the order of magnitude you are dealing with, you can also round the multipliers and you will get 10*d_model^2 for every Encoder/Decoder layer.

Conclusions

Here is a summary of all the formulas that we have deduced today.

In this article, we’ve calculated the number of parameters in Transformer Encoder/Decoder blocks, but of course, I don’t invite you to count the parameters of all new models. I have just chosen this method because I was surprised that I didn’t find such an article when I started studying Transformers.

While the number of parameters can give us an indication of the complexity of the model and the amount of data it will require to train, it is just one way to gain a deeper understanding of the architecture. I want to encourage you to explore and experiment: take a look at implementation, run the code with different hyperparameters, etc. So, keep learning and have fun!

Thank you for reading!

• I hope these materials were useful to you. Follow me on Medium to get more articles like this.
• If you have any questions or comments, I will be glad to get any feedback. Ask me in the comments, or connect via LinkedIn or Twitter.
• To support me as a writer and to get access to thousands of other Medium articles, get Medium membership using my referral link (no extra charge for you).

How to Estimate the Number of Parameters in Transformer models was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.

A graph is a relatively old mathematical data entity that is a set of connected elements. Since the graph is a very flexible structure and allows you to store information in a form familiar and convenient to humans, graphs have always been used in computer science and technology. With the rise of machine learning and deep learning, graphs have gained even more popularity by creating the field of graph machine learning.

In this post, I would like to share with you the most useful Python libraries I’ve used for graph/network analysis, visualization, and machine learning. Today, we will review:

• NetworkX for general graph analysis;
• PyVis for interactive graph visualizations right in your browser;
• PyG and DGL for solving various graph machine learning tasks.

Before that, let me tell you a few words about graph theory and graph machine learning and provide some learning resources that may be helpful to you. If you don’t know what graph or graph machine learning is, that is a great opportunity to lift the veil of secrecy!

Graph Theory and Graph Machine Learning: a Brief Introduction

The graph is simply a set of elements connected to each other.

However, the fact these elements (called nodes) can contain any information and can be connected in any way (with edges) makes the graph the most general data structure. Indeed, any complex data familiar to us can be represented as a simple graph: for example, an image — as a grid of pixels or text — as a sequence (or chain) of words.

You might wonder: are graphs really so important? Well, some tasks simply cannot be solved or even formulated without them, as some information cannot be structured in data. Imagine the following situation: you need to visit a list of cities, say for tourism or for work. You have information about the distance from one city to another, or say, the cost of tickets for different transport modes — it’s even more interesting! How to create an optimal route, that is, spend the minimum amount of money or drive a minimum distance?

For me, the task is quite practical — think at least about its application in logistics. And, this is an example of a problem that cannot be solved without the help of graphs. Think about how you will represent the data and in any case, you will still come to the weighted graph (a graph whose edges have some value, called weight). By the way, if each city needs to be visited exactly once, this task turns into the famous traveling salesman problem (TSP), which is not so easy to solve. One of the reasons is that the number of possible routes is growing very fast, and even for 7 cities, there are already 360 of them!

Graph theory (originated in the 18th century) was engaged in the study of graphs and solving various graph problems: finding a possible or optimal path in a graph, building and researching trees (a special type of graph), and so on. Graph theory was successfully used in social sciences, chemistry, biology, and other fields. But with the development of computers, the process of using graphs has reached another level.

What is really important is that this base: a set of related elements, often with different elements and types of connections, is very useful for modeling real-world tasks and datasets. This is the place where graph machine learning comes into the picture (although amazing tasks were solved before it as well). After humanity collected the appropriate datasets and developed technologies to model them (like Graph Convolutional Networks (GCNs), by analogy with Convolutional Neural Networks (CNNs)) it becomes possible to solve a wide range of graph tasks:

• Node-level tasks, like node classification — assign a label for every node in the graph. We will see an example a little below — divide a group of people into two clusters, knowing how they communicate with each other; but other applications can be very different. The intuition here comes from social science, which says that we are dependent on our environment. Indeed, any entity can be classified more effectively taking into account not only some set of features but also data about its neighborhood. For example, if your friends smoke, you are more likely to smoke, and if your friends go to the gym, you are more likely to go to the gym.
• Edge-level tasks, like edge prediction — predict if two nodes have an edge or, more often, predict edge type (graphs that have several edge types are called multigraphs). This task is very interesting for the knowledge graphs, which we see in a couple of minutes.
• Graph-level tasks. This can be graph classification, graph generation, and so on. This field is especially useful for biology and chemistry because molecules can be effectively represented as graphs. Molecule classification (determining if the molecule has certain properties) or molecule generation (and especially drug generation) sounds much cooler than some “graph-level tasks”!

Let’s take a look at examples of graphs from real life. One of the most famous graph datasets is the karate club dataset. Here, each node is a person (club member), and each edge represents the two members who interacted outside of the club.

A common problem is finding two groups of people into which the club split after an argument between two instructors (now we can treat it as binary (or 2-class) node classification). The dataset was collected back in 1977 and become a classic example of a human social network or community structure.

Another graph type, interpretable for humans, and therefore extremely useful for machine learning models is a knowledge graph. In a knowledge graph, a node is some entity or concept and an edge represents knowledge about the interaction of a pair of entities. Thus, the node-edge-node structure stores a certain fact about the world or a particular system.

The knowledge graph in the example above contains two types of edges: is and eat and is thus a multigraph we introduced earlier. The Dogs-is-Animals structure gives us the knowledge that the “dogs” set is a subset of the “animals” set, or, in simpler terms, that dogs are animals.

Wikidata is a huge free knowledge base by Wikipedia, which is constantly updated and has more than 100 million nodes now. There are more than 400 edge types, some of which are part of, different from, opposite of, population, and location, so definitely make sense.

That huge knowledge base contains a lot of information about the world around us. It’s still amazing to me how humanity has collected this data, and that machines are now able to process it!

One more thing I can’t keep silent about is wikidata's beautiful visualization capabilities. For example, here you can see the plot of connectivity of the United States states. Note that it is not drawn by anyone, it is just a subgraph of the entire wikidata graph: we took only American states as nodes and P47 (shares border with) as edges.

Take a look at Wikidata Graph Builder and other visualizations. Let me point you to some of them that I find entertaining:

• Map of the relationships between master and student from Socrates to the end of the Hellenistic Period
• Interactive timelines with historical events
• Pan Pan panda family tree

Know More about Graphs

If after that brief overview you are now interested in graphs and want to know more about them, I refer you to the wonderful Gentle Introduction to Graph Neural Networks by Google Research. In this article, you can find more examples and interactive visualizations.

Check the Graph Theory Algorithms course by freeCodeCamp.org for various graph theory algorithms overviews or Stanford CS224W: Machine Learning with Graphs course to start your graph machine learning journey.

After that brief introduction, let’s actually start with Python libraries!

NetworkX — General Graph Analysis

If you have to do some operations on graphs and you use Python as your programming language, you will most likely find the NetworkX library pretty quickly. It is probably the most fundamental and commonly used library for network analysis that provides a wide range of functionality:

• Data structures for storing and operating on undirected or directed graphs and multigraphs;
• Many graph algorithms implemented;
• Basic visualization tools.

The library is pretty intuitive and easy to use. Also, the majority of fundamentals, like graph data structures will remain the same or at least similar for all popular graph libraries. For clarity, you can create a simple graph and visualize it with the following code:

When it comes to algorithms, networkx is pretty powerful and has hundreds of graph algorithms implemented.

To summarize, this is an efficient, scalable, and powerful library, that will definitely be useful for you if you are dealing with graph analysis.

Reference

NetworkX - NetworkX documentation

PyVis — Interactive Graph Visualizations

Using networkx for graph visualization can be pretty good for little graphs but if you need more flexibility or interactivity, you better give PyVis a chance. The situation is similar to matplotlib vs plotly. Using matplotlib for quick and straightforward visualizations is perfectly fine, but if you need to interact with your chart or present it to somebody else, you better use more powerful tools.

PyVis is built on the VisJS library and produces interactive visualizations in your browser with simple code. Let’s plot the same graph as in the example above.

This code will create a graph.html file. By opening it, you will be able to interact with your visualization: zoom it, drag it, and much more.

Looks interesting, right? The library even allows you to use web UI to dynamically tweak display configurations. Definitely check the official tutorial that will walk you through the main library’s capabilities.

Reference

Interactive network visualizations - pyvis 0.1.3.1 documentation

DGL and PyG Graph Machine Learning

Let’s now switch to the more advanced topic — graph machine learning. I will mention two of the most popular libraries for it: DGL and PyG.

DGL (Deep Graph Library) was initially released in 2018. In contrast to PyG (PyTorch Geometric), which is built on top of the PyTorch and therefore supports only PyTorch tensors, DGL supports multiple deep learning frameworks, including PyTorch, TensorFlow, and MXNet.

Both libraries implement popular Graph Neural Network (GNN) cells such as GraphSAGE, GAT (Graph Attention Network), GIN (Graph Isomorphism Network), and others. It will not be difficult to build a model from pre-made blocks — the process is very similar to plain PyTorch or TensorFlow.

Here is how you can create a 2-layer GCN model for node classification in PyG:

And the same code for DGL:

Both code snippets are pretty straightforward if you are familiar with deep learning and PyTorch.

As you see, the model definition is very similar for both libraries. The training loop then can be written on the plain PyTorch for PyG and require some modifications for DGL (since DGL graph objects store the entire dataset, and you have to address train/validation/test sets using binary masks).

There is a slight difference in data representation here: you can see it at least based on the different input parameters for the forward method. Indeed, PyG stores everything as PyTorch tensors and DGL has a separate graph object that you have to use, and under the hood, it follows a more classical NetworkX style.

However, that is not a big deal — you can convert the PyG graph object to the DGL graph and vice versa with a few lines of code. The more important question is: how else are they different? And which one should you use?

DGL vs PyG

Trying to figure out which of the libraries is better, you will keep coming across the same answer — “try both and decide which works best for you”. Okay, but how are they at least different? Again, the answer that you will constantly encounter is “they are quite similar”.

And they really are! Moreover, you saw it for yourself by looking at the code a few minutes ago. But of course, you can find some differences digging deeper: here is a good resource list including a few thoughts by library authors, and here is a pretty detailed comparison on different sides.

In general, the answer is really to try both. In fact, DGL has more low-level API and can be harder to use in the sense of implementing new ideas. But this makes it more flexible: DGL is not limited to message-passing networks (classical Graph Convolutional Networks) and has the implementation of several concepts that PyG can not provide, for example, Tree-LSTM.

PyTorch Geometric, on the other hand, makes his API as easy as possible and then gains more popularity among researchers that can quickly implement new ideas, i.e. new GNN cells. In the past time, PyG becomes more and more popular due to important updates with PyG 2.0 and active and powerful teams of collaborators, including Stanford University.

So I still encourage you to try both of them, giving PyG the chance first.

If you are working on a relatively familiar graph problem (be it node classification, graph classification, etc.), both PyG and DGL have a huge amount of GNN cells implemented. Also with PyG, it will be easier for you to implement your own GNN as part of any research.

However, if you want to get full control over what is happening under the hood or implement something more complicated than the message-passing framework, your choice will most likely fall on DGL.

References

• PyG official website, PyG documentation
• DGL official website, DGL documentation

Conclusion

The target audience of this article (people interested in graphs) is quite small. Well, machine learning is a fairly young field of computer science, and graph machine learning is even younger. The last mainly attracts the attention of the research community, but, believe it or not, it is used in important real-world applications such as recommendation systems and biology/chemistry studies.

In any case, I hope these materials were interesting or helpful for you — whether you were looking for anything specific or just learned something new today. As a recap, today we briefly reviewed what graph and graph machine learning is, and took a look a the following libraries:

• NetworkX: general graph analysis;
• PyVis: interactive graph visualizations;
• PyG and DGL: machine learning on graphs.

Thank you for reading!

• I hope these materials were useful to you. Follow me on Medium to get more articles like this.
• If you have any questions or comments, I will be glad to get any feedback. Ask me in the comments, or connect via LinkedIn or Twitter.
• To support me as a writer and to get access to thousands of other Medium articles, get Medium membership using my referral link (no extra charge for you).

Graphs with Python: Overview and Best Libraries was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.

On April 18th, 2022, at Coursera’s 10th anniversary, Andrew Ng (founder of DeepLearning.AI, co-founder of Coursera, and one of the most famous popularizers of Machine Learning and AI) released a new Machine Learning Specialization. This is an updated version of the original Machine Learning Stanford course created back in 2010, which was taken by about 5 million people since it was initially released.

I took the original course some time ago and have taken the updated one as soon as it was finished in July. Let’s take a closer look at these learning materials and find out, what exactly has been updated and if is it worth taking this course in 2022.

TL;DR

Yes, it is. In my view, now this is the best way to start your machine learning journey especially if you are a complete beginner and have never heard about data science and machine learning before.

Previous article

In my previous article on this topic, I shared my thoughts about Stanford’s original Machine Learning course. There I listed my personal opinion about the drawbacks of the course and suggestions on what it would be worth to supplement it with.

Now that I have taken both versions — original and updated, in my further reasoning, I will rely on the theses expressed in this article (mostly in Comparison with the Original Course section). So, if you want to get familiar with my initial thoughts, I share the link to my previous article below.

What you need to know before taking the Machine Learning course by Stanford

Comparison with the Original Course

In this section, I’m going to compare two versions of the course based largely on what I’ve stated before. I don’t know if someone from the DeepLearning.AI team has read my review, but almost all of my claims were properly considered and satisfied. :)

Outdating

In my previous article, I said that the main drawback of this course is that it is outdated. Of course, I was not the only one who understood this, and I think this was the main reason for its update.

Although the basics that the course talks about will never change, it’s much more pleasant to know that all the information you receive, including assignments and practical examples, is topical and relevant to the present.

Based on the numerous advantages of the course, the elimination of this disadvantage only is already a sufficient condition for this course to be a very good learning material. But its creators went further, refining and improving the educational plan and ways of interacting with students, not only by updating the course but also by making it more complete, broad, and interesting.

Python Instead of Octave

In the original course, practical assignments had to be solved on Octave (MATLAB). And although there were ways around it, the official implementation didn’t use Python. Fortunately for everyone, this has been fixed, and now the course uses Python.

This means not only that you don’t need to learn a programming language that you will almost certainly never use again in your life (sorry, MATLAB programmers). It also means that together with Python, you will get the basic knowledge about the libraries that you will use almost every day if you start working in data science or machine learning: numpy, scikit-learn, and TensorFlow. It’s just great!

It is important to note that the course does not teach you Python, and you need to understand the basics of this language and programming in general — variables, conditional expressions, loops, and functions will be quite enough (classes and Object Oriented Programming in its purest form is not used in the course). So if you are a complete beginner, maybe you should take some Python basics course first.

Ensemble Learning

The original course did not mention any of the ensemble models, but now it will tell you in detail about Decision Trees (which also didn’t happen before), explain the motivation of ensemble learning, and demonstrate the Random Forest and Boosting models.

Although ensemble learning is explained here only in the context of decision trees and not other models, it is a step in the right direction that allows you to get a more complete picture of the machine learning models zoo.

You will also be shown the XGBoost library, one of the most powerful tools of classical machine learning (by classical I mean machine learning without deep neural networks), so you get it into your toolbox right away!

Categorical Variables

The original course also did not mention categorical variables at all, which seems very strange to me, because it will confuse a beginner in the very first task that he decided to solve on his own (since categorical variables are very common). This fault has been corrected here, and the course will tell you about one-hot encoding.

You will have to get acquainted with the rest of the methods by yourself, but since this course is designed for beginners, I do not consider this a disadvantage.

Unsupervised Learning Algorithms: Clustering, Anomaly Detection, and Dimensionality Reduction

As for unsupervised learning algorithms, the situation has remained about the same. K-Means only is considered for clustering, as well as Gaussian Mixture Models (GMM) for anomaly detection.

Dimensionality reduction is mentioned in the course as an auxiliary technique, and less attention is paid to it than before when we have a couple of separate lectures on Principal Component Analysis (PCA).

Update: However, in December 2022, the course was updated a little more, and several videos about PCA were returned as like in the original course.

Although it may seem that I consider many other approaches overlooked (and last time I thought so) I’ve changed my mind, which I’ll talk about a bit later.

Sorry, Support Vector Machine

Before moving on to more general considerations, to be completely honest, I should note that Support Vector Machine materials were cut out of the updated course, although the original course contained them.

Perhaps the mention of SVM did not fit into the course due to timekeeping, but I think that it was removed because of the complexity since this algorithm with its kernel trick is quite difficult to understand and is probably the most difficult aspect of classical supervised learning.

So, I think it was removed to not scare off newcomers; in any case, changing SVM to Decision Trees, Random Forest and Boosting is a good exchange.

Update: After the addition of PCA and backpropagation materials in December 2022, it is quite possible that SVM will be included in the program with some upcoming updates.

What else? More General Reasoning

Structuredness

Updated specialization has become much more structured. The original source consisted of 11 weeks, the materials inside of which were not separated from each other.

Now, the content of the specialization is divided into 3 courses, each of which has a specific topic and an internal division by weeks. So it becomes much easier to navigate inside the course, return to it and search for particular materials. By the way, these three courses are:

1. Supervised Machine Learning: introduction to machine learning, regression (linear regression), and classification (logistic regression);
2. Advanced Learning Algorithms: neural networks, decision trees, random forest, boosting, and practical advice for applying machine learning;
3. Unsupervised Learning, Recommenders, Reinforcement Learning: clustering, anomaly detection, recommender systems, and reinforcement learning.

See the specialization homepage to get a more detailed description.

More Labs, More Visualizations, and More Interactivity

Another good solution, in my opinion, is to include more practical labs and allow people to see more code. Since the code can now be executed in Jupyter Notebooks, this has freed the authors’ hands. Labs (almost all of which are optional, so no one will be scared off) have become more numerous, detailed, and diverse.

At the same time, many of them contain very intuitive and interactive visualizations. The quality of some of them, let’s be honest, could have been better (I honestly don’t understand why they used pure matplotlib even for interactive plots instead of for example plotly) but some of them really allow you to get very useful insights.

To get an idea of what kind of interactivity is meant, you can see the most recent specialization demo on YouTube.

The Third Course is Amazing

Although I said that it would be possible to mention more algorithms and approaches for unsupervised learning tasks, the third course of specialization covers much more than an analogous part before.

Clustering and anomaly detection are reviewed only during the first week. The second week is devoted to two algorithms of the most commercially successful machine learning application, recommender systems: collaborative filtering and content-based filtering, and the third week is dedicated to Deep Q-Learning (DQL), a reinforcement learning algorithm, which is really awesome!

First, tales about reinforcement learning are always very inspiring, at least to me since I think this is the most “machine”-based part of machine learning. Second, the teaching talent of Andrew Ng and the course team make this, one of the most complex fields of science, or at least its basics, understandable for everyone.

By the way, reinforcement learning is a part of machine learning, where an agent learns to do something in a given environment without knowing anything about how things work, and receiving only rewards in the case of success. This part of machine learning is also the most popular in news: that is where robots learn to play chess, go, or video games. You can also find a dozen of YouTube videos where AI agents learn different things: from walking to boxing.

Putting all together, the third course shows an extremely broad picture of real-world machine learning applications and takes specialization to the next level.

Definitely, the Specialization was Designed for Beginners

For a variety of authors' reasons, specialization was made as simple as possible, sometimes even too much. This is not a drawback, but rather an advantage for the majority of learners, but in any case, it is a feature that has several consequences.

Most of all, this concerns people who are familiar with high school mathematics (and understand what is a derivative, optimization tasks, why gradient descent really works, etc.). There will be a lot of understatement in the course for you, as if you lack additional lectures describing the math behind certain algorithms. This is because the course tries to do without math at all, but sometimes it looks strange, for example not mentioning backpropagation when talking about neural networks.

Update: A couple of lectures about backpropagation were added in the December 2022 update, so this comment has lost its relevance.

Of course, you can fill in these understatements by reading one of the many books or by taking a more advanced ML course. But if you are a computer science student, there may be more effective ways to dive into the world of AI, data science, and machine learning specifically for you. After all, you really don’t need a strong mathematical background to start exploring ML, but you may need to know a lot of mathematical points if you decide to do it seriously.

Summary

The hypothesis that I got during the course completion was completely proved when I was watching the Accelerating Your AI Career discussion by the DeepLearning.AI team.

Apparently, Andrew Ng and the team have done everything possible to give anyone the opportunity to pass this specialization and understand what machine learning is. This can be helpful for:

• students who want to find ML-related jobs,
• established professionals who want to change their field of activity,
• or just people who are interested in what ML is and whether there is a real danger from Artificial Intelligence at least for the current moment.

Andrew Ng mentioned that one of the goals of spreading knowledge about AI and ML is to make AI more accessible for people of non-technical specialties because it will still be very useful for them. AI-related products and goods are entering our lives more and more actively and by understanding the basics, you can greatly facilitate your work or see the opportunities to do so, no matter what profession you are working in. We are talking about automation, and if programming has made it possible to automate routine actions, then AI makes it possible to automate routine actions, requiring some level of intelligence right before our eyes.

This specialization is one of the steps to achieving this goal. It does not tell you at once about all approaches to training neural networks or all the pitfalls of the dimensionality reduction task. But none of the specializations, courses, or books can. Instead, it gives you the broadest possible understanding of machine learning, and it is deep enough to engage in it. The lack of math may annoy some enthusiasts, but this lowers the entry threshold so much that it has much more pros than cons.

If you:

• have never heard about machine learning,
• are afraid that your knowledge of math is not enough, or
• are not working in IT or in a technical occupation,

this specialization is an excellent choice. It will reward your decision to explore new things without scaring you with complex terms and formulas. And when you finish it, you will be able to start implementing your own project or be not afraid to continue to get acquainted with AI word with the help of more advanced materials.

In the worst case, learning about AI will be interesting, and in the best one, it can change your life for the better.

Reference

Machine Learning

Thank you for reading!

• I hope these materials were useful to you. Follow me on Medium to get more articles like this.
• If you have any questions or comments, I will be glad to get any feedback. Ask me in the comments, or connect via LinkedIn or Twitter.
• To support me as a writer and to get access to thousands of other Medium articles, get Medium membership using my referral link (no extra charge for you).

What you need to know about updated Machine Learning Specialization by DeepLearning.AI was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.

Have you ever wondered what your favorite movie looks like? Well, you definitely know what it looks like because you have most probably watched it more than once, but how does it look in static or how can it be visualized using a single image?

I present an approach to creating an image that reflects the color scheme of any video using machine learning algorithms. This is not only an interesting project, but also an endless source of gift ideas for your movie-loving friends and acquaintances: first, it’s pretty personal and unique, and second, it’s highly customizable since you can use this image as a print for anything you can order a print on: a cap, t-shirt, towel, canvas, etc., etc.

GitHub - Winston-503/movie_color_scheme

How to read this article

For your convenience, let me briefly describe all the main sections of this article so that you know where to start and can successfully navigate in the future:

• In the Problem Statement section, I will tell you what problem I am going to solve with this project;
• In the Algorithm and Implementation Details section, I will tell you how I’ve implemented the project using Python, and its libraries: moviepy and scikit-learn. Don't be afraid to skip this section if you are not familiar with Python or programming in general, since using the project code does not require any specific knowledge from you;
• In the User’s Guide section, you will find a user’s guide for the final program and a link to the repository with source code;
• In the Gallery section, you can see result images generated by the program based on different movies. Take a look here at the beginning to see what the program is able to do;
• In the Results and Conclusions section, I conclude the work done and share my thoughts about possible improvements.

Now, when you know in which order to read this article, let’s go!

Problem Statement

I want to write a program that takes some video file as input and outputs an image, reflecting the color scheme of a given video. Some characteristics of the resulting image are also set by additional parameters.

To see example images that the program is able to generate, see the Gallery section.

Algorithm and Implementation Details

Let’s decompose the initial problem into sub-problems and review the algorithm step by step:

1. First, we need to convert the proposed video to a list of images;
2. For each image in the given list, we need to extract the most popular colors by analyzing the image;
3. Finally, having the list of colors we need to compose the resulting image.

Let’s discuss these steps in more detail and see the code to perform them. Please note that the code in gists should not be used directly, and it is here only to give you a general idea. I tried to keep it as simple as possible while the resulting code in the repository is in a way optimized and wrapped in the class.

1. Convert the video to a list of images

First, we need to convert the given video to an array of images for future analysis. It is very easy with the moviepy library.

First, we read the video in the moviepy object. Getting the video duration in seconds using duration property, and setting step variable that determines how often we want to extract a video frame, we use for loop to get the image in the current timestamp (using get_frame function) and add it into the resulting list.

start variable is needed for a little offset since all videos start with a black screen at the zeroth second.

2. Analyse image colors

When we have a video “sliced” into an array of images, we need to analyze which colors are the most popular in them. Let me briefly tell you how we will do this.

General idea is to use a clustering algorithm on the image. Clustering is the task of grouping similar data points into groups (or clusters). In our case, data points are pixels of the image and features are three RGB color values.

After the clustering algorithm has calculated clusters for us (meaning that it analyzed all image pixels and create top-k pixels groups based on a color, where k is a number of clusters), we can use these cluster representatives to get the most popular colors. The easiest way to do this is to apply a clustering algorithm that is based on the centroid concept. A centroid in clustering is simply a geometric center of a cluster (that can be interpreted as the most common element for the given group). There are several centroid-based cluster algorithms; I have chosen K-Means (can be also MeanShift, Affinity Propagation, etc.): it iteratively updates cluster centroids until convergence.

Resources to Know More about Clustering

If you want to know more detail about how clustering is used to extract colors from images you can check the following article:

Color Separation in an Image using KMeans Clustering using Python

If you want to know more about clustering algorithms in general, check my clustering cheat sheet article, which will give you a big picture about clustering algorithms zoo and clustering analysis in general:

Clustering cheat sheet

Back to the practice

But let’s get back to practice. Let’s do color analysis using the example of the following image:

Running this image through a clustering algorithm with different values of k, you will get the following outputs:

As you can see, using too few colors does not reflect the real picture, and using too many colors makes the result noisy. In my experiments, I set a number of colors equals to 5.

Let’s see the code and discuss implementation details.

We first preprocess a given image: resize it to a smaller size to allow the algorithm to run faster and reshape it into a 2D array as the scikit-learn object expects. Then we train a clustering algorithm, get coordinates of the cluster centroids, and sort the resulting colors by popularity, converting them from RGB to HEX for easier visualization.

In case you want to better understand the code and each variable type and shape, please take a look at this interactive Jupyter notebook with demo code and more detailed comments.

3. Compose the resulting image

Now the matter remains small: having an array of colors, we only need to compose the final image.

To do this, we will need some simple coordinate calculations, which I have visualized below.

As we know, in matplotlib convention the lower left image corner point has coordinates (0, 0), and the upper right point has coordinates (w, h) (if the width of the image equals w and height equals h).

We need to draw n rectangles sequentially, each time shifting slightly to the right. Matplotlib Rectangle class that I will use draws a rectangle using three parameters: coordinates of the lower left point, width, and height.

The width and height of each rectangle are the same: the height is equal to the height of the image, and the width is the width of the image divided by the number of rectangles. The coordinates of the lower left point are equal to the product of the index i by the width of the image divided by the number of rectangles. Make sure of this by substituting i = 0, 1, 2 into the formula above.

Note that image width and height are set in inches, not pixels. Please refer to the same Jupyter notebook with demo code for interactive visualization.

For simplicity, in the example above, I explicitly specify a list of colors. In fact, for each image, I save all the top-k colors. With this information, you can create different ways to choose a color based on the results of this analysis.

I have implemented two options that are selected by the mode parameter. If mode="popular", we just choose the most popular color. But if mode="sample", we sample a color according to a given distribution. Thus, even for the same video, you can get different results that make the images more diverse.

User’s guide

Let’s talk about how you can use this program.

After setting up the environment and installing the required libraries, you can use the program by setting parameters as we usually do in Python scripts: python main.py --parameter1 value --parameter2 value. Let’s review the program parameters.

The list of program parameters

Required parameters:

• video_path: Path to the video file to process;
• result_path: Path to save the resulting image (including .png);
• width: Width of the resulting image, inches. Default value: 25;
• height: Height of the resulting image, inches. Default value: 5.

Less important parameters (you can just use default values):

• mode: Mode to choose a color. If 'popular' - return the most popular color, else sample a color according to distribution. Default value: 'sample';
• start: Time to start video processing (seconds). Default value: 1;
• step: Step in seconds. Default value: 1;
• number_of_colors: Number of colors to extract from each image (number of clusters). Default value: 5;
• compress_to: Number of pixels to resize the image during preprocessing. Default value: 200;
• verbose: If True, show the resulting image. Default value: False;
• delete_after: If True, delete temporary files after processing. Default value: False.

Saving intermediate results

I also need to mention one more implementation detail that can save your time. Since the analysis stage takes a long time, refusing to save intermediate results will not be a good solution.

So, I save the image array and colors array so that you can reproduce experiments later. For example, if you set mode="sample" but don’t find the resulting image very engaging, you can just run the last stage providing the path to the saved colors array file (commenting out the corresponding lines of code in main.py) without the need to run the whole analysis from scratch.

For videoname.mp4 these files will be named videoname_images.pkl and videoname_colors.pkl and will be saved in tmp folder.

Gallery

In this section, you can find program outputs based on different movies. All the provided images in high resolution you can also find in the repo.

Let me first analyze the color scheme of Shrek 2 I showed you at the beginning of the article to basically prove that this approach works and can interest real movie lovers. Let’s zoom in and review the first 10 minutes:

Here step parameter is equal to its default value of 1, so every tiny rectangle on the image above is one second in the video.

The cartoon starts with DreamWork's opening theme, where a child throws a fishing rod while sitting on the moon. This is happening in the light-blue background. A little white title on the black background with DreamWork’s presents… then follows.

This is followed by a short tale with a storybook and a moment where Prince Charming allegedly saves Fiona, after which there is a real screensaver of the movie with Fiona and Shrek, where by the way there are many references to various cultural works. After that, the narration actually begins in the swamp, in the appropriate color tones.

Let’s continue with the Lord of the Rings. The color scheme of the second film and the bright ending of the third catch my eye.

I also analyzed all Harry Potter movies. The titles of the third film are most noticeable here — they are light (against the background of the Marauders’ Map). The bright moment of Harry’s conversation with Dumbledore in the last film also stands out vividly.

With a little imagination, you can create fascinating art based on these images. I’m not very good at that, but let me share with you a general idea.

From this:

To this:

Results and Conclusions

In closing, let me discuss a little the work done.

Supported Video Formats

This program uses moviepy library for video processing, so it supports various video formats supported by FFmpeg, like .mp4, .mpeg, .avi, .mov, .mkv, etc. See the full list here (in the Muxers section).

Sometimes I have encountered moviepy video reading errors, and converting a video file from one format to another usually helped. I used a free VLC video player.

Execution Time

As we discussed above, the program consists of three main stages — extracting images from the video, analyzing them, and compiling the final image.

The first and last stages are performed fairly quickly. The compilation of the image takes place almost instantly, while the conversion of video into images usually does not take more than two minutes.

But image analysis takes a lot of time, usually, it is about the same time as the duration of the video (for a 1-second step). To reduce this time, you can:

• Set step parameter to larger values (2 or 3 for example). This will allow you to get less accurate visualizations with fewer rectangles in the final image, but will significantly increase the execution speed.
• Set number_of_colors or compress_to parameter to smaller values although it can produce poor results and is not recommended.

Since this project is amateur and made just for the sake of interest and fun, it has a number of drawbacks and huge field for future work. For example, an optimized implementation of clustering algorithms and the use of GPU can greatly reduce the running time.

Conclusion

The idea for this project came to my mind a very long time ago. I didn’t find time to implement it for a long time, but then I started to write the code, encountering errors and unknowns. When the code was ready, I spent over 20 hours processing the popular movies you see above. Finally, I structured my thoughts for a while to transfer them to this article.

And although the implementation is not without drawbacks (it could work much faster, and I am not removing black bars, which can be a problem in dark movies), I am very pleased with the result. Setting goals, and facing setbacks, but finally achieving what you want is one of the best feelings a person can experience.

Thank you for reading!

• I hope these materials were useful to you. Follow me on Medium to get more articles like this.
• If you have any questions or comments, I will be glad to get any feedback. Ask me in the comments, or connect via LinkedIn or Twitter.
• To support me as a writer and to get access to thousands of other Medium articles, get Medium membership using my referral link (no extra charge for you).

What Does Your Favorite Movie Look Like? was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.

I’ve been learning Data Science and Machine Learning for a while, and during that time I passively collected a lot of useful resources. You know how it is actually happening — you are reading a book or watching a course and the author pays your attention (or just briefly mentions) something you didn’t hear about before. This can be everything: programming language, technology, dataset, task, metric, benchmark, visualization tool, really everything. You save it to figure out a bit later and you forget about it forever.

I tried not to forget, and explore these resources, and now I am presenting to you my top 7. They are split into categories depending on the usefulness, where the first is just funny things and the last is the most useful ones:

• Interestingness
• Where to find models?
• How to start reading research papers

These are not learning recourses (e.g. books or courses). These are materials that can just make your life a bit easier or help you to look at the machine learning world from a slightly different perspective.

Interestingness

To begin with, let’s look at three resources that are suitable for people with any level of knowledge in data science and machine learning. These are just interesting stuff, familiarity with which might amuse you and at the same time bring some benefit for you.

Spurious Correlations

Do you still think that correlation means dependency? This site is the easiest way to dissuade you from this — and it needs to be done: statistics are not the last deal in DS and ML. It contains 15 cases with high correlation, which, however, just cannot cause each other.

The number of people who drowned by falling into a pool and the number of films Nicolas Cage appeared in? Well, the correlation is just 66%, looks like a coincidence. What about the divorce rate in Maine and the consumption of margarine? 99%? What the…

Spurious Correlations

Netron

Netron is one of the simplest and most convenient ways to explore neural network architecture. I talked about it in one of my previous articles. Without saying too much, it generates visualizations of the following type:

The neural networks are getting bigger and bigger, and it’s getting harder and harder to visualize them. Nevertheless, at the beginning of your ML journey, it is interesting to look at the architecture from the inside or to understand how big ResNet152 really is. Netron gives you this opportunity.

These visualizations are pleasant and interactive — you can look into each structure in more detail, and even download the weights of the particular layer. Explore the example above here.

Netron supports TensorFlow and Keras and has experimental support for PyTorch. To know more visit the official GitHub repo.

Netron

Neural Network Playground by TensorFlow

And the last resource from this section again will not surprise people with experience in machine learning. However, for those who are just getting started discovering deep learning, this interactive playground will help to understand the functional principles of neural networks better.

Understand the limitations of a one-layer neural network (and how adding extra layers can beat those limitations), experiment with different learning rates, data shapes, and activation functions in order to get visual and intuitive ideas of what a neural network is and how it works.

Tensorflow - Neural Network Playground

Where to find models?

You already realize what machine learning is, understand how neural networks or SVM are trained, and know how L1 regularization differs from L2 one.

The following step (one of the possible ones) is to see what the models are able to do, and what tasks they are solving right now. Of course, you are learning a lot from the news nowadays, but it is much more useful to see the architecture from the inside, run it with your own hands, and if the implementation is open source, get your hands dirty and see that it has the same layers and parameters like all other models.

The following three resources will help you with this.

AI Hub

AI Hub from Google is the first place where you can start. Here you can find datasets, pretrained models, and much more. But perhaps the main advantage of this resource is a large number of Google Colab notebooks that demonstrate the functioning or training process of some models.

Because notebooks are interactive, simple, and often don’t contain a lot of code, this is a good way to start not being afraid of code written by another person.

AI Hub

TensorFlow Hub

TensorFlow Hub is right for you if you are using TensorFlow (surprisingly). If you are using Keras, maybe this is a good chance to dive into neural networks, switch to TF and try implementing custom models or layers.

This site contains a large number of pre-trained models that are very easy to run.

Only by fine-tuning these models, you will be able to get very good results on many tasks. Also pay attention to the Tutorials tab, where a link to the model is often provided with more detailed descriptions and explanations.

TensorFlow Hub

Hugging Face

Finally, Hugging Face. It is the most powerful resource, containing, I think, the largest number of models, and, at the same time, having the best tools to get familiar with these models.

Hugging Face is mainly a collection of Transformers or Transformer-like models, that are primarily designed for solving various NLP tasks. However, the site provides models and datasets for other tasks in various fields — from Computer Vision to Reinforcement Learning. But as we know, the main success of Transformers has been achieved in NLP, and often right here you can find a state-of-the-art (SOTA) model that solves a specific task with the highest quality.

But what makes this site especially valuable is the community and the ability to publish demos of our models directly in the browser. And modern neural networks can do a lot, if not everything. Do you want to separate different musical parties from each other? You are welcome. Remove the background from the image? You are welcome. Turn your photo into a hero of your favorite anime or cartoon using GAN? You are welcome. Nothing needs to be downloaded and everything is absolutely free.

This is truly one of the best resources for anyone, even a non-machine learning person, and I am very happy to share it with you.

Hugging Face - The AI community building the future.

How to start reading research papers

And the last section in this article is the most important. This is the answer to the question “how to start reading ML research articles?”. And if I had something to say, I would definitely write a separate article about it. But

The best way to start reading research papers is to start reading research papers.

It sounds silly, but reread this phrase a couple of times, and everything will fall into place. In fact, many questions can be answered in this way. The best way to learn to program is to program, the best way to learn to ride a bike is to ride a bike, and the best way to read research papers is to read research papers.

Of course, I won’t leave you only with this stupid advice. The website that will help you with this is called papers with code. For some reason, I found out about it very late, as if everyone was hiding it from me, and that’s why I’m sharing it with you.

Trending Papers - Hugging Face

Intuitive interface, extensive search capabilities for datasets and tasks, and the opportunity to see the best models — this site is a real find for any ML person.

And that’s it. But wait. I’m showing you six resources in two categories, but when it comes to the most important, I have only one? Yes, that’s right. That’s because it’s enough (you probably know about arXiv without me). The rest is up to you. There is a lot to tell about this site, but it is better that you find out it yourself.

If you think that research papers are only for the elite, then this is not true. If you don’t understand something for the first time, get confused or you’re just bored — that’s fine, no one gets it initially. If you have never read these articles before, pay attention to this awesome NVIDIA blog article, it can help you a lot:

How to Read Research Papers: A Pragmatic Approach for ML Practitioners | NVIDIA Technical Blog

Summary

That’s it! Good luck in exploring these resources, and if you know any more, please share them in the comments. At the very end, let me enumerate everything again in a more compact way:

1. Interestingness

• Spurious Correlations: to get an intuitive explanation of why correlation is not a dependency;
• Netron: to visualize neural networks architectures;
• Neural Network Playground: to play with dense neural networks directly in your browser.

2. Where to find models?

• At AI Hub, TensorFlow Hub, and Hugging Face.

3. How to start reading research papers

• Start with the NVIDIA blog post
• Read papers with arXiv and papers with code.

Thank you for reading!

• I hope these materials were useful to you. Follow me on Medium to get more articles like this.
• If you have any questions or comments, I will be glad to get any feedback. Ask me in the comments, or connect via LinkedIn or Twitter.
• To support me as a writer and to get access to thousands of other Medium articles, get Medium membership using my referral link (no extra charge for you).

Top 7 Machine Learning resources I wish I knew earlier was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.

You’ve probably seen a lot of articles with this title. Each person’s experience is special, and you can get a grain of useful (and at the same time unique) information from each personal history.

Nevertheless, most of the articles on this topic that I came across sounded a little strange to me: one of the resumes in such an article consisted of two pages and the second page was literally a list of scientific publications and patents (wow, impressive!).

2023 update.

A year later, some things have certainly changed here. Most of things in this article are still true, but I should also have said about the importance of a good questions, cover letter and of course referral if it’s possible, but this is a separate (and big) topic. I also think it’s better not to use the tabular resume format that I use here because apparently (but not certainly), it is worse recognized by the Application Tracking Systems and other software.

Remember that it is more a personal story than professional advice since I don’t have professional experience in the job hunting field. When writing your resume, review information from many sources, and it is best to ask for advice from experienced people from the industry. With this in mind hope you will enjoy the story and good luck in your job search!

TL; DR

How is my story different from all the previous ones? Well, if I don’t have the worst resume, then probably one of the worst, at least in two main fields:

• Education — I’m a Bachelor of Computer Science from a city you never knew about;
• Experience — I have almost no relevant work experience.

However, I was able to find a machine learning related job in Canada, although it was difficult and took a whole year.

Education

As I already said — I’m a Bachelor of Computer Science from an unknown university. Of course, by having a good education, you have a better chance of getting a job, at least because of the connections and networking you have access to. But if you were not lucky enough to study at a top university — prove that you deserve a job by other means.

Experience

I guess this is the most important point in your resume — where you worked before and how you can present the results of your work greatly affects your chances of getting a position. Nevertheless, everyone needs to start somewhere. This is the famous Permission Paradox or Experience Paradox: if all employers require employees with work experience, how to get this work experience?

I’m not really an expert in this matter, but that is what I’ve heard from many others: you can’t leave this field empty. It might be appropriate to list the university practice here and mention elaborate coursework or a corresponding degree project. But you can’t leave it empty. If things are really bad, you can look for an open-source project to contribute to or gain some volunteer experience.

Yes, Personal Projects (again)

Since my work experience is completely unimpressive, the data science/machine learning projects section is the most valuable and powerful part of my resume. And if you have a similar situation with work experience, you should have a similar situation with projects as well.

That particular section can let you get a job. It can tell a potential employer: “Hey, I don’t have any work experience, but I’m trying, and I’m trying hard!”.

You’ve probably heard this more than once, but if you don’t have any work experience, personal projects are vital to you. Make them as interesting and complex as possible, use modern technologies, try implementing something from scratch to get a deeper understanding of how a particular algorithm works, etc.

All this shows that you are interested, that you are ready to learn new things, ready to face difficulties and overcome them.
Are you really like that? Prove it!

Here are links to my projects. I don’t know how much it helped that I wrote blogs on Towards Data Science, but it helped me first of all. Only by explaining something to another person, you can fully understand it. So writing tutorials or a personal professional blog is a very good way to start a data science career.

• Predicting the number of dislikes on YouTube videos: part 1 and part 2;
• Automatic Video Editing;
• Real-time “me-not_me” Face Detector.

Your level or interests may differ significantly, but you need to find something that will inspire you.

If you don’t have any ideas for projects, you may not be ready to implement them yet. To think and see the world in machine learning terminology you need a pretty serious knowledge base. Try to read another book or take another course to digest this knowledge and reproduce your own idea.

If you think that you are ready, but there are still no ideas, look for them on the Internet. Often the quality of such articles leaves much to be desired (you are advised to classify the iris dataset OR to solve some computer vision problem, although these tasks are of completely different levels), but this may lead you to your own thoughts.

Other Sections

Skills

Honestly, I filled in this section in such a way to list as many keywords as possible so as not to be filtered out at the very beginning. As far as I know, keywords search programs are used when selecting resumes, especially for beginner-level positions (e.g. intern or junior), because HR specialists can’t process them manually.

Also, it is important to remember that specifying skills that you don’t actually have is a bad idea.

Certifications

Here I have listed the courses from which I got the main knowledge about data science and machine learning.

I also think it doesn’t make much sense to enumerate:

• individual courses because the list will be too long and detailed; instead, you should list your programs/certifications;
• entry-level courses (like Machine Learning by Stanford) because your potential employer might think that your knowledge/qualification is insufficient.

Full Resume

Putting all the parts together, I got the resume that brought me my first job in machine learning, my first job in Canada, and my first good job in general. Thank you, resume!

Conclusions and Final Thoughts

Finding a job wasn’t easy. It took me a whole year to get one interview invitation. To be honest, I do not know whether I was unlucky all this year, or, on the contrary, I got lucky once.

While I was looking for a job, I often changed my resume, adding and removing some parts, changing formulations, or reorganizing the order of the sections. I don’t know which of these transformations were really successful, but here’s what I can say for sure:

• Don’t complicate your resume.
  Don’t list all the projects in your life or all the courses you’ve taken. I think three is quite enough for an entry-level employee.
• Do not strive to make a one-page resume.
  This is my personal view and a piece of advice from several people I’ve talked to. All online services say that a beginner’s resume must be one-page, but you should not strive for it too much.
• Don’t be afraid to ask for advice.
  Do not hesitate to ask for advice — everyone started somewhere, and many (if not all) faced problems, so people usually treat this with understanding. Perhaps some friend or professor at your university can push you on a successful path. If you can’t ask for advice in person, use the Internet and the relevant sites/forums. Sometimes it’s just important to talk to someone when you feel stuck.

And finally: never give up. I sent more than a hundred job applications (and this is far from the limit) before I received a response. It’s hard to find a first job, extremely hard sometimes, especially for immigrants or people changing career paths. But the second job will be easier to find, and the third one will find you by itself.

So good luck. Perhaps you will receive the desired offer today! So make an effort to do it.

Thank you for reading!

• I hope these materials were useful to you. Follow me on Medium to get more articles like this.
• If you have any questions or comments, I will be glad to get any feedback. Ask me in the comments, or connect via LinkedIn or Twitter.
• To support me as a writer and to get access to thousands of other Medium articles, get Medium membership using my referral link (no extra charge for you).

The Resume with which I Got My First ML Internship in Canada was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.