<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Aymone Kouame on Medium]]></title>
        <description><![CDATA[Stories by Aymone Kouame on Medium]]></description>
        <link>https://medium.com/@aymone.jk?source=rss-efb1296f9860------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*c83m2fD4pahwQdd1pCDTGA.jpeg</url>
            <title>Stories by Aymone Kouame on Medium</title>
            <link>https://medium.com/@aymone.jk?source=rss-efb1296f9860------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 04:06:39 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@aymone.jk/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Tips to Write Efficient SQL Queries]]></title>
            <link>https://medium.com/@aymone.jk/tips-to-write-efficient-sql-queries-404ab40ad521?source=rss-efb1296f9860------2</link>
            <guid isPermaLink="false">https://medium.com/p/404ab40ad521</guid>
            <category><![CDATA[sql-query-optimization]]></category>
            <category><![CDATA[sql-query-performance]]></category>
            <category><![CDATA[sql]]></category>
            <dc:creator><![CDATA[Aymone Kouame]]></dc:creator>
            <pubDate>Tue, 15 Apr 2025 13:59:00 GMT</pubDate>
            <atom:updated>2025-04-15T14:10:12.893Z</atom:updated>
            <content:encoded><![CDATA[<p>SQL Best Practices</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*M6I6u2N6IX3bVNQa" /><figcaption>Photo by <a href="https://unsplash.com/@disruptxn?utm_source=medium&amp;utm_medium=referral">Desola Lanre-Ologun</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Mastering SQL sets you apart as a data scientist. However, not all SQL queries are created equal, even though they might produce the same result. A SQL query should be efficient and easy to read (I prefer using capital letters for the commands and proper indentations). Today, I will share some tips (Dos and Don&#39;ts) for writing efficient SQL queries, saving time and cost.</p><h3>DON’T</h3><ul><li><strong>Don’t query entire table</strong>s: Avoid running `SELECT * FROM table` or queries that require scanning entire tables. `SELECT * FROM table` is acceptable when using the WHERE clause to reduce the number of rows in a clustered table. In a non-clustered table, the entire table may still be scanned with a WHERE clause.</li><li><strong>Don’t repeatedly join the same tables</strong> and use the same subqueries.</li></ul><h3>DO:</h3><ul><li><strong>Select only the columns that you need</strong> in the SQL Query.</li><li><strong>Optimize table JOINS by starting with the largest table.</strong></li><li><strong>Reduce the amount of data processed <em>before </em>a JOIN</strong> by performing pre-aggregations (use a WITH statement, for example).</li><li><strong>Use the WHERE clause</strong> to filter data. Prioritize using BOOL, INT, FLOAT, or DATE columns in the WHERE clause. They are typically faster than operations on STRING or BYTE columns.</li><li><strong>Split complex queries into smaller ones</strong>. This allows for splitting complex (time and resource-consuming) computations into multiple smaller ones instead. You can use `UNION ALL` statements, for example. You should also leverage external and temporary tables. They can handle complex SQL queries while allowing for further data manipulation and filtering.</li><li><strong>Use clustered columns, if available. </strong>This will direct SQL to scan blocks of data rather than entire tables. Filtering or aggregating by the clustered columns makes computationally intensive queries more efficient and cost-effective.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=404ab40ad521" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Memory Management in Python]]></title>
            <link>https://medium.com/@aymone.jk/memory-management-in-python-37f577d4085e?source=rss-efb1296f9860------2</link>
            <guid isPermaLink="false">https://medium.com/p/37f577d4085e</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[rams]]></category>
            <category><![CDATA[memories]]></category>
            <dc:creator><![CDATA[Aymone Kouame]]></dc:creator>
            <pubDate>Wed, 02 Apr 2025 23:19:37 GMT</pubDate>
            <atom:updated>2025-04-02T23:19:37.462Z</atom:updated>
            <content:encoded><![CDATA[<h3>How to free up memory in your work environment + a bonus tip!</h3><p>If you work with huge datasets like me, you have probably encountered memory issues, which result in your development environment crashing or becoming unresponsive. As a data Scientist, I regularly collect, wrangle, analyze, and store huge datasets (typically from a Jupyter notebook, R Studio, or SAS Studio). I have found ways to perform memory-intensive tasks without problems.</p><p>Today, I will share a few tips to:<strong> (1) check your memory usage </strong>and<strong> (2) free up memory in Python</strong>. I will demonstrate the code from a Jupyter notebook, but you can apply these principles to any development environment and/or programming language. <strong>I will link my GitHub with the code at the end.</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*LTYt6M5SIuJKvhrW.png" /></figure><p><strong>Why is memory management important?</strong></p><p>Keeping track of memory usage can help you avoid crashing your environment, run your code faster and aid in analyzing large quantities of data. For example, if you try to create a dataframe that’s larger than the available memory, your environment will crash and/or your kernel will die.</p><p><strong>How to check memory usage?</strong></p><p>The memory used in your development environment includes built-in functions, objects, modules, variables, any code, variables and dataframes created. Python offers a variety of ways to check the total memory available and used. The ‘<em>psutil</em>’ package is a good one to use. To make things, simpler I have created a Python function called `check_memory`. You can specify the unit (bytes, GB, MB or TB) and the rounding decimals. The defaults are ‘GB‘ and .3 decimals. It also prints a timestamp.</p><pre>check_memory()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/352/0*pxpz_mdfLm_akOZu.png" /></figure><pre>check_memory(&#39;mb&#39;, round_to=2)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/353/0*bzozAnjntWjdXXHf.png" /></figure><p>To check the memory used by a dataframe ‘df‘, run:</p><pre># memory usage by column<br>df.memory_usage()</pre><pre># total memory usage by column<br>df.memory_usage().sum()</pre><p><strong>How to free up memory?</strong></p><p><em>(1) Remove un-needed objects (files, dataframes, variables, …) from the memory and disk</em></p><p>You can do it manually or programmatically with :</p><pre># Files<br>!rm filename ## or !del filename </pre><pre># other objects, for example a dataframe df<br>!del df</pre><p><strong>(2) Remove all variables from memory</strong>, except for those that are defined by the user in the configuration file.</p><ul><li><em>Manually</em></li></ul><p>In your development environment, there is typically a place to shut/reset and restart the environment. In Jupyter notebook, you can do so from the menu: `Kernel → ‘Reset`. Make sure you save the files you need beforehand.</p><ul><li><em>Programmatically</em></li></ul><pre>%reset -f</pre><p>If you are in the habit of saving intermediate results to a permanent storage like a cloud bucket, you can periodically perform these actions to free up memory for the next tasks.</p><p><strong>(2) BONUS TIP: ditch CSV or Excel files for storage.</strong></p><p>Instead, use `.parquet` or `.feather` files for storage. They offer much better compression methods. Better yet, if you have write access to a relational database, write your results as a table there if they will be used frequently for more data analysis.</p><p>Happy coding!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=37f577d4085e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Finding Baby Names Using Google BigQuery Public Datasets]]></title>
            <link>https://medium.com/@aymone.jk/finding-baby-names-using-google-bigquery-public-datasets-5f87c3a83820?source=rss-efb1296f9860------2</link>
            <guid isPermaLink="false">https://medium.com/p/5f87c3a83820</guid>
            <category><![CDATA[bigquery]]></category>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[baby-names]]></category>
            <dc:creator><![CDATA[Aymone Kouame]]></dc:creator>
            <pubDate>Fri, 07 Mar 2025 21:35:46 GMT</pubDate>
            <atom:updated>2025-03-07T21:38:49.925Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-nbMaVWXQjQxxkiB" /><figcaption>Photo by <a href="https://unsplash.com/@omarlopez1?utm_source=medium&amp;utm_medium=referral">Omar Lopez</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>Follow me on </strong><a href="https://datasciencestuffwithaymone.substack.com/"><strong>Substack</strong></a><strong>!</strong></p><h3>BigQuery Public Datasets?</h3><p>During my college days, I remember scouring the internet for good (and free) data to use for my projects. The ones I found were either very outdated or a pain to download and use. I wish I had known about Google BigQuery public datasets sooner.</p><p>BigQuery public datasets are available to the public for query. The first 1 TB per month is free, subject to query pricing details (you can read more <a href="https://cloud.google.com/bigquery/public-data/">here</a>). But if you find a dataset you like, you can query it once, save it on your computer, and use it as desired. You do not need to incur any cost for simple projects.</p><h3>Tutorial Objectives</h3><p>In this tutorial, I will:</p><ol><li>Show how to use the Python API Client to query data from BigQuery public datasets</li><li>Explore the most common and rarest baby names in the United States.</li></ol><p><strong>Note</strong>: I am coding in a Jupyter Notebook.</p><h3>Step 1: Install the ‘google.cloud’ module using ‘pip’</h3><p>In a Jupyter notebook cell, run the code below. You should need to run it only once.</p><pre>!pip install google-cloud-bigquery google-cloud-bigquery-storage</pre><h3>Step 2: Create a service account.</h3><p>This step is super easy. You can do it programmatically or from the GUI console.</p><ol><li>Go to the Google <a href="https://console.cloud.google.com/projectselector2/iam-admin/serviceaccounts">Service Account</a> Page.</li><li>Select an existing project or create a new one.</li><li>Select the email address of the service account.</li><li>Click the Keys tab.</li><li>Click the Add key drop-down menu, then select Create new key.</li><li>Select JSON as the Key type and click Create. A service account key file will be downloaded as a JSON file to your local computer.</li><li><strong>Optional</strong>: Copy/paste the JSON file to the same directory where your Jupyter Notebook is located.</li></ol><h3>Step 3: Set up for querying</h3><p>1. Import the modules we need.</p><pre>from google.cloud import bigquery<br>from google.oauth2.service_account import Credentials as creds</pre><ol><li>Set the name of the project and dataset.</li></ol><p>On the <a href="https://console.cloud.google.com/bigquery">Google Cloud Console</a>, you can navigate through the public datasets, view the schemas, and preview the values in the tables. After doing that, I want to play with the ‘usa_names’ dataset (see a description of the dataset <a href="https://console.cloud.google.com/marketplace/product/social-security-administration/us-names">here</a>) from the ‘bigquery-public-data’ project.</p><pre>project = &#39;bigquery-public-data&#39;<br>dataset = f&#39;{project}.usa_names&#39;<br>print(dataset) # returns &#39;bigquery-public-data.usa_names&#39;</pre><ol><li>Set the credentials</li></ol><p>Next, we will use the ‘google.oauth2’ and python bigquery client to set up the credentials that will allow us to access the datasets.</p><p>This is where we will use the path to our JSON file containing the service account credentials. Make sure you use the correct file path. Since I copied/pasted my JSON file to the same directory as my Jupyter Notebook, I only need to use the JSON filename.</p><pre>credentials = creds.from_service_account_file(&quot;service_account.json&quot;)<br><br>client = bigquery.Client(credentials= credentials)</pre><h3>Step 4: You can now query BigQuery public datasets</h3><p>Below, we are querying the number of unique names per year in the ‘usa_1910_current’ table.</p><p>As a recap, our project is ‘bigquery-public-data’, the dataset name is ‘usa_names’, and the table name is ‘usa_1910_current’. We need all three variables to query. We have already set up the “dataset” variable to include the project. We can simply use it in the query.</p><pre>query_job = client.query(f&#39;&#39;&#39;<br> SELECT COUNT(DISTINCT name) as n_names, year <br> FROM {dataset}.usa_1910_current<br> WHERE year &gt;= 2020<br> GROUP BY 2<br> ORDER BY year<br>&#39;&#39;&#39;)<br><br>usa_names_count_per_year = query_job.result().to_dataframe()<br>usa_names_count_per_year </pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/150/0*7-PMoV7-zUN3cW3w.png" /></figure><p><strong>Note</strong>: Another way to view the schema of the tables within a dataset is to query the ‘INFORMATION_SCHEMA’ as shown below:</p><pre># Query the schema of &#39;bigquery-public-data.usa_names&#39;<br>query_job = client.query(f&#39;&#39;&#39;<br>    SELECT * <br>    FROM {dataset}.INFORMATION_SCHEMA.COLUMNS &#39;&#39;&#39;)<br><br>gbq_public_dataset_schema = query_job.result().to_dataframe()</pre><h3>Putting it all together</h3><p>We can put steps 3 and 4 in a function to easily reuse it when needed.</p><pre>from google.cloud import bigquery<br>from google.oauth2.service_account import Credentials as creds<br>project = &#39;bigquery-public-data&#39;<br>dataset = f&#39;{project}.usa_names&#39;<br><br>def query_gbq(query, dataset = dataset):<br>    <br>    credentials =creds.from_service_account_file(&quot;service_account.json&quot;)<br>    <br>    config = bigquery.QueryJobConfig(default_dataset = dataset) <br>    client = bigquery.Client(default_load_job_config= config, credentials = credentials)<br>    <br>    query_job = client.query(query)  # API request<br>    df = query_job.result().to_dataframe()<br>    <br>    return df</pre><p>Using the function to query baby names in the USA on or after 2015</p><pre>%%time<br>query = f&#39;&#39;&#39;SELECT DISTINCT * <br>            FROM {dataset}.usa_1910_current <br>            WHERE year &gt;=2015 &#39;&#39;&#39;<br><br>usnames_from2015 = query_gbq(query)<br>usnames_from2015.head() #first 5 rows</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/298/0*2sOA7Doi4IaL11rx.png" /></figure><h3>Most Common Baby Names</h3><p>Let’s see what the most common names per gender have been since 2015 in the US.</p><pre>top_numbers = usnames_from2015[[&#39;year&#39;, &#39;number&#39;, &#39;gender&#39;]]\<br>    .groupby([&#39;year&#39;, &#39;gender&#39;], as_index = False).max()<br><br>top_names = usnames_from2015[[&#39;year&#39;, &#39;number&#39;, &#39;gender&#39;, &#39;name&#39;]]\<br>     .merge(top_numbers).sort_values([&#39;year&#39;, &#39;gender&#39;, &#39;number&#39;])<br><br>top_names.set_index([&#39;year&#39;, &#39;gender&#39;, &#39;name&#39;])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/240/0*ANn4mu2JkY3IMJF7.png" /></figure><p>In 2015, Sophia was the most common name for Females, and Noah was the most common name for Males. In 2015, Olivia was the most common name for Females, and Noah was still the most common name for Males.</p><h3>Rarest baby names</h3><pre>lowest_numbers = usnames_from2015[[&#39;year&#39;, &#39;number&#39;, &#39;gender&#39;]]\<br>   .groupby([&#39;year&#39;, &#39;gender&#39;], as_index = False).min()<br><br>rare_names = usnames_from2015[[&#39;year&#39;, &#39;number&#39;, &#39;gender&#39;, &#39;name&#39;]]\<br>    .merge(lowest_numbers).sort_values([&#39;year&#39;, &#39;gender&#39;, &#39;number&#39;])</pre><p>What were the rarest baby names after 2020 for Females that start with ‘D’?</p><pre>rare_names[(rare_names.year &gt;2020) <br>            &amp; (rare_names.gender == &#39;F&#39;) <br>            &amp; (rare_names.name.str.startswith(&#39;D&#39;))].name.unique()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/552/0*Knz8XUs16n41r5hw.png" /></figure><p>I kind of like ‘Della’. It is not a common name, but it is super cute and easy to spell. What do you think?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5f87c3a83820" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GET STARTED WITH NATURAL LANGUAGE PROCESSING]]></title>
            <link>https://medium.com/data-science/get-started-with-natural-language-processing-a-multi-label-text-classification-example-in-r-part-bb8f064be5e4?source=rss-efb1296f9860------2</link>
            <guid isPermaLink="false">https://medium.com/p/bb8f064be5e4</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[r-programming]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[supervised-learning]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Aymone Kouame]]></dc:creator>
            <pubDate>Fri, 17 Aug 2018 02:58:05 GMT</pubDate>
            <atom:updated>2018-08-17T16:13:40.498Z</atom:updated>
            <content:encoded><![CDATA[<h4>A MULTI-LABEL TEXT CLASSIFICATION EXAMPLE IN R (PART 1)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_J7MtFNBb9M0oZ6G_JmN1Q.jpeg" /></figure><blockquote><strong>Text classification is a type of Natural Language Processing (NLP).</strong> NLP can be simply defined as teaching an algorithm to read and analyze human (natural) languages just like a human would, but a lot faster, more accurately and on very large amounts of data.</blockquote><p>If you are an aspiring data scientist or analyst, you must know how to build a text classification model. From email classification to information discovery, the practical applications are endless. Multi-label text classification simply means classifying text data into more than two categories.<strong> </strong>Beginner data analysts, data analysts with no experience in NLP or other data scientists who are curious to see other ways of approaching topic modeling will find this blog interesting.</p><p><strong>You need to be somewhat familiar with R and RStudio </strong>to follow along. I will use the Structural Topic Model (STM) package in R, which is an unsupervised clustering package that uses document-level covariates. I have modeled many text classifiers with STM and it gives fantastic results with minimal training. You can read more about STM <a href="https://cran.r-project.org/web/packages/stm/index.html">here</a>.</p><h4>Let’s get coding!</h4><p><strong>Step 1: Get Data. </strong>For this example, I will be using .txt documents which are news articles that I scraped from open sources using Python’s BeautifulSoup (yes, I like Python better when it comes to scraping data).</p><p>If you do not know how to do that, don’t worry! You can copy-paste free text data online and save it as .txt files (make sure to read copyrights. Use only web data that is free to use). This is what my data looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/648/1*mEUX85sInqc3G677hiHkzQ.png" /></figure><p>I saved all of my .txt files in a folder named <em>STM_trainingdata,</em> which is in my current directory.</p><p><strong>Step 2: Ingest Data in RStudio. </strong>A rule of thumb is to write functions for tasks that you will repeat often. To ingest my data into RStudio, I wrote a simple function. Depending on the project that you are working on and the desired output, your function can have different inputs/outputs. Here is what mine looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*IOy_I-FBBFLU1pzn6RBNoA.png" /></figure><p>Now, call the function as shown below and you should get “SUCCESS!”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/495/1*HnJfwYxUoUhy91QsB6rdvg.png" /></figure><p>Your data should look like this in RStudio:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/442/1*xERpb3uEpq9l3JvLDsmz5Q.png" /></figure><p>Now, every time I need to add/modify training data in my folder, all I have to do to update my data variable in RStudio is to call <em>ReadTxt()</em>.</p><p><strong>Step 3: Pre-Process Your Data. </strong>Data pre-processing is one of the most important steps in any Machine Learning process. Poorly cleansed and/or structured data will yield poor results. You can use various statistical and visualization methods during data exploration to ensure that your data is ready.</p><p>Because our data sample is very small, I wrote a basic function to cleanse it. You can read inside the function the specific cleansing steps. Also, as you will see in part 2, STM offers additional data pruning options; so I am just going to focus on the basics here.</p><p>A very important steps in NLP is to remove stopwords.<strong> </strong>Stopwords are words that are common in your corpus and therefore have no weight in determining the topics discussed within a text (they are just noise in your data). Some common stopwords are articles, pronouns, interjections, etc.</p><p>The following function is very customizable. Depending on your training data, you may have more stopwords to remove. You can add them directly to the <em>rem</em> vector. You may also not need to remove urls, for example.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/1*YCAEWAgCLRAzblB4b82oKQ.png" /></figure><p>When writing functions, it is best practice to use descriptive names and add some explanations about the inputs, outputs and processes.</p><p>Next, call the function as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/359/1*c9xwSphSk0MKJpqJBKFhFw.png" /></figure><p>Your clean data should look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/629/1*LqjWO9yPFqD1_pBPfFkulw.png" /></figure><p>At this point, you should have 4 items in your RStudio’s global environment. See below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/593/1*v-SEOjx3R0me3NBNBDuivA.png" /></figure><p>That’s it for Part 1! In Part 2, we will get into the interesting part: training and testing our STM model as well as some visualizations. Stay tuned!</p><p><em>If you like this post, please give it a clap and feel free to add me on </em><a href="https://www.linkedin.com/in/aymonekouame/"><em>LinkedIn</em></a><em> and T</em><a href="https://twitter.com/Aymone_K"><em>witter</em></a><em>!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb8f064be5e4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/get-started-with-natural-language-processing-a-multi-label-text-classification-example-in-r-part-bb8f064be5e4">GET STARTED WITH NATURAL LANGUAGE PROCESSING</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>