<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jeferson Marques on Medium]]></title>
        <description><![CDATA[Stories by Jeferson Marques on Medium]]></description>
        <link>https://medium.com/@marquesjef?source=rss-5a8178286789------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*6hxLjGtOHLkYMNJs6tuG3A.jpeg</url>
            <title>Stories by Jeferson Marques on Medium</title>
            <link>https://medium.com/@marquesjef?source=rss-5a8178286789------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 14:44:39 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@marquesjef/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Unveiling Statistics: First Reading]]></title>
            <link>https://medium.com/@marquesjef/unveiling-statistics-first-reading-396201d2dd34?source=rss-5a8178286789------2</link>
            <guid isPermaLink="false">https://medium.com/p/396201d2dd34</guid>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[book-review]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[reading]]></category>
            <dc:creator><![CDATA[Jeferson Marques]]></dc:creator>
            <pubDate>Fri, 26 Apr 2024 14:48:45 GMT</pubDate>
            <atom:updated>2024-04-26T14:48:45.372Z</atom:updated>
            <content:encoded><![CDATA[<p>As I sit here, freshly closing the pages of “Naked Statistics” by Charles Wheelan, I am compelled to share the profound impact this book has had on my understanding of statistics and its relevance in the modern world. Wheelan masterfully guides readers through the labyrinth of statistical theory, unraveling complex concepts with clarity and wit.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tTHJevac6IARaTB9Hq5lFQ@2x.jpeg" /></figure><p>From the outset, Wheelan sets the stage by emphasizing the ubiquity of statistics in our daily lives. Whether it’s in political polling, sports analytics, or medical research, statistics play a crucial role in shaping our understanding of the world around us. With this foundation laid, Wheelan embarks on a journey through the essentials of statistical analysis.</p><p>One of the book’s greatest strengths lies in its ability to make statistics accessible to readers of all backgrounds. Wheelan eschews jargon and technical language in favor of clear, straightforward explanations. Concepts like probability distributions, hypothesis testing, and regression analysis are demystified through real-world examples that resonate with readers. Whether it’s explaining the concept of standard deviation using the height of NBA players or illustrating the pitfalls of p-hacking with medical studies, Wheelan’s examples are both informative and entertaining.</p><p>But “Naked Statistics” is more than just a primer on statistical theory; it is a call to arms for critical thinking. Throughout the book, Wheelan emphasizes the importance of questioning assumptions, scrutinizing methodologies, and distinguishing between correlation and causation. In an age of misinformation and fake news, these skills are more essential than ever. By arming readers with the tools of statistical literacy, Wheelan empowers us to navigate the sea of data that inundates us daily and make informed decisions based on evidence, not conjecture.</p><p>As an analyst in the field of Business Intelligence, the lessons learned from “Naked Statistics” have proven invaluable. Armed with a deeper understanding of statistical principles, I am better equipped to extract meaningful insights from the data at my disposal. Whether it’s conducting A/B tests to optimize website performance or using regression analysis to forecast sales trends, the concepts learned from Wheelan’s book form the bedrock of my analytical toolkit.</p><p>In conclusion, “Naked Statistics” is a must-read for anyone seeking to demystify the world of statistics and harness its power in their personal and professional lives. Through its engaging prose, real-world examples, and emphasis on critical thinking, Wheelan’s book equips readers with the knowledge and skills needed to navigate the complexities of our data-driven world. So if you’ve ever felt intimidated by statistics, fear not – with “Naked Statistics” as your guide, the world of data analysis is within reach.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=396201d2dd34" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Predicting with Regression in Python: A Simple Guide]]></title>
            <link>https://medium.com/@marquesjef/predicting-with-regression-in-python-a-simple-guide-4ff2d340f502?source=rss-5a8178286789------2</link>
            <guid isPermaLink="false">https://medium.com/p/4ff2d340f502</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[predictions]]></category>
            <category><![CDATA[regression]]></category>
            <dc:creator><![CDATA[Jeferson Marques]]></dc:creator>
            <pubDate>Mon, 11 Mar 2024 13:26:05 GMT</pubDate>
            <atom:updated>2024-03-11T13:26:05.236Z</atom:updated>
            <content:encoded><![CDATA[<p>Imagine you have a Spotify playlist full of rock songs. Each song has different features like duration, loudness, tempo, and more. It would be interesting to predict a song’s popularity based on these features. I will do it using regression.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I9r7tMDRimLlMuojLi2j1Q.jpeg" /></figure><h4>What is Regression?</h4><p>Regression is a statistical method that helps us understand the relationship between variables. In our case, we want to understand how song features relate to their popularity.</p><h4>Getting Started with Python</h4><p>Python is a great language for data analysis. We’ll use two key libraries: pandas for data handling and sklearn for regression.</p><pre>import pandas as pd<br>from sklearn.model_selection import train_test_split<br>from sklearn.linear_model import LinearRegression</pre><h4>Loading the Data</h4><p>Let’s assume our rock_songs.csv file has the following data for five songs:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fsheetsu.com%2Ftables%2F4cd857241c&amp;dntp=1&amp;display_name=Sheetsu&amp;url=https%3A%2F%2Fsheetsu.com%2Ftables%2F4cd857241c&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=sheetsu" width="700" height="307" frameborder="0" scrolling="no"><a href="https://medium.com/media/735e7e4060f4fdb34cf42a4a19788d45/href">https://medium.com/media/735e7e4060f4fdb34cf42a4a19788d45/href</a></iframe><pre>df = pd.read_csv(&#39;rock_songs.csv&#39;)</pre><h4>Preparing the Data</h4><p>We’ll split our data into two parts: features (X) and target (y). The target is what we want to predict - song popularity.</p><pre>X = df.drop(&#39;popularity&#39;, axis=1)<br>y = df[&#39;popularity&#39;]</pre><p>Next, we split our data into training and test sets.</p><pre>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)</pre><h4>Building the Regression Model</h4><p>Now, we’re ready to create our regression model and fit it to our training data.</p><pre>model = LinearRegression()<br>model.fit(X_train, y_train)</pre><p>After running our regression model, let’s say we get the following coefficients for each feature:</p><ul><li>Duration: 0.05</li><li>Loudness: -2</li><li>Tempo: 0.5</li></ul><p>This means that for every second increase in duration, the popularity increases by 0.05. For every dB increase in loudness, the popularity decreases by 2. And for every BPM increase in tempo, the popularity increases by 0.5.</p><h4>Making Predictions</h4><p>With our model trained, we can predict the popularity of songs in our test set.</p><pre>predictions = model.predict(X_test)</pre><p>Now, let’s predict the popularity of a new song F with a duration of 230 seconds, loudness of -5 dB, and tempo of 120 BPM.</p><pre>new_song = pd.DataFrame({&#39;Duration&#39;: [230], &#39;Loudness&#39;: [-5], &#39;Tempo&#39;: [120]})<br>popularity = model.predict(new_song)<br>print(f&#39;The predicted popularity for song F is {popularity[0]}&#39;)</pre><p>Let’s say the output of this code is 72.5. This means that based on our model, we predict that song F would have a popularity score of 72.5.</p><h4>Conclusion</h4><p>Using regression for predictions opens up a world of possibilities. It allows us to understand and quantify the relationships between different variables. In our case, we were able to see how song features such as duration, loudness, and tempo can influence a song’s popularity.</p><p>The beauty of regression lies in its versatility and applicability across various domains. From predicting stock prices to forecasting weather, from assessing risk in insurance to optimizing marketing strategies, regression serves as a powerful tool.</p><p>Moreover, with Python and its libraries, implementing regression models has become more accessible than ever. It empowers us to make data-driven decisions and predictions, which are crucial in today’s data-centric world.</p><p>So, whether you’re a music enthusiast trying to understand what makes a song popular, a business leader making strategic decisions, or a budding data scientist exploring different statistical methods, regression can provide valuable insights and predictions for your goal.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ff2d340f502" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring Python Syntax: Understanding Variables, Keywords, Operators, Expressions, Functions…]]></title>
            <link>https://medium.com/@marquesjef/exploring-python-syntax-understanding-variables-keywords-operators-expressions-functions-and-34678d5d97f0?source=rss-5a8178286789------2</link>
            <guid isPermaLink="false">https://medium.com/p/34678d5d97f0</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[python-programming]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Jeferson Marques]]></dc:creator>
            <pubDate>Mon, 26 Feb 2024 20:05:47 GMT</pubDate>
            <atom:updated>2024-02-26T20:23:52.838Z</atom:updated>
            <content:encoded><![CDATA[<h3>Exploring Python Syntax: Understanding Variables, Keywords, Operators, Expressions, Functions, Conditional Statements and Object-Oriented Programming</h3><p>As someone deeply passionate about programming and fascinated by the elegance of Python syntax, I find joy in delving into its various components and understanding how they work together to create powerful and expressive code. In this article, I’ll take you on a journey through Python syntax, highlighting the differences between variables, keywords, operators, expressions, functions, conditional statements and object-oriented programming (OOP) concepts..</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/826/1*XMnr7IGqr-E0lF4v12UEaQ.png" /></figure><h4>Variables:</h4><p>Variables in Python serve as containers for storing data values. They can hold different types of data, such as numbers, strings, or even more complex objects. Declaring a variable in Python is straightforward — simply choose a name for the variable and assign a value to it using the equal sign (=). For example:</p><pre>song_title = &quot;Stairway to Heaven&quot;</pre><h4>Keywords:</h4><p>Keywords, also known as reserved words, are predefined words in Python that have special meanings and purposes. These words cannot be used as variable names because they are reserved for specific tasks within the language. Examples of Python keywords include if, else, for, while, def, return, and import.</p><h4>Operators:</h4><p>Operators in Python are symbols that perform operations on variables and values. Python supports various types of operators, including arithmetic operators (+, -, *, /), comparison operators (==, !=, &lt;, &gt;), logical operators (and, or, not), and assignment operators (=, +=, -=, *=). Here’s an example of using operators to manipulate data:</p><pre>total_streams = rock_album1_streams + rock_album2_streams</pre><h4>Expressions:</h4><p>Expressions in Python are combinations of variables, values, and operators that are evaluated to produce a result. Expressions can be simple, such as 2 + 3, or more complex, involving multiple variables and operators. Python evaluates expressions according to operator precedence and associativity rules.</p><h4>Functions:</h4><p>Functions in Python are blocks of reusable code that perform a specific task. They help in organizing code, improving readability, and promoting code reuse. Functions can take input parameters, perform operations, and return results. Defining a function in Python involves using the def keyword followed by the function name and parameters. Here&#39;s an example:</p><pre>def calculate_total_streams(album1_streams, album2_streams):<br>    total_streams = album1_streams + album2_streams<br>    return total_streams</pre><h4>Conditional Statements:</h4><p>Conditional statements in Python allow you to execute different blocks of code based on certain conditions. The if, elif (else if), and else keywords are used to create conditional statements. These statements help in making decisions and controlling the flow of the program. Here&#39;s an example of a conditional statement:</p><pre>if total_streams &gt; 1000000:<br>    print(&quot;This artist is rocking the charts!&quot;)<br>elif total_streams &gt; 500000:<br>    print(&quot;Solid performance on the streaming platforms.&quot;)<br>else:<br>    print(&quot;Time to promote the artist for more streams.&quot;)</pre><h4>Object-Oriented Programming (OOP):</h4><p>Object-oriented programming is a programming paradigm that uses objects and classes to structure code. In Python, everything is an object, and classes are used to create new types of objects. A class defines the properties (attributes) and behaviors (methods) of objects. Here’s an example of defining a class in Python:</p><pre>class Song:<br>    def __init__(self, title, artist):<br>        # Constructor method (__init__) initializes the object with attributes<br>        self.title = title  # Attribute to store the title of the song<br>        self.artist = artist  # Attribute to store the artist of the song<br><br>    def play(self):<br>        # Method to play the song<br>        print(&quot;Playing&quot;, self.title, &quot;by&quot;, self.artist)</pre><h4>Key takeaway:</h4><p>In conclusion, Python syntax offers a clear and concise way to express ideas and solve problems in programming, including object-oriented programming concepts. Its versatility and readability make it a favorite among developers worldwide. As Tim Peters famously said in “The Zen of Python”:</p><p>Beautiful is better than ugly.</p><p>Explicit is better than implicit.</p><p>Simple is better than complex.</p><p>Complex is better than complicated.</p><p>Flat is better than nested.</p><p>Sparse is better than dense.</p><p>Readability counts.</p><p>Special cases aren’t special enough to break the rules.</p><p>Although practicality beats purity.</p><p>Errors should never pass silently.</p><p>Unless explicitly silenced.</p><p>In the face of ambiguity, refuse the temptation to guess.</p><p>There should be one — and preferably only one — obvious way to do it.</p><p>Although that way may not be obvious at first unless you’re Dutch.</p><p>Now is better than never.</p><p>Although never is often better than *right* now.</p><p>If the implementation is hard to explain, it’s a bad idea.</p><p>If the implementation is easy to explain, it may be a good idea.</p><p>Namespaces are one honking great idea — let’s do more of those!</p><p>In essence, Python’s principles highlight the beauty and clarity of its syntax, offering a seamless experience for programmers of all levels. As we navigate the intricacies of coding, let’s aim for clarity and precision in our Python projects.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=34678d5d97f0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring Common Table Expressions (CTEs) and Subqueries in SQL: A Comparative Analysis]]></title>
            <link>https://blog.devgenius.io/exploring-common-table-expressions-ctes-and-subqueries-in-sql-a-comparative-analysis-19a05be7c8df?source=rss-5a8178286789------2</link>
            <guid isPermaLink="false">https://medium.com/p/19a05be7c8df</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[sql]]></category>
            <dc:creator><![CDATA[Jeferson Marques]]></dc:creator>
            <pubDate>Thu, 15 Feb 2024 13:04:17 GMT</pubDate>
            <atom:updated>2024-02-16T17:08:43.926Z</atom:updated>
            <content:encoded><![CDATA[<p>As a data enthusiast and SQL aficionado, I’ve often found myself immersed in the world of database querying, constantly exploring different techniques to extract insights from data efficiently. In this article, I’ll delve into two powerful tools in SQL: Common Table Expressions (CTEs) and Subqueries, examining their strengths, differences, and best use cases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/765/1*qNjJgrDneVwD-L5QsMlZvA.png" /></figure><h4>Understanding Common Table Expressions (CTEs):</h4><p>CTEs, also known as “WITH clauses,” provide a concise and readable way to define temporary result sets within a SQL statement. They offer a structured approach to break down complex queries into more manageable parts, enhancing code readability and maintainability.</p><p>One of the key advantages of CTEs is their ability to reuse defined queries within the same statement, reducing redundancy and improving query performance. By defining a CTE at the beginning of a query, you can reference it multiple times throughout the rest of the statement, promoting code reuse and simplifying complex logic.</p><p>Let’s consider an example in the context of a music application database. Suppose we want to identify artists with more than three albums. Using a CTE, we can define a temporary result set of popular artists and then join it with the main artists table to retrieve relevant information:</p><pre>WITH PopularArtists AS (<br>    SELECT artist_id, COUNT(*) AS num_albums<br>    FROM albums<br>    GROUP BY artist_id<br>    HAVING COUNT(*) &gt; 3<br>)<br><br>SELECT artists.artist_id, artists.artist_name, PopularArtists.num_albums<br>FROM artists<br>INNER JOIN PopularArtists ON artists.artist_id = PopularArtists.artist_id;</pre><h4>Exploring Subqueries:</h4><p>Subqueries, on the other hand, are nested SQL queries embedded within a larger query. They provide a powerful mechanism to filter or manipulate data based on the results of another query. While subqueries are versatile and widely supported across different SQL implementations, they can sometimes lead to complex and less readable code, especially in scenarios involving multiple levels of nesting.</p><p>Continuing with our music application database example, let’s rewrite the previous query using a subquery to identify artists with more than three albums:</p><pre>SELECT artist_id, artist_name<br>FROM artists<br>WHERE artist_id IN (<br>    SELECT artist_id<br>    FROM albums<br>    GROUP BY artist_id<br>    HAVING COUNT(*) &gt; 3<br>);</pre><h4>Comparing CTEs and Subqueries:</h4><p>Both CTEs and subqueries serve similar purposes, but they have distinct characteristics that make them suitable for different scenarios. Here’s a brief comparison:</p><ul><li><strong>Code Reusability:</strong> CTEs excel in promoting code reuse by allowing defined queries to be referenced multiple times within a statement. Subqueries, while capable, may require duplication of logic when used in multiple parts of a query;</li><li><strong>Readability and Maintainability:</strong> CTEs often result in more readable and maintainable code, as they allow for the modularization of complex logic. Subqueries, while powerful, can sometimes lead to cluttered and harder-to-understand queries, especially in deeply nested scenarios;</li><li><strong>Performance:</strong> Both CTEs and subqueries can be optimized by the database engine, but performance may vary depending on the specific implementation and query complexity. In some cases, CTEs may offer performance benefits, particularly when used to optimize recursive queries or complex joins.</li></ul><h4>Key takeaways:</h4><p>In conclusion, both Common Table Expressions (CTEs) and Subqueries are valuable tools in the SQL developer’s arsenal, offering different approaches to achieving similar outcomes. While CTEs excel in readability, maintainability, and code reuse, subqueries provide versatility and flexibility in crafting intricate queries. Understanding the strengths and nuances of each approach is crucial for leveraging them effectively in different scenarios, ultimately empowering developers to extract meaningful insights from data with efficiency and clarity.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=19a05be7c8df" width="1" height="1" alt=""><hr><p><a href="https://blog.devgenius.io/exploring-common-table-expressions-ctes-and-subqueries-in-sql-a-comparative-analysis-19a05be7c8df">Exploring Common Table Expressions (CTEs) and Subqueries in SQL: A Comparative Analysis</a> was originally published in <a href="https://blog.devgenius.io/">Dev Genius</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Resolving Date Hierarchy Issues in Power BI with MySQL Data]]></title>
            <link>https://medium.com/@marquesjef/resolving-date-hierarchy-issues-in-power-bi-with-mysql-data-c9ca135d042f?source=rss-5a8178286789------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9ca135d042f</guid>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[power-bi]]></category>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[etl]]></category>
            <dc:creator><![CDATA[Jeferson Marques]]></dc:creator>
            <pubDate>Wed, 24 Jan 2024 13:10:39 GMT</pubDate>
            <atom:updated>2024-01-24T13:10:39.983Z</atom:updated>
            <content:encoded><![CDATA[<p>Power BI is a powerful tool for data visualization and analysis, but users may encounter challenges when dealing with date hierarchies. In this article, we will explore a common issue involving date hierarchy in a table loaded from MySQL and the step-by-step resolution of this problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/756/1*6XOGZwGKgCKGatPQBp5NkA.png" /></figure><h3>The Initial Problem</h3><p>When loading data from MySQL into Power BI, I identified an issue with the date hierarchy. The created_at column, of date type, was not being recognized correctly by Power BI to automatically create day, month, quarter, and year hierarchies. This impacted my ability to analyze and visualize data over time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/915/1*NktLokh2WySTh36InHbSTQ.png" /><figcaption>How the problem is and how it should be.</figcaption></figure><p>Below is an image depicting the current state of our data represented in a bar chart:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/517/1*ufEBdfbxquAkeqVlz9TdAQ.png" /><figcaption>Current situation.</figcaption></figure><h3>Initial Attempts to Solve</h3><p>The original MySQL query extracted only active tokens from the token_histories table, which stored the history of token statuses. The original query looked like this:</p><pre>SELECT th.*<br>FROM token_histories th<br>INNER JOIN (<br>    SELECT token_id, MAX(created_at) AS max_created_at<br>    FROM token_histories<br>    GROUP BY token_id<br>) latest ON th.token_id = latest.token_id AND th.created_at = latest.max_created_at<br>WHERE th.token_status_id NOT IN (3, 4);</pre><h3>Enhancing Date Hierarchy</h3><p>To address the hierarchy issue, I decided to add new columns to the MySQL query. Initially, I added columns for the year and month:</p><pre>SELECT <br>    th.*,<br>    YEAR(th.created_at) AS year,<br>    MONTH(th.created_at) AS month<br>FROM token_histories th<br>INNER JOIN (<br>    SELECT token_id, MAX(created_at) AS max_created_at<br>    FROM token_histories<br>    GROUP BY token_id<br>) latest ON th.token_id = latest.token_id AND th.created_at = latest.max_created_at<br>WHERE th.token_status_id NOT IN (3, 4);</pre><p>After incorporating the new columns into the Power Query, the data table within Power BI underwent a transformation. The year and month columns provided a clearer temporal representation of the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/479/1*IerYp5dJH0ZOtup5DlG5BQ.png" /><figcaption>New columns in Power Query.</figcaption></figure><p>In the Power BI visualization, the addition of these columns allowed for a more granular analysis of token histories. The enhanced date hierarchy improved the accuracy of trend analysis and provided a solid foundation for creating insightful visualizations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/529/1*Xte6iPjlxErbjrD2igxtUw.png" /><figcaption>New bar chart with the new month column.</figcaption></figure><p>However, the chart did not effectively convey the intended information because the numeric representation of months on the X-axis lacked clarity. It was challenging for viewers to discern that the numerical values corresponded to months, and the distinction that Month 1 belonged to 2024, while the other months were associated with 2023, was not clearly evident.</p><p>To improve it, I transformed the month into its full name using the MONTHNAME function:</p><pre>SELECT <br>    th.*,<br>    YEAR(th.created_at) AS year,<br>    MONTHNAME(th.created_at) AS month_name<br>FROM token_histories th<br>INNER JOIN (<br>    SELECT token_id, MAX(created_at) AS max_created_at<br>    FROM token_histories<br>    GROUP BY token_id<br>) latest ON th.token_id = latest.token_id AND th.created_at = latest.max_created_at<br>WHERE th.token_status_id NOT IN (3, 4);</pre><p>The chart’s X-axis still lacked the correct chronological order, and currently, it only maintains an alphabetical order, either ascending or descending.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/529/1*WhS5B8qsM2Om0NtqI_XSQw.png" /><figcaption>X-axis in the alphabetic order.</figcaption></figure><h3>Ordering the Months and Adding Concatenated Columns</h3><p>To address this, I added the month’s numerical representation before each month using the CONCAT and LPAD functions:</p><pre>SELECT <br>    th.*,<br>    YEAR(th.created_at) AS year,<br>    CONCAT(LPAD(MONTH(th.created_at), 2, &#39;0&#39;), &#39; - &#39;, MONTHNAME(th.created_at)) AS formatted_month<br>FROM token_histories th<br>INNER JOIN (<br>    SELECT token_id, MAX(created_at) AS max_created_at<br>    FROM token_histories<br>    GROUP BY token_id<br>) latest ON th.token_id = latest.token_id AND th.created_at = latest.max_created_at<br>WHERE th.token_status_id NOT IN (3, 4);</pre><p>The LPAD function plays a crucial role in formatting the month numbers. It ensures that the month values are consistently represented with two digits by left-padding with zeros, addressing potential issues with sorting and display. The resulting format, like &quot;01 - January,&quot; is more suitable for maintaining a chronological order and improving visual clarity in representations, especially when creating charts or reports in tools like Power BI.</p><p>This application of LPAD is particularly beneficial in scenarios where numeric representations of months are required to follow a standardized two-digit format, ensuring a proper chronological sequence and facilitating clear data visualization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/961/1*_dQxL1YujsmoMqKnrlIPoA.png" /></figure><p>Despite the improvements made to the new query, it still falls short in accurately representing the data reality. The current visualization does not differentiate between the years, as months from June to December are associated with 2023, while January is attributed to 2024. This lack of year distinction can lead to potential misinterpretations in data analysis, particularly in scenarios where the temporal context is crucial. Further refinements may be necessary to address this limitation and provide a more accurate depiction of the data timeline.</p><p>To further enhance the visual representation, a new column was added by concatenating the month number with the year:</p><pre>SELECT <br>    th.*,<br>    YEAR(th.created_at) AS year,<br>    CONCAT(LPAD(MONTH(th.created_at), 2, &#39;0&#39;), &#39; - &#39;, MONTHNAME(th.created_at)) AS formatted_month,<br>    CONCAT(YEAR(th.created_at), &#39;-&#39;, LPAD(MONTH(th.created_at), 2, &#39;0&#39;)) AS year_month_concatenated<br>FROM token_histories th<br>INNER JOIN (<br>    SELECT token_id, MAX(created_at) AS max_created_at<br>    FROM token_histories<br>    GROUP BY token_id<br>) latest ON th.token_id = latest.token_id AND th.created_at = latest.max_created_at<br>WHERE th.token_status_id NOT IN (3, 4);</pre><h3>Conclusion</h3><p>The implemented refinements in the query have resulted in a more robust data visualization, adhering to best practices in data visualization by accurately reflecting chronological order and providing a clearer representation of the temporal context in the generated chart.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/649/1*lVC9gXLcsyjIOYzcGEtCCg.png" /><figcaption>The chart in the chronological order.</figcaption></figure><p>By following these steps, I successfully resolved the date hierarchy issue and improved the visual representation of the data in Power BI. This detailed approach ensures a clearer understanding of the process for addressing similar challenges in the future.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9ca135d042f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Visualization: Crafting Stories with Data]]></title>
            <link>https://medium.com/@marquesjef/data-visualization-crafting-stories-with-data-12611cb0b3b?source=rss-5a8178286789------2</link>
            <guid isPermaLink="false">https://medium.com/p/12611cb0b3b</guid>
            <category><![CDATA[storytelling]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[power-bi]]></category>
            <category><![CDATA[tableau]]></category>
            <category><![CDATA[business-intelligence]]></category>
            <dc:creator><![CDATA[Jeferson Marques]]></dc:creator>
            <pubDate>Sat, 23 Sep 2023 17:47:37 GMT</pubDate>
            <atom:updated>2023-09-23T17:47:37.468Z</atom:updated>
            <content:encoded><![CDATA[<p>In today’s data-driven world, the ability to effectively convey insights from complex datasets is a highly sought-after skill. Data visualization serves as a powerful tool for transforming raw numbers into compelling narratives. It allows us to tell stories with data, making information more accessible, engaging, and actionable.</p><p><strong>Use Data to Develop Stories</strong></p><p>Data alone can be overwhelming and lacks context. To harness its true potential, we must use data to develop narratives that resonate with our audience. This process involves understanding the data’s nuances, identifying trends, outliers, and correlations, and crafting a storyline that connects the dots. Data-driven storytelling begins with a clear understanding of the message we want to convey and the questions we aim to answer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/804/1*JVhF8eg7688yAxYqJf0DcQ.png" /></figure><p><strong>Use Tableau/Power BI Dashboards</strong></p><p>Tableau and Power BI are two leading tools that empower data professionals to create interactive and visually appealing dashboards. These platforms enable us to transform data into actionable insights by designing dynamic and customizable visualizations. With drag-and-drop simplicity, we can build charts, graphs, maps, and tables that convey complex information at a glance.</p><p>Dashboards offer interactivity, allowing users to explore data from different angles, drill down into specific details, and gain a deeper understanding of trends and patterns. By using these tools effectively, we can enhance data storytelling by providing a more engaging and interactive experience.</p><p><strong>Sharing Data Stories</strong></p><p>Once we’ve crafted compelling data stories, the next crucial step is sharing them with the intended audience. Effective data communication involves choosing the right medium and format for dissemination. Whether it’s through reports, presentations, or online dashboards, the goal is to make data stories accessible and understandable.</p><p>Furthermore, data stories should be tailored to the audience’s level of expertise. Executives may require high-level summaries, while analysts may appreciate in-depth insights. Visualization should be clear and concise, with a focus on key takeaways.</p><p>In conclusion, data visualization is more than just creating charts and graphs; it’s about crafting stories that resonate and drive action. By using data to develop narratives, leveraging Tableau/Power BI dashboards for interactive experiences, and sharing data stories effectively, we can unlock the true power of data and make informed decisions that impact our organizations and the world. Data visualization is not just a tool; it’s a language that transcends borders and speaks to the human desire for understanding and insight in an increasingly data-rich landscape.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=12611cb0b3b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Leveraging Subqueries in SQL: Enhancing Data Aggregation with CASE and HAVING]]></title>
            <link>https://medium.com/@marquesjef/leveraging-subqueries-in-sql-enhancing-data-aggregation-with-case-and-having-bbfc9aaeda2?source=rss-5a8178286789------2</link>
            <guid isPermaLink="false">https://medium.com/p/bbfc9aaeda2</guid>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Jeferson Marques]]></dc:creator>
            <pubDate>Mon, 04 Sep 2023 22:23:04 GMT</pubDate>
            <atom:updated>2023-09-05T00:25:56.484Z</atom:updated>
            <content:encoded><![CDATA[<p>Structured Query Language (SQL) is the backbone of data manipulation and retrieval in relational databases. Subqueries, often referred to as nested queries, are a fundamental SQL concept that allows for versatile and intricate data retrieval. In this article, we will delve into the world of subqueries, exploring their purpose, best practices, and how to utilize subqueries in combination with the CASE and HAVING functions to perform advanced data aggregation. We will illustrate these concepts with practical examples using the Spotify dataset available on Kaggle, showcasing the power of subqueries, as well as functions like AVG, COUNT, and JOIN.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/820/1*_joyD7Dg6IiuYaY-WdPsTQ.png" /></figure><p><strong>Understanding Subqueries</strong></p><p>A subquery is a SQL query nested within another query, allowing you to retrieve data from one or more tables based on the result of an inner query. Subqueries can be used in various SQL statements, including SELECT, INSERT, UPDATE, and DELETE, but they are most commonly found in the WHERE clause of a SELECT statement.</p><p><strong>Syntax of a Basic Subquery:</strong></p><pre>SELECT column1, column2<br>FROM table1<br>WHERE column3 operator (SELECT column4 FROM table2 WHERE condition);</pre><p>In this syntax:</p><ul><li><em>table1</em> is the outer query&#39;s table.</li><li><em>column1</em>, <em>column2</em>, and <em>column3</em> are columns from table1.</li><li><em>condition</em> is a condition that filters rows from table2.</li><li><em>column4</em> is a column from <em>table2</em>.</li><li><em>operator</em> is a comparison operator such as =, &gt;, &lt;, etc.</li></ul><p>There are a few rules that subqueries must follow:</p><ul><li>Subqueries must be enclosed within parentheses</li><li>A subquery can have only one column specified in the SELECT clause. But if you want a subquery to compare multiple columns, those columns must be selected in the main query.</li><li>Subqueries that return more than one row can only be used with multiple value operators, such as the IN operator which allows you to specify multiple values in a WHERE clause.</li><li>A subquery can’t be nested in a SET command. The SET command is used with UPDATE to specify which columns (and values) are to be updated in a table.</li></ul><p>Now, let’s explore how subqueries can be used to aggregate data effectively and how the CASE and HAVING functions can enhance this process.</p><p><strong>Using Subqueries for Data Aggregation</strong></p><ol><li><strong>Using Subqueries with AVG and COUNT:</strong></li></ol><p>Subqueries can be employed to dynamically calculate averages and counts. Consider a scenario where we have a Spotify dataset, and we want to find the average duration of tracks by a specific artist.</p><pre>SELECT artist_name, AVG(duration_ms) AS avg_duration<br>FROM tracks<br>WHERE artist_name = &#39;Adele&#39;<br>GROUP BY artist_name;</pre><p>To find the overall average track duration for all artists, we can utilize a subquery:</p><pre>SELECT artist_name, (SELECT AVG(duration_ms) FROM tracks) AS avg_duration<br>FROM tracks<br>GROUP BY artist_name;</pre><p>Here, the subquery (SELECT AVG(duration_ms) FROM tracks) calculates the overall average track duration, which is then presented for each artist.</p><p><strong>2. Using Subqueries with CASE:</strong></p><p>The CASE statement is a valuable tool for conditional aggregation. Suppose we want to count the number of tracks in each genre that have a popularity score higher than 80.</p><pre>SELECT genre, COUNT(*) AS popular_tracks_count<br>FROM tracks<br>WHERE popularity &gt; 80<br>GROUP BY genre;</pre><p>To also include genres with no highly popular tracks, we can utilize a subquery with the CASE statement:</p><pre>SELECT genre, <br>       SUM(CASE WHEN popularity &gt; 80 THEN 1 ELSE 0 END) AS popular_tracks_count<br>FROM tracks<br>GROUP BY genre;</pre><p>In this query, the CASE statement categorizes tracks as highly popular (popularity &gt; 80) or not (popularity &lt;= 80), and SUM aggregates the counts.</p><p><strong>Leveraging HAVING for Advanced Filtering</strong></p><p>The HAVING clause works in conjunction with GROUP BY to filter aggregated data based on specified conditions. Let’s consider a scenario where we want to identify genres with an average popularity score greater than 70 and more than 100 tracks:</p><pre>SELECT genre, AVG(popularity) AS avg_popularity<br>FROM tracks<br>GROUP BY genre<br>HAVING AVG(popularity) &gt; 70 AND COUNT(*) &gt; 100;</pre><p>Here, the HAVING clause filters the results post-aggregation, ensuring that only genres meeting the specified criteria are included in the output.</p><p><strong>Using Subqueries with JOIN for Data Aggregation</strong></p><p>In this query, we’ll find the top artists who have released more than 10 tracks and display their most popular tracks, along with a category indicating whether the track is highly popular or not.</p><pre>SELECT<br>    t1.artist_name,<br>    t1.track_name,<br>    t1.popularity,<br>    CASE<br>        WHEN t1.popularity &gt;= 80 THEN &#39;Highly Popular&#39;<br>        ELSE &#39;Not Highly Popular&#39;<br>    END AS popularity_category<br>FROM tracks t1<br>JOIN (<br>    SELECT artist_name, COUNT(*) AS track_count<br>    FROM tracks<br>    GROUP BY artist_name<br>    HAVING COUNT(*) &gt; 10  -- Only consider artists with more than 10 tracks<br>    ORDER BY track_count DESC<br>    LIMIT 5<br>) t2<br>ON t1.artist_name = t2.artist_name<br>ORDER BY t2.track_count DESC, t1.popularity DESC;</pre><p>In this query:</p><ol><li>We have added a CASE statement within the SELECT clause. This CASE statement categorizes tracks as “Highly Popular” when their popularity is greater than or equal to 80 and as “Not Highly Popular” otherwise.</li><li>The resulting column popularity_category provides this categorization for each track, allowing us to distinguish between highly popular and less popular tracks for the top artists with more than 10 tracks.</li></ol><p>This query demonstrates how the CASE statement can be used alongside subqueries, JOINs, and the HAVING clause to perform complex data analysis tasks in SQL, providing insights into track popularity for specific artists.</p><p><strong>Conclusion</strong></p><p>Subqueries, when combined with the CASE and HAVING functions, empower SQL users to perform advanced data aggregation and analysis tasks. They enable dynamic calculations, conditional aggregations, and post-aggregation filtering, providing valuable insights from complex datasets like the Spotify database from Kaggle. By mastering these SQL features, users can construct intricate queries that reveal profound insights and support informed decision-making in various data-driven scenarios.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bbfc9aaeda2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Database, Data Warehouse, and Data Lake: Differences, Advantages, and Transitions]]></title>
            <link>https://medium.com/@marquesjef/understanding-database-data-warehouse-and-data-lake-differences-advantages-and-transitions-f83f0b05620?source=rss-5a8178286789------2</link>
            <guid isPermaLink="false">https://medium.com/p/f83f0b05620</guid>
            <category><![CDATA[data-warehouse]]></category>
            <category><![CDATA[data-lake]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Jeferson Marques]]></dc:creator>
            <pubDate>Fri, 01 Sep 2023 13:09:09 GMT</pubDate>
            <atom:updated>2023-09-01T13:09:09.419Z</atom:updated>
            <content:encoded><![CDATA[<p>In today’s data-driven world, effective data management is paramount for businesses seeking to gain insights, make informed decisions, and remain competitive. Three key components in the data ecosystem are databases, data warehouses, and data lakes. This article aims to differentiate these entities, highlight their respective advantages, and discuss scenarios for transitioning between them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/818/1*1v8hpmWXqlOfgFbbt46zyg.png" /></figure><p><strong>Databases</strong></p><p>Databases are structured repositories designed for efficient data retrieval and storage. They excel at handling structured data with predefined schemas. Common databases include MySQL, Oracle, and PostgreSQL. Their advantages lie in rapid access, ACID compliance, and suitability for transactional applications. Databases are ideal for scenarios like e-commerce websites, where real-time, structured data is crucial.<br>Top tools and solutions commonly use:</p><ol><li><strong>MySQL:</strong> A popular open-source relational database management system known for its reliability and performance;</li><li><strong>Microsoft SQL Server:</strong> A robust database system with strong integration capabilities for Windows-based applications;</li><li><strong>PostgreSQL:</strong> An open-source, object-relational database system known for its extensibility and support for advanced data types;</li><li><strong>Oracle Database:</strong> A highly scalable and secure database solution suitable for enterprise-level applications.</li></ol><p><strong>Data Warehouses</strong></p><p>Data warehouses are specialized databases optimized for analytical queries. They consolidate data from various sources into a single repository, enabling complex reporting and analysis. Advantages include historical data storage, data transformation, and support for business intelligence tools. Businesses often transition to data warehouses as they grow and require in-depth analysis of historical data. For example, an e-commerce company might move from a database to a data warehouse when analyzing years of sales data for trend analysis. <br>Top tools and solutions commonly use:</p><ol><li>Amazon Redshift: A fully managed data warehouse service in the cloud, designed for high-performance analytics.</li><li>Snowflake: A cloud-based data warehousing platform known for its ease of use and scalability.</li><li>Google BigQuery: A serverless, highly scalable, and cost-effective data warehouse service that integrates well with Google Cloud services.</li><li>Microsoft Azure Synapse Analytics: A unified analytics service that combines big data and data warehousing capabilities.</li></ol><p><strong>Data Lakes</strong></p><p>Data lakes are versatile, storage-based repositories that can accommodate structured, semi-structured, and unstructured data. They’re cost-effective and scalable. While they offer flexibility, they require careful governance to prevent data sprawl. Organizations typically opt for data lakes when dealing with vast and diverse data sources, such as social media feeds, IoT data, and logs. Transitioning to a data lake may be necessary when integrating data from various sources, like sensor data from multiple factories, for predictive maintenance.<br>Top tools and solutions commonly use:</p><ol><li>Amazon S3: Amazon Simple Storage Service provides scalable object storage suitable for building data lakes in AWS.</li><li>Azure Data Lake Storage: Part of Microsoft’s Azure Data Lake, it offers a secure and massively scalable data lake solution.</li><li>Google Cloud Storage: Google’s object storage service that can serve as the foundation for building data lakes on Google Cloud.</li><li>Apache Hadoop: An open-source framework that can be used to build and manage data lakes, often in combination with tools like Apache Hadoop Distributed File System (HDFS).</li></ol><p><strong>Transitioning Between Them</strong></p><p>Transitioning between these components is driven by business needs. Start with a database for structured data and transition to a data warehouse as analytical requirements grow. When dealing with a wide variety of data sources or storing vast amounts of raw data, consider implementing a data lake alongside your data warehouse. Data lakes can serve as a staging area for raw data before refining and transforming it in the data warehouse for analytics.</p><p><strong>Conclusion</strong></p><p>Databases, data warehouses, and data lakes play distinct but complementary roles in modern data management. The choice depends on data type, usage, and scalability requirements. By understanding their differences and advantages, businesses can make informed decisions and transition seamlessly between these data management components as their needs evolve.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f83f0b05620" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Significance of Data Cleaning in Data Analytics: Leveraging Google Sheets and BigQuery]]></title>
            <link>https://medium.com/@marquesjef/the-significance-of-data-cleaning-in-data-analytics-leveraging-google-sheets-and-bigquery-4ca5d52d8816?source=rss-5a8178286789------2</link>
            <guid isPermaLink="false">https://medium.com/p/4ca5d52d8816</guid>
            <category><![CDATA[data-driven]]></category>
            <category><![CDATA[data-cleaning]]></category>
            <category><![CDATA[changelog]]></category>
            <category><![CDATA[google-sheets]]></category>
            <category><![CDATA[bigquery]]></category>
            <dc:creator><![CDATA[Jeferson Marques]]></dc:creator>
            <pubDate>Thu, 24 Aug 2023 20:34:48 GMT</pubDate>
            <atom:updated>2023-08-24T20:34:48.352Z</atom:updated>
            <content:encoded><![CDATA[<p>In the realm of data analytics, the importance of clean and accurate data cannot be overstated. Raw data rarely arrives in pristine condition; it often carries various forms of “dirty data” that can compromise the quality of analysis and decision-making. Dirty data encompasses various types of imperfections, including duplicate data, outdated data, incomplete data, incorrect/inaccurate data, and inconsistent data. To ensure the reliability of insights, it’s crucial to address these issues through meticulous data cleaning processes.</p><p>Data cleaning involves identifying and rectifying errors, inconsistencies, and inaccuracies in datasets. It’s an essential precursor to data analysis, as any flawed data could lead to misleading conclusions. One of the widely accessible tools for data cleaning is Google Sheets, offering various functions that aid in this process. Google Sheets provides an array of data cleaning tools, including data validation to restrict inputs to specified criteria, conditional formatting to visually highlight data anomalies, and functions like “COUNTIF” to tally occurrences.</p><p>Sorting and filtering are also indispensable techniques for data cleaning in Google Sheets. Sorting data alphabetically or numerically brings to light irregularities, while filtering allows the isolation of specific subsets for closer scrutiny. These tools are particularly useful for identifying outliers and inconsistencies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*f1v99d-OmvFmZL9e8PFxdw.png" /></figure><p>Moving to more complex scenarios and larger datasets, Google BigQuery comes into play. This serverless, scalable data warehouse employs SQL for data cleaning. SQL’s data cleaning capabilities encompass filtering out unwanted records using the WHERE clause, handling missing values with the COALESCE function, and combining data from diverse sources using JOIN operations. The “GROUP BY” clause allows aggregating data for summary statistics, and “ORDER BY” arranges data for better insights. Moreover, the “CASE” statement facilitates conditional transformations, while “CAST” ensures data types are aligned. String manipulation functions like CONCAT and SUBSTR help in concatenating and extracting substrings respectively, contributing to data refinement. Typecasting ensures that data is interpreted correctly by transforming it into appropriate formats.</p><p>Effective data cleaning involves not only the technical aspects but also robust documentation. The process of tracking changes, additions, deletions, and errors involved in your data-cleaning effort is crucial. This documentation ensures transparency, accountability, and reproducibility. Moreover, creating a changelog detailing each cleaning action taken allows for easy communication and collaboration among team members, making it simpler to understand the evolution of the dataset.</p><p>The data cleaning process in BigQuery excels not only in efficiency but also in facilitating collaboration. Multiple users can work on the same dataset simultaneously, enhancing productivity and maintaining consistency across cleaning operations.</p><p>In conclusion, the significance of data cleaning in data analytics cannot be overstated. Accurate insights and informed decisions hinge on the quality of data inputs. Google Sheets and BigQuery offer a robust toolkit for data cleaning. While Google Sheets accommodates smaller-scale tasks with user-friendly functions like conditional formatting, sorting, and filtering, BigQuery steps in for heavy-duty data cleaning with the power of SQL on massive datasets. Investing time and effort in the data cleaning phase, along with diligent documentation practices and changelog creation, ultimately lays the groundwork for more robust and reliable data analytics outcomes.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ca5d52d8816" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transitioning from Spreadsheets to Relational Databases: Making Informed Choices for Data Analysis]]></title>
            <link>https://medium.com/@marquesjef/transitioning-from-spreadsheets-to-relational-databases-making-informed-choices-for-data-analysis-b7713a6cbf59?source=rss-5a8178286789------2</link>
            <guid isPermaLink="false">https://medium.com/p/b7713a6cbf59</guid>
            <category><![CDATA[bigquery]]></category>
            <category><![CDATA[relational-databases]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[spreadsheets]]></category>
            <category><![CDATA[mysql]]></category>
            <dc:creator><![CDATA[Jeferson Marques]]></dc:creator>
            <pubDate>Tue, 22 Aug 2023 20:31:03 GMT</pubDate>
            <atom:updated>2023-08-22T20:31:03.994Z</atom:updated>
            <content:encoded><![CDATA[<p>In the ever-evolving landscape of data analysis, the choice between working with spreadsheets or transitioning to a robust relational database like BigQuery, MySQL, or SQL Server is a pivotal decision that can significantly impact the efficiency, scalability, and accuracy of your analyses. In this article, we delve into the scenarios where spreadsheets suffice and when the transition to relational databases becomes imperative. Additionally, we highlight the key distinctions between popular relational database options.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FCXMFjtWwKzKG_7fh2zhxA.png" /></figure><p><strong>When to Stick with Spreadsheets</strong></p><p>Spreadsheets have long been the go-to tool for managing and analyzing data due to their simplicity and ease of use. For small-scale projects, quick calculations, or personal analysis, spreadsheets like Microsoft Excel or Google Sheets can serve as adequate solutions. They are particularly useful when:</p><ol><li>Data Volume is Small: Spreadsheets are suitable for handling modest amounts of data that can be managed comfortably within a single file;</li><li>Individual Usage: For personal or individual analysis tasks, spreadsheets offer an intuitive interface and quick access;</li><li>Quick Data Manipulation: Spreadsheets excel in quick data manipulation, sorting, filtering, and simple calculations;</li><li>Limited Collaborative Needs: When collaboration is minimal and the focus is on individual tasks, spreadsheets can be convenient.</li></ol><p><strong>Transitioning to Relational Databases</strong></p><p>As datasets grow in size and complexity, spreadsheets may become cumbersome and inadequate. The transition to relational databases offers several advantages that cater to the evolving demands of data analysis:</p><ol><li>Scalability: Relational databases can handle massive datasets with ease, ensuring performance remains consistent as data volume increases;</li><li>Data Integrity: Relational databases enforce data integrity through relationships, preventing inconsistencies and errors;</li><li>Structured Querying: Relational databases enable complex queries, filtering, and aggregations, enabling deeper insights from the data;</li><li>Concurrency: Collaborative environments benefit from the concurrent access and data sharing capabilities of relational databases;</li><li>Security and Access Control: Relational databases provide robust security features, ensuring data remains protected and accessible only to authorized users.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/705/1*FK5gmboG9aX4iKklxxHDlg.png" /></figure><p><strong>Distinguishing Relational Databases: BigQuery, MySQL, and SQL Server</strong></p><p>Each relational database option offers distinct features suited to various needs:</p><ol><li>Google BigQuery: BigQuery is a cloud-based data warehousing solution known for its scalability and speed in analyzing massive datasets. It excels in handling analytics workloads and supports complex queries with impressive processing power;</li><li>MySQL: MySQL is an open-source relational database management system renowned for its reliability and versatility. It’s well-suited for smaller to medium-sized applications and projects;</li><li>SQL Server: Developed by Microsoft, SQL Server is a comprehensive and robust relational database system designed for a wide range of applications, from small projects to enterprise-level systems.</li></ol><p><strong>Choosing Wisely: When to Opt for Which</strong></p><p>The choice between BigQuery, MySQL, or SQL Server depends on your specific requirements:</p><ol><li>BigQuery: Ideal for data warehousing, analytics, and handling extremely large datasets where query speed is crucial;</li><li>MySQL: Well-suited for smaller to medium-sized projects that require a reliable and cost-effective database solution;</li><li>SQL Server: Suitable for projects of varying sizes, particularly those requiring comprehensive features, strong security, and integration with Microsoft ecosystems.</li></ol><p>In Conclusion, while spreadsheets are valuable for small-scale analysis and quick calculations, transitioning to relational databases like BigQuery, MySQL, or SQL Server becomes essential as data complexity and volume grow. The choice between these databases hinges on your project’s requirements and specific needs. By making informed decisions about when to use spreadsheets and when to embrace relational databases, data professionals can elevate their analytical capabilities, maximize efficiency, and unlock deeper insights from their datasets.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b7713a6cbf59" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>