<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Lorela Blaka on Medium]]></title>
        <description><![CDATA[Stories by Lorela Blaka on Medium]]></description>
        <link>https://medium.com/@lorelablaka?source=rss-88355778bd1d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ymJoePfqQIxasPCbegafrA.png</url>
            <title>Stories by Lorela Blaka on Medium</title>
            <link>https://medium.com/@lorelablaka?source=rss-88355778bd1d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 11:39:50 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@lorelablaka/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Create table with random data in SQL]]></title>
            <link>https://medium.com/@lorelablaka/create-table-with-random-data-in-sql-7bf102de088?source=rss-88355778bd1d------2</link>
            <guid isPermaLink="false">https://medium.com/p/7bf102de088</guid>
            <dc:creator><![CDATA[Lorela Blaka]]></dc:creator>
            <pubDate>Mon, 08 Nov 2021 01:59:08 GMT</pubDate>
            <atom:updated>2021-11-08T02:00:51.723Z</atom:updated>
            <content:encoded><![CDATA[<p>The goal of this article is to create a customizable framework to generate a table with random data that has the following columns:</p><ol><li>Date</li><li>ID</li><li>Amount</li><li>Category</li></ol><p>These columns cover all major column types so that there is more room for practice analysis (or any other purpose for the random data).</p><p>The first step is to create the table. For this example, we will call it “sales”:</p><pre>create table sales (<br>offer_id integer not null,<br>offer_date date not null,<br>amount integer not null,<br>category varchar not null<br>);</pre><p>Now that the table is created, we can use the INSERT statement to populate the random to the table. This code is written in the form of CTEs (common table expressions), with each CTE representing a column, except the “category” column. The category column is defined in the main SELECT statement at the bottom. Each column is joined by the row_n, which represents the row_number column in each CTE. Below is the code:</p><pre>with dates as (<br>select<br>row_number() over() as r_num<br>, t.day::date as offer_date<br>from generate_series(timestamp ‘2020–04–01’<br>, timestamp ‘2020–04–30’<br>, interval ‘1 day’) AS t(day)<br>)</pre><pre>, offer as(<br>select<br>row_number() over() as r_num<br>, o.offer_id as offer_id<br>from generate_series(100, 150) AS o(offer_id)<br>)</pre><pre>, amount as(<br>select<br>r_num<br>, cast(amount*random() as int) as amount<br>from (<br>select row_number() over() as r_num<br>, a.amount as amount<br>from generate_series(100000, 4000000) AS a(amount) ) am<br>)</pre><pre>INSERT INTO sales(<br>select offer_id<br>, offer_date<br>, amount<br>, case when mod(amount,2)=0 then ‘shirts’ else ‘pants’ end as category<br>from dates, offer, amount<br>where dates.r_num = offer.r_num<br>and offer.r_num = amount.r_num )</pre><p>This code can be modified in many ways, from changing the column names to the data the category values.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7bf102de088" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Moving/Trailing/Rolling Average in SQL]]></title>
            <link>https://medium.com/@lorelablaka/moving-trailing-rolling-average-in-sql-d1405ebd0c92?source=rss-88355778bd1d------2</link>
            <guid isPermaLink="false">https://medium.com/p/d1405ebd0c92</guid>
            <dc:creator><![CDATA[Lorela Blaka]]></dc:creator>
            <pubDate>Fri, 05 Nov 2021 21:37:18 GMT</pubDate>
            <atom:updated>2021-11-05T21:46:19.998Z</atom:updated>
            <content:encoded><![CDATA[<p>The goal of this article is to gain a better understanding of how to calculate moving (i.e. trailing, rolling) average in SQL.</p><p><strong>Background: SQL and Trend Types</strong></p><p>SQL has grown to be la lingua franca for data analysis and engineering. And oftentimes when we have a raw dataset, we need to perform calculations that help us extract some valuable insight from it. Part of this involves identifying trends in the raw data.</p><p>There are a few main types of trends:</p><ol><li><strong>Linear</strong> — downward or upward (ex. average sales are increasing month over month)</li><li><strong>Periodic</strong> — changes based on a pattern (ex. average sales are spiking only in the summer)</li><li><strong>Exponential</strong> — changes based on a rate (ex. average sales are increasing at an increasing or decreasing rate overtime)</li><li><strong>Variational </strong>— changes in variance increase or decrease overtime (ex. average sale <em>spikes</em> are increasing overtime)</li></ol><p><strong>Rolling Statistics</strong></p><p>The most popular examples of rolling statistics include the rolling average and standard deviation, specifically in time series analysis. We will only focus on rolling average, which represents a series of averages, with each average being taken from a subset of the data. This subset of data has a consistent size; for example, last 7 days, or the last 30 days. This is so important because the rolling average helps identify linear, periodic, and exponential trends. The rolling standard deviation helps identify trends in variance, something we can focus on in a later article.</p><p><strong>Let’s start with an example:</strong><br>We own an e-commerce business that sells inventory to retailers. Below shows April 2020 daily sales:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/0*O7SIsy3DOyc6Bn3j" /></figure><p>Table Columns:</p><ul><li>“Offer_id” — ID of the sale</li><li>“Offer_date” — date of the sale</li><li>“Amount” — selling price of the sale, and the metric we will take the rolling average of</li></ul><p>As mentioned before, a rolling statistic represents a series of that statistic based on a subset of the data. Because we are looking at daily sales, we can choose a 7-day rolling average.</p><p>We can think of this subset of data as a “window” of 7 days .. spoiler alert, we will be using window functions to achieve this.</p><p>Window functions in SQL perform a calculation based on a set of rows that are somehow related to the current row. In our case, any row that falls within the last 7 days of the current row will be included in the calculation.</p><p><strong>Now for the SQL code.</strong></p><p>There are two ways to do this:</p><ol><li>Assuming there is a sale for each day</li><li>Assuming there isn’t a sale for each day</li></ol><p><strong>Here is the SQL code for the first way:</strong></p><pre>SELECT offer_date<br>, amount<br>, avg(amount) OVER (ORDER BY offer_date ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) as moving_average<br>FROM sales;</pre><p>And the results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/710/1*45f-Dj01NbQUcWWkM7PrYQ.png" /></figure><p><em>(Note: Window functions do not aggregate data, so there are still a total of 30 rows. I’m showing the first 14 for display purposes.)</em></p><p>This actually achieved exactly what we wanted because the data just so happens to have a sale for each day of April.</p><p>But what if this isn’t the case (assumption 2)? We need to modify the current code to grab rows specified directly by the date.</p><p><strong>Here is the SQL code for the second way:</strong></p><pre>SELECT date(offer_date — interval ‘6 days’) as start_date, <br>offer_date, <br>amount<br>FROM sales<br>ORDER BY offer_date;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/616/0*RU6_qFSwX0Dab51n" /></figure><p>We can see that the start_date and offer_date represent the time window we are looking for in each row.</p><p>To subset the data directly by date, all we have to do is combine the first code snippet and the interval code above:</p><pre>SELECT offer_date<br>, amount<br>, moving_average</pre><pre>FROM (SELECT amount<br>, offer_date<br>, AVG(amount) OVER (ORDER BY offer_date RANGE BETWEEN interval ‘6 days’ PRECEDING AND CURRENT ROW) as moving_average<br>FROM sales<br>) subquery</pre><pre>ORDER BY offer_date;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/710/1*45f-Dj01NbQUcWWkM7PrYQ.png" /></figure><p>And we finally have our desired moving average, which matches the results of the first method!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d1405ebd0c92" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Model Training With Data Parallelism]]></title>
            <link>https://medium.com/@lorelablaka/model-training-with-data-parallelism-9395d75642bc?source=rss-88355778bd1d------2</link>
            <guid isPermaLink="false">https://medium.com/p/9395d75642bc</guid>
            <dc:creator><![CDATA[Lorela Blaka]]></dc:creator>
            <pubDate>Tue, 12 Oct 2021 17:35:33 GMT</pubDate>
            <atom:updated>2021-10-12T17:35:33.083Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Background</strong></p><p>Handling larger datasets has become more prevalent in the recent years of data science. Large datasets are very valuable, especially in deep learning, but they have posed the challenge of increased computational complexity. A solution to this challenge was outlined in this research paper called<a href="https://arxiv.org/pdf/2006.15704v1.pdf"> “PyTorch Distributed: Experiences on Accelerating Data Parallel Training”</a>.</p><p><strong>The DDP Library</strong></p><p>This solution is using the PyTorch Distributed Data Parallel (DDP) module. For some background, Pytorch is a machine learning library primarily developed by Facebook’s AI Research Lab. Within the library, the DDP module performs this distributed model training by parallelizing data. First, every single GPU across every machine will get a copy of the model and a subset of the data depending on the number of nodes and GPUs. Then to optimize the model hyper parameters, the gradients are averaged out independently amongst the machines during the model training phase, which is done via forward &amp; backward passes.</p><p>The study went into explaining the DDP system design, which at the top has a Python API frontend, gradient reduction core algorithm, and the collective communication library.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/328/0*Kkupu8EYqycT9D_D" /></figure><p>The API has two goals:</p><ol><li>Ease-of-use if they need to expand the DDP training to other machines</li><li>Exposure of as many optimization opportunities as possible in the internal implementation</li></ol><p>The paper then went into technical depth about the evolution of the Gradient Reduction algorithm, which strives to find the optimal parameters. For background, during the distributed training, each machine has its own local model replica and local optimizer. To ensure accuracy, all local models start from the exact same model state, and see the same parameter gradients after every backward pass. Each local optimizer should then be able to bring their local model replicas to the same state at the end of every iteration.</p><p>The DDP is built on communication libraries like NCCL and Gloo. The APIs from these libraries are wrapped into one API called ProcessGroup API. In DDP, each local machine is expected to join a process group for communication primitives to work on.</p><p>The paper also conducted performance evaluations on the NCCL and Gloo communication libraries using 2 different models. DDP accelerates training by aggregating gradients into buckets for communication (gradient algorithm), overlapping communication with computation, and skipping synchronizations. More specifically:</p><p>“Results showed that DDP with NCCL backend can achieve near-linear scalability on 256 GPUs when configured properly. The measurements also revealed that the backward pass in DDP is the most expensive step in training and requires efforts from both framework developers to enable optimization algorithms and application developers to empirically configure the knobs.”</p><p><strong>Why should anyone care about this?</strong></p><p>The DDP library should definitely interest an aspiring data scientist because it is built for handling large data in a computationally optimized way. This allows the data scientist to run higher quality models more effectively.</p><p>And from a business standpoint, efficiently running models with a lot of data will allow for:</p><ol><li>Enhanced model results faster (enhanced because the machine can handle higher volume of data)</li><li>Decrease costs and frustration since the appropriate system is utilized</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9395d75642bc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Extract Data using Spotify API]]></title>
            <link>https://medium.com/@lorelablaka/extract-data-using-spotify-api-889222835bf4?source=rss-88355778bd1d------2</link>
            <guid isPermaLink="false">https://medium.com/p/889222835bf4</guid>
            <dc:creator><![CDATA[Lorela Blaka]]></dc:creator>
            <pubDate>Mon, 20 Sep 2021 12:54:21 GMT</pubDate>
            <atom:updated>2021-11-04T04:01:10.766Z</atom:updated>
            <content:encoded><![CDATA[<h3>Extract Data Using the Spotify API and Python Requests</h3><p>The goal of this article is to gain a better understanding of API documentation by going through an example using the Spotify API.</p><p><strong>Background: APIs</strong></p><p>API stands for Application Programming Interface and has been a turning point in the development of technology. It allows different programs to work together in various ways by creating a line of communication between them. APIs can be used to gain access to data from third parties, like how Apple uses The Weather Channel API to access weather forecasts. APIs can also extend functionality of a product when a vendor integrates with an application. For example, when an online showroom extends functionality to also process transactions.</p><p>There are three approaches companies take to API release policies.</p><ol><li>Private: the API can only be used for internal purposes</li><li>Partner: the API can be shared with specific business partners</li><li>Public: the API can be used by anyone to gain access to information for innovative or other purposes</li></ol><p>Amongst the different types of APIs, we will focus on extracting data using the Spotify Web API.</p><p><strong>Using the Spotify API</strong></p><p>We will start by going to<a href="https://developer.spotify.com/dashboard/applications"> Dashboard section</a> of Spotify for Developers, which is where we have to Login and create an app by specifying some details:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*r6qCtWlHAM9wx1fP" /></figure><p>Within the app, there is the Client ID and Client Secret, which is needed to authenticate with the service. You can reset your Client Secret by just clicking the red reset button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DmFYBt8Sjy_h60ve" /></figure><p>Next, we will open up a new Jupyter Notebook file and install the library <em>requests, </em>which is needed to actually use the API. We then authenticate once by specifying the Client ID and Client Secret, and receive a token to access future sessions.</p><p>After token authentication, client authorization is needed based on this<a href="https://developer.spotify.com/documentation/general/guides/authorization/client-credentials/"> Client Credentials Flow</a> of the Authorization Guide. We will need to specify:</p><p>The Token URL for the URL request, the method of POST, and the <em>required</em> token parameters. The token parameters can be specified in dictionary form. Notice our second parameter requires our Authorization (client ID and client secret) to be turned into base64 encoded string. So we first need to import <em>base64</em> and modify the Authorization key to bring the base64 encoded credentials</p><p>Then we import the <em>requests</em> library and create and run our request using the post method. From here, we can get our access token.</p><p>So far, our notebook should look something like this (Note: our token expires in 3600 seconds):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nLkmnhowCkB-ftRk" /></figure><p>Next, I’m going to organize this code in a class object called Spotify and run some print statements to visualize the outputs within the class:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Xhm3lFn_zx52jizh" /></figure><p>Everything looks good! We can finally retrieve some Spotify data. How we actually use the Access Token to access the data goes back to the<a href="https://developer.spotify.com/documentation/general/guides/authorization-guide/"> Authorization Guide.</a> Notice our token_type is “Bearer” now instead of “Basic”, which was our original token_type during authentication. So moving forward, we will use the Bearer type.</p><p>We also need to specify an endpoint from their<a href="https://developer.spotify.com/documentation/web-api/reference/"> list</a> .. the endpoint for this example will be <a href="https://developer.spotify.com/documentation/web-api/reference/#/operations/search">Search</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cMr4iQh0CEe4TK4z" /></figure><p>Notice our only required<em> </em>header parameter is Authorization and our required query parameters are “q” and “type”. If we accurately set up our code to include these parameters, we should be getting a Status Code of 200. The code for making requests to the Search endpoint will look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1uP3tnqfXxbunCel" /></figure><p>One thing to point out is we need to import <em>urlencode</em> (cell 11 of the picture above). Urlencode allows us to pass multiple url parameters, like the ones in the <em>data </em>dictionary above, while keeping the url string format.</p><p>We then print the <em>data</em> to make sure that the parameters are aligned appropriately in the url (which it looks like they are).</p><p>Lastly, we make a request with the specified <em>lookup_url </em>and <em>headers. </em>In this example, calling the json object displays all of the results of the “The Weeknd” tracks.</p><p>Moving forward, we can switch up the Search parameters, or even switch up the endpoints.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=889222835bf4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Assessing Snowflake: Top-Down Approach]]></title>
            <link>https://medium.com/@lorelablaka/assessing-snowflake-top-down-approach-9d70273e48c4?source=rss-88355778bd1d------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d70273e48c4</guid>
            <dc:creator><![CDATA[Lorela Blaka]]></dc:creator>
            <pubDate>Mon, 23 Aug 2021 12:46:12 GMT</pubDate>
            <atom:updated>2021-08-26T03:45:56.981Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WxLe6zMxeuEtzw0KnIGewQ.jpeg" /></figure><p><strong>Cloud Services in General</strong></p><p>Who needs water or oil when data is now the most valuable resource on earth? Our reality is that in 2020, the daily average of data generated was a jaw-dropping 2.5 exabytes of data (2.5 with 18 zeros). By 2025, this number is expected to be <a href="https://techjury.net/blog/how-much-data-is-created-every-day/#gref">463 exabytes</a>, a ~casual~ <em>184.2x</em> increase. This includes everything from your posts, likes, and stories on social media, to a company’s sales records and conversion rates. With these mammoth numbers, it obviously demands for data-handling solutions with a focus on scalability, security, and performance. This is where the idea of cloud computing services comes in. These services include Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). A good understanding of each requires more than just a few sentences, and for the purpose of this article, I want to focus on PaaS data warehousing products.</p><p><strong>Data Warehousing Market</strong></p><p><a href="https://www.marketsandmarkets.com/Market-Reports/data-warehouse-as-a-service-market-191544663.html">Over half of all organization data</a> is expected to be switched over to cloud storage due to an increasing need for large scale deployment. (Side note, an unsurprising facilitator of this market growth is covid, but a barrier is lack of resources and expertise). With growth comes opportunity. The top industries using data warehousing tools include business services, retail, manufacturing, finance, and insurance. And the <a href="https://www.datanyze.com/market-share/data-warehousing--240">top 5 tools</a> with the largest market share are Snowflake (16.8%), Apache Hive (10.23%), Amazon Redshift (9.23%), IBM Netezza (9.23%), and Oracle Autonomous (6.64%). Google BigQuery came in 10th with 3.05% market share. The fact that there are so many data warehouse services in this market and each solution isn’t totally the same means products <em>will</em> be improving. What is Snowflake and what makes it so successful? What are its shortcomings?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/225/1*JtDH-dGnX1mjWQ6dVCQMpg.png" /></figure><h4><strong>What is Snowflake?</strong></h4><p>A data warehouse in general (including Snowflake) is a system used for data analysis purposes and it is imperative for any company that wants to perform analytics. For example, a company has a list of data sources (i.e. Salesforce for customer data, Postgres for their specific product-generated data, etc). How do they migrate this data from sources like Postgres to an end-user tool for BI and analysis usage? First, something called ETL code is written, which <em>extracts</em> data from a database (i.e. PostresSQL), <em>transforms</em> it based on the code written, and <em>loads</em> the newly formatted data into the data warehouse. This is called Online Analytical Processing (OLAP). From here, businesses can improve their decision making capability, measure KPIs, enhance data quality, etc.</p><p>Snowflake in particular is built on top of AWS and Microsoft Azure cloud infrastructure. And at the end of 2019, <a href="https://cloud.google.com/blog/products/data-analytics/announcing-snowflake-on-google-cloud-platform">Google Cloud also announced its partnership</a> with Snowflake for better performance and reliability, so its customer base is vast.</p><p><strong>What makes it unique?</strong></p><p>Snowflake architecture allows for ease-of-use and scalability, two important pillars for data teams. There are three layers to snowflake architecture and each are<a href="https://www.stitchdata.com/resources/snowflake/"> independently scalable: storage, compute, and cloud services.</a></p><p>The database storage layer houses all the loaded data, and runs independently from the compute layer. This setup allows resources to be scaled up or down without the extra cost of computing resources. Compared to a solution like Redshift, which does not separate these two layers, additional storage means needing to increase the node size on the cluster, leading to paying for more compute power.</p><p>The compute layer consists of multiple virtual clusters and each cluster can access all the data in the storage layer without competing for resources. You can specify the size based on your needs. In contrast, with a solution like Redshift, you need to maintain a cluster size that is appropriate for your peak load, and either manually change your cluster size later, or waste resources keeping the node size large. Snowflake also has the ability to auto detect when scaling is needed for a more dynamic approach to maintenance — you can choose this mode during setup.</p><p>Secondly, within the compute layer, there isn’t a concept of partitions, indexes, or tunables. This minimizes the time you would spend on optimization in another cloud solution.</p><p>Lastly, in cloud services, Snowflake stores all its customers metadata in a centralized manner. It allows for cloning tables and whole databases just by cloning their metadata which can be a quick solution if you ever need this feature.</p><p><strong>What are its shortcomings?</strong></p><p>Snowflake’s shortcomings can be mainly attributed to its incipience and dependency on other cloud tools.</p><p>For one, <a href="https://coursedrill.com/snowflake-advantages/">Snowflake owns the deployment infrastructure</a>. This comes with a couple of cons. First, if you are used to having control and optimizing your own warehouse, then Redshift might be a better option. Second, dependence on Snowflake’s support is risky during an emergency. You can’t perform tasks such as data recovery or migration unless there is a backup data warehouse. And because it only works on 3 cloud tools, there are only so many options for backup.</p><p>Snowflake’s ease-of-maintenance <a href="https://fridaynightanalytics.com/snowflake-data-warehouse-pros-cons/#Top_Detriment_2_-_Data_Engineering_Tools">comes at a cost</a>. Unlike Redshift, which is cheaper due to the required self-maintenance, the support bills definitely can add up based on your own IT resources and needs.</p><p><strong>In conclusion…</strong></p><p>While this article is mainly all about Snowflake, there is no clear winning data warehouse service until you dig down to see what each offers in terms of scalability, security, and ease-of-use. Assess your own business needs, and then you’ll start to see how the differentiators are valuable to you. Hey, maybe one day you want to go multi-cloud (which <a href="https://www.nubeasoft.com/insights/top-six-challenges-with-multicloud-approach">is not the easiest feat</a> since cloud vendors are built to lock you in).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d70273e48c4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My personal journey // rant to pursuing Data Science]]></title>
            <link>https://medium.com/@lorelablaka/my-personal-journey-rant-to-pursuing-data-science-272de96af144?source=rss-88355778bd1d------2</link>
            <guid isPermaLink="false">https://medium.com/p/272de96af144</guid>
            <category><![CDATA[growth]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[career-development]]></category>
            <category><![CDATA[personal-development]]></category>
            <dc:creator><![CDATA[Lorela Blaka]]></dc:creator>
            <pubDate>Mon, 02 Aug 2021 00:10:14 GMT</pubDate>
            <atom:updated>2021-08-02T01:09:41.060Z</atom:updated>
            <content:encoded><![CDATA[<p>This modern phenomenon called data science has been pinned as one of the “hottest” fields to be in. It probably makes sense since we are in the Era of Information (aka Data), and our lives are shaped by technological advances. On a more personal level, I wanted to share my own journey in this field, which is ongoing.</p><p>My interest in this field has been more of an evolution and arguably dates back to my first internship for an NYC startup. This is where I became drawn to the tech industry and culture. I was an Econ student at the time, and 3x per week after morning classes would walk to West Houston St, grab my coffee, and start work. This was a new feeling for me because I had never been in this type of professional setting. But I quickly developed an affinity for this working environment, which influenced my next pre-senior summer internship, and ultimately my post-undergrad full-time job.</p><p>Having only worked at startups has its pros and cons, but it’s greatly contributed to the skills I have developed today. As an economics major, I got an analyst job for the same startup I interned for the past summer. I was in this role for 10 months until an opportunity to join the data team came. Of course, there was a qualification process of interviews, and afterward, the data lead offered me the opportunity. I was elated, to say the least because just a few weeks before receiving this offer, I made an active effort (and failed) to join the Product team. I had a pretty long-lasting interest in Product (i.e. read tons of books, attended speeches, ideated over the ownership they have over literally everything). But after interviewing for that role, the only executable feedback I received from the team was how clearly analytical and data-oriented I was.</p><p>Back to my new gig on the Data team, I started with simple tasks/tickets of pulling data for internal purposes and occasionally updating the SQL code in our Business Intelligence tool. Then when my manager left the company, and because we were a team of two, my responsibilities completely changed. I was now responsible for the design, development, and maintenance of our BI tool and the ongoing metrics in our reporting database. My knowledge of data integrity, data flow, debugging (amongst other things) faced a steep learning curve. Another invaluable experience I gained was from collaborating and receiving help from our backend engineers when I needed to debug. Without an engineering background, I felt lost almost all the time. But I was consistently driven to solve the company’s analytical problems daily. This persisted for about two months before hiring my new manager, who had a decade’s worth of math-heavy machine learning experience. The dynamic of the team changed from my last manager, but I still evolved in my role. After about a year, I decided that I needed to up-skill, which is what brought me to my next pursuit in completing the Flatiron Data Science Bootcamp. Because I was heavily using SQL in my last role, I wanted to gain more Python experience (per industry standards). It turns out that the majority of the Flatiron Data Science curriculum is based on Python, so this was a logical next step for me.</p><p>It’s incredible how my understanding of data science has changed over time. It’s been shaped mainly by my conversations with people in the field and articles I’ve read. Some have described it as advanced math and machine learning, which does not describe my skillset (yes, I’ve been more math-oriented, but I was an economics major). As a side note, the complexity of machine learning algorithms varies greatly, adding to my general confusion. So I think the more realistic meaning of data science is the “study of data ‘’, which I’ve seen is used a lot in articles. The professions that embody this broader definition include ML engineers and data engineers, various types of data analysts, and more. This whole slew of professional options explains why this industry’s labor market is increasingly growing, especially on the demand side. That said, the next paragraph will be my attempt to convince you as the reader that this field is worth transitioning to (without any assumptions made about your personal career goals and endeavors).</p><p>We obviously didn’t have a 2020 vision of the economic shambles of 2020, but the career trends based on 2021 are very promising. A job in data science is generally accessible, since, as aforementioned, there is a healthy demand for data-skilled workers. This is because data scientists work within most major industries, especially ones pursuing technological advances (i.e. healthcare, retail, financial). As a result, data needs will continue to grow, and that’s when the increasing need for cloud infrastructure comes in. Personally, working for a startup helped me understand the importance of having solid data infrastructure, which varies from company to company. But with the right setup, analysts and scientists can then do their jobs more efficiently. Depending on the sector, they will work on impactful projects that can help: detect cancer earlier, save companies tons of $$$, and curate buyer preferences, to name a few. These types of projects are growing in number and will require more data specialists. Our obstacle is to ensure that the supply meets the demand, which I’m sure it will eventually, but it has some work to do. Currently, <a href="https://www.datascienceprograms.org/">about 500 universities have data science program</a>s (~2% of total worldwide universities). While you <em>don’t </em>need a data science degree to get a job in data science, the current education available can be a limiting factor to having supply meet the growing demand.</p><p>To say the least, data science has a very healthy labor market. And I hope that sharing my experience helped shed some light on the wide range of skills involved in this field.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=272de96af144" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>