<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jubert Roldan on Medium]]></title>
        <description><![CDATA[Stories by Jubert Roldan on Medium]]></description>
        <link>https://medium.com/@jubertroldan?source=rss-4af303089aef------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*GvYXNkodtkmwB3SB.jpg</url>
            <title>Stories by Jubert Roldan on Medium</title>
            <link>https://medium.com/@jubertroldan?source=rss-4af303089aef------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 16:12:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jubertroldan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[What can we learn from the emergence of analytics engineers?]]></title>
            <link>https://jubertroldan.medium.com/what-can-we-learn-from-the-emergence-of-analytics-engineers-fcd0f1abb506?source=rss-4af303089aef------2</link>
            <guid isPermaLink="false">https://medium.com/p/fcd0f1abb506</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[analytics-engineering]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Jubert Roldan]]></dc:creator>
            <pubDate>Sun, 14 Apr 2024 22:40:01 GMT</pubDate>
            <atom:updated>2024-04-14T23:27:50.987Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KyJHVsEcmf6hn5Ge" /><figcaption>Photo by <a href="https://unsplash.com/@clemono?utm_source=medium&amp;utm_medium=referral">Clem Onojeghuo</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>What can we learn from the emergence of analytics engineers? bridging data gaps and building insights</h3><p>The emergence of analytics engineering opens doors to exciting possibilities in the ever-evolving landscape of data careers. It’s a field suited for individuals with a diverse skill set and a knack for taking initiative. Exploring this career path not only unveils its potential value but also triggers personal reflection, guiding individuals to understand their strengths and aspirations more deeply.</p><p>What exactly is an analytics engineer? Coined by leaders in dbt (<a href="https://www.getdbt.com/">data build tool </a>), it is an evolution of the role data analyst doing more than whats expected to resolve common issues working on the field.</p><p>Understanding the term “analytics engineer” requires a closer look at the fusion of skills and responsibilities encapsulated within. Unlike traditional roles, this new breed of professionals marries analytical acumen with engineering precision. They not only analyze data but also construct robust pipelines and systems, akin to data engineers. This multifaceted approach is particularly vital in dynamic environments like startups, where the first data hire often shoulders the entire data workflow, laying the foundation for future scalability and insights.</p><p><strong>⚠️ “The Gap”</strong></p><p>On my experience, the gap between data infrastructure and analytics operations remains an evident challenge. Despite the abundance of structured data, navigating through it often leads to hurdles like establishing a source of truth, avoiding duplicated analyses, reconciling conflicting numbers, and managing an endless backlog of tasks.</p><p><strong>🔩 “Engineering”</strong></p><p>The word “engineering” makes the term “analytics engineer” more than just a title combining both data engineer and data analyst; it embodies a commitment to robust solutions and scalability. Central to analytics engineering is its adherence to engineering principles, ensuring solutions are functional, safe, reliable, and scalable. Comparing it to building a house, while a carpenter can construct a basic roof over one’s head, scaling that to a large 100-floor story building demands engineering finesse working with multiple teams and considering scale. Analytics engineers provide scalable and manageable solutions critical for evolving data needs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JQJEhG8QYMDOImcw" /><figcaption>Photo by <a href="https://unsplash.com/@mikitayo?utm_source=medium&amp;utm_medium=referral">Mikita Yo</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>👾 “As and early game hero and a mid-late game support/lead”</strong></p><p>At the inception of data projects or innovation opportunities, businesses require individuals with diverse skill sets capable of handling various tasks in the unknown. The data practitioner, whether a data analyst/engineer/scientist who discovers the opportunity, may end up building a Proof of Concept (POC) to kickstart something. The skillset of analytics engineers may fill this void, offering comprehensive expertise that spans from initial data pipelines to sophisticated analysis — a spectrum where data analysts/scientists or engineers may specialise over time based on demand.</p><p>On well-established teams, I envision analytics engineers making significant contributions by elevating the working methodologies of analytics teams. They provide guidance and support on implementing best practices such as code collaboration (using version control), maintaining data integrity and quality assurance, as well as ensuring scalability and modularity of the code base. This transition isn’t without its challenges and demands, but integrating such skill sets into a team can greatly increase productivity, particularly in enhancing the capabilities of an analytics team.</p><p><strong>🔨 “Tools”</strong></p><p>Tools like dbt play a pivotal role in bridging this gap by providing a holistic solution encompassing testing, managed SQL runs, robust documentation, and ease of adoption for analytics professionals. This comprehensive approach resonates strongly with analytics backgrounds, providing efficient workflows and data-driven decision-making.</p><p>I have written a quick 3-min read as I was really amazed by this framework, heres the link:</p><p><a href="https://medium.com/p/f1b88be4e22b">https://medium.com/p/f1b88be4e22b</a></p><p>While businesses may question the necessity of this evolving role and face challenges in integrating such skill sets, the undeniable gap between data infrastructure and analytics demands attention. Analytics engineers offer a strategic advantage, ensuring self-serve analytics, addressing critical data gaps, and infusing established data teams with refined skills and leadership.</p><p>In my view, the analytics engineer role is poised for substantial growth across startups and established organizations due to its ability to fulfill demands for self-serve analytics, bridge infrastructure-analytic divides, and enrich data teams with enhanced skills and leadership capabilities.</p><p><strong>References:</strong></p><p><a href="https://www.getdbt.com/what-is-analytics-engineering">https://www.getdbt.com/what-is-analytics-engineering</a></p><p><a href="https://www.xomnia.com/post/bridging-the-data-gap-unraveling-the-role-of-the-analytics-engineer/">https://www.xomnia.com/post/bridging-the-data-gap-unraveling-the-role-of-the-analytics-engineer/</a></p><p><a href="https://www.linkedin.com/pulse/rising-star-understanding-role-analytics-engineer-matthew-spuffard?utm_source=share&amp;utm_medium=member_ios&amp;utm_campaign=share_via">https://www.linkedin.com/pulse/rising-star-understanding-role-analytics-engineer-matthew-spuffard?utm_source=share&amp;utm_medium=member_ios&amp;utm_campaign=share_via</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fcd0f1abb506" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Finding further efficiency in your data pipeline? Try exploring DBT for data transformations]]></title>
            <link>https://jubertroldan.medium.com/finding-further-efficiency-in-your-data-pipeline-try-exploring-dbt-for-data-transformations-f1b88be4e22b?source=rss-4af303089aef------2</link>
            <guid isPermaLink="false">https://medium.com/p/f1b88be4e22b</guid>
            <category><![CDATA[etl]]></category>
            <category><![CDATA[dbt]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Jubert Roldan]]></dc:creator>
            <pubDate>Sun, 07 Apr 2024 22:02:00 GMT</pubDate>
            <atom:updated>2024-04-07T23:06:56.395Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NcZuI1FVVQl3XNa9" /><figcaption>Photo by <a href="https://unsplash.com/@solenfeyissa?utm_source=medium&amp;utm_medium=referral">Solen Feyissa</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>As a data professional navigating through various ETL processes, I’ve found DBT (Data Build Tool) to be a game-changer in efficiently managing transformations and analytics pipelines.</p><p>In my experience working with several ETL (Extract transform Load) tools and working in the analytics/reporting space, DBT stands out for its versatility and effectiveness in data transformation tasks. (Note: I am not affiliated in any form with DBT but just a user that appreciates the solution it provides)</p><p>DBT is an open-source framework that focuses on data transformation within the ETL workflows. the framework treats the data transformation as code which enables version control, testing and documentation for maintaining quality and reproducibility.</p><p><strong>Benefits of using DBT</strong></p><p><strong>1.) DBT’s simplicity makes it easy to learn</strong>, especially for team members familiar with SQL syntax. This accessibility ensures quick onboarding and a consistent coding approach across analytics projects. Personally, I grasped the basics of DBT in just a few hours, swiftly constructing pipelines and appreciating its streamlined workflow.</p><p>an example DBT model, first 4 lines shows configuration that tells dbt to “materialize” or “create” a table out of the standard SQL script format.</p><pre>{{ config(<br>   materialized=&quot;table&quot;,<br>   schema=&quot;staging&quot;<br>) }}<br>sq<br>WITH source AS (<br>   SELECT<br>       item_id,<br>       quantity,<br>       price,<br>       description<br>   FROM source_table<br>)<br>SELECT<br>   *<br>FROM source</pre><p><strong>2.) DBT also integrates testing features</strong> to ensure data integrity throughout the transformation process. With predefined tests and the ability to customize checks, teams can validate data accuracy seamlessly. This built-in testing capability not only saves time but also reinforces confidence in analytics outputs.</p><p>example of a built in test are unique, not null and accepted values which would validate on the columns where these tests have been defined. this can easily be executed by running ‘dbt test’.</p><pre>version: 2<br><br>models:<br>  - name: orders<br>    columns:<br>      - name: order_id<br>        tests:<br>          - unique<br>          - not_null<br>      - name: status<br>        tests:<br>          - accepted_values:<br>              values: [&#39;placed&#39;, &#39;shipped&#39;, &#39;completed&#39;, &#39;returned&#39;]<br>      - name: customer_id<br>        tests:<br>          - relationships:<br>              to: ref(&#39;customers&#39;)<br>              field: id</pre><p>More on dbt test: <a href="https://docs.getdbt.com/docs/build/data-tests">https://docs.getdbt.com/docs/build/data-tests</a></p><p><strong>3.) Documentation is inherently woven into DBT</strong>, aligning with the ethos of “documenting as you code.” The platform generates comprehensive documentation automatically as you build, eliminating the need for separate documentation efforts. This not only enhances collaboration but also streamlines knowledge sharing within the team.</p><p>An example of DBT documentation, when the code is well-written and detailed, showcases self-documentation. This includes descriptions in important fields and a directed acyclic graph (DAG) that illustrates the lineage of how a model has been built</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZT4e63hs-umGQ_duu9Qvmg.png" /><figcaption>Auto generated DBT docs website (source: <a href="https://docs.getdbt.com/docs/collaborate/documentation">https://docs.getdbt.com/docs/collaborate/documentation</a>)</figcaption></figure><p>DBT’s flexibility extends beyond its core functionalities, seamlessly integrating with other tools like Airflow for orchestrating complex workflows. This interoperability empowers data teams to leverage their preferred tech stack while benefiting from DBT’s robust transformation capabilities.</p><p>Exploring DBT has shown promising potential in streamlining data workflows and encouraging effective collaboration within team setting. Its intuitive design and feature-rich environment present opportunities for enhancing data engineering processes and analytics pipelines as we continue to onboard and explore its capabilities.</p><p>I have listed down some points I have personally experienced while working with DBT, and its impact is nothing short of transformative.</p><p>to learn more, the website provides a lot of information available to get started: <a href="https://docs.getdbt.com/docs/introduction">https://docs.getdbt.com/docs/introduction</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f1b88be4e22b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Looking into the Starbucks dataset — an example Analysis on transactional data from Starbucks…]]></title>
            <link>https://jubertroldan.medium.com/looking-into-the-starbucks-dataset-an-example-analysis-on-transactional-data-from-starbucks-3e66830a1aa0?source=rss-4af303089aef------2</link>
            <guid isPermaLink="false">https://medium.com/p/3e66830a1aa0</guid>
            <dc:creator><![CDATA[Jubert Roldan]]></dc:creator>
            <pubDate>Sun, 28 Feb 2021 12:49:46 GMT</pubDate>
            <atom:updated>2021-03-01T07:45:38.307Z</atom:updated>
            <content:encoded><![CDATA[<h3>Looking into the Starbucks dataset — an example Analysis on transactional data from Starbucks mobile app to predict who picks up a discount.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*56BSYEx1aEroiLAYn8X8pw.jpeg" /></figure><p><strong>Introduction</strong></p><p>In this analysis we look into how we can build a model to predict whether or not we would get a successful promo.</p><p>the dataset used here is a simulated data that mimics customer behaviour on the Starbucks rewards mobile app. the mobile app sends out an offer and/or informational material to its customer such as discounts (%), BOGO — Buy one get one free, and informational material about whats happening in Starbucks.</p><p>This analysis, we would focus only on the actionable offers such as BOGO and discounts to look at whats happening and try to predict if the offer would be successful or not. reason being is, we would like to see the complete journey taken by the customer to see the offer effectiveness.</p><p>This analysis can be similar in other transactional data/industry, such as banking transaction, mobile transaction,..etc. which can potentially be useful as a transferrable skills which can be used on the data science career journey.</p><p><strong>Metrics</strong></p><p>The metrics that we will be using is the AUC. it indicates how well the probabilities from the positive class are separated against the negative class.</p><p>The aim is to get an AUC will be better than random for the baseline model. from there we try to improve the model.</p><p>to ensure the metrics are consistent, I also implemented KFold cross validation. the KFold cross validation algorithm divides the dataset into k non overlapping fold which is then used as a held back test set. the mean of each fold is then calculated showing a better metric for the model.</p><p><strong>Preparing the Data</strong></p><p>The dataset comprises of 3 main tables which are json files</p><ul><li>portfolio — details about the offers available</li><li>profile — customer demographics</li><li>transcript — transactions/offers the customer got from the mobile app.</li></ul><p>in order to prepare the data, we needed to assign data types and understand what is happening to the dataset. looking out for null values. below are the summary on what has been done on each dataset pre-merging them together.</p><p>portfolio — we needed to make sense of the channels available. basically this has 4 values and best to make them as dummy variables (1’s and 0’s) for them to be used later as a feature.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*3SvOwP2vcp_7LtC2XgqgYQ.png" /></figure><p>profile — the profile data is a bit sparse, however, the data that has the null values are very straightforward to fix. dropping the age = 118 removes the unecessary null values. (will be shown on the visualization part)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/699/1*c1h18wB1QyA3fBz-oVkPeg.png" /></figure><p>transcript — the transactional data is the main dataset containing the events and person. in order to process this we need to take out the key value pair within the dictionaries in the value column. this is also where we would create our target variable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/1*vfiOBbzc_PxlaDm2_mkSMw.png" /></figure><p>Merging this together, we get the below dataframe with a target variable “offer_successful” which is defined as a complete journey for a customer who received an offer, viewed the offer and completed the offer. which to me is a legit definition on how an end to end successful could look like.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WXqXTMcLf_SIFw70KrvHtA.png" /></figure><p>Steps to define offer successful:</p><ol><li>merge the 3 tables together using its specific id’s</li><li>get the dummy variables based on the event (in transcript)</li><li>group by person and offer_id</li><li>identify which customers that have passed across all the journey from offer received, offer viewed and offer completed.</li><li>For all the customers who have crossed the full journey, offer received = 1, offer viewed = 1 and offer completed = 1 then assign a number =1 otherwise assign as zero. (this will be the target variable for the model we will build in the next part.</li></ol><p>all code can be referenced from the github repo below:</p><p><strong>Analyse the data — Data Visualization</strong></p><p>just a quick look at the dataset that we have.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/420/1*S6DIcSaUT4oZz0knwWn8vw.png" /><figcaption>histogram — income demographics</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/428/1*dtsMiiuTfC-8f59uXjy_PA.png" /><figcaption>gender demographics</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/417/1*2X3Q3fh1PM5TuUsF1Vb0yA.png" /><figcaption>age profile (histogram) — removing outlier (age = 118) will show normally distributed data</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/452/1*hc5uRSDEjkPAq3KPxavWXQ.png" /><figcaption>event — shows multiple transactions compared to offer. mainly because a customer can transact even not receiving nor viewing or even completing an event.</figcaption></figure><p><strong>Data Exploration — correlation plots</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RripMgu-oMxES_CnkwDJYQ.png" /><figcaption>correlation plot does not show indication of any highly correlated data except for duration and difficulty.</figcaption></figure><p><strong>Descriptive Statistics</strong></p><p>as part of data cleaning, it is good to highlight that the data isnt always perfect, therefore outliers and null values should be dealth with accordingly.</p><ol><li>There are null values on the profile data -</li></ol><p>2. Descriptive stats.</p><p><em>portfolio</em> —</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/922/1*IQ-M43xd45LjqEp9xnHipA.png" /><figcaption>portfolio descriptive stats</figcaption></figure><p><em>profile </em>— shows around 2175 customers with age =118 which is unrealistic and having null values on gender. removing this will still provide sufficient data to use for prediction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/438/1*ve8XwuJZtZUhmanotfePtg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/790/1*EcX9OPALUJzmDVAuwCdCEA.png" /></figure><p><em>Transcript — no null values on transcript</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/366/1*w2npY7_Z50Igzh4fCWHpIw.png" /></figure><p>Amount customers are spending have a maximum of ~$1k with a mean of $13</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/670/1*x-XuNhBkxop2FD6l65NZuQ.png" /></figure><p><strong>Methodology</strong></p><p>Build a model that would predict better than random as the baseline model. then optimise using alternative model for comparison</p><p><strong><em>Data Preprocessing</em> </strong>— Note that the null values has been dropped to handle the spare dataset. still enough samples has been identified thus it is an acceptable solution.</p><p><strong>Model</strong></p><p><em>Logistic Regression — </em>is a statistical <strong>model</strong> that in its basic form uses a <strong>logistic</strong> function to <strong>model</strong> a binary dependent variable, although many more complex extensions exist. In <strong>regression</strong> analysis, <strong>logistic regression</strong> (or <strong>logit regression</strong>) is estimating the parameters of a <strong>logistic model</strong> (a form of binary <strong>regression</strong>). (source wikipedia)</p><p>initial logistic regression model provides a 0.75 Area under the curve. this means</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/456/1*1NB96e_QL_tMQ8_fA_BGSQ.png" /><figcaption>logistic regression model — ROC curve</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/575/1*FlwvESk6jiBlQu0tUHIDXg.png" /><figcaption>ROC AUC score — .67</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/507/1*Ys55xgkcqEYGXIbFH1OVOA.png" /><figcaption>showing around .7 f1 score. which is better than random.</figcaption></figure><pre>KFold validation metric (n_splits =10)<br>Accuracy: 0.672 (0.007) - baseline</pre><p><strong>Data Refinement </strong>— can we improve it a little bit further?</p><p><em>Random Forest Classifier — is an ensemble of decision trees for classifying. thought its prone to overfitting its one of the more popular machine learning algorithms out there. alongside with other variants of tree based model such as GBM, Extra Trees, XGBoost.</em></p><p>— the random forest classifier has been optimized using GridsearchCV with grid parameters. AUC 0.79 which is a better model compared to the previous.</p><p>max_depth: 100</p><p>min_samples_leaf: 5</p><p>n_estimators: 1000</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/435/1*0T-YJ9gnYSlyebpB1MiEiQ.png" /><figcaption>Random Forest classifier — a better AUC 0.79</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/566/1*3IsYGNSl_QlHNj62MMOO-Q.png" /><figcaption>ROC AUC Score</figcaption></figure><pre>KFold validation metric (n_splits =10)<br>Accuracy: 0.705 (0.010)<br><em>*improved!</em></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/508/1*6El6pI2UrHpMVN4QNUiJqg.png" /><figcaption>showing slightly better results compared to the previous model. (Logistic regression</figcaption></figure><p><strong>Conclusion/Call to action</strong></p><p>In summary, this exercise has analysed, cleaned and modelled the available data to predict members that received an effective offer.</p><p>The use of a baseline model can be used to classify such as logistic regression. logistic regression is easy to implement, and very efficient to train. though it provides a decent AUC (Area under the curve) there are models that would be able to improve the results due to the way the algorithm work. a logistic regression constructs on a linear boundary.</p><p>to use a Random Forest on the other hand, it does not construct to linear boundaries are based on ensemble of Trees which can go through the data at a specified depth. (can be optimized using GridsearchCV).</p><p>An AUC of 0.79 compared to an AUC of 0.67 does not really make a lot of difference in the real world, however its good to note that there can be a better opportunity provided there is time and processing power to perform higher number of grid parameters on the grid search.</p><p>The result can be further refined and enriched with data which would be then of value to Starbucks. other models can be tested out to improve model performance.</p><p><strong>Reflection</strong></p><p>the problem I am trying to solve is to create a model that would be able to predict if an offer would be successful or not. this is specifically defined as a received, viewed and completed. if the customer does not receive nor viewed but completed an offer it will not be valid.</p><p>this analysis can be an initial start to provide an initial model to the business, if Starbucks has no model yet and it would be better than random. if a score is provided to the business for a successful offer, it would really help out.</p><p>this model can be still further optimised by enriching features however an trying out advanced models such as XGBoost and GBM. this can be exported as a pickle file which can then be used in a production ready environment. however, it is outside of the scope of this exercise.</p><p><strong>github repository</strong></p><p><a href="https://github.com/jubertroldan/Starbucks">https://github.com/jubertroldan/Starbucks</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3e66830a1aa0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring the data HR Job Change for Data Scientist]]></title>
            <link>https://jubertroldan.medium.com/exploring-the-data-hr-job-change-for-data-scientist-b7ee0fc6edfc?source=rss-4af303089aef------2</link>
            <guid isPermaLink="false">https://medium.com/p/b7ee0fc6edfc</guid>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Jubert Roldan]]></dc:creator>
            <pubDate>Wed, 06 Jan 2021 04:16:47 GMT</pubDate>
            <atom:updated>2021-01-07T10:12:31.224Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*J5KsKjytZSoQj9UKtKeZ4A.jpeg" /></figure><h3>Peeking into the lens of a Data Scientist’s data — why they try to search for new jobs.</h3><p><strong>Introduction</strong></p><p>we have seen the rampant demand for data driven technologies in this era and one of the key major careers that fuels this are the data scientist’s gaining the title ‘sexiest jobs’ out there.</p><p>with this demand and plenty of opportunities drives a greater flexibilities for those who are lucky to work in the field. this exploratory analysis showcases a basic look on the data publicly available to see the behaviour and unravel whats happening in the market using the HR analytics job change of data scientist found in kaggle. <a href="https://www.kaggle.com/arashnic/hr-analytics-job-change-of-data-scientists/tasks?taskId=3015">https://www.kaggle.com/arashnic/hr-analytics-job-change-of-data-scientists/tasks?taskId=3015</a></p><p>There are 3 things that I looked at. (including answers)</p><ol><li>Exploring the potential numerical given within the data — what are to correlation between the numerical value for city development index and training hours?</li><li>Exploring the categorical features in the data using odds and WoE.</li></ol><p>3. Take a shot on building a baseline model that would show basic metric.</p><p><strong>Preparing the data</strong></p><p>In preparation of data, as for many Kaggle example dataset, it has already been cleaned and structured the only thing i needed to work on is to identify null values and think of a way to manage them.</p><p>as this is only an initial baseline model then i opted to simply remove the nulls which will provide decent volume of the imbalanced dataset 80% — not looking, 20% — looking. which to me as a baseline looks alright :)</p><p><strong>Analyse the data</strong></p><p>The simplest way to analyse the data is to look into the distributions of each feature. with this I have used pandas profiling. I also wanted to see how the categorical features related to the target variable. with this I looked into the Odds and see the Weight of Evidence that the variables will provide.</p><p><em>Odds — shows experience / enrolled in the unversity tends to have higher odds to move</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zYpS34nsmNAWpsY08ET07g.png" /></figure><p><em>Weight of evidence — shows the same experience and those enrolled in university.;[</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EzOQ22Ri_FTJUpAJsXI5kw.png" /></figure><p><strong>Modelling the baseline</strong></p><p>as a very basic approach in modelling, I have used the most common model — Logistic regression. though i have also tried Random Forest. for the purposes of exploring, lets just focus on the logistic regression for now.</p><p>The model i created shows an AUC (Area under the curve) of 0.75, however what i wanted to see though are the coefficients produced by the model found below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/794/1*z-nZ76D8Dxj-ydeiNTvvRg.png" /></figure><p>Coefficients (plotted)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zxiut24by5eLUcSCEoVR1Q.png" /></figure><p>this gives me a sense and intuitively shows that years of experience are one of the indicators to of job movement as a data scientist.</p><p><strong>Conclusion/Call to Action:</strong></p><p><strong>Question 1. Answer</strong> — In relation to the question asked initially, the 2 numerical features are not correlated which would be a good feature to use as a predictor.</p><p><strong>Question 2. Answer </strong>— looking at the categorical variables though, Experience and being a full time student shows good indicators.</p><p><strong>Question 3. Answer</strong> — Trying out modelling the data, Experience is a factor with a logistic regression model with an AUC of 0.75</p><p>Of course, there is a lot of work to further drive this analysis if time permits. but just to conclude this specific iteration. we have seen that experience would be a driver of job change… maybe expectations are different? maybe job satisfaction? well personally i would agree with it.</p><p>Github link — all code found in this link.</p><p><a href="https://github.com/jubertroldan/hr_job_change_ds">jubertroldan/hr_job_change_ds</a></p><p>Jupyter notebook below:</p><p><a href="https://github.com/jubertroldan/hr_job_change_ds/blob/master/HR_Analytics_DS.ipynb">https://github.com/jubertroldan/hr_job_change_ds/blob/master/HR_Analytics_DS.ipynb</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b7ee0fc6edfc" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>