<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Nagendra Nukala on Medium]]></title>
        <description><![CDATA[Stories by Nagendra Nukala on Medium]]></description>
        <link>https://medium.com/@nagendranukala?source=rss-5526408e3a82------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ZMWLFrTdeGV4zyQOY2u-tw.jpeg</url>
            <title>Stories by Nagendra Nukala on Medium</title>
            <link>https://medium.com/@nagendranukala?source=rss-5526408e3a82------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 23:30:20 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@nagendranukala/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Onboarding as a Data Scientist]]></title>
            <link>https://nagendranukala.medium.com/onboarding-as-a-data-scientist-4298e5481c92?source=rss-5526408e3a82------2</link>
            <guid isPermaLink="false">https://medium.com/p/4298e5481c92</guid>
            <category><![CDATA[product-analytics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[onboarding]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Nagendra Nukala]]></dc:creator>
            <pubDate>Thu, 23 Jun 2022 16:02:27 GMT</pubDate>
            <atom:updated>2022-06-23T16:02:27.391Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="image of the process of data science" src="https://cdn-images-1.medium.com/max/1024/1*XAzd98T6sRDKXozuzvgjWA.jpeg" /></figure><p>It’s been a few weeks since I started my new role in a product data science team as a data science engineer. This is a role that I really look up to and have been preparing on the side to learn the skills required for the job for some time now. Thanks to Adobe’s policies encouraging people to look for careers changes internally and thanks to my new manager, who agreed to give me a chance to work on his team, the hope of getting on a data science team has become reality. There is a lot of content out there on how to land a data scientist’s job, but in this blog, I plan to write my experience on getting onboarded and delivering value as a data scientist on a team with focus on product led growth (PLG).</p><p>In the past year or so, I have gone through trainings and bootcamps on python, sql, statistics, watched a lot of videos on use cases and techniques to build, tune, deploy models, worked with data science teams with my clients to help them build data engineering pipelines, schemas, data collection techniques to build and maintain data pipelines, so in my mind I was ready for this role, where I had to work with product managers to help them understand the impact of the features launched and the overall product’s success using the right data. I do have a great amount of experience in data collection, analysis, A/B testing, audience management, etc. from my previous roles at Adobe. However, onboarding into this new team has been a completely new experience, nothing less than exciting, I hope to document these learnings, so they could be of help to the larger community.</p><p>In this new data science team, the stack used is pretty standard, hadoop (hive, presto) for data storage and analysis, jenkins for data pipelines, power BI/tableau, mostly power BI for data visualization, github for version control and collaboration, jira for project management.</p><p>Most of my experience has been with SaaS solutions, when you have a SaaS solution to solve a problem, it is usually the only solution that you work on, so accesses, data lineage, data dictionary, learnings, all of it is focussed around that one SaaS solution. Since SaaS solutions are purpose built and solve one thing very well, usually there is more than one SaaS solution in play at any given point of time, that makes things more complex, that’s beyond the scope of this conversation for now.</p><p>In this or any other large organization’s data science team, there is more of a modern data stack approach, using a grouping of open source solutions to achieve the data science use cases, this is mostly because the volume of data the teams handle is huge, the presence of ninja data scientists, who can write magical sql queries and build excellent dashboards for executive reporting, also IT plays a very important role in commissioning solutions, since the storage and compute resources are all provisioned and maintained by IT.</p><p>While I practised python and sql, hoping to start wrangling, analyzing, visualizing and modelling data before coming to the team, I found out that most of my initial days went in setup and acclimatization of existing processes. One of the things that my manager asked me to do was to document all my learnings during onboarding while I am at it; I think that is a very smart move, especially because this setup is one time and once you are over the hump, there is no way you remember the pain you went through to get all possible accesses to deliver effectively. Lets go through the learnings one by one now:</p><p><strong>Hive/Presto: </strong>our team is a small part of a large organization, so we share the big data infrastructure with the larger organization, this means there are more tables that you read your data from than you own, so there is a dependency on others, so you need to know those tables and request access, since this data is sensitive, the access is maintained through a central mechanism and accessible behind the firewall. There is a good mechanism to request table access, but as with all things, providing access to another team member is not a priority for any table owner, so there will be delays in getting access, one work around I found was to email the owners for tables that I needed urgent access, that helped in getting access quickly. There will also be cases where you don’t know what you don’t know, i.e. you don’t know all the possible tables you need access to, so going over existing queries with a fine comb helps with all possible table names.</p><p><strong>Jira: </strong>In the past, I have mostly working with jira in the capacity of being a issue reporter, we used other tools for project management and time tracking, but in this team, like in any other agile development teams, people literally talk via jira, so I had to take a crash course on all things jira, this training helped me get a perspective: <a href="https://www.linkedin.com/learning/learning-jira-server-edition/get-started-with-the-server-edition-of-jira">learning jira server edition</a>. After the training, I have setup my own board, issue tracker by stage of an issue, put story points for each issue assigned, essentially, I was able to leverage jira effectively for my delivery.</p><p><strong>Github</strong>: I have been working on and off with github during my trainings, reviewing code for internal tools at Adobe, however setting up my corporate git to work with our team’s remote repo was a challenge in itself. Corporate git is only accessible behind the firewall for security reasons and in our case, push to a remote repo was restricted to ssh, also the remote repo access maintained by the repo owner and needs to provide both read and write permissions. I did struggle a bit with being able to push to the remote repo and create a pull request (PR) initially, however succeeded after a few attempts. This training on O’reilly helped with a lot of details on the inner working of github and some advanced functions: <a href="https://learning.oreilly.com/videos/complete-git-guide/9781800209855/">complete git guide</a>.</p><p><strong>Presto and sql</strong>: If anybody has aspirations to become good at data science, the one thing they should absolutely master is SQL syntax. I had some experience in sql in the past, mostly related to developing web applications and running some ad hoc queries, however brushing up the advanced sql commands like CTEs, window functions, joins of all kinds comes to rescue to help understand the queries written by the team and also to define your own queries. In addition to this, understanding the table schema, variable definitions, helps a lot when you are actually asked to run a query for solving a business question. Here is a book that will help brush through the basics of learning sql from a data scientist’s perspective: <a href="https://learning.oreilly.com/library/view/sql-for-data/9781119669364/">SQL for data scientists</a>.</p><p><strong>Splunk</strong>: Data collection/instrumentation is the achilles heal of a data scientist, it is not mainstream data science, definitely not related to modelling or statistics, so often overlooked. However we need to remember that a model output is only as good as the input data, else GIGO prevails. One of the ways to understand the quality of data instrumented is to query using splunk, you could see all the data points collected via a hit and run stat functions, counts for key value pairs. Splunk gets data in real time, so connecting an user action to the variables in the hits is possible via SQL (Splunk Query Language). Find a way to identify and isolate your hits, I did that by using my guid, helps look for hits against specific actions. Here is a great training on getting started with splunk: <a href="https://learning.oreilly.com/videos/practical-splunk-for/9780137435364/">practical splunk for beginners</a>.</p><p><strong>Power BI</strong>: This was the most challenging to setup, especially because power BI desktop only works on windows and I have a mac. There are a plethora of options to setup windows on mac, citrix machine, VMware, cloud desktop, each of them had some or the other limitation, so I ended up onboarding my personal windows machine and setup power BI desktop on it. The data layer is hadoop, so we need to setup a ODBC driver in power BI to run presto queries, there was great documentation internally, so I was able to set that up after a few tries. Once the setup hurdle was crossed, I was amazed at the punch that power BI packs for data visualization, everything is fluid and the report format can be conjured up in any possible way. Once the reports are created on power BI desktop, it can be shared with any workspace, so it can be viewed by end users. There were similar challenges as above with access control to workspaces, but again it was all one time. Here is a starter training on power BI that is helpful: <a href="https://learning.oreilly.com/videos/introduction-to-microsoft/9780137456314/">Introduction to power BI</a></p><p><strong>Data Dictionary</strong>: Our team was working on an initiative to centrally document the variables that are instrumented for our application, I was given a chance to work on that project. It gave me a chance to go over our entire application and connect each system/user generated event with the variables and values that are collected, this exercise was a massive blessing in disguise, especially since I was not very well versed with all the functionalities that are present in our product and the rationale behind the data collection for each of these functionalities. We used an application called <a href="https://www.avo.app/"><strong>Avo</strong></a><strong> </strong>for creating the data dictionary. The expectation here is that once we have onboarded all the events and properties into Avo, we will not need to use excel sheets to maintain the variable map, a.k.a data dictionary. I can see the value this centralized data dictionary is going to provide to our product team and also to the engineering team, who can take the output of the Avo events and check for the event/property values collected before a release to make sure that the data collection pipeline is not broken during release.</p><p><strong>Data pipelines</strong>: Our data pipelines are orchestrated using jenkins, though I have used some other tools like airflow and such in my earlier experience, jenkins, with all its CI/CD capabilities is a good enough solution for creating data pipelines, I didn’t have a lot of experience with jenkins, so I went back to learning the basics, this content is very helpful: <a href="https://learning.oreilly.com/videos/introduction-to-jenkins/9780136584148/">Introductions to Jenkins DevOps</a>. There is a lot of talk about moving to different setup internally by IT, but I do hope I get to learn and use jenkins for creating my pipelines, this has been on lower priority for me in this team because we have a jenkins ninja in the team already and I have been spending majority of my time consolidating the data dictionary.</p><p>As you can see it has been a learning whirlwind the last three weeks with so and there is much more to do, this was after all supposed to be an uneventful setup phase. I look forward to moving past this phase into building meaningful data visualizations and models to help our product team provide better features to serve our users better.</p><p>There is a lot more for me to learn and implement here, running and making sense of A/B tests, uplift calculation and modelling for feature sets, hoping to get to these larger topics as soon as possible.</p><p>PS: most of the links shared are from O’reilly learning, which is behind a paywall, I had the good fortune to get a subscription of O’reilly online sponsored by Adobe, I strongly suggest to get your own, it’s a great investment in your own learning.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4298e5481c92" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Sonification, an alternative to visualization?]]></title>
            <link>https://nagendranukala.medium.com/data-sonification-an-alternative-to-visualization-318f156e6c53?source=rss-5526408e3a82------2</link>
            <guid isPermaLink="false">https://medium.com/p/318f156e6c53</guid>
            <category><![CDATA[data-storytelling]]></category>
            <category><![CDATA[exploratory-data-analysis]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[data-sonification]]></category>
            <dc:creator><![CDATA[Nagendra Nukala]]></dc:creator>
            <pubDate>Wed, 09 Mar 2022 22:11:34 GMT</pubDate>
            <atom:updated>2022-03-10T16:44:13.215Z</atom:updated>
            <content:encoded><![CDATA[<h3>Augment Data Visualization with Sonification for better Data Story Telling</h3><figure><img alt="various wave forms" src="https://cdn-images-1.medium.com/max/1024/1*_peZta2N-0fkGgfWRUsftg.jpeg" /></figure><p>For the last few months, I have been talking to <a href="https://medium.com/u/fbf4949b4ffe">Jeff McSpadden</a> —CEO and co-founder of <a href="https://www.composure.care/">Composure</a> about his approach to using music for healing purposes, currently his focus is on seniors in specialized care facilities. I loved this idea; in my opinion, catering to the seniors is an excellent proposition and it serves a great cause. During our discussions about his business strategy for building out the sonic journeys based on preset business rules, he setup a call with folks at <a href="https://www.sonify.io/"><strong>Sonify</strong></a>, they have created an open source setup called <a href="https://www.sonify.io/twotone/"><strong>twotone</strong></a> for creating music compositions based on data as input — the concept is called data sonification, similar to data visualization. This blog is about my experience with data sonification.</p><p>Being at Adobe, I keep hearing about various ways of data visualization all day, there is <a href="https://experienceleague.adobe.com/docs/analytics-learn/tutorials/analysis-workspace/analysis-workspace-basics/analysis-workspace-overview.html?lang=en"><strong>Adobe Analytics Workspace</strong></a><strong> </strong>— web based data visualization solution, very useful for online data collected through the Adobe sdk. It boasts quick visualizations of massive amounts of data and is a world class solution. There is <a href="https://experienceleague.adobe.com/docs/analytics-platform/using/cja-overview/cja-overview.html?lang=en"><strong>Customer Journey Analytics</strong></a>, which is based on a similar UI as the workspace, but lets users join the online data collected via analytics with other offline sources and helps with enterprise user journey based reporting. There are other generally available purpose built visualization solutions like looker, Power BI, Tableau and a whole host of others. Data visualization is a mainstream activity in the data analysis process.</p><p>So when I first heard about data sonification, I started to wonder why is there not much emphasis on the sound aspect in data analysis, after all, hearing is one of our five senses. Here are some points that came to my mind when comparing both the mediums for data analysis:</p><ul><li>Let’s compare the optic and auditory connections to the brain: ear takes more energy for perceiving a sound as opposed to what an eye takes to perceive light, since the middle and inner ear functionality is mostly mechanical. Comparing the frequency of seeing and hearing, eye can sense light ranging from 460THz to 750 THz, about 0.7 octaves, where as the ear can hear sounds from 20 Hz to 20 KHz, around 10 octaves. Also, hearing is a tiny fraction of the process of audition, i.e. the entire process our brain follows to make sense of the world based on sound is much more than the sound itself and is complex and sophisticated as compared to the process of making sense of light, e.g. did you know we are blind at the optic disc, i.e. when the light makes it’s way from the retina to the optic disc, there is a dark spot in the middle, but our brain processes the images and covers up the dark spot with its projections based on the images around the dark spot.</li><li>Higher availability of trained analysts in visualization techniques — we all have been taught about understanding patterns in data using graphs and other visual techniques from secondary school onwards, so understanding a pattern in data by looking at a graph is easier for a lot of people, comes naturally to some. However, to understand a data pattern from a sonification journey needs a trained ear, I ll share some examples as part of this article to further clarify this point.</li></ul><p>I used a free dataset available on kaggle: the movie database, to answer the question: is there any trend on the budget of movies over time? below is a scatter plot to answer this question drawn using seaborn in python, we can see clearly that there is a positive correlation between time and the budget, basically the budget of movies have steadily increased over time. It took me around 10 minutes or so to get the data from kaggle, load, clean and visualize, understanding the pattern in data took a few seconds.</p><figure><img alt="scatter chart with budget in the y axis and release year in the x axis" src="https://cdn-images-1.medium.com/max/879/1*sc7euvRXjnexODcS9QAoPQ.png" /></figure><ul><li>The same dataset was sonified using the no code UI based application: <strong>twotone;</strong> here is the file: <a href="https://soundcloud.com/nukala-nagendra/sonify1?utm_source=clipboard&amp;utm_medium=text&amp;utm_campaign=social_sharing"><strong>soundcloud link</strong></a></li><li>Explanation of the above audio: violin tones represents the release_year and the piano tones represent the budget, you ll notice that the piano tones become significantly audible as the composition progresses (after the 6 minute mark, there is a brief period of silence around the 4 min mark), indicating a positive correlation. I was however a little lost when I was built another journey to answer the question: ‘do larger budget movies produce larger profits?’ didn’t get a coherent output. While creating the dashboard, the initial version was not really musical, so I had to take help from Jeff to make the composition more meaningful and melodious, since he has over three decades of music composition experience, it took him less than a few minutes to make some quick adjustments. I do think for two column corelation kind of problems in exploratory data analysis, sonification could help in some form. Thanks for the no code application, I was able to generate the audio file from the processed data in minimal time, but to understand the correlation, I had to hear the entire 10 minute clip. The point I am trying to make is there will be significant ETL involved in generating the audio file and the time of conducting analysis would be atleast equal to the length of the play time in the audio file.</li></ul><p>I couldn’t stop myself with this dataset; so I tried with another dataset. Since I deal with retail clients, I created a test dataset for a product funnel report (micro conversions on the various steps the user takes from seeing a product in the page to buying it on the site) of fictitious retail company and created a musical dashboard, <a href="https://soundcloud.com/nukala-nagendra/product-category?utm_source=clipboard&amp;utm_medium=text&amp;utm_campaign=social_sharing"><strong>take a listen</strong></a>. I don’t know what can be made out of it, but I did like the groove of the composition. However, if you have to analyze the data for this piece, you ll need to know to recognize the sounds of each type of instrument and also remember which instrument corresponds to which column of data, for this audio file, below is the mapping:</p><ul><li>product views on the piano</li><li>Cart additions on double bass</li><li>Orders on church organ</li><li>Units on electric guitar</li><li>Revenue on mandolin</li><li>Orders/Visits on marimba</li></ul><p>After trying a few combinations with a few datasets, tempo, bpm, octaves, apperegio and feeling like on top of the world, I started to wonder where can data sonification fit in real world scenarios, here are scenarios that come to mind:</p><ul><li>Exploratory data analysis in the sonic medium will open the data analysis process to a whole host of users with vision challenges; along with inclusion, it will help to get new perspectives from the visually challenged. In fact, I would love to see this application as a browser extension that could convert the graphs on the screen into musical notes, so it makes visual reports accessible to all.</li><li>I have developed a huge respect for audio books lately and the main reason is due to the fact that I get to hear it when I am doing something completely unrelated; like mundane household chores. So if we are able to create sonic dashboards related to data observability and monitoring, then the network operations center or similar teams in charge of monitoring data pipelines could have one less screen to monitor, instead the room could be playing a musical composition and the anomalies could be spotted by an erratic sound beat. So it would certainly be useful in monitoring and observability kind of scenarios.</li><li>Data storytelling — we all have heard about this term more than we need to, however in today’s parlance, data story telling does not have a musical component in it, other than the voice of the data story teller. What if we can design self sustaining stories with audio-visual components to be presented to the audience. We have a lot of qualified musicians and sound engineers with great knowledge in music and how it can relate to a given dataset, so if we can create opportunities for the data analysts and the music pros to get together, I am sure we can come up with intuitive ways to use the auditory medium to speak the language of data. Most of the dominant species in the animal kingdom have used the audio-visual cues to make sense of their surroundings and take the right calls, so using two of our senses will help the brain a get a better sense of the dataset than just visualization.</li></ul><p>To get a sense of sonified data, one needs to understand the basics of music composition, like tempo, melody, timbre, panning, rhythm, etc. and getting a primer on MIDI is also absolutely required. MIDI is like the http of music, it’s a format that the electronic music is coded and communicated on.</p><p>All in all; I was really intrigued with the learning experience of data sonification, I can’t wait to see stories/musicals being told based on the data. Won’t that be music to the ears!</p><p>I would recommend trying out the beta version of the application on the sonify website to get a hang of this concept.</p><p>If you are interested in further reading, below are some recommendations:</p><ul><li>Follow <a href="https://medium.com/u/c22bfb74bc0d">Hugh McGrory</a> here is one of the articles from him on data sonification: <a href="https://medium.com/sonify/data-driven-storytelling-making-civic-data-accessible-with-audio-1bc4fc17429f">https://medium.com/sonify/data-driven-storytelling-making-civic-data-accessible-with-audio-1bc4fc17429f</a></li><li>MIDI: one of the books that I read to get an understanding on MIDI was <a href="https://learning.oreilly.com/library/view/modern-recording-techniques/9781317356653/">Modern Recording Techniques, 9th Edition</a> published by O’Reilly Media — Ch 9 is related to MIDI.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=318f156e6c53" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Use Data Mesh pattern to stitch Modern Data Stack and specialized systems together]]></title>
            <link>https://nagendranukala.medium.com/use-data-mesh-pattern-to-stitch-modern-data-stack-and-specialized-systems-together-7a88700c0ad6?source=rss-5526408e3a82------2</link>
            <guid isPermaLink="false">https://medium.com/p/7a88700c0ad6</guid>
            <category><![CDATA[modern-data-stack]]></category>
            <category><![CDATA[data-mesh]]></category>
            <category><![CDATA[modern-data-platform]]></category>
            <category><![CDATA[adobe-experience-platform]]></category>
            <dc:creator><![CDATA[Nagendra Nukala]]></dc:creator>
            <pubDate>Wed, 23 Feb 2022 16:45:53 GMT</pubDate>
            <atom:updated>2022-02-23T16:48:01.093Z</atom:updated>
            <content:encoded><![CDATA[<h4>When an organization has specialized system like Adobe Experience Platform for supporting marketing use cases in conjunction with a modern data stack, then using a data mesh architecture pattern can help with orchestrating the solution integration.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AyQw5i8C_0lmv5aX9LXP_w.jpeg" /><figcaption>Image of people interacting with various systems</figcaption></figure><p>Off late, content about the modern data stack, modern marketing data stack and data mesh have been catching my attention. Considering that I work for a company that designs and builds data intensive SaaS applications, I have had the fortune to understand the design patterns of our applications, so in this article, I want to provide a quick overview on each of these topics and I will try to connect these concepts with the working of the Adobe solutions.</p><p>Lets start with understanding the core components of a modern data stack, below is a diagram of a typical modern stack setup:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*T0EBvdvg5bV8B9Ux" /><figcaption>Architecture diagram for a modern data stack</figcaption></figure><p><strong>Data ingestion: </strong>With all the online systems generating data, the modern data systems need to have ways to capture all sorts of data in all sorts of formats. Capturing online and real time data needs tooling on the end points, i.e. mobile and web to capture the data, also since it is real time, the volume is directly proportional to the traffic on the properties. Batch data ingestion has been around for a while, but the modern systems need to support different file formats, encoding, ingest faster at scale. There are some Saas solutions like fivetran, segment, stitch that operate in this space.</p><p><strong>Enterprise Data Storage: </strong>We are talking about an enterprise data lake here, storing the data in raw format as it arrives. It could be an Azure blob, S3, Google cloud storage. Some of the data could be ported into a cloud datawarehouse like a bigquery, redshift or snowflake to be used for data processing on scale. There is also a lakehouse model that databricks deltalake provides, this sits on top of the enterprise data lake, facilitates various access patterns without having to manage complex data pipelines.</p><p><strong>Data Transformation: </strong>this is where the data pipelines and plumbing is done. There are atleast two well known data transformation strategies, ETL — mostly ‘schema on write’ kind of workflows. ELT — this format is more relevant in a datalake kind of an environment, where the data is loaded first in any format received and then transformed on read (‘schema on read’). There is no right way, depends on the use case in hand. There are some open source and managed service tools like dbt, matellion in coordination with Apache Air Flow that are used for transformation purposes.</p><p>A new concept emerged in the recent past: <strong>reverse ETL —</strong>the idea here is to build solutions that make it easy for pushing the data in a required format from the enterprise data storage to SaaS solutions, it is a good tool set to have in the modern data stack. Hightouch is one of the SaaS solutions that operates in this field.</p><p><strong>Data Out/Egress: </strong>There are different access patterns that use the data, so the transformation layer needs to be able to provide as needed:</p><p><strong>BI:</strong> this is the most used access pattern, mostly used for exploratory data analysis, standard reporting, building KPIs, ad hoc reporting. There are lots of SaaS solutions that operate in this space, but some of the most known ones that satisfy a variety of use cases are: Looker, Tableau, Mode, Power BI.</p><p><strong>Data Science:</strong> There is a lot of development happening in this section, there is a new breed of solutions that serve the new breed of users: citizen data scientists, basically these are low code/no code data science and machine learning solutions to build and push models to production in a quick and seamless manner. Some of well known solutions are Dataiku, DataRobot, Alteryx.</p><p><strong>Marketing Activation:</strong> This section is particularly important to the marketers who run top of the funnel campaigns based on audiences, using reverse ETL solutions, the marketers could land home built audiences into the advertiser domains like FB, google, pinterest. Doing this helps with marketing campaign personalization and spend optimization.</p><p><strong>Data Catalog &amp; Observability: </strong>With huge amounts of data stored in the data lake, having a robust data cataloging solution is imperative, it’s like having a catalyst to speed up the usage of the data and minimize time spent by users on looking for the right datasets. There are some SaaS solutions like Altan that provide data cataloging capabilities. A lot of the engineering companies have created and open sourced them like Lyft’s Admundson, Uber’ Databook, Meta’s Nemo.</p><p>Data Observability is another core requirement in a modern data stack. With a lot of data pipelines moving lots of data across the ecosystem, it is important to understand the metrics on each step of the data flow, setup triggers and act on failures. Ability to do this effectively increases the confidence in the data workflows.</p><p><strong>Data Privacy &amp; Governance: </strong>Data privacy tools have become an absolute requirement in the modern data stack. Data lineage needs to be maintained for all the data points used by models, reporting, activation. When the user requests for a data processing based on GDPR or CCPA regulation, capability needs to be present to remove that user’s data from all possible places it is used in. Access controls should be setup so that users with the right privilege should only be able to access the eligible datasets.</p><p>Now that we are aware of the core components of a modern data stack, let’s take a look at the architecture and components of a <strong>specialized system</strong> that is equivalent to a <strong>Modern Marketing Data Stack</strong>. Generally speaking, one could stand up a marketing use case leveraging the modern data stack, but in large/medium sized organizations where marketing teams move at a fast pace with fewer resources, purpose built marketing stacks are useful to achieve the desired use cases.</p><p>Let’s look at the architecture diagram of the <strong>Adobe Experience Platform</strong> that is built on the modern data stack principles:</p><figure><img alt="Modern marketing stack architecture diagram" src="https://cdn-images-1.medium.com/max/1024/0*8Uwe7h0C-UlkphNL" /><figcaption>Architecture diagram for Adobe Experience Platform</figcaption></figure><p>Since this stack is purpose built to support the marketing efforts of large organizations at scale, there are a few components that are worth calling out:</p><p><strong>Real Time Data Ingestion: </strong>Adobe’s experience cloud solutions have a robust data collection tool set ranging from compact sdks that can be deployed within the web and mobile end points to a mature tag management solution that allows analytics engineers to manage the code on the pages independent to the application developers. This mature tool set provides the foundation to enable real time data collection at scale into the experience platform (AEP) which can then be leveraged to perform real time data processing and activation. Edge side data collection is another paradigm that is becoming mainstream. In this concept, the relevant data for any marketing related activity is captured only once and passed on to an edge, the middleware in the edge is used to push data to multiple end points, this helps with trimming the page load time.</p><p><strong>Flexible Schema Definition: </strong>Data into AEP is facilitated by schema on write. This is a lakehouse pattern to make sure that all the data is structured in a meaningful and usable format. AEP is not meant to replace an exist datalake or datawarehouse, hence schema on write makes perfect sense, since the assumption is that the data entering the system is valuable and hence required for immediate processing. AEP supports the XDM schema that has rketing destinations like Meta, google, pinterest and others, building customer journeys — this is a canvas where the marketers could write rule based logic to provide next best action for a selected segment of users. The combined output of real time and batch data for a user segment could be exported to the enterprise data lake as well.</p><p><strong>Exploratory data analysis (EDA) and data visualization</strong> can be done in various different forms. Customer Journey Analytics (CJA) provides a web based analysis and visualization solution that provides the user ability to run reports on dimensions, metrics, time based events and standard visualization capabilities like bars, stacks, fallout, cohorts, scatter plots, etc. This solution is also purpose built to support the enterprise reporting use case from a customer journey reporting perspective. Some data analysts are python/R savvy and would like to do EDA in a jupyter notebook and do specialized operations, this access pattern is supported via the data science workbench in AEP, it is not a typical CJA use case. If there are specialized use cases that require the data to be wrangled into different holding patterns, then it is best to pull out the data into a data lake and perform wrangling to do specialized one time analysis.</p><p>Looking at the above architectures, there is a question that pops up, is it better to use a modern data stack, i.e. a combination of ETL + data warehouse + reverse ETL to achieve the marketing use cases or is it better to use a fully managed modern marketing data stack. Data warehouse has a lot more data and the models are usually built on top of it using the data science applications, so it makes sense to activate directly from there. In some use cases where we need pure batch processing capabilities and the longevity of the models is higher, this combination makes sense and is recommended.</p><p>However, when we are looking at event data being collected in real time and that needs to be used to react to user’s signals in near real time, in combination with the relevant offline datawarehouse data and we need segment creation, activation, rule based journey implementation capabilities, it is better to lean on a fully managed purpose built system, so that the marketing teams can achieve the scale they need. The output of the customer segments could always be fed back into the datawarehouse via ETL processes that could in turn help in training more relevant models.</p><p>Another question that also comes to mind is about data duplication, when a fully managed marketing stack is used to solve for the marketing use cases along with a modern data stack. This is another valid point that needs to be handled. Recently I have been following the concept of <strong>data mesh: </strong>it picks up concepts from the devops world and is influenced by domain driven design. Basically the concept talks about building data products like applications that are owned by individual groups, each group publishes contracts via supported APIs to expose reliable information that any of the other groups can subscribe to. This is better explained in the following diagram:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*46O2_yHJxd-VK6xe" /><figcaption>At a high level, a data mesh is composed of three separate components: data sources, data infrastructure, and domain-oriented data pipelines managed by functional owners. Underlying the data mesh architecture is a layer of universal interoperability, reflecting domain-agnostic standards, as well as observability and governance. (Image courtesy of Monte Carlo Data.)</figcaption></figure><p>Data generated by a specialized system like a marketing stack is setup as a product that exposes the data catalog and APIs, which can be used by any other domain or can be pulled into the enterprise datawarehouse. If there is an enterprise event hub available, then the marketing stack can push the events into the event hub that can be an event gateway that other applications can sign up for, this would work if the enterprise an event driven architecture setup. Any domain specific SaaS solutions like AEP, Salesforce, SAP, could be integrated into the data enterprise event hub using this paradigm.</p><p>There were a lot of concepts discussed in this article, however the main points I want to drive home is to provide the capability to the readers to understand a modern data stack and the criteria to integrate with a specialized system like a fully managed marketing stack and start thinking about the data mesh pattern to see if it fits into the organizations’ architectural landscape.</p><p>Here are some books/articles for further reading about topics discussed in this article:</p><ul><li>Modern Cloud Platform by Alice LaPlante — published by O’Reilly Media</li><li>Data Mesh by Zhamak Dehghani — published by O’Reilly Media</li><li>Architecting Data Intensive SaaS Applications — published by O’Reilly Media</li><li>Adobe Experience Platform Overview — <a href="https://experienceleague.adobe.com/docs/experience-platform/landing/home.html?lang=en">https://experienceleague.adobe.com/docs/experience-platform/landing/home.html?lang=en</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7a88700c0ad6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Collaborative Filtering Similarity Calculations]]></title>
            <link>https://nagendranukala.medium.com/collaborative-filtering-similarity-calculations-a974ae4650?source=rss-5526408e3a82------2</link>
            <guid isPermaLink="false">https://medium.com/p/a974ae4650</guid>
            <category><![CDATA[recommendation-engine]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[recommender-systems]]></category>
            <category><![CDATA[collaborative-filtering]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Nagendra Nukala]]></dc:creator>
            <pubDate>Tue, 21 Dec 2021 23:51:31 GMT</pubDate>
            <atom:updated>2021-12-22T19:16:21.539Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4-AIyan_aMlCmTLeYIAVBQ.jpeg" /><figcaption>image of a matrix with user ratings</figcaption></figure><p>In the <a href="https://nagendranukala.medium.com/collaborative-filtering-fundamentals-d2d845e1de75">last article</a>, we went over the high level overview of all the components that make up an item-item collaborative filtering process. In this article, we will go over the math of doing the similarity calculations using an example of movie ratings by a sample of people. These calculations are simplified version of what happens in the complex recommender systems of today, but it does give the reader an idea on the base level math involved in developing a collaborative filtering recommender system.</p><p>Recommender systems consist of multiple processes, they are all stitched to work together and hence we refer to the whole thing as a pipeline, below are the major steps that we will cover in our version of the pipeline:</p><ol><li>For an active user who is a part of a rating matrix, calculate the similarity between the item to predict and all the rest of the items.</li><li>Order the items by similarity.</li><li>Select the neighbourhood.</li><li>Calculate the predicted rating.</li></ol><p>The first thing that we need to do here is to select a similarity method. Remember, we had talked about different kinds of these in the l<a href="https://nagendranukala.medium.com/collaborative-filtering-fundamentals-d2d845e1de75">ast article</a>. For this calculation, we will use the cosine similarity method.</p><p>Example Rating Matrix, 1 being the lowest and 5 being the highest rating for a movie:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/999/1*fvHMEpRY8VpyZ8XNHWMc6g.png" /><figcaption>Movie rating matrix for 6 users rating 6 movies</figcaption></figure><p><strong>Similarity calculations:</strong> All the users have rated Home Alone (HA), so we will look at each user in turn. The user is Abhilash, who also rated The Matrix (TM), Avengers: Endgame (AE), Shawshank Redemption (SR) and The Godfather (TG). We will add his rating to the list of items with Home Alone (HA), so we have: HA: [TM, AE, SR, TG]. Next is Disha, who rated the movies in the above list plus The Hangover (TH). An item should be only added once. We will go row by row and put the movies that have been rated together by users. Lets calculate the list for all the movies rated together, it would be as follows:</p><ul><li>HA: [TM, TH, AE, SR, TG]</li><li>TM: [HA, TH, AE, SR, TG]</li><li>TH: [HA, TM, AE, SR, TG]</li><li>SR: [HA, TM, TH, AE, TG]</li><li>TG: [HA, TM, TH, AE, SR]</li></ul><p>With these lists, we can calculate the similarity of each item against the other. We will calculate this using adjusted cosine similarity function. To prepare for this method, we have to normalize the item ratings based on the users’ average rating. The formula is defined as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/924/0*xN5ImT8X2c_JbwhG" /><figcaption>formula for normalizing the user rating</figcaption></figure><p>Below id the rating matrix after normalizing the ratings by average user rating:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZWofmZfuH9YoWaMqQLPXRA.png" /></figure><p>Formula for calculating similarity between two movies (Ha, TM) as highlighted in the above table is as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/663/1*HUhE3OGyDSZmCFtaafv-Aw.png" /></figure><p>Lets put some numbers to this equation and derive the similarity between both these movies based on the above ratings:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2xPup0Kfh4COOz_Q" /><figcaption>calculation for the similarity rating between two items</figcaption></figure><p>Cosine similarity gives values between -1 and 1.</p><p>Below is the full rating matrix that can be derived based on the math we did for one pair.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0lDwiWwrPrvanWCgi3ExMg.png" /><figcaption>similarity matrix for the rating array, negative correlation has been marked progressively red and positive has been marked green</figcaption></figure><ul><li><strong>Close to 1</strong>: If the two items are strongly similar, then the values tend to be closer to 1, e.g. TH and HA, these are strongly related to each other and no correlation to anything else on the chart.</li><li><strong>Close to -1:</strong> if they are strongly dissimilar, then the values tend to be closer to -1, e.g. HA and TM are strongly dissimilar to each other</li><li><strong>Close to 0: </strong>if there is not much correlation, then the values tend to be around 0. TM and TG interestingly don’t seem to have any correlation, even story wise, they seem polar opposites, both are my favourites though :-)</li></ul><p>Based on the calculation we did on the ratings, we do see there is a strong dissimilarity between HA and TM, the eager data scientist in me can’t agree more, but we need to remember that these similarity calculations are solely based on ratings, which is an explicit value, we go by what the users say. When we have less overlapping ratings or lesser ratings from users, the calculations are inconclusive, referred to as cold start problems. In the above matrix, we replaced the empty values with 0, usually there are more scientific imputation methods used to treat missing values, that can be a whole blog series in itself.</p><p>Here is python code snippet I used to calculate the column wise cosine similarity, feel free to try it out.</p><pre>import numpy as np</pre><pre>from sklearn.metrics.pairwise import cosine_similarity</pre><pre>from scipy import sparse</pre><pre>A = np.array(</pre><pre>[[-1.80, 1.20, 0, 0.20, -0.80, 1.20],</pre><pre>[0.80, -0.20, 0.80, -1.20, 0, -0.20],</pre><pre>[-0.83, 1.17, -1.83, 1.17, 0.17, 0.17],</pre><pre>[2, 0, 2, 0, -2, -2],</pre><pre>[0.67, -0.33, 0.67, -0.33, -2.33, 1.67],</pre><pre>[-1.17, 0.83, -0.17, 0.83, 1.83, -2.17]])</pre><pre>A_sparse = sparse.csr_matrix(A)</pre><pre>similarities = cosine_similarity(A_sparse.transpose())</pre><pre>print(&#39;pairwise dense output:\n {}\n&#39;.format(similarities))</pre><p><strong>Finding neighbours: </strong>Now that we have the similarity matrix, we will move to the next step in the recommendation pipeline, finding neighbours. There are mainly two types, Top N and threshold, details about it in my <a href="https://nagendranukala.medium.com/collaborative-filtering-fundamentals-d2d845e1de75">earlier article</a>. For simplicity, we will use top 2 and 0.5 as threshold, so we see how the neighbours line up in both methods:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v3Q-OMrk2fUK5MZfvqZ9BQ.png" /><figcaption>neighbour calculation using Top 2 in Top-N and 0.5 as threshold</figcaption></figure><p>Choosing the right method is important based on the use case the recommendation solution will solve for, as you can see that threshold based neighbours show stronger similarities, but there are lesser results. Top N type guarantees output, but the quality is low and the relations are weak, since we have some negative correlations also showing up.</p><p><strong>Calculating Predictions:</strong> there are a few ways to run prediction calculations, two of them that we discussed earlier are regression and classification methods. More details on these in my <a href="https://nagendranukala.medium.com/collaborative-filtering-fundamentals-d2d845e1de75">earlier article</a>. We will go with regression method for the prediction calculation here, the formula is as below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*UGymLltMr5KuY945" /><figcaption>prediction score calculation formula and explanation of terms</figcaption></figure><p>Let’s use the formula to calculate Raman’s rating of The Matrix (TM). For this calculation, we will use the movies in the neighbourhood, we know from the neighbourhood table above that TM: AE: 0.72, SR:0.21. Let’s use these numbers in the above function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/1*fm3V0-SMcrikJ7j1NigC5A.png" /><figcaption>calculation of user’s prediction for a movie by plugging values from the formula above</figcaption></figure><p>We came pretty close to the actual rating, Raman had rated TM as a 3. Now that we have the prediction, it’s easy to generate recommendation, we repeat this calculation for a number of similar items and return top N back to the client that requested the recommendations and voila! the recommendations widget is populated with relevant items.</p><p>Of course there is a lot of operations that need to be done when the recommendations are executed at scale, this is where the world of ML ops is introduced. We need to evaluate how much of this compute intensive calculations can be done offline, how to qualify data for training and testing, setting up continuous validation of the input data and model output, model drift and data drift calculations, what triggers a model retraining pipeline. All these items are a whole world in themselves, but for now, let’s be happy that we know the basic math behind the recommender engine.</p><p>If you are interested in reading about recommender systems, the book mentioned below is an absolute must read, most of the content I wrote in this article is a summary of a single chapter in this book.</p><ul><li><strong>Practical Recommender Systems — by Kim Falk — available on o’reilly online.</strong></li><li>Item-based Collaborative Filtering Recommendation Algorithms” by Badrul Sarwar et al — <a href="http://files.grouplens.org/papers/www10_sarwar.pdf"><em>http://files.grouplens.org/papers/www10_sarwar.pdf</em></a></li></ul><p><em>PS: I had a lot of fun writing this article, partly because this topic is dear to me, but mostly because the people mentioned in the rating matrix are my very dear friends, so this writing this article took me back to my school days and reminded me of all the great times we all had. Also, the people are real, ratings are not theirs though, so please don’t judge their choices with the above ratings ;-)</em></p><p><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a974ae4650" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Collaborative Filtering Fundamentals]]></title>
            <link>https://nagendranukala.medium.com/collaborative-filtering-fundamentals-d2d845e1de75?source=rss-5526408e3a82------2</link>
            <guid isPermaLink="false">https://medium.com/p/d2d845e1de75</guid>
            <category><![CDATA[recommender-systems]]></category>
            <category><![CDATA[collaborative-filtering]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Nagendra Nukala]]></dc:creator>
            <pubDate>Thu, 25 Nov 2021 18:58:58 GMT</pubDate>
            <atom:updated>2021-11-26T03:32:23.295Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="sorting items into the right categories" src="https://cdn-images-1.medium.com/max/1024/1*ST9929GM2l9A_ZVJcIlvdA.jpeg" /></figure><p>I have been trying to learn about recommender systems; we all have interacted with recommenders in some way or form in every walk of life. It could be, asking our like-minded friend for a movie suggestion, seeing product recommendation on eCommerce sites like Amazon, etc., recommenders have been around forever.</p><p>With the advent of the likes of Netflix, Spotify, etc.; recommender systems have taken off to another level all together. It makes sense for all of us to learn and understand the basics of such ubiquitous applications like recommender systems. Since this topic is so vast, in this article, I am summarizing about one of the most straight forward pieces of recommender systems: item — item collaborative filtering.</p><p>Item — item collaborative filtering shows up to the end consumers as widgets where items are recommended based on the items that the users or cohorts of similar users have interacted/rated. In Netflix kind of scenario, it is the widget that shows: “Users who liked what you like also like..”. Let’s see what are the core components to build this functionality out.</p><p>These are the main steps for building out item based filtering:</p><ol><li>Calculate similarity between the item to predict and all the rest</li><li>Order items by similarity</li><li>Select the Neighbourhood</li><li>Calculate the predicted rating</li><li>Use the predictions to calculate recommendations</li></ol><p><strong>Similarity calculations:</strong> this is mostly done offline in a batch, since calculating item similarity can be a memory intensive process, especially if we are talking about all the SKUs in the product catalog. Types of similarity functions:</p><ol><li>Jaccard Similarity: when we are calculating similarity on unary or binary dataset like bought (1), not bought (0), we use the Jaccard similarity function.</li><li>Cosine similarity: This is used mostly when there is quantitative data, like rating a product/movie (7/10). There is a derivative of this function called adjusted cosine, that averages out the individual user rating of each item to remove rating bias.</li><li>Pearson Similarity: this algorithm is also used for quantitative data, it helps identify likes, dislikes and unrelated items/users, also removes rating bias and is gives very similar results as adjusted cosine.</li></ol><p>All these functions are statistical methods that we learn in school. I ll write another article with details on calculating similarity using the adjusted cosine function. For now, we will assume we were able to run code to generate the similarity matrix and store it in a table ranked in the order or most similar to most dissimilar.</p><p><strong>Clustering: </strong>This is usually done to make the recommender respond faster. Basically, with clustering we set a criteria that helps look for less than all the items tagged as similar. There are various ways to do it, but for starters and to keep things simple, we can go with the following:</p><ol><li>Top N: we tell the system that N number of items should be in the neighbourhood, so you define an item as the center and consider N closest values to it in a 2D plot to be a part of the neighbourhood.</li><li>Threshold: In this method, we define a cut off value in the similarity matrix to be a part of the neighbourhood, e.g. we want all the items have a similarity score above 0.7 (scores range: -1 to 1) to be a part of my cluster, then no matter how many values fall in the range, they all become a part of the cluster.</li></ol><p>The choice of Top N and threshold is a trade off between quantity and quality, the business use case needs to define the choice, either of them work well.</p><p><strong>Prediction: </strong>The most common ways to make predictions are regression and classification.</p><ol><li>Regression: this basically means you take an average of all the ratings and make the prediction, mostly used in real estate setting, when coming up with a list price of an house based on comparables of the neighbourhood.</li><li>Classification: in this method, you take rank the items by frequency of occurrence and predict the value of the item that occurs most. For example, when you see an item mostly rated 4, then you give it a 4 too, basically you have rated it based on the classification method.</li></ol><p>Use one of the above methods to predict the rating of an item in the neighbourhood. To generate a recommendation, we repeat the calculation for as many items as we need, lets say we need to send out top 5 recommendations, then we run the prediction rating on a set of items, rank them in the order of high to low and send out the top 5 back to the application.</p><p>Some other details to know about collaborative filtering:</p><ul><li><strong>Sparcity:</strong> If the items don’t have a lot of ratings, then the recommendations will mostly fall back to recommend the most popular items, since the dataset won’t be dense enough to get a relevant recommendation.</li><li><strong>Gray Sheep: </strong>this refers to a type of users who have unusual tastes and would be impossible to find related users or items, this is possibly what happens when my kids watch paw patrol from my account, while I mostly watch documentary and crime series, the recommender systems must have put me in the gray sheep cohort. :-)</li><li><strong>Cold Start:</strong> collaborative filtering needs data to do the predictions, going by a thumb rule, apparently a good collaborative filter algorithm needs upto 20 ratings on an item before being able to run predictions, so what happens to new items? Applications either run a separate widget for new arrivals (I have seen this in Netflix, Amazon) or this could possibly be a reason to run the explore/exploit method for introducing newer products.</li><li><strong>Similarity: </strong>Collaborative filtering is content agnostic, it follows users’ behavioural trends. This again pushes the output towards more popular content.</li></ul><p>The main positive about collaborative filtering is that is content agnostic. Only ratings and interactions between items are needed to get started, hence this is a great way to start learning about recommender systems.</p><p>Want to learn more about recommenders, here are some places to find more information:</p><ul><li>Item-based Collaborative Filtering Recommendation Algorithms” by Badrul Sarwar et al — <a href="http://files.grouplens.org/papers/www10_sarwar.pdf"><em>http://files.grouplens.org/papers/www10_sarwar.pdf</em></a></li><li>Practical Recommender Systems — by Kim Falk — available on o’reily online.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d2d845e1de75" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding ITP and its effects]]></title>
            <link>https://nagendranukala.medium.com/understanding-itp-and-its-effects-67c137cd4d84?source=rss-5526408e3a82------2</link>
            <guid isPermaLink="false">https://medium.com/p/67c137cd4d84</guid>
            <category><![CDATA[adobe-experience-cloud]]></category>
            <category><![CDATA[third-party-cookie]]></category>
            <category><![CDATA[first-party-data]]></category>
            <category><![CDATA[itp]]></category>
            <category><![CDATA[adobe-analytics]]></category>
            <dc:creator><![CDATA[Nagendra Nukala]]></dc:creator>
            <pubDate>Tue, 03 Aug 2021 18:35:59 GMT</pubDate>
            <atom:updated>2021-08-03T23:51:46.909Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Eh7HV-Hm6pgBitJhNq5G2g.jpeg" /></figure><h3>What is covered in this article?</h3><ul><li>ITP overview</li><li>Browser treatment of cookies and Adobe cookies’ lifetime</li><li>How to see what ITP is blocking in your browser</li><li>Next steps and thoughts on future of tracking</li><li>References</li></ul><p><strong>Note:</strong> <em>the details in the article are my thoughts based on the articles I have read and the research I have done, this is not an official Adobe statement on ITP and its effects on Adobe solutions.</em></p><h3>ITP — what is it?</h3><p>ITP stands for Intelligent Tracking Prevention, the underlying concept has been introduced by the webkit.org, this is a nonprofit institution that defines the core behavior of browsers. Safari and all major browsers running on iOS platform comply with these principles. The main goal of tracking protection is to remove user fingerprinting and scripts silently tracking user data without appropriate consent.</p><p>In safari this is implemented based on algorithmic model that trains on the user’s browser locally, hence is most effective for that user.</p><h3>Browser restrictions based on the latest release — ITP 2.3:</h3><p><strong>1st party context:</strong><em> </em>if I am browsing adobe.com, all the assets and links with adobe.com as the domain are considered first party context.</p><p><strong>3rd party context:</strong> if I am browsing adobe.com and it is loading an image from google, then the url for that asset is considered third party context while on adobe.com.</p><p>Kinds of storage available in a browser:</p><ul><li>Cookies</li><li>indexedDB</li><li>localStorage</li><li>sessionStorage</li></ul><p>Below table provides an understanding of the how each browser treats a cookie and classifies trackers. I have taken information available on various blogs to compile this table, details of the referred sites are given in the references below. One additional thing that I have added in this table is to provide the details on how each browser treats the Adobe Experience Cloud cookies, User scenario: user visited adobe.com home page from browsers, e.g. chrome, firefox and safari.</p><p>To make complete sense of the below table, you ll need to know the definition of each cookie set by Adobe solutions, which can be found here: <a href="https://experienceleague.adobe.com/docs/id-service/using/intro/cookies.html?lang=en">https://experienceleague.adobe.com/docs/id-service/using/intro/cookies.html?lang=en</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/794/1*cQyu_UMQ1vShNIZSkvLb9A.png" /></figure><p><strong>ITP release timeline:</strong></p><ul><li>ITP 2.1 — Safari — Cookie 7 days (first party JS cookie) (only for those sites identified as having cross tracking capabilities)</li><li>ITP 2.2 — Safari — Cookie 1 day (first party JS cookie) (only for those sites identified as having cross tracking capabilities)</li><li>ITP 2.3 — Safari — All data capped to 7 days (script writable — from JS)</li><li>CName Cloaking — Blocked in Firefox and Safari and Brave (rollout on the way)</li></ul><h3>Seeing what ITP blocks:</h3><p>There are a few ways of seeing the ITP classification on a browser, since the algorithm runs locally, the restriction is not uniform for all users on safari, below are the ways to look at the ITP restrictions:</p><p><strong>Step 1: </strong>The privacy report: you can look at the privacy report for any site by clicking on the small shield icon to the left of the address bar, it is not necessary that all the domains shown in the privacy report are being blocked, since the blocking happens via the algorithm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tupcOOAUpJXIM-6dG5b_Rg.png" /></figure><p>If you want to see more details, click the (i) button on the right side of the dialog box.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/659/1*YeKokyh_GUHrYi1oUkv1XQ.png" /></figure><p><strong>Step 2:</strong> There is a feature in iOS 14+ to see the domains that are classified as. trackers. To see that, follow the instructions below which are based on Mac OS big sur 11.4 and Safari 14.1.1.</p><p>a. Open Safari</p><p>b. Open ‘develop’ option in the main menu</p><p>c. Click on ‘intelligent tracking prevention debug mode’</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/419/1*2LI3AlvngUc3a5DiZvHPdQ.png" /></figure><p><strong>d.</strong> Go to any website: e.g. adobe.com</p><p><strong>e.</strong> Open the web console on Safari by going to develop and select ‘show javascript console’</p><p><strong>f.</strong> Filter the messages by ‘ITP’</p><p><strong>g.</strong> 3rdpartytestwebkit.org is set by default in ITP messages, so you should see. at least that entry</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z_89W-oJ2bn4re8_rmKIbg.png" /></figure><h3>Impact of ITP on Adobe solutions:</h3><p>Any ad tech or mar tech solution leveraging visitor identification via browser cookies would get affected. Below is a possible list of issues that could be seen in the Adobe solutions due to the cookie restrictions:</p><ol><li>ID service — Visitor identification is set to 7 days in Safari, as opposed to 2 years in other browsers. When a visitor comes back after 7 days, the visitor gets a new ecid value.</li><li>Analytics — Since Analytics depends on ECID for visitor identification, the visitors in Analytics are counted as new visitors if they visit the site after 7 days gap. More details on ITP effects on Analytics and method to build ITP affected visitors report on this link: <a href="https://experienceleague.adobe.com/docs/analytics/technotes/cookies/cookies.html?lang=en">https://experienceleague.adobe.com/docs/analytics/technotes/cookies/cookies.html?lang=en</a></li><li>Target — The active tests that run over 7 days are affected. When a user comes back to the site any time after 7 days, the user is identified as a new visitor and would be evaluated for the test again. More details on the effects of ITP on target in this link: <a href="https://experienceleague.adobe.com/docs/target/using/implement-target/before-implement/privacy/apple-itp-2x.html?lang=en">https://experienceleague.adobe.com/docs/target/using/implement-target/before-implement/privacy/apple-itp-2x.html?lang=en</a></li><li>Audience Manager — Profile merge rules across devices get affected when the last authenticated profile option is used. The segments the user qualified for in the existing device would work, given that the user comes back in 7 days. If the segment is written against the user profile, then the segments would be returned in authenticated state only.</li></ol><h3>What Next:</h3><p>Any cookies that are set on the browsers with HTTP response, i.e. using the Set-Cookie header in the HTTP response from a first party domain are not affected by the ITP 2.3 rules, as long as there is no cname cloaking being done. In the below screenshot from apple.com, you can see that there is a cookie called ‘dssid2’ that has around 2-year validity:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Oz6rrhSBnycgYr4yWRs09w.png" /></figure><p>We should be able to leverage first party cookie set using a response header method for visitor identification and mitigate the 7 day cookie drop issue. However, this method would work currently only for Analytics, when the data is sent to Analytics via bulk data ingestion API. Audience Manager and Target cannot consume a client side first party identity as the solutions’ primary identity just yet.</p><p>To set this First-party Cookie, client-side IT teams need to develop and implement a visitor identification service that would ID the users in a first-party cookie context, and Adobe solutions need to develop the capability to ingest data from the clients’ server side based off the client maintained first party identity.</p><p>I have seen some posts talking about using a first party domain as a proxy to route the Analytics data eventually to Adobe servers, however this route is implementation heavy, since we are talking about setting up a first party proxy to be scaling to process all the requests from the client, once all the hard work is done, it’s a matter of time before the safari ITP engine picks up the proxy as a tracking domain and cuts the cookie lifetime. Remember, we are up against an ML model that runs on the individual browsers, so the domain exclusion is specific to each individual’s browsing behavior.</p><p>I am hoping the capability to capture data on a client specific first party identifier using the server-side launch methodology is launched soon, so that we can enable Adobe clients to capture their first party data securely.</p><h3>References:</h3><p>Following articles were referenced during writing my article, please go over these articles for further knowledge on ITP impact in general:</p><ul><li><a href="https://www.simoahava.com/privacy/intelligent-tracking-prevention-ios-14-ipados-14-safari-14/">https://www.simoahava.com/privacy/intelligent-tracking-prevention-ios-14-ipados-14-safari-14/</a></li><li><a href="https://www.cookiestatus.com/">https://www.cookiestatus.com/</a></li><li><a href="https://magicpixel.io/introduction-to-intelligent-tracking-prevention-with-tag-management/">https://magicpixel.io/introduction-to-intelligent-tracking-prevention-with-tag-management/</a></li><li><a href="https://medium.com/adobetech/safari-itp-2-1-impact-on-adobe-experience-cloud-customers-9439cecb55ac">https://medium.com/adobetech/safari-itp-2-1-impact-on-adobe-experience-cloud-customers-9439cecb55ac</a></li><li><a href="https://webkit.org/blog/11338/cname-cloaking-and-bounce-tracking-defense/">https://webkit.org/blog/11338/cname-cloaking-and-bounce-tracking-defense/</a></li><li><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1598969">https://bugzilla.mozilla.org/show_bug.cgi?id=1598969</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=67c137cd4d84" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Significance of Size & Power Calculation in A/B Tests]]></title>
            <link>https://medium.com/adobetech/significance-of-size-power-calculation-in-a-a-b-test-8c21ab9ce171?source=rss-5526408e3a82------2</link>
            <guid isPermaLink="false">https://medium.com/p/8c21ab9ce171</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[a-b-testing]]></category>
            <category><![CDATA[adobe-experience-cloud]]></category>
            <category><![CDATA[adobe-target]]></category>
            <dc:creator><![CDATA[Nagendra Nukala]]></dc:creator>
            <pubDate>Tue, 13 Jul 2021 15:41:01 GMT</pubDate>
            <atom:updated>2021-07-20T20:53:09.838Z</atom:updated>
            <content:encoded><![CDATA[<p>A/B testing is a common mechanism to test out a hypothesis. When A/B tests are run on the web/mobile or other interfaces using solutions like <strong>Adobe Target</strong>; size, power, statistical significance calculations are taken care in the solution itself. That is a significant advantage of going with a seasoned SaaS solution. Lift and confidence is calculated inline for each alternative in the experiment, <strong>Adobe Target</strong> lets the user know when is the right time to push the winner to 100%. There is also functionality to smoothen the outcome metric by removing outliers, so that nuance in the power and size calculation is taken care of. There are advanced features like <a href="https://experienceleague.adobe.com/docs/target/using/activities/auto-allocate/automated-traffic-allocation.html?lang=en"><em>auto-allocate</em></a> that even push the winner to 100% when the statistical significance is hit. It is recommended to use this feature if you have access to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5OrSOE-v4oLorn96u089ag.jpeg" /></figure><p>However, A/B tests are not just limited to the web or mobile app, they are run for comparing audience performance in <strong>Adobe Audience Manager</strong> using the <a href="https://experienceleague.adobe.com/docs/audience-manager/user-guide/features/audience-lab/audience-lab.html?lang=en"><em>audience lab feature</em></a>, for emails and push messages in <strong>Adobe Campaign</strong>, templates in <strong>Adobe Experience Manager</strong>, page versions on web servers, machine learning algorithm performance comparison, they are every where. So the understanding of size and power calculations as explained above comes in handy. It will help the practitioners guide the business teams pick the right experiment that will give the most benefit. Understanding the math behind, size, power, statistical significance, etc. would make sure right alternative is chosen with a great degree of confidence and more importantly, the wrong alternative is not pushed to production, more commonly referred as type I and type II errors in statistical parlance.</p><p>Recently I was reading a book (<em>Analytical Skills for AI and Data Science by Daniel Vaughn</em>) that gave a primer on A/B testing process, there were a few details that stood out, so I thought of putting it down along with resources from <a href="https://experienceleague.adobe.com/#home">Adobe experience league</a> to help with estimating the size and power for a given A/B test.</p><p><strong>Definitions of size/power calculations in A/B tests:</strong> A size of a statistical test is the probability of encountering a false positive. The power of the test is the probability of correctly finding the difference between treatment and control.</p><p>The formula of calculating sample size for a test with two alternatives is as below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/382/1*QSbPhVc8yC1YxczSHzRIxA.png" /><figcaption>sample size calculation</figcaption></figure><p>Here t(k) is a critical value to reject hypothesis with probability k according to a t distribution, α and 1−β are size and power of the test. MDE is the minimum detectable effect of the experiment, N are the number of users in the test, P is the fraction assigned to the treatment group and Var(Outcome) is the variance of the outcome metric you are using to decide if the test is successful or not. Full details of the calculations can be found in this <a href="https://github.com/dvaughan79/analyticalskillsbook/blob/master/jupyter_notebooks/appendix.ipynb">github</a> link posted by the author of the analytics skills book.</p><p><strong>Two key takeaways from this calculation:</strong></p><ol><li><strong>The noisier the outcome metric chosen, the larger the sample size needed for each recipe of the test required, hence the test needs to run longer to achieve statistical significance.</strong></li><li><strong>The smaller the MDE between alternatives, the more the sample size required to conclude the test result with a high statistical significance.</strong></li></ol><p>Lets’ calculate the sample size that is required to achieve a given MDE with an example. Lets’ say your hypothesis is that if we provide a discount of 10% of the seasonal products on your catalog, the sales will increase further. Since there is a variety of products in the seasonal category, the variance (Var(outcome)) is at $ 6000 (Standard Deviation S is about $ 77). We choose the size and power (5% and 80%).</p><p>In discussions with business team, we came to a conclusion that the alternative would only make sense if we see a minimum effect (MDE) of $ 11 (or 15% of 1 standard deviation (s)). Assuming we run both the alternatives at 50%, when we run our size calculator, we use the formula as below:</p><p>N = ((t_alpha + t_oneminusbeta)^2 * var_outcome)/(MDE^2 * P * (1-P))</p><p>Calculating t_alpha and t_oneminusbeta from size and power values need the degrees of freedom input as well, I ll not include those details in this article. Details of the calculation are also present in the <a href="https://github.com/dvaughan79/analyticalskillsbook/blob/master/jupyter_notebooks/appendix.ipynb">github link as mentioned earlier</a>.</p><p>Based on the above formula, we see that we need at least 1,228 participants in the experiment. If we have anything lesser than this number, our outcome would be prone to a type I or type II error, which is not desirable.</p><p>In the above example, lets’ assume that we are going to provide the discount offer for seasonal products in an email and the average open rate of emails is around 5%, then to get at least 1,228 participants for our experiment, then we ll have send around 24,560 emails to achieve statistically significant MDE. Since you know the math now, you can objectively tell the email ops team to select an audience that meets the sample size criteria.</p><p>One another resource that is available for running the size and power calculations is this link:<em> </em><a href="https://docs.adobe.com/content/target-microsite/testcalculator.html"><em>https://docs.adobe.com/content/target-microsite/testcalculator.html</em></a>. Feel free to use the size and power calculator available on this link to estimate the performance of your prospective A/B test, so you don’t have to run complex size and power calculations offline before ranking your A/B tests in the order of impact.</p><p>Here is a great explanation about the various statistical inputs that are used for calculating the outcome of an A/B test: <a href="https://experienceleague.adobe.com/docs/target/using/activities/abtest/sample-size-determination.html?lang=en"><em>https://experienceleague.adobe.com/docs/target/using/activities/abtest/sample-size-determination.html?lang=en</em></a>. It gives a good understanding of the under the hood calculations.</p><p>Now that we have a basic understanding of the size and power calculations, lets’ make it a practice to estimate the sample size before qualifying the hypothesis for A/B testing.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8c21ab9ce171" width="1" height="1" alt=""><hr><p><a href="https://medium.com/adobetech/significance-of-size-power-calculation-in-a-a-b-test-8c21ab9ce171">The Significance of Size &amp; Power Calculation in A/B Tests</a> was originally published in <a href="https://medium.com/adobetech">Adobe Tech Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>