<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[ShopStyle Engineering - Medium]]></title>
        <description><![CDATA[Building the ultimate fashion shopping destination - Medium]]></description>
        <link>https://medium.com/shopstyle-engineering?source=rss----b45652c2366---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>ShopStyle Engineering - Medium</title>
            <link>https://medium.com/shopstyle-engineering?source=rss----b45652c2366---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 21:55:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/shopstyle-engineering" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Javascript Equality Cheatsheet]]></title>
            <link>https://medium.com/shopstyle-engineering/javascript-equality-cheatsheet-fb75412d6e4b?source=rss----b45652c2366---4</link>
            <guid isPermaLink="false">https://medium.com/p/fb75412d6e4b</guid>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Loïc Combis]]></dc:creator>
            <pubDate>Wed, 16 Jun 2021 17:03:15 GMT</pubDate>
            <atom:updated>2021-06-16T17:03:15.705Z</atom:updated>
            <content:encoded><![CDATA[<p>If you have experience developing JavaScript applications, you probably already know how confusing this is. 😭</p><p>Truthy, Falsy… == or === … <strong>What the heck?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AVyFU4Z4lvW59bAb1YvlgQ.png" /><figcaption>Source: <a href="https://pixabay.com/illustrations/programming-coding-laptop-6119584/">Pixabay</a>.</figcaption></figure><h4>Truthy vs Falsy</h4><p>First, an important notion. Every value in <strong>javascript</strong> is either Truthy or Falsy. It means that, unlike other modern languages, you don’t need to actually compare a value or have a boolean value to create a <strong>boolean expression</strong>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2ce2e4816c1d79072a1ec0d3e9c31dc2/href">https://medium.com/media/2ce2e4816c1d79072a1ec0d3e9c31dc2/href</a></iframe><p>Meaning that you can directly pass a non-boolean variable into boolean expressions (e.g.: if or while clauses). The value will automatically default to true if the value is truthy and to false if it is falsy.</p><p>🚨🚨 <strong>Note:</strong> All javascript values are truthy except for undefined, null, NaN, &#39;&#39; (empty string) &amp; 0 (as a number).</p><p>And this is what it looks like in a slightly more complex expression.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7c17057afc45e1fc40ae00ec34e12989/href">https://medium.com/media/7c17057afc45e1fc40ae00ec34e12989/href</a></iframe><p>Ok! We now know that all javascript values resolve to either a truthy or falsy value when used in boolean expressions. Let’s now have a look at equality.</p><h3>Equality</h3><p>There are two operators checking equality in javascript == &amp; === (with respectively != &amp; !==).</p><p><strong>🚨🚨 Note:</strong> Both equality (and difference) operators are symmetric, meaning that for all A &amp; B, A == B and A === B are respectively identical to B == A and B === A.</p><h4>Loose Equality (or double equality)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sy4ScX41XJ7XLAmbHW4kqQ.png" /><figcaption>Javascript Loose Equality Table.</figcaption></figure><p>So, a couple of thoughts here. First…</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FpPhyAv5t9V8djyRFJH%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FpPhyAv5t9V8djyRFJH%2Fsource.mp4&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FpPhyAv5t9V8djyRFJH%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="317" frameborder="0" scrolling="no"><a href="https://medium.com/media/b834229046c5bc2e3eea40264a76b464/href">https://medium.com/media/b834229046c5bc2e3eea40264a76b464/href</a></iframe><p>Second: please don’t use ==. Never 😡. Unless I’m missing something, I really can’t think of a scenario where this is better than ===(feel free to comment if you have examples).</p><p>Now some explanations about the table above. Loose equality will verify equality after converting both values to a common type. Here are some examples:</p><p>→ For a comparison between a number with a boolean or string , the values are converted to numbers.</p><pre>// convert true to a number.<br>true == 1 // is true</pre><pre>// convert &quot;1&quot; to a number.<br>1 == &quot;1&quot; // is true</pre><p>→ For a comparison between a string with a boolean, both values are converted to number.</p><pre>// true is converted to the number 1, so is &quot;1&quot;.<br>true == &quot;1&quot; // is true</pre><pre>// true is converted to the number 1, the conversion of &quot;true&quot; to a // number results in NaN<br>true == &quot;true&quot; // is false</pre><p>→ For a comparison of a number or string with an object, the object is converted into a primitive value using combinations of .toString() &amp; .valueOf().</p><pre>// [1] is converted to &quot;1&quot; using .toString() then &quot;1&quot; is converted to a number as described above.<br>1 == [1] // is true</pre><pre>// [[]] is converted to &quot;&quot;.<br>&quot;0&quot; == [[]] // is false.</pre><p>→ For a comparison between a boolean with an object , the object is converted to a primitive and the boolean value into a number.</p><pre>// false is converted to 0, [[]] to &quot;&quot; and then to a number.<br>false == [[]] // is true</pre><p><strong>🚨🚨 Note:</strong> The loose equality operator == does <strong>NOT</strong> have the <em>transitive</em><strong> </strong>property — if A == B &amp; B == C then A == C— For example, 0 == [[]] &amp; 0 == &quot;0&quot; are true but &quot;0” == [[]] is false.</p><p>Finally, as you can see in the table above, the comparison of two object values always results in false. So is any comparison with NaN. The reason being that javascript compares the references and not the actual value for complex object values (better performances).</p><pre>// This creates two different references of the value [0]<br>[0] == [0] // is false</pre><p>⚠️⚠️ If two variables share the same reference, the comparison would return true.</p><pre>// Create a uniq instance<br>const array = [0]</pre><pre>// array &amp; array2 reference the same value in the memory.<br>const array2 = array</pre><pre>array == array2 // is true.</pre><h4>Strict Equality (or triple equality)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-wQ4ON7UURAIUOtw4eYaVg.png" /><figcaption>Javascript Strict Equality Table.</figcaption></figure><p>Ok… I feel better, this table makes way more sense to me 😋…</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FVgldw2H2uRLRXTtWhS%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FVgldw2H2uRLRXTtWhS%2Fsource.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FVgldw2H2uRLRXTtWhS%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="362" frameborder="0" scrolling="no"><a href="https://medium.com/media/1582d2964aacb7ff8f52ac7caf6ee4c8/href">https://medium.com/media/1582d2964aacb7ff8f52ac7caf6ee4c8/href</a></iframe><p>The strict equality operator === will first compare the type of the operands. If different → false. If identical → Compare the values (or references for objectvalues).</p><p>Not only it is safer to use the triple-equality operator because it reduces the risk of side effects, but it also gives better performances as no conversion occurs.</p><h4>What about inequalities?</h4><p>Javascript also allows us to use &gt;, &lt;, &gt;= &amp; &lt;= to compare non-numeric values and different types.</p><p>⚠️⚠️ I would recommend NOT to use inequality operators for non-numeric values. Even less to compare different types. It is already very confusing for equality 😫. It adds the complexity of the order on top of it. Please, use well-defined custom comparators instead.</p><p>If you still want to risk it… Here are a few examples:</p><p>→ string value comparison matches the alphabetical order.</p><pre>&quot;a&quot; &gt; &quot;b&quot; // is false</pre><pre>&quot;a&quot; &gt; &quot;A&quot; // is true</pre><pre>&quot;a&quot; &gt; &quot;1&quot; // is true</pre><pre>&quot;A&quot; &gt; &quot;1&quot; // is true</pre><pre>&quot;€&quot; &gt; &quot;£&quot; // is true</pre><p>→ The comparison with undefined and NaNis always false.</p><pre>1 &gt; undefined // is false</pre><pre>1 &lt; undefined // is false</pre><pre>1 &gt; NaN // is false</pre><pre>1 &lt; NaN // is false </pre><p>→ The comparison with null is equivalent to the comparison with 0.</p><pre>1 &gt; null // is true</pre><pre>-1 &gt; null is false</pre><p>You can find more examples on <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Greater_than">MDN’s “Greater Than (&gt;)” documentation page</a>.</p><p>This is it! 😎</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2F8UhGWWnohtzva%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2F8UhGWWnohtzva%2Fsource.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2F8UhGWWnohtzva%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/25895883be741990bc6b6d6afe23f0ae/href">https://medium.com/media/25895883be741990bc6b6d6afe23f0ae/href</a></iframe><p>Thanks for taking the time to read this article! Always a pleasure to publish new content!</p><p>Feel free to clap if you liked this post 😁. If not, you can always yell at me on Twitter: <a href="https://twitter.com/CombisLoic">@CombisLoic</a></p><p>Check out my other posts on <a href="https://loic-combis.medium.com/">Medium</a> 🥰</p><h3>References</h3><p><a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Equality_comparisons_and_sameness">Equality comparisons and sameness</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fb75412d6e4b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/shopstyle-engineering/javascript-equality-cheatsheet-fb75412d6e4b">Javascript Equality Cheatsheet</a> was originally published in <a href="https://medium.com/shopstyle-engineering">ShopStyle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI tales, your first Machine Learning model with Python, Conda, Scikit-learn& Jupyter.]]></title>
            <link>https://medium.com/shopstyle-engineering/ai-tales-your-first-machine-learning-model-with-python-conda-scikit-learn-jupyter-8a8ce7b151c?source=rss----b45652c2366---4</link>
            <guid isPermaLink="false">https://medium.com/p/8a8ce7b151c</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[scikit-learn]]></category>
            <dc:creator><![CDATA[Loïc Combis]]></dc:creator>
            <pubDate>Wed, 16 Jun 2021 17:03:13 GMT</pubDate>
            <atom:updated>2021-06-16T17:03:13.499Z</atom:updated>
            <content:encoded><![CDATA[<p>Wow! This is very exciting!! The first (of many) AI-related posts. 😁 In just a few minutes, you will have your first <strong>Machine Learning</strong> model running. This is the first tale of series focusing on Artificial intelligence.</p><figure><img alt="Artificial intelligence tales by Loïc Combis" src="https://cdn-images-1.medium.com/max/1024/1*eb87Rml6WiKDEV8XMPyobg.jpeg" /><figcaption>Source: <a href="https://pixabay.com/illustrations/artificial-intelligence-brain-think-4550606/">Pixabay</a></figcaption></figure><p>Once upon a time, a young and curious engineer wanted to get a better understanding of <em>“What is AI?”,</em> get hands-on experience with real-life tools, and of course, have fun (why not after all 😃)! So, little by little, building knowledge, and models (lots of them…), I started looking back, and retrospectively, a couple of questions popped into his mind…</p><blockquote>How do you actually get started? And how can you navigate in such a large &amp; diverse ecosystem as AI?</blockquote><p>Well, the answer is not as trivial as you’d imagine… One needs to ask about theory vs practice, what tools are the best to achieve their objectives, how much time does one has… Which is even more questions than before…</p><p>In the end, I figured that you want to build AI right? Use real-life tools. Not just build knowledge about the mathematical models involved 🤯… Therefore, these tales will focus on hands-on experience. We’ll try to dispatch theoretical ideas here and there but <strong>we’ll mainly spend time actually developing AI</strong>. Because I believe that with practice, the theory will naturally come along the way.</p><p>🚨 Let’s take a deep breath and remember that moment 🧘‍♂️… From zero to hero… Let’s cross that bridge together.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_UtdIM2R1abzeORWcdNuTA.jpeg" /><figcaption>Source: <a href="https://pixabay.com/photos/bridge-park-garden-japanese-garden-53769/">Pixabay</a></figcaption></figure><h3>Setup</h3><p>We’ll need a bunch of tools to get started: <a href="https://www.python.org/">Python</a>, <a href="https://pandas.pydata.org/">Pandas</a>, <a href="https://numpy.org/">NumPy</a>, <a href="https://matplotlib.org/">MatPlotLib</a>, <a href="https://scikit-learn.org/stable/">Scikit-Learn</a>, <a href="https://jupyter.org/">Jupyter</a>…</p><p>OMG what’s all this 😩… Don’t worry, we’re about to discover it. As you can imagine, it would be quite painful to install all these one by one, so we’ll use a tool manager called <a href="https://docs.conda.io/en/latest/">Conda</a>. There are different versions (Anaconda, the full manager, and MiniConda, the light version). We’ll use ⚠️⚠️ <strong>MiniConda </strong>⚠️⚠️. You can download it <a href="https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html">here</a>.</p><p>🚨 <strong>Note:</strong> Make sure to download MiniConda for python 3.8+.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SIFj5TaH93vmZtC0L_jOoQ.png" /><figcaption><a href="https://docs.conda.io/en/latest/miniconda.html">MiniConda Installer Downloa Page</a></figcaption></figure><p>Once you’ve downloaded the installer, open the package and follow the instructions (the process might differ if you’re on Windows or Linux):</p><figure><img alt="MiniConda installation window" src="https://cdn-images-1.medium.com/max/1024/1*03LHf7AXIneJTt3nBRQauw.png" /></figure><p>After finishing installing Conda, you should be able to run the following command in a terminal:</p><pre>conda --version</pre><p>In my case, I’m using <strong><em>conda 4.9.2. </em></strong>If you run into issues while installing conda, refer to the <a href="https://conda.io/projects/conda/en/latest/user-guide/install/index.html">installation guide</a>. You can also find similar issues/solutions on <a href="https://stackoverflow.com/">Stackoverflow</a>.</p><p>Now that we have conda running, we can create the ecosystem containing all the tools we need and get started. To do so go to your tutorial root directory and run the following command:</p><pre>conda create --prefix ./first-model/env/</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*huHkLIOEX1okDl7omCFATQ.png" /></figure><p>We just created an empty environment. Now you can activate this environment and install the tools we’ll need.</p><pre>// Activate first-model environment.<br>conda activate ./first-model/env/</pre><pre>// Install the tool we need.<br>conda install python jupyter pandas numpy scikit-learn matplotlib seaborn</pre><p>It will ask you if you want to install a bunch of packages, approve and let conda’s magic happen.</p><p>🚨<strong> Note: </strong>Deactivate an environment by running: conda deactivate. Which will default to the “base” environment.</p><p>You just set up your AI ecosystem. Well done! 👏👏👏</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FfcsrmKkYPtZhC%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Fhow-i-met-your-mother-barney-stinson-self-five-fcsrmKkYPtZhC&amp;image=https%3A%2F%2Fmedia1.giphy.com%2Fmedia%2FfcsrmKkYPtZhC%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/ceb1dc0f33d581e4ddf84701b35dfefa/href">https://medium.com/media/ceb1dc0f33d581e4ddf84701b35dfefa/href</a></iframe><h3>Your first notebook</h3><p>AI model development is quite different in the way we approach it. It’s not just about writing a succession of methods/classes that, when inputted a set of parameters will logically output a consistent result. The actual “code” or logic is pretty straightforward in our case. The most important aspect is data analysis and preparation.</p><p>→ To build a good model, we need to understand the data we have. We need to explore, get familiar with the meaning of each feature (or type of data), and get an overview of the general trends. We need to make sense out of the raw data.</p><p>→ Machine Learning algorithms expect formatted data. We can’t just take raw data and plug it into an algorithm. At best, we’d get random results but for most cases, it just won’t work.</p><p>→ After we’ve trained our model, we want to measure the results so we can iterate with different parameters and optimize the results.</p><p>All this process mixes both logic writing and data visualization. So instead of separating the analysis/visualization from the code, we will use a <a href="https://jupyter.org/">Jupyter Notebook</a>. Here is how Jupyter describes its notebook feature:</p><blockquote>The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more.</blockquote><p>So basically, using a notebook, we will build AI in a sensical way. In today’s article, we won’t go into much details asthis is more like a setup and get started post. But in the following, we are going go through these steps several times and explain more about the underlying reasoning for each one of them.</p><p>→ Setup the notebook and define its purpose.</p><p>→ Exploratory data analysis (EDA).</p><p>→ Data preparation.</p><p>→ Model configuration &amp; training.</p><p>→ Result measurement.</p><p>No need for more explanations, let’s get started!! In your terminal, run the following command (make sure the correct environment is activated).</p><pre>jupyter notebook</pre><p>This will open a window in your default browser that should look like this:</p><figure><img alt="Jupyter Notebook browser" src="https://cdn-images-1.medium.com/max/1024/1*VGvsVzsghHp3wyUI40UDig.png" /><figcaption>Jupyter Web client</figcaption></figure><p>To create a new notebook, click new &gt; Python 3. This will open a new tab with an empty notebook. You can rename it “first-notebook” as I did so you can find it more easily later.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eUiAoNIG5P1_BrKotB-0ow.png" /><figcaption>First Jupyter Notebook</figcaption></figure><p>If you go back to the first tab, you will notice that the notebook appears in the file list. Same if you go to your project folder, the notebook will be there as well.</p><p>So, here is our empty notebook. A notebook works with cells, following these rules:</p><p>→ <strong>Each cell has two modes, edit, and view</strong>. Use your keyboard arrows to navigate cells and press Enteron a selected cell to switch to the edit mode. Press ESC to switch back to the view mode.</p><p>→ <strong>There are two types of cells, Code (to run python code) &amp; Text (to write markdown)</strong>. You can switch between the two with ESC + yor ESC + m when the cell is selected. Type CTRL + Enter (or CMD + Enter on Mac) to run a cell.</p><p>→ <strong>The “code” cells share the same runtime.</strong> Meaning that if you can access &amp; modify the state defined in other cells.</p><p>Let’s play with all this for a moment. Select in the first cell, type the following:</p><pre># First Notebook</pre><p>Then type ESC + m (text cell) and then type CTRL/CMD + Enter (run the cell).</p><p>In a second cell below, type ESC + y (code cell) and paste the following:</p><pre>print(&quot;Hello World&quot;)</pre><pre>a = 1<br>b = 2</pre><p>Then run the cell. Finally, paste the following in a third cell:</p><pre>print(a + b)</pre><pre>b = 3</pre><pre>print(a + b)</pre><p>Run this cell as well, and normally your notebook should look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nzeD47W4omyW4Pt2GuUvmQ.png" /><figcaption>Jupyter Notebook with Code &amp; Markdown cells.</figcaption></figure><p>🚨 <strong>Note:</strong> If you face difficulties using the notebook, I would recommend reading this <a href="https://jupyter-notebook.readthedocs.io/en/stable/notebook.html"><em>Introduction to Jupyter Notebooks.</em></a><em> </em>You will find all the documentation you need to get unblocked.</p><p>Feel free to experiment more and get familiar with the tool. The next section will focus on building your first AI model…</p><blockquote>What?! Already?!!</blockquote><p>Indeed…</p><h3>Your first model</h3><p>Let’s start by creating a new notebook (as explained in the previous section), and call it “<em>first-model</em><strong>”</strong>.</p><h4>Load the data</h4><p>The first part consists in loading our dataset and peaking at the different features, to understand the data. In the first cell, paste the following:</p><pre>from sklearn import datasets<br>import pandas as pd<br>import matplotlib.pyplot as plt</pre><p>sklearn package contains numbers of Machine learning models representing different algorithms used to train on a dataset, as well as useful methods to preprocess data. In addition, it contains several example datasets that can be used to learn machine learning. More about <a href="https://scikit-learn.org/stable/"><em>Scikit-Learn</em></a>.</p><p>pandas is a Python library defining two main objects — DataFrame &amp; Serie — which facilitate data analysis. More about <a href="https://pandas.pydata.org/docs/"><em>Pandas</em></a>.</p><p>matplotlib is another library used to print all sorts of charts. More about <a href="https://matplotlib.org/">MatplotLib</a>.</p><p>Then we load the <strong><em>Iris dataset.</em></strong></p><pre># Load Scikit Learn Iris Demo dataset.<br>iris = datasets.load_iris()</pre><p>Turn the dataset into a DataFrame and print the first five rows.</p><pre>df = pd.DataFrame(iris.data, columns=iris.feature_names)</pre><pre>df[&#39;target&#39;] = iris.target</pre><pre>df.head()</pre><p><strong>🚨🚨 Note:</strong> Don’t forget to run the cells in the correct order!! You can see the index at which the cell was run for the last time In [...].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zZFTxRTuidifvImHbKqvRQ.png" /><figcaption>First Model, Data Analysis.</figcaption></figure><p>As you can see on the screenshot above, the dataset contains 4 features: <em>sepal length, sepal width, petal length &amp; petal width</em>. The target columns determines the type of Iris the row represent.</p><p>0 → Setosa</p><p>1 → Versicolour</p><p>2 → Virginica</p><pre>len(df) # gives us the number of rows in the dataframe.</pre><p>We also notices that the dataset is not missing any value so we’re good to go for the data preparation. 😁</p><p><strong>🚨🚨 Note:</strong> As we’ll see in the following posts, this does not constitute an actual <em>Exploratory Data Analysis (EDA)</em>. We face a pretty straightforward dataset, but in the real world, the data is generally way more complex… And requires to take more time studying the different features, there shape etc…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3cDEGHP5G0AIoIdceqCkvA.png" /></figure><p>Add &amp; run the following cell:</p><pre>from sklearn.utils import shuffle</pre><pre># Shuffle data set rows <br>shuffled = shuffle(df)</pre><pre># Separate features (X) from the prediction target.<br>X = shuffled.drop(&#39;target&#39;, axis=1)<br>y = shuffled[&#39;target&#39;]</pre><p>As the dataset is ordered (all rows with 0, then all rows with 1…), we shuffle the dataset. ⚠️ This is very important!! Otherwise the model could pick up misleading patterns while training.</p><p>Second, we’re going to split the data set in two parts: <strong>Training &amp; Testing.</strong> To do so, we’ll use the very handy method train_test_split from sklearn.model_selection.</p><pre>from sklearn.model_selection import train_test_split</pre><pre># Split the data set for training &amp; validation<br>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)</pre><p>→ test_size defines the ratio of the dataset to be used for testing (20% here).</p><p>→ random_state defines the seed to be used for randomizing the split. It gives us the ability to reproduce the same results despite the randomness of the split.</p><p>Finally verify the different subsets.</p><pre>len(X_train), len(X_test), len(y_train), len(y_test)<br># (120, 30, 120, 30)</pre><p>Almost there… T minus 2 minutes before your first model becomes reality!!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FMREZajNc2bbkl3gIQv%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FMREZajNc2bbkl3gIQv%2Fsource.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FMREZajNc2bbkl3gIQv%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/cbc1e9c7d94b2bbaad97fd485f8edb1e/href">https://medium.com/media/cbc1e9c7d94b2bbaad97fd485f8edb1e/href</a></iframe><h4>Train &amp; test the model</h4><p>Although we already prepared the data, to reach a better accuracy, we’re going to normalize the values (scale everything between 0 and 1). It is meant to prevent features to weigh more than others due to scale/unit differences (e.g.: compare a human height in meters with their weight in grams).</p><p>The good thing is that sklearn already has everything we need to create a pipeline normalizing the data and then fitting (training) the model.</p><pre>from sklearn.pipeline import make_pipeline<br>from sklearn.ensemble import RandomForestClassifier<br>from sklearn.preprocessing import StandardScaler</pre><pre># Create a pipeline to normalize the data &amp; fit the model.<br>pipe = make_pipeline(StandardScaler(), RandomForestClassifier())</pre><pre># Train the model<br>pipe.fit(X_train, y_train)</pre><pre># Score the model with the test data<br>pipe.score(X_test, y_test)</pre><p>In the cell above, StandardScaler will take care of normalizing the data and RandomForestClassifier is our actual Machine Learning model!!!</p><p>Run the cell and you should obtain an accuracy around 93.33%.</p><blockquote>OMG.. 3 lines of codes to scale our data, train &amp; score the model… Are you kidding me?</blockquote><p>Unfortunately no… sklearn makes it really easy for us to build AI… ⚠️But, remember that we are using a small demo dataset. You’ll see in the next posts that it gets trickier really quick!</p><p>Let’s go a bit in further in the assessment of our model. First, let’s get the actual predictions:</p><pre>y_preds = pipe.predict(X_test)</pre><pre>y_preds</pre><pre># array([1, 0, 1, 1, 0, 1, 1, 2, 2, 0, 2, 0, 2, 1, 0, 2, 1, 1, 1, 0, # 2, 2, 0, 1, 2, 2, 2, 0, 1, 2])</pre><p>Now, using matplotlib and seaborn (UI lib for matplotlib), we are going to print out the confusion_matrix .</p><pre>from sklearn.metrics import confusion_matrix, classification_report<br>import seaborn as sns</pre><pre>sns.set(font_scale=1.5) # Increase font size</pre><pre>fig, ax = plt.subplots(figsize=(3, 3))<br>ax = sns.heatmap(confusion_matrix(y_test, y_preds),<br>                 annot=True, # Annotate the boxes<br>                 cbar=False)<br>plt.xlabel(&quot;True Label&quot;)<br>plt.ylabel(&quot;Prediction&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/830/1*dWYQqXny_f5IRJO0QiCE5w.png" /><figcaption>Model Confusion Matrix.</figcaption></figure><p>But what does that mean? This matrix compares the prediction per label vs the actual label.</p><p>→ On the diagonal, you have all the count of correct prediction(e.g.: 0x0, the model predicted 0 and the actual label is 0).</p><p>→ The sum of each row (minus diagonal value) gives us the false positives (e.g.: Row with prediction 1, we predicted twice 1 whereas the true label is 2).</p><p>→ The sum of each column (minus diagonal value) gives the the false negatives (e.g.: Column with true label 2, we predicted twice something other than two where as the true label is 2).</p><p>The confusion matrix can give us interesting insights on labels that are the most difficult to choose from.</p><p>Another interesting table to asses our model is the classification.</p><pre>print(classification_report(y_test, y_preds))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0HmFu3Rk8UBiy78QMEvWCw.png" /><figcaption>Model Classification Report.</figcaption></figure><p>→ <em>Precision. </em>Ratio of true positives / occurences.</p><p>→ <em>Recall</em>. Ratio of true positives / (true positives + false negatives). It is the number of time we predicted a label versur how many times we should have.</p><p>→<em> F1-Score</em>. Weighted average of the precision and recall.</p><p>→ <em>Support</em>. Number of occurrences to “support” the scores (e.g.: There was 8 occurrences of the label 0 in y_test).</p><h4>Conclusion</h4><p>We’ve done it! Our first Machine Learning Model… Today, we saw how one can quickly get started with AI, install all the tools (Conda, Python, Scikit-Learn, Pandas). How to create Jupyter notebooks, load and prepare a dataset, and finally, how to create a pipeline allowing you to scale your data, train and assess am model. But… This just the beginning. There is so much more going on… So many more ML models to build… Next time, we’ll dive deep into <strong>Exploratory Data Analysis, Machine Learning models, Fine Tuning and Model Assessment</strong>.</p><p>All the notebooks in this post are available on the <a href="https://github.com/loic-combis/ai-tales">AI tales repo</a>.</p><p>I really hope you enjoyed this first step into the world of AI. Do not hesitate to clap if it’s case, If you didn’t, feel free to yell at me on me on <a href="https://twitter.com/CombisLoic">Twitter</a>.</p><p>⚠️⚠️⚠️ Comming Soon — AI tales, Machine Learning in Depth.⚠️⚠️⚠️</p><h3>More Links</h3><p>If you’re interested in other things I might be doing 😋, here is my <a href="https://loic-combis.github.io/">personal website</a>.</p><p>My other posts on <a href="https://loic-combis.medium.com/">medium</a>.</p><p>My <a href="https://github.com/loic-combis">Github</a>.</p><p>What do we do at <a href="https://about.shopstyle.com/">ShopStyle</a>?</p><p>ShopStyle <a href="https://www.shopstyle.com/">main website.</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8a8ce7b151c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/shopstyle-engineering/ai-tales-your-first-machine-learning-model-with-python-conda-scikit-learn-jupyter-8a8ce7b151c">AI tales, your first Machine Learning model with Python, Conda, Scikit-learn&amp; Jupyter.</a> was originally published in <a href="https://medium.com/shopstyle-engineering">ShopStyle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Set Up a MEAN Stack with Nx in Less than 20 Minutes]]></title>
            <link>https://medium.com/shopstyle-engineering/how-to-set-up-a-mean-stack-with-nx-in-less-than-20-minutes-fb0a1108ae67?source=rss----b45652c2366---4</link>
            <guid isPermaLink="false">https://medium.com/p/fb0a1108ae67</guid>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[angular]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Loïc Combis]]></dc:creator>
            <pubDate>Wed, 16 Jun 2021 17:03:11 GMT</pubDate>
            <atom:updated>2021-06-16T17:03:11.042Z</atom:updated>
            <content:encoded><![CDATA[<p>20 minutes, 3 steps, 1 running JavaScript full-stack TODO app! Let’s do this! 😎</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/504/1*aeZEGTqhTryFLfzIIaxeXQ.png" /></figure><p>When starting a new project, there is always a bunch of annoying, time-consuming tasks required to set up your app, install the tools, go through the doc, remember the shell commands… 😩.</p><p>I’ve personally started dozens of apps and every time, it’s the same thing all over again. The worst part is that I never remember the process by heart and feel like wasting my time because it’s literally identical.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FBY8ORoRpnJDXeBNwxg%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FBY8ORoRpnJDXeBNwxg%2Fsource.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FBY8ORoRpnJDXeBNwxg%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="362" frameborder="0" scrolling="no"><a href="https://medium.com/media/b6e504f1e0ee5ea60937c7f20c4fa75a/href">https://medium.com/media/b6e504f1e0ee5ea60937c7f20c4fa75a/href</a></iframe><p>So, for this article (and my mental health ✌️), I’ve decided to build and share with you the “groundwork” for a MEAN stack app, using Nx. Just as a quick reminder, MEAN — means (sorry I had to… 😂) — stands for <a href="https://www.mongodb.com/">MongoDB</a>, <a href="https://expressjs.com/">Express</a>, <a href="https://angular.io/">Angular</a> &amp; <a href="https://nodejs.org/en/">Node.js</a>.</p><p><a href="https://nx.dev/">Nx</a> is a tool meant to build, architect, and test a monorepo (several apps in one single codebase). After the setup, we will implement a simple end-to-end TODO app.</p><h4>Prerequisites</h4><p>There are two prerequisites, being the installation of NodeJS and MongoDB. We’ll focus on the app more than the installation. You can download NodeJS <a href="https://nodejs.org/en/">here</a> and MongoDB <a href="https://docs.mongodb.com/manual/installation/">here</a>.</p><h4>My setup</h4><pre>node -v<br>//  v12.20.0</pre><pre>npm -v<br>// v6.14.8</pre><pre>mongo --version<br>// MongoDB shell version v4.2.3</pre><blockquote>Ready, get set…</blockquote><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2Fdr6toZX3D1O8%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2Fdr6toZX3D1O8%2Fsource.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2Fdr6toZX3D1O8%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="234" frameborder="0" scrolling="no"><a href="https://medium.com/media/a4cad2655dc0c1ae71358596bcb9f1f7/href">https://medium.com/media/a4cad2655dc0c1ae71358596bcb9f1f7/href</a></iframe><h4>1&#39; — 5&#39;: Setup</h4><p>Run the following commands to install npx.</p><pre>npm install -g npx</pre><p>Create the monorepo and follow the prompts. Feel free to use your own names instead of “groundwork” &amp; “client”.</p><pre>npx create-nx-workspace --preset=angular</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EPalhEn8ZDad1seUv9HJDQ.png" /></figure><p>You should now have your repo initialized with two apps: client &amp; client-e2e (end-to-end test app). Go into your repo and run the following commands to create the Express API.</p><pre>cd groundwork</pre><pre>npm install -D <a href="http://twitter.com/nrwl/express">@nrwl/express</a></pre><pre>ng g <a href="http://twitter.com/nrwl/express">@nrwl/express</a>:app api</pre><p>Here is how your workspace should look like at this stage:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R68-12SMF3MzTPIJkBnIwg.png" /></figure><p>🚨🚨 <strong>Note:</strong> I created a .nvmrc file to manage the version of NodeJS used. This is not required, but if you want to know more please look up <a href="https://github.com/nvm-sh/nvm">nvm</a>.</p><p>Let’s install a bunch of packages that we’ll be using later when developing building the TODO app.</p><pre>npm install --save mongoose body-parser cors</pre><pre>npm install --save-dev <a href="http://twitter.com/types/mongoose">@types/mongoose</a></pre><p>That’s it for the setup 🎉. Let’s continue with the API.</p><h4>6&#39; — 12&#39;: Build the API</h4><p>First, we are going to link the API to MongoDB. Under apps/api/src/app/helper create a db.helper.ts file. It will manage the connection to MongoDB (replace test with your database name).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f57e3bdb328969a4881aff1f27afd490/href">https://medium.com/media/f57e3bdb328969a4881aff1f27afd490/href</a></iframe><p>Now, in apps/api/src/main.ts, import DBHelper and call the init function.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4259c1b25257f7d2d98d2d42b8ac6886/href">https://medium.com/media/4259c1b25257f7d2d98d2d42b8ac6886/href</a></iframe><p>Now you can run mongo with mongod --dbpath your-path-to-storage, start the API with npm start api to verify the connection to the database.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QkhRKwr1FMTWNkYFWTtYWw.png" /></figure><p>Let’s create the define the TODO resource for the database, as well as the API endpoints for our Angular App.</p><p>Under apps/api/src/app/ create a “todo” folder containing the following files:</p><p>→ todo.model.ts</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/432d9a69670e089f5576102dd7da3732/href">https://medium.com/media/432d9a69670e089f5576102dd7da3732/href</a></iframe><p>→ todo.schema.ts. This describes the structure of the resource to be stored in the database as well as the contract regarding each property (e.g.: Required).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5c4d06abc939a627e2b2736bb50ed7cd/href">https://medium.com/media/5c4d06abc939a627e2b2736bb50ed7cd/href</a></iframe><p>→ todo.service.ts. The service is using the interface created right above to interact with the database. More about <a href="https://mongoosejs.com/">mongoose</a>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/05ab0568e04fb2b8559fa78fc97a60f9/href">https://medium.com/media/05ab0568e04fb2b8559fa78fc97a60f9/href</a></iframe><p>Whoa! A lot of things happened there. 🔥 Nothing crazy though. And yet, we are almost done with the API!</p><p>The last thing we want to do on the server is registering the routes so our angular client can reach the methods defined in todo.service.ts.</p><p>Update main.ts to contain the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5d6c164b565c7155a9a122cff863aec7/href">https://medium.com/media/5d6c164b565c7155a9a122cff863aec7/href</a></iframe><p>Here it is! The API is ready!! 😍 If you want to test it out before we build the client, you can download <a href="https://www.postman.com/">Postman</a> and play with the API.</p><h4>13&#39; — 19&#39;: Build the Client</h4><p>A few things to do here. Let’s start by creating a service that will communicate with the API.</p><p>Under apps/client/src/app create todo.service.ts containing the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/42bbbe7580cf836ccdd534dd20beafea/href">https://medium.com/media/42bbbe7580cf836ccdd534dd20beafea/href</a></iframe><p><strong>🚨🚨 Note:</strong> The service contains three methods, matching the one we created in the API earlier.</p><p>Under apps/client/src/app create todo.interface.ts containing the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c004355e0e6b291dac7ddfa95f9940dd/href">https://medium.com/media/c004355e0e6b291dac7ddfa95f9940dd/href</a></iframe><p><strong>🚨🚨 Note:</strong> Duplicating the same interface both in the client and the API is a bad practice 😇, especially in a JavaScript monorepo where we can actually share those interfaces thanks to libraries. This is just for simplicity (remember we have less than 20mins to complete the app).</p><p>We’re almost there!! Just a few more steps. 😁</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FlSbTmUmQwxUmiExV4h%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FlSbTmUmQwxUmiExV4h%2Fsource.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FlSbTmUmQwxUmiExV4h%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="362" frameborder="0" scrolling="no"><a href="https://medium.com/media/53d0e3608c8dcd5688b0930611915de3/href">https://medium.com/media/53d0e3608c8dcd5688b0930611915de3/href</a></iframe><p>Now replace the content of app.component.ts with the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7b24d3b03cfcdd1a83f3b91af1c91fbb/href">https://medium.com/media/7b24d3b03cfcdd1a83f3b91af1c91fbb/href</a></iframe><p>Now, the corresponding template in app.component.html:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e9255dd780d9b1e8f784e7c6b3a2aa3c/href">https://medium.com/media/e9255dd780d9b1e8f784e7c6b3a2aa3c/href</a></iframe><p>A bit of style, in app.component.scss (forgive me this is very basic and messy; but once again, that’s not the point of the article 😅).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/97948edbb3bf71974eaefd40c7c6b317/href">https://medium.com/media/97948edbb3bf71974eaefd40c7c6b317/href</a></iframe><p>And in styles.scss:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/80efed25a61e83c0a0a72c301812dd72/href">https://medium.com/media/80efed25a61e83c0a0a72c301812dd72/href</a></iframe><p>Finally, we import a couple of modules to use ngModel and HttpClient in app.module.ts:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d704a6d37541920567c7147d01c0a9f2/href">https://medium.com/media/d704a6d37541920567c7147d01c0a9f2/href</a></iframe><p>Run, npm start client, go to <a href="http://localhost:4200/">http://localhost:4200</a> and this is what you get!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_jZT5AsAoZjVvo2htCW9ZQ.png" /></figure><h4>What about the 20th minute?</h4><p>Easy…</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FxT1R9MyOQLEB2WY6Oc%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FxT1R9MyOQLEB2WY6Oc%2Fsource.mp4&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FxT1R9MyOQLEB2WY6Oc%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="183" frameborder="0" scrolling="no"><a href="https://medium.com/media/54442d2326b140db79dbcbfec06ba498/href">https://medium.com/media/54442d2326b140db79dbcbfec06ba498/href</a></iframe><p>Thanks for taking the time to read this article! Always a pleasure to publish new content.</p><p>All the code is available on GitHub: <a href="https://github.com/loic-combis/groundwork">https://github.com/loic-combis/groundwork</a>.</p><p>If you’re facing issues, please let me know so I can update the post to cover the part in detail and/or fix the error.</p><p>Feel free to clap if you liked this post 😁. If you didn’t, you can always yell at me on Twitter: <a href="https://twitter.com/CombisLoic">https://twitter.com/CombisLoic</a></p><p>Check out my other posts <a href="https://loic-combis.medium.com/">https://loic-combis.medium.com/</a> 🥰.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fb0a1108ae67" width="1" height="1" alt=""><hr><p><a href="https://medium.com/shopstyle-engineering/how-to-set-up-a-mean-stack-with-nx-in-less-than-20-minutes-fb0a1108ae67">How to Set Up a MEAN Stack with Nx in Less than 20 Minutes</a> was originally published in <a href="https://medium.com/shopstyle-engineering">ShopStyle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Angular Injection Token for sharing environment across Nx libraries & applications.]]></title>
            <link>https://medium.com/shopstyle-engineering/angular-injection-token-for-sharing-environment-across-nx-libraries-applications-befb9d3c6c67?source=rss----b45652c2366---4</link>
            <guid isPermaLink="false">https://medium.com/p/befb9d3c6c67</guid>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[nx]]></category>
            <category><![CDATA[injectiontoken]]></category>
            <category><![CDATA[angular]]></category>
            <dc:creator><![CDATA[Loïc Combis]]></dc:creator>
            <pubDate>Wed, 17 Feb 2021 19:21:44 GMT</pubDate>
            <atom:updated>2021-03-09T19:11:06.336Z</atom:updated>
            <content:encoded><![CDATA[<h3>Environment sharing across libraries with Angular Injection tokens.</h3><p>Hi there! Let’s look at how you — as we do at <a href="https://about.shopstyle.com/">ShopStyle</a> — can use <strong>Angular injection tokens</strong> to share your application-specific environment with Nx libraries and build complex, sophisticated apps that scale like a charm👌.</p><figure><img alt="Angular developer with red hoody." src="https://cdn-images-1.medium.com/max/640/1*6UMN07iTpTtSoMHyrJ3tyA.jpeg" /></figure><p>As your app grows, you will face a dilemma on how to manage your increasing amount of features. 🤯 A solution would be to properly segment your<strong> angular modules</strong> within the app. But what if for example, you want to create another application? The chances are high that at least some of the features/behaviors are similar between the apps. It doesn’t matter if it’s only the design or entire modules. Would you copy/paste the code, reimplement everything? 😖💩 Of course not!</p><blockquote>Remember, a good engineer is a lazy one.</blockquote><p>Instead, you will create <strong>beautiful and reusable libraries!! 🎉 </strong>But as you already know, with great power comes great responsibility. Your libraries <strong>MUST </strong>stay isolated<strong>. </strong>Meaning you cannot import anything from the actual app into the library. This is a one-way street.</p><p>But then… How do I use my application-specific configuration without compromising library isolation? Well, let’s see a quick and efficient way to deal with environment variables with <a href="https://angular.io/api/core/InjectionToken"><strong><em>Angular Injection Tokens</em></strong></a><strong>.</strong></p><p>Although<strong> </strong>Angular supports libraries out of the box (see the <a href="https://angular.io/guide/creating-libraries"><em>documentation</em></a>), it is less convenient when it comes to sharing features across multiple apps. For this tutorial, we’ll be using <strong>Nx</strong>.</p><h3>What is Nx?</h3><figure><img alt="Nx extensive dev tools logo" src="https://cdn-images-1.medium.com/max/1024/1*mFJGmWVqj8AAIMbnlrHxSQ.png" /></figure><blockquote><strong>Nx</strong> is a suite of powerful, extensible dev tools to help you architect, test, and build at any scale — integrating seamlessly with modern technologies and libraries while providing a robust CLI, caching, dependency management, and more.It has first-class support for many frontend and backend technologies, so its documentation comes in multiple flavours.</blockquote><p>This is how Nx describes itself on its <a href="https://nx.dev/">website</a>. It is basically a set of tools allowing you to manage several applications and their dependencies in one single codebase called <a href="https://www.atlassian.com/git/tutorials/monorepos"><em>“Monorepo”</em></a>.</p><h3>Let’s get started</h3><h4>Setup</h4><pre>$ node --version</pre><pre>v10.11.0</pre><pre>$ npm --version</pre><pre>6.14.7</pre><pre>$ npx --version </pre><pre>6.14.7</pre><pre>$ nx --version</pre><pre>11.2.12</pre><p>🚨 <strong>Note:</strong> You can easily install npx &amp; nxusing npm install -g npx nx.</p><h4>Create an Nx Workspace</h4><pre>$ npm install -g create-nx-workspace</pre><pre>$ npx create-nx-workspace@latest</pre><figure><img alt="create-nx-workspace command flow" src="https://cdn-images-1.medium.com/max/1024/1*ouTEBkur6EG5ul23v_l6rw.png" /></figure><pre>$ cd environment-token/</pre><p>You can run nx list in your repo to check the list of plugins installed in the workspace. As we created an empty project, we should only have base plugins installed:</p><figure><img alt="Nx workspace installed plugins" src="https://cdn-images-1.medium.com/max/494/1*wiAVgLkwxFuqvyFEZUsjtw.png" /></figure><h4>Add Angular schematics and create an application.</h4><pre>$ npm install --save @nrwl/angular</pre><p>Let’s run nx list again:</p><figure><img alt="Nx workspace installed plugins after adding angular" src="https://cdn-images-1.medium.com/max/486/1*nJe6Vrz1Azs6RRzcHzQvvg.png" /></figure><p>🚨<strong> Note:</strong> In addition to Angular, Nx also installed <a href="https://www.cypress.io/"><em>Cypress</em></a> which is an End to End testing framework. We use it at <a href="https://about.shopstyle.com/"><em>ShopStyle</em></a> and it’s actually very powerful as it allows you to test complete user stories. 💯 👍</p><h4>Create the applications and libraries</h4><p>Let’s first generate two applications that will share common libraries.</p><pre>$ nx g @nrwl/angular:application first-app</pre><pre>$ nx g @nrwl/angular:application second-app</pre><p>The actual setup for each project doesn’t really matter for this tutorial. But here is what configuration I used.</p><figure><img alt="Angular app initial configuration" src="https://cdn-images-1.medium.com/max/1024/1*d8w169cvQWN_aQmqOLZh1A.png" /></figure><p>Now let’s create two shared libraries. The first one will be called <em>“home” </em>and will contain a shared home.component. The second will be the called “environment” and will hold the <strong>Injection Token</strong> and the<strong> environment interface</strong> definition.</p><pre>// Create the &quot;home&quot; library<br>$ nx g <a href="http://twitter.com/nrwl/angular">@nrwl/angular</a>:lib home</pre><pre>// Add the &quot;home&quot; component to the &quot;home&quot; library and export it.<br>$ nx g @nrwl/angular:component home --export --project home</pre><pre>// Create the environment library.<br>$ nx g <a href="http://twitter.com/nrwl/angular">@nrwl/angular</a>:lib environment</pre><p>We’re all set!! 😍 Your project structure should look like the following at this point (2 applications + e2e apps, 2 libraries with a home component).</p><figure><img alt="Monorepo structure" src="https://cdn-images-1.medium.com/max/1024/1*BWnfY0XEG7B0XTkeCRaewA.png" /></figure><h4>Create the injection token</h4><p>Under ./libs/environment/src/lib/ create the two following files:</p><p>→ environment.interface.ts</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7c8658a0179bc336203fc7a0a0852f9e/href">https://medium.com/media/7c8658a0179bc336203fc7a0a0852f9e/href</a></iframe><p>Here we’re defining the contract that the token value must respect (so we don’t get surprises when using the token).</p><p>→ environment.token.ts</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bfc58fb65e98699f40664ce9301030b3/href">https://medium.com/media/bfc58fb65e98699f40664ce9301030b3/href</a></iframe><p>🚨 <strong>Note:</strong> Think of the token as a wildcard. The token holds a value that we don’t know yet but will be provided at runtime.</p><p>Let’s export those two files. Under ./libs/environment/src/index.ts paste the following.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/da8cbc1ac95b5a644202b6736a3dee0b/href">https://medium.com/media/da8cbc1ac95b5a644202b6736a3dee0b/href</a></iframe><p>We just created the <strong>Injection Token. 🎉 </strong>Awesome!! Hold on tight, just a few more steps to complete!</p><h4>Provide the token</h4><p>We need to let Angular know about this token so we can use it in components. Under both ./apps/first-app/src/app/app.module.ts and ./apps/second-app/src/app/app.module.ts provide the injection token as follows:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/49f8a6f1bd8ad46a133541042b231024/href">https://medium.com/media/49f8a6f1bd8ad46a133541042b231024/href</a></iframe><p>Angular now knows what value use when the token is injected into a component/service. We can add custom properties to the value to match the environment.interface .</p><p>→ Replace the content of./apps/first-app/environments/environment.ts with the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ea31436b88058824d86c0f206dcd80f0/href">https://medium.com/media/ea31436b88058824d86c0f206dcd80f0/href</a></iframe><p>→ Replace the content of./apps/second-app/environments/environment.ts with the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b380b2f60b0c68ac3766f4bb752219cd/href">https://medium.com/media/b380b2f60b0c68ac3766f4bb752219cd/href</a></iframe><p>And boom!! That’s how you provide an injection token to an angular app.</p><h4>Use the token in a library</h4><p>Our environment token is now ready to be used. So let’s use it in our home.component! 😃</p><p>To inject it into the component, replace the content of ./libs/home/src/lib/home/home.component.ts with:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/34ffdd8ad507c88d0c5474fbf869c581/href">https://medium.com/media/34ffdd8ad507c88d0c5474fbf869c581/href</a></iframe><p>Add a little bit of HTML to display the values. In home.component.html (of the same folder), add :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1bf0226847f504f69eb31130bb199951/href">https://medium.com/media/1bf0226847f504f69eb31130bb199951/href</a></iframe><p>And a little bit of style of course (in home.component.scss):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b5b276d381efc3003e566b9321253e5b/href">https://medium.com/media/b5b276d381efc3003e566b9321253e5b/href</a></iframe><p>Finally, let’s use our home component in the applications.</p><p>In both ./apps/first-app/src/app/app.component.html &amp; ./apps/second-app/src/app/app.component.html replace the content with:</p><pre>&lt;environment-token-home&gt;&lt;/environment-token-home&gt;</pre><p>🚨 <strong>Note:</strong> This assumes you already imported the home.module in both app.module.ts when providing the environment token.</p><p>Let’s see the results:</p><pre>$ npm start first-app</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DAQskbykqQxFX9xqm3XkOw.png" /></figure><pre>$ npm start second-app</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zVuyWDPuom_zzrHg7ksG2A.png" /></figure><p>And voila!! 😉</p><p>I hope you liked this tutorial! Feel free to fork the repo and play with it. If you did please clap 😊👏! Otherwise, any constructive comment will be welcomed!</p><p>You can check out my other stories on <a href="https://medium.com/@loic-combis">medium</a> (more to come soon). I’ll also try to post every new story to my <a href="https://twitter.com/CombisLoic">Twitter account</a>.</p><h3>Sources</h3><p>Github repo: <a href="https://github.com/loic-combis/environment-token">https://github.com/loic-combis/environment-token</a></p><p>Nx website: <a href="https://nx.dev/">https://nx.dev/</a></p><p>Angular Libraries: <a href="https://angular.io/guide/creating-libraries">https://angular.io/guide/creating-libraries</a></p><p>Angular Injection Tokens: <a href="https://angular.io/api/core/InjectionToken">https://angular.io/api/core/InjectionToken</a></p><p>Monorepos: <a href="https://www.atlassian.com/git/tutorials/monorepos">https://www.atlassian.com/git/tutorials/monorepos</a></p><h3>More Links</h3><p>Personal website: <a href="https://loic-combis.github.io/">https://loic-combis.github.io/</a></p><p>Twitter: <a href="https://twitter.com/CombisLoic">https://twitter.com/CombisLoic</a></p><p>Medium: <a href="https://loic-combis.medium.com/">https://loic-combis.medium.com/</a></p><p>Github: <a href="https://github.com/loic-combis">https://github.com/loic-combis</a></p><p>ShopStyle main website: https://www.shopstyle.com/</p><p>More about what we do at ShopStyle: <a href="https://about.shopstyle.com/">https://about.shopstyle.com/</a></p><p>Pixabay header image: <a href="https://pixabay.com/photos/red-matrix-matrix-matrix-code-5031496/">https://pixabay.com/photos/red-matrix-matrix-matrix-code-5031496/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=befb9d3c6c67" width="1" height="1" alt=""><hr><p><a href="https://medium.com/shopstyle-engineering/angular-injection-token-for-sharing-environment-across-nx-libraries-applications-befb9d3c6c67">Angular Injection Token for sharing environment across Nx libraries &amp; applications.</a> was originally published in <a href="https://medium.com/shopstyle-engineering">ShopStyle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Developing a log processing pipeline on Amazon Web Services]]></title>
            <link>https://medium.com/shopstyle-engineering/developing-a-log-processing-pipeline-on-amazon-web-services-fb14fb603c9a?source=rss----b45652c2366---4</link>
            <guid isPermaLink="false">https://medium.com/p/fb14fb603c9a</guid>
            <category><![CDATA[cloudfront]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[kinesis-firehose]]></category>
            <category><![CDATA[kinesis]]></category>
            <category><![CDATA[redshift]]></category>
            <dc:creator><![CDATA[Derrick Petzold]]></dc:creator>
            <pubDate>Thu, 19 Mar 2020 20:40:43 GMT</pubDate>
            <atom:updated>2020-03-19T21:46:34.728Z</atom:updated>
            <content:encoded><![CDATA[<p>It can be advantageous to have your own log processing pipeline to provide insights into your traffic. For example, suppose after an DDOS event you wanted to see which networks the attack originated from. Having the tooling for that will also allow you to create your own custom dashboards to visualize your traffic. Below is a Grafana panel showing the top twenty networks by requests using Redshift as its data source. In this post we will discuss the components required to create this pipeline.</p><figure><img alt="Top Requests by Network" src="https://cdn-images-1.medium.com/max/1024/1*0t9OoqxhDUM1qdZVj9mfYA.png" /><figcaption>Top Requests by Network</figcaption></figure><p>With visualization noticing the outlier becomes incredibly simple. Below is the architecture diagram of the pipeline infrastructure:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TStE9LapBXjmgkLGJdJ4BA.png" /></figure><p>At ShopStyle, our pipeline starts at CloudFront which is configured to write logs to S3. If you’re creating your CloudFront distributions with Terraform here’s an example to add logging:</p><pre>logging_config {<br>  bucket = &quot;cloudfront.s3.amazonaws.com&quot;<br>  prefix = &quot;www&quot;<br>}</pre><p>Once logs are written to S3, an event notification triggers a Lambda function to stream the data. The Lambda function performs the following actions:</p><ol><li>Downloads the log file from S3.</li><li>Uncompresses it.</li><li>Streams the contents to Kinesis.</li></ol><p>Now that the logs are in Kinesis we have our data stream. To augment the data there is a StatefulSet in a Kubernetes cluster. The number of replicas in the StatefulSet map to the number of shards in the stream. Below is the Kubernetes task definition for the StatefulSet:</p><pre>apiVersion: apps/v1<br>kind: StatefulSet<br>metadata:<br>  name: log-streamer-augmenter<br>  labels:<br>    app: log-streamer-augmenter<br>spec:<br>  serviceName: log-streamer-augmenter<br>  replicas: 10<br>  selector:<br>    matchLabels:<br>      app: log-streamer-augmenter<br>  template:<br>    metadata:<br>      labels:<br>        app: log-streamer-augmenter<br>    spec:<br>      containers:<br>      - name: log-streamer-augmenter<br>        image: xxxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/ss/log-streamer:xxxxxx<br>        resources:<br>          requests:<br>            cpu: 100m<br>            memory: 200Mi<br>        env:<br>        - name: POD_NAME<br>          valueFrom:<br>            fieldRef:<br>              fieldPath: metadata.name<br>        args: [&quot;--augmenter&quot;, &quot;--shard-idx&quot;, &quot;-1&quot;]</pre><p>To augment the data the following operations are performed:</p><ul><li>Combine the date and time fields into a timestamp.</li><li>Reverse DNS lookup of the ip address.</li><li>RADb lookup of the originating network from the ip address.</li><li>Country of origin from a geolookup of the ip address.</li><li>User-Agent parsing via python-user-agents</li></ul><p>Once the data has been augmented it is streamed to another Kinesis data stream encoded into Protocol Buffers using Pavel Perestoronin’s absolutely excellent Python module, pure-protobuf. Below are the data models used:</p><pre>from dataclasses import dataclass<br>from pure_protobuf.dataclasses_ import (<br>    field,<br>    message,<br>)<br>@message<br>@dataclass(frozen=True)<br>class BaseAugmentedLogEntry:<br>    augmented: datetime.datetime = field(0)<br>    timestamp: datetime.datetime = field(1)<br>    client_ip: str = field(2)<br>    country: str = field(3)<br>    ip_source: str = field(5)<br>    producer: str = field(6)<br>    sent_bytes: int = field(7)<br>    ssl_cipher: str = field(8)<br>    ssl_protocol: str = field(9)<br>    user_agent: str = field(10)<br>    device_type: str = field(11)<br>    network: str = field(11)<br>    url_param__pid: str = field(12)<br>    url_param__mpid: str = field(13)<br></pre><pre>@message<br>@dataclass(frozen=True)<br>class CloudFrontAugmentedLogEntry(BaseAugmentedLogEntry):<br>    cookie: str = field(20)<br>    edge_location: str = field(21)<br>    edge_request_id: str = field(22)<br>    edge_response_result_type: str = field(23)<br>    edge_result_type: str = field(24)<br>    fle_encrypted_fields: str = field(25)<br>    forwarded_for: str = field(26)<br>    host: str = field(27)<br>    host_header: str = field(28)<br>    http_method: str = field(29)<br>    hostname: str = field(30)<br>    protocol: str = field(31)<br>    protocol_version: str = field(32)<br>    received_bytes: int = field(33)<br>    referrer: str = field(34)<br>    status_code: int = field(35)<br>    time_taken: float = field(36)<br>    uri: str = field(37)<br>    uri_query: str = field(38)<br>    network: str = field(39)<br></pre><pre>@message<br>@dataclass(frozen=True)<br>class LoadBalancerAugmentedLogEntry(BaseAugmentedLogEntry):<br>    actions_executed: typing.List[str] = field(20)<br>    chosen_cert_arn: str = field(21)<br>    domain_name: str = field(22)<br>    elb: str = field(23)<br>    elb_status_code: int = field(24)<br>    error_reason: str = field(25)<br>    http_type: str = field(26)<br>    matched_rule_priority: int = field(27)<br>    received_bytes: int = field(28)<br>    redirect_url: str = field(29)<br>    request: str = field(30)<br>    request_creation_time: datetime.datetime = field(31)<br>    request_processing_time: float = field(32)<br>    response_processing_time: float = field(33)<br>    target_group_arn: str = field(34)<br>    target_ip: str = field(35)<br>    target_processing_time: float = field(36)<br>    target_status_code: str = field(37)<br>    trace_id: str = field(38)<br>    network: str = field(39)<br>    hostname: str = field(40)</pre><p>Now that the modeled data is being streamed another Lambda function sends the data to Redshift via a Kinesis Data Firehose. The Firehose writes the data to S3 and Redshift will copy data from it.</p><h3>Visualization</h3><p>With the data in Redshift it can be used as a data source in Grafana. Below is the query to generate the data in the initial screenshot:</p><pre>WITH data AS (<br>  SELECT<br>    $__timeGroup(&quot;timestamp&quot;, $Grouping),<br>    count(*) as count,<br>    network as metric<br>  FROM<br>    cloudfront_logs<br>  WHERE<br>    $__unixEpochFilter(&quot;time&quot;)<br>  GROUP BY 1, 3<br>  ORDER BY count DESC<br>)<br>SELECT<br>    &quot;time&quot;,<br>    count,<br>    metric<br>FROM<br>    data<br>WHERE count &gt; (<br>    SELECT threshold FROM (<br>        SELECT<br>            MAX(count) as threshold,<br>            metric<br>        FROM data<br>        GROUP BY 2<br>        ORDER BY 1 DESC<br>        OFFSET 20<br>        LIMIT 1<br>    ) sub<br>)<br>ORDER BY 1;</pre><p>If you’re interested in working on problems like these, ShopStyle is hiring! Check out our <a href="https://about.shopstyle.com/careers">open positions</a> and apply!</p><h3>References</h3><ul><li><a href="https://aws.amazon.com/blogs/big-data/build-a-serverless-architecture-to-analyze-amazon-cloudfront-access-logs-using-aws-lambda-amazon-athena-and-amazon-kinesis-analytics/">Build a Serverless Architecture to Analyze Amazon CloudFront Access Logs Using AWS Lambda, Amazon</a></li><li><a href="https://www.arundhaj.com/blog/getting-started-kinesis-python.html">Getting started with AWS Kinesis using Python</a></li><li><a href="https://medium.com/@aoc/kinesis-firehose-to-redshift-pipeline-with-terraform-2261b5afd29d">Kinesis Firehose to Redshift pipeline with Terraform</a></li><li><a href="https://github.com/dpetzold/aws-log-parser">https://github.com/dpetzold/aws-log-parser</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fb14fb603c9a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/shopstyle-engineering/developing-a-log-processing-pipeline-on-amazon-web-services-fb14fb603c9a">Developing a log processing pipeline on Amazon Web Services</a> was originally published in <a href="https://medium.com/shopstyle-engineering">ShopStyle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sign In with Apple for Angular 8 and NodeJS server.]]></title>
            <link>https://medium.com/shopstyle-engineering/sign-in-with-apple-for-angular-8-and-nodejs-server-f37e6a81545a?source=rss----b45652c2366---4</link>
            <guid isPermaLink="false">https://medium.com/p/f37e6a81545a</guid>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[apple-sign-in]]></category>
            <category><![CDATA[angular]]></category>
            <category><![CDATA[angular8]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Loïc Combis]]></dc:creator>
            <pubDate>Thu, 17 Oct 2019 21:42:39 GMT</pubDate>
            <atom:updated>2021-03-09T19:11:48.797Z</atom:updated>
            <content:encoded><![CDATA[<h3>Sign In with Apple for Angular 8 with a NodeJS server.</h3><figure><img alt="Sign In with Apple feature announcement at the 2019 Apple world wide developer conference in California." src="https://cdn-images-1.medium.com/max/800/1*Ds9aGCGjs9yENSjhVGCJ3A.jpeg" /><figcaption>Presentation of “Sign In with Apple” feature, at WWDC in June 2019.</figcaption></figure><p>Last June in California at the WWDC, Apple announced its new feature: “Sign In with Apple”. This solution aims to provide a <strong>fast</strong>, <strong>secure</strong> and <strong>privacy-focused</strong> way to authenticate users to your application.</p><blockquote>“Sign in with Apple makes it easy for users to sign in to your apps and websites using their Apple ID. Instead of filling out forms, verifying email addresses, and choosing new passwords, they can use Sign in with Apple to set up an account and start using your app right away. All accounts are protected with two-factor authentication for superior security, and Apple will not track users’ activity in your app or website.”</blockquote><p>At <a href="https://www.shopstyle.com/"><em>ShopStyle</em></a>, we decided to implement this feature on our <strong>WEB </strong>and<strong> IOS platforms. </strong>Indeed, a non-negligible amount of today’s users already have an Apple Id.<strong> </strong>Let’s have a quick look at the <strong>Javascript library</strong> and how to integrate it within your <strong>Angular 8 app </strong>with a<strong> NodeJS server:</strong></p><h4>1. Sign In with Apple Component</h4><p>First, you’ll need to set up a nice <strong>Sign In with Apple</strong> button to trigger the login flow. I would recommend you to follow <a href="https://developer.apple.com/design/human-interface-guidelines/sign-in-with-apple/overview/"><em>Apple’s guidelines</em></a> as they provide a comprehensive documentation on this matter.</p><figure><img alt="Apple Sign In button layout example." src="https://cdn-images-1.medium.com/max/434/1*4TmfqxwQ3mTXRTM4VDesUw.png" /><figcaption>Example of Sign In with Apple button.</figcaption></figure><p>Once you’ve created the component, what we want to do is binding the button’s click to trigger Apple’s Sign In flow. To do so, you’ll need to load the <strong>JS library</strong> provided by Apple. You can do it by adding a script tag in your index.html:</p><pre>&lt;script type=&quot;text/javascript&quot; src=&quot;https://appleid.cdn-apple.com/appleauth/static/jsapi/appleid/1/en_US/appleid.auth.js&quot;&gt;&lt;/script&gt;</pre><p>Because the user may never use Apple Sign In, we would rather load the library only when we need it. So what we’ll do instead is creating a new service to manage all the Apple Sign In related logic.</p><p>First, run the following command to create the service (don’t forget to provide it in app.module.ts) :</p><pre>ng g s AppleLoginService</pre><p>Then install the following dependency that we’ll use in the service:</p><pre>npm install --save scriptjs</pre><p>What we want is to load the library when the service is instantiated and then init the library as soon as it is available:</p><pre>private readonly appleScriptUrl: string = ‘apple_lib_script_url&#39;;</pre><pre>private ready: Promise&lt;boolean&gt; = new Promise(resolve =&gt; {</pre><pre>if (typeof window !== ‘undefined’) {<br>        const script = require(‘scriptjs’);<br>        script(this.appleScriptUrl, () =&gt; resolve(true));</pre><pre>} else {<br>        resolve(false);<br>    }</pre><pre>});</pre><pre>constructor() {<br>    this.ready.then(isReady =&gt; {<br>        if (isReady) {<br>            this.init();<br>        }<br>    });<br>}</pre><p>Several things happen here. First, we declared the URL of the JS library script <a href="https://appleid.cdn-apple.com/appleauth/static/jsapi/appleid/1/en_US/appleid.auth.js">https://appleid.cdn-apple.com/appleauth/static/jsapi/appleid/1/en_US/appleid.auth.js</a> Following this, we defined a promise that will load the library and resolve it once it’s done! Thanks to this, we can safely call the init method detailed below.</p><pre>private init(): void {</pre><pre>AppleID.auth.init({</pre><pre>clientId: &#39;your client id&#39;,</pre><pre>scope: ‘name email’,</pre><pre>redirectURI: `${window.location.origin}/apple-signin`,</pre><pre>state: ‘what ever string to be remembered’</pre><pre>});</pre><pre>}</pre><p>This is an important part of the process because this is where we tell Apple who we are, what data we want and where to send them.</p><p>→ <strong>clientId:</strong> this is your <strong>WWDR client identifier</strong>. If you don’t have an Apple developer account, you can create one following the <a href="https://developer.apple.com/programs/enroll/"><em>enrollment instructions</em></a></p><p>→ <strong>scope:</strong> this the data you want Apple to retrieve after the user authenticates. In our case, we asked for the name and the email. One important thing to know is that Apple retrieves these pieces of information only the first time the user authenticates (see testing section below).</p><blockquote><strong>Note:</strong> While logging in, the user may refuse to share their real email address. In this case, Apple will provide you a <a href="https://support.apple.com/en-us/HT210425"><em>proxy email address</em></a>. To be able to email the user through this anonymous address, you’ll need to register your contact email (the one used by your application) in your Apple developer account app.</blockquote><p>→ <strong>redirectURI:</strong> this is the URI to which Apple will post the authentication result. Like the contact email address, the redirectURI must be registered in your Apple developer account app. Note that Apple won’t let you define local URLs as redirect URIs.</p><p>→ <strong>state:</strong> this is a string that you want Apple to remember for when the user will be authenticated. For example, you could set the URL of the page to redirect after the login is successful.</p><p>If you want more information about the config object please check the <a href="https://developer.apple.com/documentation/signinwithapplejs/clientconfigi"><em>official documentation</em></a><em>.</em></p><p>Finally, the last step to trigger the login flow is to bind your Sign In with Apple button’s click to an actual call of the library signIn method:</p><pre>AppleID.auth.signIn();</pre><blockquote><strong>Note:</strong> You can override the previous configuration by passing another config object to the signIn method.</blockquote><p>Now, when a user clicks on the Sign In with Apple button, they will be <strong>redirected to Apple Sign In</strong>’s page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CrMvlYtcWW6GMIwUYICZtA.png" /></figure><p>If the user uses Safari and its device matches the requirements, the user will be able to sign in with <strong>touch/face id</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zylr5N0uZFqo9ctv4Km73A.png" /></figure><h4>2. Handle Apple’s response.</h4><p>Unlike Google or Facebook’s APIs, Apple Sign In doesn’t open a popup window which resolves in promise containing the authentication status and the user info. Once the user signed in (or canceled), <strong>Apple will send a POST</strong> request directly to your <strong>redirect URI</strong>.</p><p>So what we want to do is create the following <strong>middleware in our NodeJS server</strong>. It will be responsible for the token validation and the redirection logic.</p><pre>app.post(‘/apple-login’, appleLogin);</pre><p>Install the dependency used in the middleware:</p><pre>npm install —-save request</pre><p>And in apple-login.middleware.ts:</p><pre>export function appleLogin(req: Request, res: Response, next: Function) {</pre><pre>if (req.body.error === ‘user_cancelled_authorize’) {<br>        // The user cancelled the sign in process.       <br>        return res.redirect(req.body.state || ‘/’);<br>    }<br>    <br>    // Validate the token thanks to Apple Rest API<br>    request({<br>        url: , <br>        method: &#39;POST&#39;,<br>        json: true,<br>        body: body, // See below the content of the body.<br>        headers: {<br>            &#39;User-Agent&#39;: req.get(&#39;user-agent&#39;),<br>        }<br>    },<br>    (error, response, body) =&gt; {</pre><pre>if (error) {<br>            // Token is invalid<br>            return next(error);<br>        }</pre><pre>// The token is valid<br>        // Authenticate the user with your own API and redirect the    <br>        // user to page registered in the state.<br>        return res.redirect(req.body.state || &#39;/&#39;);<br>    });</pre><pre>}</pre><p>Many things happening in this middleware:</p><p>→ check if the user decided to cancel the authentication process.</p><p>→ if not, we verify the token received using the <strong>Apple Auth REST API</strong>. Regarding the content of the body of the received POST request, please check the <em>“Handle the Authorization Response” </em>section in the <a href="https://developer.apple.com/documentation/signinwithapplejs/configuring_your_webpage_for_sign_in_with_apple"><em>documentation</em></a>. Regarding the body to be sent to Apple Auth REST API, the response received and more information about how validating a token, consult <a href="https://developer.apple.com/documentation/signinwithapplerestapi/generate_and_validate_tokens"><em>this page</em></a>.</p><p>→ if the token is invalid we handle the error in another middleware.</p><p>→ finally, if the token is valid, we perform our authentication and redirect the user according to the state param of the config object <em>(part 1)</em>.</p><blockquote><strong>Note:</strong> Sign In with Apple is cross platform solution. This means the credentials received after the user authenticates may be reused across several platforms (i.e. If the user logs in with the web app, the credentials will be consistent with the mobile app). Moreover, every time the user uses another device and / or platform, Apple adds an extra layer of security for the user with 2-factor-authentication.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rOtPn1q4JFeDxUNTMILoVQ.png" /><figcaption>Sign In with Apple platforms availability.</figcaption></figure><h4>3. Testing</h4><p>As it is explained above, Apple will retrieve the requested information only the first time the user signs in. This may be a problem when testing your application as deleting your account in your app won’t be enough.</p><p>Fortunately, you can revoke the permissions given to your application and make Apple “forget” that you previously logged in to your app with Apple ID. To do so you need to follow these steps:</p><p>→ Go to <a href="https://appleid.apple.com/#!&amp;page=signin"><strong>Apple ID’s website</strong></a> and authenticate.</p><p>Once you’re on your account page (picture below), click the <em>“Manage…”</em> link of the “Security” section.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bPClOL6wLFV7Ds-BGP9WoA.png" /><figcaption>Apple ID account page.</figcaption></figure><p>This will open a modal with the list of the applications you signed in with your Apple ID. Choose your app.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3rFuXsEZvRyw-OJt" /><figcaption>Authorized Applications list.</figcaption></figure><p>Click on “Stop using Apple ID”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BSEX5j2k7aSftHO_" /><figcaption>Application details.</figcaption></figure><p>Finally, confirm that you want to revoke the permissions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tDyI4rpaPbCQlCj0" /><figcaption>Stop Using Apple ID confirmation window.</figcaption></figure><p>As a consequence, you will revoke the permissions given to your application. If you try logging in again into your app, Apple will treat this user as if was authenticating for the first time and therefore send the information set in the authentication configuration.</p><h4>Conclusion</h4><p>In today’s article, we implemented <strong>Sign In with Apple for Angular 8 </strong>with a NodeJS server. As we saw, we can split the process into 3 different parts.</p><p>First, we create the login component that is responsible for loading and initializing <strong>Apple’s JS library.</strong> When the user triggers the Apple Sign process, they are redirected to Apple’s Sign In page where they will perform the authentication.</p><p>Then, once the user authenticated, Apple sends a POST request that we handle in middleware on our NodeJS server.</p><p>Finally, we verify the token received using <strong>Apple Auth REST API</strong>. Once it’s done, all there is to do is redirecting the user to the app and update the application state.</p><p>If you’re interested in working on problems like these, ShopStyle is hiring! Check out our <a href="https://boards.greenhouse.io/shopstyle">open positions</a> and apply!</p><h3>More documentation about Sign In with Apple</h3><p>Another very interesting tutorial on <a href="https://medium.com/techulus/how-to-setup-sign-in-with-apple-9e142ce498d4"><em>how to set up Sign In with OAuth and Javascript</em></a> by Arjun Komath.</p><p>If you want more explanations about Apple Sign In flow, read this article, <a href="https://developer.okta.com/blog/2019/06/04/what-the-heck-is-sign-in-with-apple"><em>“What the heck is sign in with apple”</em></a><em> by </em>Aaron Parecki.</p><p>Here is the link of Apple official documentation on <a href="https://developer.apple.com/documentation/signinwithapplejs/configuring_your_webpage_for_sign_in_with_apple"><em>how to configure your webpage for sign in with Apple</em></a></p><h3>More Links</h3><p>Personal website: <a href="https://loic-combis.github.io/">https://loic-combis.github.io/</a></p><p>Twitter: <a href="https://twitter.com/CombisLoic">https://twitter.com/CombisLoic</a></p><p>Medium: <a href="https://loic-combis.medium.com/">https://loic-combis.medium.com/</a></p><p>Github: <a href="https://github.com/loic-combis">https://github.com/loic-combis</a></p><p>ShopStyle main website: <a href="https://www.shopstyle.com/">https://www.shopstyle.com/</a></p><p>More about what we do at ShopStyle: <a href="https://about.shopstyle.com/">https://about.shopstyle.com/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f37e6a81545a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/shopstyle-engineering/sign-in-with-apple-for-angular-8-and-nodejs-server-f37e6a81545a">Sign In with Apple for Angular 8 and NodeJS server.</a> was originally published in <a href="https://medium.com/shopstyle-engineering">ShopStyle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Improving Render Time and TTFB in with server-rendered Angular]]></title>
            <link>https://medium.com/shopstyle-engineering/improving-render-time-and-ttfb-in-with-server-rendered-angular-5fe8b87f4cd7?source=rss----b45652c2366---4</link>
            <guid isPermaLink="false">https://medium.com/p/5fe8b87f4cd7</guid>
            <category><![CDATA[angular]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[performance]]></category>
            <dc:creator><![CDATA[Caleb Amsden]]></dc:creator>
            <pubDate>Tue, 08 Oct 2019 19:22:07 GMT</pubDate>
            <atom:updated>2019-10-08T19:22:07.008Z</atom:updated>
            <content:encoded><![CDATA[<h3>Improving Render Time and TTFB with server-rendered Angular</h3><h3>Introduction</h3><p>At ShopStyle, we’ve found that improving site performance can have a huge impact on business success. While we measure performance in different ways, this article will focus on the early stage of the request lifecycle, metrics we call <strong>render time</strong> and <strong>time to first byte</strong> (TTFB).</p><p>We use render time as an internal measurement of how long it takes our Node server to render the final response payload. TTFB is an external measurement of how long it takes from the initial request to the client receiving the first byte of response data.</p><p>Render time and TTFB are critical because they determine how long it takes to display content to users, and for SEO crawlers like Googlebot or Bingbot slow TTFB can impact your ranking in search results.</p><p>In the next section, I’ll explain how we’ve reduced our average render time at ShopStyle from <strong>680ms </strong>to<strong> 175ms</strong> over the last year.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MT-b5esNSzFC77MN" /></figure><h3>Rendering Strategies</h3><p>ShopStyle uses a universal Angular app with server-rendering in Node via Express. This means that our application code is built to run both in Node as well as the browser. We chose this approach because it:</p><ol><li>has the flexibility and client performance of a single page app,</li><li>allows fast perceived performance by showing users content quickly, and</li><li>simplifies development by allowing us to quickly write components that can be consumed by bots and users.</li></ol><p>This flexibility comes with the potential downsides that TTFB can be slower than a client-rendered app, and it could take more time for the site to be fully interactive (time-to-interactive, or TTI). As Addy Osmani warns in <a href="https://medium.com/dev-channel/the-cost-of-javascript-84009f51e99e">The Cost Of JavaScript</a>:</p><blockquote>Many sites optimize content visibility at the expensive of interactivity. To get a fast first paint when you do have large JavaScript bundles, developers sometimes employ server-side rendering; then “upgrade” it to attach event handlers when the JavaScript finally gets fetched.</blockquote><blockquote>Be careful — this has its own costs. You 1) generally send down a larger<em> </em>HTML response which can push our interactivity, 2) can leave the user in an uncanny valley where half the experience can’t actually be interactive until JavaScript finishes processing.</blockquote><p>In a later post, we’ll discuss strategies on reducing JS payload size to reduce time users spend in the “uncanny valley”. But for the rest of this article, we’ll focus on addressing the issues Addy describes: ensuring we have fast initial responses (TTFB) and minimizing the size of the response.</p><p>Using a server-rendered app with client rehydration isn’t ideal for every site or web app. Server-rendering helps with perceived performance, but can negatively impact actual load times. If you can get away with client rendering only, for example for internal applications or applications behind an authentication gate, it’s often preferable to skip server-rendering.</p><p>Alternatively, if your content is static and you can pre-render and cache the HTML in a CDN, then it doesn’t make sense to pay the performance cost of dynamic rendering.</p><p>For more information on rendering strategies, check out another <a href="https://developers.google.com/web/updates/2019/02/rendering-on-the-web">excellent post</a> by Addy Osmani and Jason Miller.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*p5HRZ0y_t_Ojpriv" /><figcaption>Source: <a href="https://developers.google.com/web/updates/2019/02/rendering-on-the-web">https://developers.google.com/web/updates/2019/02/rendering-on-the-web</a></figcaption></figure><h3>CPU Profiling</h3><p>The first step in improving our performance was to understand what parts of our system were slow. As we started to dig into our application performance, we found that CPU profiling was critical in measuring how Node rendered our application.</p><p>To gather a CPU profile from a Node app, you’ll want to pass in the --inspect flag:</p><pre>node --inspect dist/app/server.js</pre><p>Once your local server is running, follow these steps to record a CPU profile:</p><ol><li>open chrome://inspect in chrome</li><li>click the link called “Open dedicated DevTools for Node”</li><li>navigate to the “Profiler” tab</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*WqGTCGO180aOHtuft88laQ.png" /></figure><p>The “Start CPU profiling” button is a solid circle in the top left of the Profile tab. Click on the button to start recording, then navigate to your local server address in a new tab.</p><p>Your CPU chart will look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3r9tzAIj9vYkcWLh6IfTYg.png" /></figure><p>You may have to scroll down and dig until you find some entries for your application code, but you can also search with cmd-f (or ctrl-f) until you find your component code:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H20ADaXh-LQPSNIgJPqVRg.png" /><figcaption>example of a component method for the ShopStyle home page in our CPU profile</figcaption></figure><h3>Reducing Render Time</h3><p>After running a few CPU profiles, we discovered a rogue tracking request that was blocking render and causing a spike in response times. The request was introduced to send our performance data to a tracking service, but in fact, ended up causing a performance regression.</p><p>For background, Angular uses a library called Zone.js to wrap asynchronous actions so it knows when to run change detection. This also applies when running Angular in Node. For example, think about a component that looks like:</p><pre>@Component({<br>  selector: &#39;app&#39;,<br>  templateUrl: &#39;./app.component.html&#39;,<br>  styleUrls: [&#39;./app.component.css&#39;],<br>})<br>export class AppComponent {<br>  rendered = false;</pre><pre>constructor() {<br>    setTimeout(() =&gt; {<br>      this.rendered = true;<br>    }, 30000)<br>  }<br>}</pre><p>Zone wraps the setTimeout method, so the page won’t be rendered until the 30-second timer has cleared. The same concept applies to asynchronous API requests; if you are waiting on a slow response that doesn’t impact the initial render you should remove it. By queueing the tracking requests and sending them after the response was sent, we <strong>shaved off 300ms</strong> from our average response times.</p><p>When reviewing the compiled instructions in the CPU profile, we also noticed some component code was running that wasn’t needed for the initial render. Any dynamic content that isn’t visible on the first page load and isn’t needed for SEO should be skipped. In our case, we were rendering hidden content in the header and search bar. Removing this from the server-rendered app saved roughly <strong>100ms per request</strong>.</p><p>The last issue we found in the CPU profiles were methods that seemed to run over and over. This was an ngFor issue, and we quickly realized it was related to the product cells on our &quot;/browse” pages:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h1f74frZkKpJyUDgykzJZQ.png" /><figcaption>Example of a page with multiple product cells; while you can only see 8, we render a total of 40 per page.</figcaption></figure><p>Even though most users will only see 6–10 products above the fold, we render 40 products — for SEO reasons and so users will see additional products as they scroll. If anything is even slightly slow inside this ngFor then it would have an impact on render times.</p><p>In our case, we weren’t able to identify a single binding or method that was slow. It turned out that the sheer number of bindings led to higher than ideal render times. By carefully combing through component code, we were able to remove unused bindings, move style logic to CSS and replace complicated binding logic with pipes (<a href="https://blog.mgechev.com/2017/11/12/faster-angular-applications-pure-pipes-memoization-pure-functions-part-2/">see this article</a> from Angular team member Minko). The refactor wasn’t quick and easy, but it did reduce our average response times by <strong>another 100ms</strong>.</p><p>To summarize, here are some of the lessons learned by performing a deep dive into profiles of our Node rendering:</p><ol><li>Only run asynchronous tasks on the server when necessary.</li><li>Skip server-rendering of content that’s not visible on the first render.</li><li>Be cautious of template bindings when inside an ngFor, use pure Pipes where you can.</li></ol><h3>Reducing Latency</h3><p>While working on reducing render times, we also wanted to reduce the size of our rendered response to optimize the time it takes to send them <a href="https://medium.com/@asolove/the-new-fast-advanced-web-performance-best-practices-for-2017-28d1684414e0">over the wire</a>. In effect, having a smaller payload and fast render times will lead to fast TTFB measurements.</p><p>At ShopStyle, we decided to implement third party Real User Monitoring (RUM) as well as internal tracking using window.performance.timing metrics to view performance data from a sample of users. Also, we implemented load tests using <a href="https://artillery.io/">Artillery</a> to measure TTFB before we release new code.</p><p>We’ve found several keys to reducing the size of the response are:</p><ol><li>Minifying HTML: remove comments, whitespace, source maps, etc</li><li>Rehydrating browser state: minimize data in transferState</li><li>Test compression strategies</li></ol><p>You’ll also want to ensure you minify rendered HTML. If you are using the Angular CLI, set the optimization option to true in your angular.json file.</p><pre>&quot;server&quot;: {<br>  &quot;builder&quot;: &quot;@angular-devkit/build-angular:server&quot;,<br>    &quot;options&quot;: {<br>      &quot;optimization&quot;: true,<br>       ...<br>    }<br>  }<br>}</pre><p>If you are using Angular server rendering, you’re likely using <a href="https://angular.io/api/platform-browser/TransferState">transferState</a> to rehydrate application state from the server to the client. What this means is Angular saves some data in a script tag with an id called “app-state”:</p><pre>&lt;script id=”app-state” type=”application/json”&gt;</pre><p>When Angular loads in the browser, it reads the data inside the app-state script to quickly bootstrap your application state. The caveat to this is to only put data in transferState that’s absolutely necessary to render your application, any extra data is wasted bloat that will slow down the HTML response.</p><p>Lastly, ensure you compress the response after rendering. You can easily add Node middleware to do this:</p><pre>const compression = require(&#39;compression&#39;);<br>const express = require(&#39;express&#39;);</pre><pre>const app = express();</pre><pre>app.use(compression({ level: 6 }));</pre><p>The default level is 6, but you should experiment with different levels of compressing. Lower levels of compression will be faster but have larger output.</p><p>The Express compression library is currently <a href="https://github.com/expressjs/compression/pull/150">working on</a> support for brotli, a newer compression algorithm from Google. We use brotli to compress static assets and are excited to look into testing the middleware for rendered responses when it’s available.</p><h3>Caching</h3><p>Many developers are familiar with caching assets in a CDN or in the browser, but when it comes to render performance, caching HTML can supercharge your response time.</p><p>In an ideal world, we would pre-render every page and cache it in a CDN. However ShopStyle has millions of products that can potentially change every 30 minutes; it would be extremely expensive to set up a pre-rendering strategy to support this amount of dynamic content.</p><p>Alternatively, we could put a CDN in front of our dynamically rendered content to cache responses. However, we like to run ab tests that impact the HTML output and potentially return different output depending on user state.</p><pre>generateKey(req: Request, res: Response) {<br>  // logic to pull device, url, country and <br>  // matching a/b tests off the request go here...</pre><pre>return hash({<br>    device,<br>    url,<br>    country,<br>    abtests: JSON.stringify(matchingTests),<br>  });<br>}</pre><p>A good alternative to using our custom Redis service would have been to use a <a href="https://varnish-cache.org/intro/index.html#intro">Varnish</a> caching layer. We decided to go with Redis since we could easily write custom caching logic in TypeScript via Node middleware.</p><h3>Coming soon…</h3><p>One of the more <a href="https://github.com/vikerman/ivy-universal">exciting projects</a> we’ve seen in the Angular ecosystem shows how to use Web Components and the new Angular Ivy rendering engine to dynamically load application code only when users interact with the component or related components.</p><h3>Houssein Djirdeh on Twitter</h3><p>Here&#39;s a visual of how @vikerman and the @angular team is exploring progressive hydration with Ivy In here, Angular runtime code is only fetched AFTER the user begins interacting with page This could enable shipping a very small bundle upfront ⚡️ https://t.co/EUUroi5WT2 https://t.co/Ohyq2JhUCr</p><p>This allows you to dynamically server-render a lightweight page with a minimal JavaScript file (~12kb) responsible for bootstrapping the rest of the application. We plan on conducting similar experiments to see if we can apply this approach to a large-scale application like <a href="https://www.shopstyle.com/">ShopStyle.com</a>.</p><p>At the same time, we are currently testing a Progressive Web App (PWA) with the <a href="https://developers.google.com/web/updates/2015/11/app-shell">App Shell Architecture</a>. We’ll still dynamically server-render responses for bots, but when we detect a user request we send back a small, pre-rendered app shell. In other words, we’ll pre-render static content like the header/footer, and render the rest of the page in the browser.</p><p>Because the pre-rendered shell is mostly static, it’ll be easier to cache the shell early in the request lifecycle. For returning users, the PWA will cache the shell in the browser so it will be loaded instantly as soon as users make a request. For new users, we use Lambda Edge functions to detect and serve the shell to users via AWS servers closest to them.</p><p>In summary, we’re happy to see results from our initial performance work, but are even more excited about the future of web performance at ShopStyle!</p><p>If you’re interested in working on performance problems like these, ShopStyle is hiring! Check out our <a href="https://boards.greenhouse.io/shopstyle">open positions</a> and apply!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5fe8b87f4cd7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/shopstyle-engineering/improving-render-time-and-ttfb-in-with-server-rendered-angular-5fe8b87f4cd7">Improving Render Time and TTFB in with server-rendered Angular</a> was originally published in <a href="https://medium.com/shopstyle-engineering">ShopStyle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Event Log Processing in Spark: Dealing with Data Schema Change]]></title>
            <link>https://medium.com/shopstyle-engineering/event-log-processing-in-spark-dealing-with-data-schema-change-5c2584f4d898?source=rss----b45652c2366---4</link>
            <guid isPermaLink="false">https://medium.com/p/5c2584f4d898</guid>
            <category><![CDATA[data-modeling]]></category>
            <category><![CDATA[spark]]></category>
            <category><![CDATA[data-processing]]></category>
            <dc:creator><![CDATA[Phil Kallos]]></dc:creator>
            <pubDate>Tue, 17 Nov 2015 21:14:36 GMT</pubDate>
            <atom:updated>2015-11-17T23:26:53.730Z</atom:updated>
            <content:encoded><![CDATA[<p>The core component of our data infrastructure here at ShopStyle is our event log. In order to improve our product, we want to have a clear picture of how shoppers use ShopStyle to find what they are looking for. We also want to log events from our content ingestion platforms to monitor things like product price changes and inventory changes.</p><h4>Basic Schema</h4><p>We have chosen to model our events as simple <a href="https://thrift.apache.org/">Thrift</a> schemas, which we stream through <a href="https://aws.amazon.com/kinesis/">Amazon Kinesis</a> and persist in Amazon S3 for long-term storage. We then have streaming and batch processes, running on<a href="http://spark.apache.org/"> Apache Spark</a>, that process this raw data into meaningful output.</p><p>Here’s a much simplified version of our event schema.</p><pre>struct RawEvent {<br> 10: required string eventId,   // Unique UUID<br> 11: required i64 timestamp,    // Millisecond timestamp<br> 20: required string eventType, // The type of event e.g. “Search”<br> 30: optional i64 userId,       <br> 40: optional string country,<br> 41: optional string language,<br> 50: optional map&lt;string, string&gt; extraValues<br>}</pre><p>For the most part, this model serves us very well, as it supports events initiated by a user (Search for products, Click on a product) as well as events that represent other types of log data (a Product’s price was updated). Thrift structure allows us to add new fields in a backwards compatible manner, so if we needed to add a new field to record the IP of the originating event, we could add a new attribute.</p><pre>12: optional string ipAddress </pre><p>And we’re good to go.</p><h4>Flexible Schema</h4><p>This works nicely, but what we quickly noticed during instrumentation is that our application developer teams really wanted a lot of flexibility in the metadata that they record.</p><p>This is where the <em>extraValues</em> field comes into play. Although the field backwards compatibility in Thrift is really nice, in practice to add fields you need to change the Thrift schema, recompile and redistribute the generated sources, and likely do a code push to production in order for your change to propagate. For long-lived fields like <em>ipAddress</em>, this is fine, and we benefit from the type definition and structure that Thrift offers. But for other shorter-lived fields, or fields that are less predictable, we use a simple <em>map&lt;string, string&gt;</em>.</p><p>As an example, say we want to let new users know about our awesome Sale Alert feature where you can receive email notifications when a product goes on sale. We are testing a dialog that prompts them with instructions on how to set up a Sale Alert, and we want to record that they saw it, and later if they interact with it or if they simply dismiss it.</p><p>We can fire this off as a JSON event from the web front-end:</p><pre>{<br>  eventType: “Impression”,<br>  country: “us”,<br>  language: “en”,<br>  timestamp: new Date(),<br>  page: {<br>    url: “<a href="http://www.shopstyle.com/">http://www.shopstyle.com</a>&quot;,<br>    title: “ShopStyle Home”<br>  },<br>  modal: {<br>    id: “sale-alert-signup-test1”,<br>    name: “Sale Alert Signup”<br>  },<br>  user: {<br>    id: 12341234<br>  }<br>  // …<br>}</pre><p>And later down the pipeline, we can expect that this JSON will be translated to our type-safe, compact Thrift schema. The fields that correspond 1-to-1 with the Thrift schema are populated (e.g. <em>userId</em>), and the fields that don’t correspond are shredded out of JSON and become key/value pairs in <em>extraValues</em>.</p><pre>RawEvent(<br>  userId: 12341234<br>  timestamp: 1234712341234<br>  country: “us”<br>  language: “en”<br>  extraValues: map(<br>    “page.url” : “<a href="http://www.shopstyle.com/">http://www.shopstyle.com</a>&quot;<br>    “page.title”: “ShopStyle Home”<br>    “modal.id”: “sale-alert-signup-test1”<br>    “modal.name”: “Sale Alert Signup”<br>  )<br>  // …<br>)</pre><p>Great! This gives us a bit more flexibility in instrumentation, and later if we decide to stop running tests with new modals, we won’t be stuck with the hyper-specific <em>modalId: String</em> field on our Thrift schema.</p><h4>Dealing with Change</h4><p>Now, you might be looking at the <em>extraValues</em> map and wondering what happens when you need to move or rename a field. How do you maintain backwards compatibility? Say for example we have a field in our <em>extraValues </em>map where we store a product id, and we sensibly decide to call it “productId”. For whatever reason a few months later it turns out we need to call it “product.id”. Given that we have batch processing code that needs to potentially run against both the “productId” and “product.id” event logs, we’re in a bit of a pickle. Do we have to remember the exact date and time we made the switchover, for every change we make?</p><p>The answer is to abstract and pack away the logic for extracting fields when reading from the RawEvent objects in the event log. Scala’s handy <a href="https://coderwall.com/p/k_1jzw/scala-s-pimp-my-library-pattern-example">pimped out classes</a> to the rescue! We build a thin implicit class wrapper around RawEvent, and tuck away the potentially complex logic for finding values you care about.</p><pre>object RawEventImplicits { <br>  implicit def toRichRawEvent(e: RawEvent): RichRawEvent = new RichRawEvent(e)<br>}</pre><pre>class RichRawEvent(val e: RawEvent) extends AnyVal {<br>  // Thrift required fields<br>  def eventId: String = e.getEventId<br>  def timestamp: Long = e.getTimestamp<br>  def eventType: String = e.getEventType<br> <br>  // Thrift optional fields become Scala Options<br>  def userId: Option[String] = Option(e.getUserId)<br>  def country: Option[String] = Option(e.getCountry)<br>  def language: Option[String] = Option(e.getLanguage)<br>  def extraValues: Option[Map[String, String]] =<br>    Option(e.getExtraValues.asScala.toMap)<br> <br>  // Utility methods for extracting fields<br>  // The newer field name comes first<br>  def productId: Option[Long] =<br>    strFromExtraValues(“product.id”).orElse(<br>      strFromExtraValues(“productId”)).map(_.toLong)<br> <br>  // Helper methods<br>  private def strFromExtraValues(key: String): Option[String] = <br>    e.extraValues.flatMap(_.get(key))<br>}</pre><p>Such an approach gives us a robust event schema that holds up to changes and makes our lives easier if we need to revisit certain initial assumptions about our instrumentation. We are able to maintain Spark batch and streaming code that can run interchangeably against both old and new event logs.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5c2584f4d898" width="1" height="1" alt=""><hr><p><a href="https://medium.com/shopstyle-engineering/event-log-processing-in-spark-dealing-with-data-schema-change-5c2584f4d898">Event Log Processing in Spark: Dealing with Data Schema Change</a> was originally published in <a href="https://medium.com/shopstyle-engineering">ShopStyle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[To Train Or Not To Train]]></title>
            <link>https://medium.com/shopstyle-engineering/to-train-or-not-to-train-623ef432f5a?source=rss----b45652c2366---4</link>
            <guid isPermaLink="false">https://medium.com/p/623ef432f5a</guid>
            <dc:creator><![CDATA[Maria Kissa]]></dc:creator>
            <pubDate>Sat, 14 Nov 2015 02:41:06 GMT</pubDate>
            <atom:updated>2015-11-17T21:17:42.070Z</atom:updated>
            <content:encoded><![CDATA[<h3>To Train Or Not To Train?</h3><h4>a philosophical inquiry about machine learning and an oversimplified introduction to the latter …</h4><p>Soon enough after entering the beautiful world of Natural Language Processing (NLP) I realized how massively machine learning is used. So much, that sometimes people think NLP is actually a subfield of machine learning, when in fact machine learning is just some of the means to facilitate successful NLP.</p><p>Hey, wait a minute … What is NLP? To keep it short, simply think it as making sense out of raw text (or at least this is how I think of it) and there are loads of ways to do so, including machine learning.</p><p>And what is machine learning? According to <a href="https://en.wikipedia.org/wiki/Machine_learning">Wiki</a> “<strong>Machine learning</strong> is a subfield of computer science … explores the study and construction of algorithms that can learn from and make predictions on data.”</p><p>Now, let’s go through the basic steps of machine learning, (or at least how I perceive them):</p><ol><li>have a prediction problem</li><li>build a dataset (decide on the features)</li><li>build the algorithm (ideally) / pick existing algorithm(s) (in real life — outside academia) and focus on step 2</li><li>train the algorithm on the dataset</li><li>there you go … you now have a model</li><li>evaluate the model (aka, compute F-measure)</li><li>if F-measure high stop, if not go back to step 2 and 3 (these two go more or less hand in hand).</li></ol><p>(<em>Attention!</em> The description of the steps is purposefully written with the form of these high level and oversimplified statements. )</p><p>What I would like to focus on is Step 1, aka “<em>have a prediction related problem</em>”. What is a prediction related problem? Well, to put it simply: we want to predict one value in a distribution (that’s linear regression), we want to classify an element in two or more classes (logistic regression, random forests, support vector machines are some algorithms for that), we want to group elements into categories (clustering algorithms will do the trick).</p><p>What are the NLP problems where machine learning is used?</p><p>They are a lot and they are very interesting, but hardly is machine learning a pure solution to them. Really often, machine learning has to paired with rule based systems to generate high performance results. Lately, Stanford articles have made this fact clearer more than ever. Coreference resolution, tokenization and word sense disambiguation, are just a few examples. Maybe, topic modeling, meaning assign a topic to a piece of text is a problem that machine learning can entirely tackle (aka, Latent Dirichlet Allocation) but again, there are other ways with comparable performance that can do the trick. I’ve seen Latent Semantic Indexing and generalized Vector Space Model work wonders when documents are from a particular context and hence disambiguation is not a issue anymore.</p><blockquote>Study your problem first, explore your data!</blockquote><p>And that is exactly the point: why should I use LDA for document retrieval, spend my time training a model now and then, depending on that, when all my documents come from the world of nutrition where an apple refers to an apple and not the technology company? (Again, an oversimplification to make a point but still…)</p><p>Both, during my PhD and also in a recent project, the problems to be tackled were suggesting the use of machine learning. I avoided it and it worked both times with the use of statistical formulas, traditional algorithms and rule-based techniques. With high performance (close to 100%). Why? Simply, because when studying the problem and my data exhaustively, YES, the solution was there. Simply, because I do not want to be dependent on a model that I have to retrain every now and then (e.g., for a different language, which something so common in NLP). Simply, because building a dataset of 15k sentences/elements to train on (according to Stanford’s recommendation) is not a “survivable” option for me (especially, when I am not convinced that this is the last resort :P).</p><p>My point is machine learning is great and utterly useful, but not all problems are “machine learning” ones. Just, because something might be big data, doesn’t always suggest machine learning. And that would be my advice: think twice. Think thoroughly if your problem is a machine learning one, and then move forward.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=623ef432f5a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/shopstyle-engineering/to-train-or-not-to-train-623ef432f5a">To Train Or Not To Train</a> was originally published in <a href="https://medium.com/shopstyle-engineering">ShopStyle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>