<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[dubizzle Engineering - Medium]]></title>
        <description><![CDATA[Tech and Product News from the boiler room. - Medium]]></description>
        <link>https://medium.com/dubizzletechblog?source=rss----dafa6a20811c---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>dubizzle Engineering - Medium</title>
            <link>https://medium.com/dubizzletechblog?source=rss----dafa6a20811c---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 11:33:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/dubizzletechblog" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Continuous Testing for iOS Apps using XCUITest, Bitrise & Browserstack]]></title>
            <link>https://medium.com/dubizzletechblog/continuous-testing-for-ios-apps-using-xcuitest-bitrise-browserstack-795c7c1af391?source=rss----dafa6a20811c---4</link>
            <guid isPermaLink="false">https://medium.com/p/795c7c1af391</guid>
            <category><![CDATA[bitrise]]></category>
            <category><![CDATA[xcuitest]]></category>
            <category><![CDATA[continuous-integration]]></category>
            <category><![CDATA[browserstack]]></category>
            <category><![CDATA[ios]]></category>
            <dc:creator><![CDATA[Satheesh Kumar]]></dc:creator>
            <pubDate>Thu, 01 Aug 2019 10:40:35 GMT</pubDate>
            <atom:updated>2019-10-17T21:51:59.584Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EKEtEMZW7iB0UkFSRqkktg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@andreas_haslinger?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Andreas Haslinger</a> on <a href="https://unsplash.com/search/photos/ios-testing?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>iOS Apps Continuous Testing journey is really exciting for anyone who is trying to implement it using different tools/framework combinations.</p><p>In this article, I will share about how it can be implemented using XCUITest, Bitrise and Browserstack platforms.</p><p>We used <strong>XCUITest </strong>automation framework provided by Apple and POM design pattern for writing tests which helped us to write clean and maintainable tests.</p><p><strong>Bitrise </strong>continuous Integration system enabled us to create pipelines which can build both iOS and android projects without worring about managing build environments/machines.</p><p><strong>Browserstack </strong>served the need of a Cloud Testing platform well to enable “Continuous Testing” for iOS and android apps.</p><p>Let me share few of the tips that will make “Continuous Testing” seamless in your Continuous Delivery pipelines:</p><ol><li><strong>Avoid building “UI Automation Scheme/Tests” in the release pipelines</strong>: If you are building your xcuitests as well in the release pipelines (release to beta/appstore), this will increase the build time and might act like additional build step which would take almost same time of building your ipa. The optimized approach would be to build your “XCUITests” scheme on a nightly basis(we created scheduled jobs in Bitrice-CI) and publish the xcuitests to your cloud testing platform. Browserstack has handy apis using which you can publish your xcuitests on the cloud, which can be used for executing the tests later in the release pipelines.</li></ol><p><strong>Bitrise-CI: Scheduled Job View:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fA-lNrpCqNynlnVCZwEQmQ.png" /></figure><p>Now your release pipeline steps will be optimized like below: (ie: once the ipa is built, just publish it to Browserstack and then start executing tests using previously published xcui_test_suites)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YsFr88MmahT-PONm2ycFnQ.png" /></figure><p><strong>2. Group your tests:</strong></p><p>We classified our tests into BVT, Smoke, Sanity and Regression and created separated files to store tests details and device to execute the test as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qSfhYSsGYCbEE3pyooMkSA.png" /></figure><p><strong>3. Use the newly available Bitrise step “Browserstack iOS UI Test” to specify test configurations like ipa url, xcuitest url, tests to execute etc. [ Really excited to become the creator/maintainer of this step :) ]</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s6B9Wyesi4QpIQUD4WTFbQ.png" /></figure><p>Following are the configuration options available on this step:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RosSL2Jef41w7G3IsXdUdw.png" /></figure><p>As we all know that “UI Automation” is sometimes flaky due to different reasons, automatic re-try for failed tests is a must have feature in your CI pipelines. This step supports automatic retry as well to make ui test execution seemless in the continous testing pipelines.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ICpz2865neZb04ll7spjBQ.png" /></figure><p>Also, “Browserstack local testing feature also is supported via this step”. One requirement is, you need to make sure Browserstack local binary is running before executing this step and jq pacakge is installed on the node on which this step is running.</p><p>That’s it. Now you should be able to run your XCUITests on Browserstack successfully.</p><p><strong>Test Results from Browserstack cloud testing platform:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sEaY5VyFhGSNgBNYGCVp1w.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=795c7c1af391" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dubizzletechblog/continuous-testing-for-ios-apps-using-xcuitest-bitrise-browserstack-795c7c1af391">Continuous Testing for iOS Apps using XCUITest, Bitrise &amp; Browserstack</a> was originally published in <a href="https://medium.com/dubizzletechblog">dubizzle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Setting up Prettier and ESlint for JS and React Apps]]></title>
            <link>https://medium.com/dubizzletechblog/setting-up-prettier-and-eslint-for-js-and-react-apps-bbc779d29062?source=rss----dafa6a20811c---4</link>
            <guid isPermaLink="false">https://medium.com/p/bbc779d29062</guid>
            <category><![CDATA[eslint]]></category>
            <category><![CDATA[prettier]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Mahmoud Felfel]]></dc:creator>
            <pubDate>Wed, 20 Feb 2019 14:21:50 GMT</pubDate>
            <atom:updated>2020-05-07T20:00:55.711Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*83PZeBAFQkP1XyOfDigxsg.png" /><figcaption>Prettier + ESLint</figcaption></figure><p>In any Frontend project, especially with big teams, it is important to standardize the code style and formatting, this has many benefits:</p><p>1- It will ensure clean code which follows the best practices across the team, especially with interns or junior team members who might not know why a specific way of writing code if better than another.</p><p>2- Will help everyone to learn and level up, as every linting rule is there for a purpose and each JS standard has the list of reasons with examples why any specific rule is needed.</p><p>3- Reduces code review time and makes it more focused on code quality and optimizations instead of syntax and style, so you don’t have to add a comment again to ask your colleague to use const instead of var, or remove an unused variable or module.</p><p>4- Will give you more confidence in your code, as you will be sure there will never be a production issue because someone had a typo in some variable name or be embarrassed by your console.log(&#39;LOL&#39;) slipping into production.</p><p>So, I hope I convinced you or gave you enough reasons to convince your team to adopt linting and styling tools for your project.</p><p>The most famous tools in the JS community for code formatting and syntax linting are Prettier and ESlint. We will see now how to get your project set up and ready to use these tools in three steps:</p><p>1- Adding the required npm modules and configurations.<br>2- Adding a pre-commit hook to make sure no code is pushed with linting errors.<br>3- Handling legacy code that doesn’t follow linting rules and not autofixable.</p><h3><strong>1-Adding the required npm modules and configurations:</strong></h3><pre>&gt; npm install --save-dev babel-eslint eslint eslint-config-airbnb eslint-config-prettier eslint-plugin-react eslint-plugin-import prettier pretty-quick</pre><p>Or using yarn:</p><pre>Yarn add -D babel-eslint eslint eslint-config-airbnb eslint-config-prettier eslint-plugin-react eslint-plugin-import prettier pretty-quick</pre><p>Here we added <strong><em>eslint</em></strong> for code-quality rules and we will follow the <a href="https://github.com/airbnb/javascript">Airbnb js</a>/<a href="https://github.com/airbnb/javascript/tree/master/react">react</a> standard for that.</p><p>Also, we added <strong><em>prettier</em></strong> which will format your code to follow a specific set of formatting rules.</p><p><strong><em>Pretty-quick</em></strong> is used to apply prettier to the staged files only, we will use this for the pre-commit hook.<br>And the job of <a href="https://github.com/prettier/eslint-config-prettier"><strong><em>eslint-config-prettier</em></strong></a> is to turn off all ESLint rules that are unnecessary anymore or might conflict with Prettier rules.</p><p>Now you would need to add some configuration files for your app to start following those rules.</p><p><strong>Add a .prettierrc file:</strong></p><pre>{ <br> “printWidth”: 100,<br> “trailingComma”: “all”,<br> “tabWidth”: 2, <br> “semi”: true, <br> “singleQuote”: true<br>}</pre><p>This is just an example if you want to change some default rules based on your or your team’s preference.</p><p>You can do the same with <strong><em>eslint</em></strong> by adding <strong>.eslintrc</strong></p><pre>{</pre><pre>    &quot;parser&quot;: &quot;babel-eslint&quot;,</pre><pre>    &quot;parserOptions&quot;: {</pre><pre>    &quot;sourceType&quot;: &quot;module&quot;,</pre><pre>    &quot;allowImportExportEverywhere&quot;: false,</pre><pre>    &quot;codeFrame&quot;: false</pre><pre>},</pre><pre>&quot;extends&quot;: [&quot;airbnb&quot;, &quot;prettier&quot;],</pre><pre>&quot;env&quot;: {</pre><pre>    &quot;browser&quot;: true,</pre><pre>    &quot;jest&quot;: true</pre><pre>},</pre><pre>&quot;rules&quot;: {</pre><pre>    &quot;max-len&quot;: [&quot;error&quot;, {&quot;code&quot;: 100}],</pre><pre>    &quot;prefer-promise-reject-errors&quot;: [&quot;off&quot;],</pre><pre>    &quot;react/jsx-filename-extension&quot;: [&quot;off&quot;],</pre><pre>    &quot;react/prop-types&quot;: [&quot;warn&quot;],</pre><pre>    &quot;no-return-assign&quot;: [&quot;off&quot;]</pre><pre>  }</pre><pre>}</pre><p>You notice there we are extending <strong><em>airbnb</em></strong>, and <strong><em>prettier</em></strong> to extend the default <strong><em>eslint</em></strong> rules with their rules for code quality and formatting.</p><p>Also notice that we are using babel-eslintas the parser, which will allow <strong><em>eslint</em></strong> to lint all your valid babel (es6,7, …etc) code.</p><h4>Adding Eslint to your code editor:</h4><p>I will talk here about <strong><em>vscode</em></strong>, but I believe you can do the same with any code editor.</p><p>First add a <strong><em>.editorconfig</em></strong> file in the root directory of your project, for example:</p><pre># <a href="http://editorconfig.org/">http://editorconfig.org</a></pre><pre>root = true</pre><pre>[*]</pre><pre>charset = utf-8</pre><pre>end_of_line = lf</pre><pre>indent_size = 2</pre><pre>indent_style = space</pre><pre>insert_final_newline = true</pre><pre>max_line_length = 100</pre><pre>trim_trailing_whitespace = true</pre><pre>[*.md]</pre><pre>max_line_length = 0</pre><pre>trim_trailing_whitespace = false</pre><pre>[{Makefile,**.mk}]</pre><pre># Use tabs for indentation (Makefiles require tabs)</pre><pre>indent_style = tab</pre><pre>[*.scss]</pre><pre>indent_size = 2</pre><pre>indent_style = space</pre><p>Then you can use the <a href="https://marketplace.visualstudio.com/items?itemName=EditorConfig.EditorConfig"><strong><em>editorconfig vscode extension</em></strong></a> to override your editor’s default settings to follow the <strong><em>.editorconfig</em></strong> file.</p><p>Also, you can use the <a href="https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode"><strong><em>Prettier vscode extension</em></strong></a> to make prettier the default formatter for your js modules.</p><p>I also recommend you to enable <strong>onSave</strong> formatting and <strong>autofixing</strong>, you can do that by going to your <strong>vscode</strong> <strong>user settings</strong> Command + , or <strong>Code &gt; Preferences &gt; settings</strong>. Then add these two lines to your settings object:</p><pre>“editor.formatOnSave”: true,</pre><pre>“eslint.autoFixOnSave”: true</pre><p>Now your project has all the configurations and required to follow <strong><em>eslint</em></strong> and prettier.</p><h3><strong>2- Adding a pre-commit hook to make sure no code is pushed with linting errors.</strong></h3><p>You can easily add a <strong>pre-commit</strong> hook to your project using <a href="https://www.npmjs.com/package/husky"><strong><em>husky</em></strong></a> npm module, but there is another module that makes it even easier to lint your staged files before committing which is <a href="https://www.npmjs.com/package/lint-staged"><strong><em>lint-staged</em></strong></a>.</p><pre>npm install — save-dev lint-staged husky</pre><p>Then add to your <strong><em>package.json</em></strong> file:</p><pre>“husky”: {</pre><pre>    “hooks”: {</pre><pre>        “pre-commit”: “lint-staged”</pre><pre>    }</pre><pre>},</pre><pre>“lint-staged”: {</pre><pre>    “src/**/*.{js,jsx}”: [“eslint”, “pretty-quick — staged”, “git add”]</pre><pre>},</pre><p>This will run <strong><em>eslint</em></strong> and <strong><em>prettier</em></strong> on your staged files whenever you attempt to create a new commit. So all your new code now will be following the eslint quality and prettier formatting rules.</p><h3><strong>3- (Optional) Handling legacy code that doesn’t follow linting rules and not autofixable.</strong></h3><p>You mostly are all set by now, and you might not need this step. But if you have a large code base and it is being touched a lot by different team members then whenever anyone tries to edit a file, he will get linting errors for all the code in the file, as that will be the staged file now which will make it harder for others to refactor everything in every module they need to change.</p><p><strong>There is a solution to that situation and we will do it on two steps:</strong></p><p>First, we can use prettier to format all our code base to follow the prettier rules:</p><p>You can solve this by first running prettier on all your codebase:</p><pre>prettier --write src/</pre><p>Then do the same with eslint:</p><pre>eslint — fix src/</pre><p>Run your code to make sure nothing is broken, which ideally it shouldn’t be.<br><strong>Hint</strong>: You can always do git rebase --hard to reset your local branch and files to the latest commit, which is the status before you run the above commands. Just in case anything went wrong.</p><p>But even then, if you try to commit now, you will find the number of eslint errors is less but still (in a large code base) you might still be getting thousands of errors.</p><p>I fixed that by writing a small script that silences the broken rules only in the same file not globally over the whole codebase. And you can consider these as FIXMEs that you can clean up gradually at later point whenever you have time.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4ad26ad88adbd8b6ef42e71b738d5f6b/href">https://medium.com/media/4ad26ad88adbd8b6ef42e71b738d5f6b/href</a></iframe><p>So you will end up with files like this:</p><pre>/* eslint no-shadow:0 */<br>/* eslint react/require-default-props:0 */<br>/* eslint max-len:0 */<br>/* eslint react/destructuring-assignment:0 */<br>/* eslint jsx-a11y/label-has-for:0 */</pre><pre>// the module&#39;s code</pre><p><strong>Note:</strong> there is a downside of using the above technique which is your git history will be missed up now, you can’t `git blame` correctly now because all the code was sorta re-written!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bbc779d29062" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dubizzletechblog/setting-up-prettier-and-eslint-for-js-and-react-apps-bbc779d29062">Setting up Prettier and ESlint for JS and React Apps</a> was originally published in <a href="https://medium.com/dubizzletechblog">dubizzle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tips to enhance/optimize appium automation framework.]]></title>
            <link>https://medium.com/dubizzletechblog/tips-to-enhance-optimize-appium-automation-framework-2f066b10ebe6?source=rss----dafa6a20811c---4</link>
            <guid isPermaLink="false">https://medium.com/p/2f066b10ebe6</guid>
            <category><![CDATA[appium]]></category>
            <category><![CDATA[uiautomation]]></category>
            <category><![CDATA[mobile-automation-testing]]></category>
            <category><![CDATA[android-testing]]></category>
            <category><![CDATA[continuous-testing]]></category>
            <dc:creator><![CDATA[Satheesh Kumar]]></dc:creator>
            <pubDate>Tue, 12 Feb 2019 10:36:05 GMT</pubDate>
            <atom:updated>2019-02-12T10:36:05.471Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OQZ5LXy03Ql2xKIQ914W8A.png" /><figcaption>Mobile Testing Automation Tool</figcaption></figure><p>The most commonly used stack for android automation is [<strong>Appium,Java,TestNG]</strong>. Following are few tips to enhance/optimize this framework.</p><p><strong>A) Setup up Re-Try logic using TestNg: </strong>Retry logics will ensure any random flakiness with the test is handled automatically to ensure seamless running of automation tests in the CD Pipeline.</p><p><strong>Implementation Steps:</strong></p><p>Step1: Create a RetryAnalyzer class which implements IRetryAnalyzer and can monitor the test results and informs TestNg runner to re-run failed test case.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0fyg6G68__GNZVKcLuXarw.png" /></figure><p>Now, your test methods needs to be annotated with “RetryAnalyzer.class”, so that it can send the tests results status to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*apxWE3yjZNbgyzJsgY7rPg.png" /></figure><p><strong>B) Enable Parallelism using Latest Appium Driver: </strong>If you are using in-house testing infrastructure, (Note: Preference will be to use Cloud Testing solutions), you need to enable Parallelism using Latest Appium Driver to keep the implementation strategy simpler.</p><p>In the older versions of appium server, we would require to start separate Appium server instances to enable paralell testing on different real devices/simulators, however the latest version of appium server supports paralellism using single appium server itself.</p><p><strong>Implementation Steps:</strong></p><p><strong>Step 1</strong>: Update your base driver method to accept parameters like udid and systemport.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*943qL0clznCknK24n_rfZw.png" /></figure><p><strong>Step2</strong>: TestNg.xml file needs to be updated, to pass these parameter values to base driver method. [Note: You can specify any port number between8200 to 8299]</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ab6KZpnhQwP7C3OMpw6k8g.png" /></figure><p><strong>C) Implement UI Scrolling using UIAutomator(UiScrollable): </strong>Latest versions of Appium server doesnt have swipe/scrollTo methods. You need to use UIAutomator for this. Different ways of implementing scrolling using UIScrollable class are mentioned below:</p><pre><strong>public void </strong>scrollForwardByResourceID(String resourceID) {<br><br>    <strong>driver</strong>.findElementsByAndroidUIAutomator(<strong>&quot;new UiScrollable(new UiSelector().resourceId(&#39;&quot;</strong>+resourceID+<strong>&quot;&#39;)).scrollForward()&quot;</strong>);<br><br>}<br><br><strong>public void </strong>scrollForwardByResourceID(String resourceID, <strong>int </strong>steps) {<br><br>    <strong>driver</strong>.findElementsByAndroidUIAutomator(<strong>&quot;new UiScrollable(new UiSelector().resourceId(&#39;&quot;</strong>+resourceID+<strong>&quot;&#39;)).scrollForward(&quot;</strong>+steps+<strong>&quot;)&quot;</strong>);<br><br>}<br><br><strong>public void </strong>scrollBackwardByResourceID(String resourceID, <strong>int </strong>steps) {<br><br>    <strong>driver</strong>.findElementsByAndroidUIAutomator(<strong>&quot;new UiScrollable(new UiSelector().resourceId(&#39;&quot;</strong>+resourceID+<strong>&quot;&#39;)).scrollBackward(&quot;</strong>+steps+<strong>&quot;)&quot;</strong>);<br><br>}</pre><p>Other important methods available in UiScrollable class are as follows: <strong>scrollDescriptionIntoView</strong>(String text), <strong>scrollTextIntoView</strong>(String text), scrollIntoView(<strong>UIObject</strong> obj). Complete list of referene can be found <a href="https://developer.android.com/reference/android/support/test/uiautomator/UiScrollable">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2f066b10ebe6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dubizzletechblog/tips-to-enhance-optimize-appium-automation-framework-2f066b10ebe6">Tips to enhance/optimize appium automation framework.</a> was originally published in <a href="https://medium.com/dubizzletechblog">dubizzle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Achieving Continuous Testing in the Continuous Delivery Pipeline for Web-apps]]></title>
            <link>https://medium.com/dubizzletechblog/achieving-continuous-testing-in-the-continuous-delivery-pipeline-for-web-apps-b3181900e4c9?source=rss----dafa6a20811c---4</link>
            <guid isPermaLink="false">https://medium.com/p/b3181900e4c9</guid>
            <category><![CDATA[continuous-testing]]></category>
            <category><![CDATA[test-automation]]></category>
            <category><![CDATA[continuous-delivery]]></category>
            <category><![CDATA[continuous-integration]]></category>
            <category><![CDATA[continuous-deployment]]></category>
            <dc:creator><![CDATA[Satheesh Kumar]]></dc:creator>
            <pubDate>Tue, 06 Nov 2018 09:55:22 GMT</pubDate>
            <atom:updated>2018-12-10T02:32:24.274Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RrBgSZrczhHPVR-LMzFx6Q.jpeg" /><figcaption>Achieving Continuous Testing in the Continuous Delivery Pipeline</figcaption></figure><p>“<strong>Continuous Testing</strong>” is the buzzword at the moment in the QA/Test Automation world. Let’s have a look at how this can be achieved in an efficient way.</p><p><strong>What is </strong>Continous Testing?</p><p>Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate. [ref: wikipedia]</p><p>Following <strong>Test pyramid</strong> explains an efficient strategy to achieve Continuous Testing:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wxjkLya4aVc9nISRLmowHw.png" /><figcaption><strong>TEST PYRAMID</strong></figcaption></figure><p>As shown in the diagram above, code coverage via <strong>Unit Tests </strong>plays a major role on this. Maximizing coverage for your front-end/backend services via unit tests will give confidence at the foundation level itself.</p><p>Next step is to set-up CI pipelines, which will be triggered every time developer commits code to source control repository to enable code compliance with the quality guidelines set and also ensures immediate feedback to Developers.</p><p>Another advantage of CI Integration is as follows: once the developer creates a Pull Request(PR) in the Source control System, the reviewers of the PR can easily get information about the CI Status on the PR itself, which will reduce the feedback time required to approve the PR.</p><p>In the next stage of Continuous Testing model, we need to write integration tests and API end point tests[RestAssured is the commonly used library for validating api response, schema and other tests]. For web frontend, adding integration level cypress tests, which can mock specific pages and validate particular functionality, will also help to improve the coverage.</p><p>For UI Automation, considering its complexity and maintainability, try to pick specific set of test cases from your Master Test Cases, and automate it using Stable frameworks. To ensure Browser and OS compatibility these tests can be integrated with any cloud testing platforms and execute it in your CD Pipeline.</p><p>Adding Performance Tests scripts [Jmeter scripts will do the job] and Security Tests [OWASP ZAP is a good open source tool to use], will make Continuous Testing more powerful in the Continuous Delivery Pipeline.</p><p>Sample Implementation of such pipeline will be as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yCeklCV2FuxVaVtwJsN73w.png" /><figcaption>CI/CD Pipeline with Continuous Testing</figcaption></figure><p><strong>10 Key Points to </strong>consider while implementing <strong>UI/Functional Test Automation Frameworks</strong>:</p><ol><li>Use stable automation frameworks</li><li>Use POM model to organize your tests</li><li>Try to use robust element locators which are less likely to change and are fast.</li><li>Organize your Test Data.</li><li>Use BDD frameworks, which can help to easily convert your user stories/test scenarios into Automation Test Scripts and provides many other benefits.</li><li>Lets not duplicate same tests as part of different test suites.</li><li>Avoid long running tests and split it into separate tests for reliability</li><li>CI/CD Support and other key features like parallel execution, failed tests auto-retry, reporting tests results etc.</li><li>Implement traceability between your Manual Test Cases and Automated Test Cases. [Ex: Test_case_id can be used for mapping]</li><li>Cloud Testing platform support is a major factor and needs to be considered in the starting of the “Test Automation framework” design itself.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b3181900e4c9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dubizzletechblog/achieving-continuous-testing-in-the-continuous-delivery-pipeline-for-web-apps-b3181900e4c9">Achieving Continuous Testing in the Continuous Delivery Pipeline for Web-apps</a> was originally published in <a href="https://medium.com/dubizzletechblog">dubizzle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Debugging “Django migraiton is not running”]]></title>
            <link>https://medium.com/dubizzletechblog/debugging-django-migraiton-is-not-running-f805429aa92a?source=rss----dafa6a20811c---4</link>
            <guid isPermaLink="false">https://medium.com/p/f805429aa92a</guid>
            <category><![CDATA[migration]]></category>
            <category><![CDATA[django]]></category>
            <dc:creator><![CDATA[Gaurav Toshniwal]]></dc:creator>
            <pubDate>Tue, 30 Oct 2018 11:47:04 GMT</pubDate>
            <atom:updated>2018-11-08T10:05:34.840Z</atom:updated>
            <content:encoded><![CDATA[<h3>Debugging “Django migration is not running”</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*grQtRddJZ6pLErj2yeRYPA.png" /></figure><p>I’ll not cover the common case of some migration that was not applied but somehow the fields were created and you should rollback the migration or fake the previous migration and re-run the migration etc.</p><p>A simple migration of mine — which was just creating some fields for an existing model was not executing. This is an unexpected behavior where I had tried all of the above and I started feeling dizzy.</p><p>In the course of debugging I figured out that the allow_migrate method of database router was returning False when it should have returned None.</p><p>Django has a setting <a href="https://docs.djangoproject.com/en/2.1/ref/settings/#database-routers">DATABASE_ROUTERS</a> which will be used to determine which database to use when performing a database query.</p><p>From the docs:</p><blockquote><em>if you want to implement more interesting database allocation behaviors, you can define and install your own database routers.</em></blockquote><p>A database router class implements up to four methods:</p><ul><li>db_for_read(model, **hints)</li><li>db_for_write(model, **hints)</li><li>allow_relation(obj1, obj2, **hints)</li><li>allow_migrate(db, app_label, model_name=None, **hints)</li></ul><p>From the <a href="https://docs.djangoproject.com/en/2.1/topics/db/multi-db/#allow_migrate">documentation</a>:</p><blockquote><strong><em>allow_migrate(db, app_label, model_name=None, **hints)</em></strong></blockquote><blockquote><em>Determine if the migration operation is allowed to run on the database with alias db. Return True if the operation should run, False if it shouldn’t run, or None if the router has no opinion.</em></blockquote><p>In my case, someone had installed a database router that was returning false when it should have returned None:</p><pre><strong>def </strong>allow_migrate(self, db, model):<br>    <em>&quot;&quot;&quot; Make sure the x app only appears on the &#39;x&#39; db &quot;&quot;&quot;<br>    </em>response = False<br>    <strong>if </strong>model._meta.app_label == self.APP_LABEL:<br>        response = db == settings.DB_X_NAME<br>    <strong>return </strong>response</pre><p>I just changed the part response = False to response = None and bam! It worked like a charm.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f805429aa92a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dubizzletechblog/debugging-django-migraiton-is-not-running-f805429aa92a">Debugging “Django migraiton is not running”</a> was originally published in <a href="https://medium.com/dubizzletechblog">dubizzle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Setting up your Development Environment with Docker Compose]]></title>
            <link>https://medium.com/dubizzletechblog/setting-up-your-development-environment-with-docker-compose-4001e39975e?source=rss----dafa6a20811c---4</link>
            <guid isPermaLink="false">https://medium.com/p/4001e39975e</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[docker-compose]]></category>
            <dc:creator><![CDATA[dubizzle Techie]]></dc:creator>
            <pubDate>Mon, 29 Oct 2018 08:19:18 GMT</pubDate>
            <atom:updated>2018-10-29T08:19:18.235Z</atom:updated>
            <content:encoded><![CDATA[<p>Story by Ara Hayrabedian</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*pNl7QPSzNU_iwvHPpKSXrA.jpeg" /></figure><p>At dubizzle we lean towards a service-oriented architecture whenever it makes sense, this means that when you&#39;re developing, you may need to run several server stacks. Some of us use docker-compose, a super-convenient way of coordinating containers and easily spinning up and taking down services on your development machine (and elsewhere). This guide will walk you through some of the thicker parts of setting up such an environment. In this tutorial, we assume some basic familiarity with docker.</p><h3>Before we begin</h3><p>We assume you&#39;re running docker natively on Linux for the most part (hereby referred to as &#39;your_docker_host&#39;), however, docker-compose works just as well on OSX with docker-machine, including phenomenally smooth support for mounting volumes.</p><h3>My First docker-compose Stack</h3><p>As a primer, we will first spin up a group of containers with dependencies between them. Something like this should do nicely:</p><pre>version: &#39;2&#39;<br>services:<br>  webapp:<br>    image: &lt;your_web_app_here&gt;:latest<br>    ports:<br>      - &quot;80:80&quot;<br>    depends_on:<br>      - postgres<br>      - memcached<br><br>  memcached:<br>    image: memcached:latest<br><br>  postgres:<br>    image: postgres:9.3</pre><p>A word of warning before we continue, the above configuration would have no long-term persistence, I won&#39;t get into persistency for the sake of brevity, that&#39;s a blog post on its own, but two ways to do it are: data-only containers and named volumes (docker version &gt; 1.9.0), though I haven&#39;t tried the latter myself yet.</p><p>So the above configuration, when you run docker-compose up, will be nice enough to make sure that both Memcached and Postgres are up and running before bringing up your webapp, older versions of docker-compose required that you explicitly declare every network dependency as well. More recent versions of docker-compose use an internal DNS mechanism that allows every container to talk to every other container unless explicitly configured otherwise (see <a href="https://docs.docker.com/compose/networking/">docker networking documentation</a>). Personally, I have found that if you declare your really hard dependencies in depends_on(say, mainly backing stores), you&#39;ll generally be okay.</p><p>If I were to run something like ping postgresfrom within the webappcontainer, I&#39;d expect a response at this point. The Postgres and Memcached containers are, by default configured to only expose their relevant ports within the docker-compose network. We have also explicitly declared that the host&#39;s port 80 should be mapped to port 80 of our webapp. If I were to curl your_docker_host:80, I&#39;d expect the webapp container to respond.</p><p>Make sure you understand what&#39;s happening above, as we&#39;ll be slowly evolving the above config throughout this tutorial.</p><h3>My Second, more useful, docker-compose Stack</h3><p>I&#39;ll now throw in a few useful docker-compose tricks, while still fairly basic, are leaps ahead of the first stack.</p><pre>version: &#39;2&#39;<br>services:<br>  webapp:<br>    image: &lt;your_web_app_here&gt;:latest<br>    ports:<br>      - &quot;80:80&quot;<br>    depends_on:<br>      - postgres<br>      - memcached<br>    volumes:<br>     - /path/to/my/code/on/my/dev/machine:/path/to/the/code/in/the/container<br>    env_file: env_vars/webapp_env_vars.env<br><br>  memcached:<br>    image: memcached:latest<br><br>  postgres:<br>    image: postgres:9.3</pre><p><strong>env_vars/webapp_env_vars.env:</strong></p><pre>DB_HOST=postgres<br>DB_PORT=5432<br>DB_USER=some_cool_dev<br>DB_PASSWORD=OMG_PLS_K33P_4_S3CRET<br>MEMCACHED_HOST=memcached<br>MEMCACHED_PORT=11211<br>SOME_VARIABLE_MY_WEBAPP_READS=a_value_i_use_to_configure_it</pre><p>You should notice two new things here:</p><p>1) webappdeclares an env_file from which to pick up env vars. This is important because we configure a lot of our applications with environment variables so that the same docker image can run in different environments, taking inspiration from <a href="http://12factor.net/">12-factor apps</a>. The variables in this env file are loaded up into the container when it runs. It also means, in line with best practices, we can avoid committing checking in secrets and passwords into our code and source control.</p><p>2) webappalso declares a volume. In this case, we want to get some development done, so we mount our source code in the place the webapp container expects to serve it, this will vary depending on your setup so I only provide nonsense paths. But what it means is that making a change on your own machine will reflect inside the container, how cool is that!? It&#39;s super cool. Because we don&#39;t want to build a new image every time we want to test a code change.</p><h3>DOUBLE TIME!</h3><p>You guessed it, it&#39;s time to fire up two, independent but dependent stacks. Let&#39;s say microservice webapponeand microservice webapptwo. They will talk to each other, the outside web will also want to talk to them. You can already guess we have an issue here. Port 80. Port 80 is an issue... Solution? Toss in a Load Balancer/reverse proxy to handle port 80 and route requests based on hostnames.</p><pre>version: &#39;2&#39;<br>services:<br>  haproxy:<br>    image: haproxy:1.6.4<br>    volumes:<br>      - ./conf/haproxy:/usr/local/etc/haproxy # assuming you have that folder structure on your own machine<br>    ports:<br>      - &quot;80:80&quot;<br>    depends_on:<br>      - webappone<br>      - webapptwo<br><br>  webappone:<br>    image: &lt;your_web_app_one_here&gt;:latest<br>    depends_on:<br>      - postgresone<br>      - memcachedone<br>    volumes:<br>      - /path/to/my/code/on/my/dev/machine/code_for_one:/path/to/the/code/in/the/container<br>    env_file: env_vars/webapp_one_env_vars.env<br><br>  memcachedone:<br>    image: memcached:latest<br><br>  postgresone:<br>    image: postgres:9.3<br><br>  webapptwo:<br>    image: &lt;your_web_app_two_here&gt;:latest<br>    depends_on:<br>      - postgrestwo<br>      - memcachedtwo<br>    volumes:<br>      - /path/to/my/code/on/my/dev/machine/code_for_two:/path/to/the/code/in/the/container<br>    env_file: env_vars/webapp_two_env_vars.env<br><br>  memcachedtwo:<br>    image: memcached:latest<br><br>  postgrestwo:<br>    image: postgres:9.3</pre><p><strong>env_vars/webapp_one_env_vars.env:</strong></p><pre>DB_HOST=postgresone<br>DB_PORT=5432<br>DB_USER=THE_BEST_ALPHA<br>DB_PASSWORD=OMG_PLS_K33P_4_S3CRET<br>MEMCACHED_HOST=memcachedone<br>MEMCACHED_PORT=11211<br>SOME_VARIABLE_MY_WEBAPP_READS=a_value_i_use_to_configure_it</pre><p><strong>env_vars/webapp_two_env_vars.env:</strong></p><pre>DB_HOST=postgrestwo<br>DB_PORT=5432<br>DB_USER=THE_BEST_BETA<br>DB_PASSWORD=OMG_PLS_K33P_4N0TH3R_S3CR3T<br>MEMCACHED_HOST=memcachedtwo<br>MEMCACHED_PORT=11211<br>SOME_VARIABLE_MY_WEBAPP_READS=a_value_i_use_to_configure_it</pre><p><strong>conf/haproxy/haproxy.cfg:</strong></p><pre># This is a DEVELOPMENT haproxy configuration to be run inside a docker<br># container, please refrain from thinking about using it in production.<br><br>global<br>        user haproxy<br>        group haproxy<br><br>defaults<br>        log     global<br>        mode    http<br>        option  httplog<br>        balance roundrobin<br>        contimeout 5000<br>        clitimeout 50000<br>        srvtimeout 50000<br><br>frontend http<br>        bind 0.0.0.0:80<br>        default_backend host_one # we feel like putting priority on webappone<br>        acl host_one hdr(host) -i one.something.local<br>        acl host_two hdr(host) -i two.something.local<br>        use_backend one if host_one<br>        use_backend two if host_two<br><br># Reminder, docker-compose provides the hostnames via internal DNS, we&#39;ll explicitly<br># specify the ports though because we feel like it. Look at how beautiful the<br># uniformity is though. Every webapp exposes port 80. #NO #CONFLICT #NO #WAR<br>backend one<br>        server webapp_one webappone:80<br><br>backend two<br>        server webapp_two webapptwo:80</pre><p>To explain: we have two webapps, each listening (in its own container) on port 80, we now have the ENTIRE STACK being represented by HAProxy listening on port 80 and routing requests based on headers. Why HAProxy? because. you can toss in whatever reverse proxy you like, nginx or otherwise.</p><p>So a request lifecycle looks roughly like:</p><p>Browser hits up your_docker_host (because you put one.something.local and two.something.local in your hosts file pointing there (if dev)):</p><blockquote><em>hey, I&#39;d like two.something.local, bro, please serve.</em></blockquote><p>haproxy:</p><blockquote><em>oh wow, this guy wants two.something.local, I should totally hit up webapptwo:80 on my internal network to see what he/she says.</em></blockquote><p>webapptwo:</p><blockquote><em>request has come in with host two.something.local, That&#39;s actually me! I should totes mcgotes serve that, what path did they want again, OH! Here it is!, RETURNIFY</em></blockquote><p>Traverse in reverse and you have a response to the browser. Done.</p><h3>Bonus section: local, self-signed SSL.</h3><p>So your website is mainly in SSL and you&#39;ve ALWAYS had difficulty duplicating it locally. Fear no more. You can now serve your stack locally, with SSL to boot.</p><p>HAProxy supports SSL termination since 1.5.X, <a href="https://www.digitalocean.com/community/tutorials/how-to-implement-ssl-termination-with-haproxy-on-ubuntu-14-04">see here</a>. You&#39;ll notice in the previous config we mounted an haproxy config. We&#39;ll reuse this mount (not the cleanest config, but we&#39;re just demonstrating here) in order to also serve HTTPS traffic. We need to do 2 things:</p><ol><li>Start the haproxy container listening on port 443 (if you&#39;ve gotten this far, you should know how to allow haproxy to listen on port 443)</li><li>Push our self-signed certificate into our haproxy container (as already stated, we&#39;ve already mounted a volume, so we only need to add our .crt file into that folder, next to the haproxy config)</li><li>Tell our haproxy.cfg to start using this self-signed the certificate for termination, add:bind 0.0.0.0:443 ssl crt /usr/local/etc/haproxy/self.pem to your frontend httpblock.</li></ol><p>and there you have it. Two webapps served via https with unified ports under a docker stack. Note that your webapps themselves will still receive regular HTTP traffic. Hence why it&#39;s called SSL termination</p><h3>Lessons learned the hard way</h3><h3>docker-compose is quirky:</h3><p>you may need to take things into your own hands and completely rmcontainers if:</p><ol><li>you change env vars</li><li>you change ports</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4001e39975e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dubizzletechblog/setting-up-your-development-environment-with-docker-compose-4001e39975e">Setting up your Development Environment with Docker Compose</a> was originally published in <a href="https://medium.com/dubizzletechblog">dubizzle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to javascript iterables, iterators, and generators]]></title>
            <link>https://medium.com/dubizzletechblog/introduction-to-javascript-iterables-iterators-and-generators-a26be413dfd9?source=rss----dafa6a20811c---4</link>
            <guid isPermaLink="false">https://medium.com/p/a26be413dfd9</guid>
            <category><![CDATA[es6]]></category>
            <category><![CDATA[es7]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Mahmoud Felfel]]></dc:creator>
            <pubDate>Mon, 29 Oct 2018 08:13:42 GMT</pubDate>
            <atom:updated>2018-10-29T08:13:42.491Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HXE6j32Vsq1wPv7ea3pMhA.jpeg" /></figure><iframe src="https://cdn.embedly.com/widgets/media.html?url=https%3A%2F%2Fplay.ht%2Farticles%2Fa26be413dfd9&amp;src=https%3A%2F%2Fplay.ht%2Fembed%2F%3Farticle_url%3Dhttps%3A%2F%2Fmedium.com%2F_p%2Fintroduction-to-javascript-iterables-iterators-and-generators-a26be413dfd9&amp;type=text%2Fhtml&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;schema=play" width="700" height="185" frameborder="0" scrolling="no"><a href="https://medium.com/media/9c427db303d3775a965f90d161575400/href">https://medium.com/media/9c427db303d3775a965f90d161575400/href</a></iframe><h3>Iterators and Iterables:</h3><p>ES6 (aka ES2015) has introduced two new concepts in javascript: <strong><em>iterables</em></strong>, and <strong><em>iterators</em></strong>.<br>This is a brief on what iterables, iterators, and generators are and how to make use of them.</p><h4>What are iterables?</h4><p>In general, it is a data structure that allows its data to be consumed. It does so by implementing a method whose key is <strong><em>Symbol.iterator</em></strong> which returns an iterator.</p><h4>Why do we need iterables?</h4><p>Pre-ES6 in Javascript if you have some data source, depending on how you structure your data, you need to consume it in a different (sometimes hacky!) way. <br>Iterating and consuming the data of an Array is different from a key/value object and as ES6 is adding new data sources like <strong><em>Map</em></strong> and <strong><em>Set</em></strong>, so it doesn’t make sense and is not practical to implement different ways of consuming all those different data sources.<br>So the language must have an interface that all the data sources can implement, so all the consumers have the same way to consume data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/687/1*aw0ODP5vE6ajNe2A3xH8ig.png" /><figcaption>image source <a href="http://exploringjs.com/es6/ch_iteration.html">here</a>, not exactly but inspired from it</figcaption></figure><h4>How to implement an iterable?</h4><p>To call any object <strong><em>iterable</em></strong>, it needs to implement the <strong><em>iterable</em></strong> interface, which as <a href="http://www.ecma-international.org/ecma-262/6.0/index.html#sec-iterable-interface">per ecmascript specification</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HG_lhsM5lzLm2o5oprsutw.png" /></figure><p>So for an object to be an <strong><em>iterable</em></strong>, it needs a method that returns an <strong><em>iterator</em></strong>, and this methods property is<strong><em> @@iterator (Symbol.iterator</em></strong>)<strong><em>.</em></strong></p><h4>What is an iterator?</h4><p>Iterator is a way to pull data from a data structure on one-at-a-time fashion, it needs to implement the <strong><em>iterator</em></strong> interface.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pOTL_06NCO6WTUzOelrPPg.png" /><figcaption><a href="http://www.ecma-international.org/ecma-262/6.0/index.html#sec-iterator-interface">source</a></figcaption></figure><p><strong>Hint: </strong>the <strong><em>iterator</em></strong> interface has another method called <strong><em>return()</em></strong>, which is being called when the iteration reaches the last value, or stopped manually by calling it explicitly or for example <strong><em>break;</em></strong> a for loop.</p><p>And the <strong><em>iteratorResult</em></strong> that returns from the iterator <strong><em>next()</em></strong> method needs to follow this interface:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZMD-pcNxJwQhM-LTiU7KjQ.png" /><figcaption><a href="http://www.ecma-international.org/ecma-262/6.0/index.html#sec-iteratorresult-interface">source</a></figcaption></figure><p>So each time we call <strong><em>next()</em></strong> on an <strong><em>iterator</em></strong> we should get<strong><em> {value: the current iteration value, done: false/true}</em></strong></p><p>As we see in the above first image, one of the ways to consume iterables is <strong><em>for..of </em></strong>loop, so lets use it on one of the language built-in iterables, an <strong><em>array</em></strong>.</p><pre>const arr = [1,4,2];</pre><pre>for(v of arr){<br>    console.log(arr); // 1 4 2<br>}</pre><p>But can we call<strong><em> .next</em></strong> to pull values from an array:</p><pre>const arr = [1,4,2];<br>arr.next() // Error</pre><p><strong>Why?</strong> because it is an <strong><em>iterable</em></strong> not an <strong><em>iterator</em></strong>, and to be able to use <strong><em>.next</em></strong> we need to call the <strong><em>iterator</em></strong> method this iterable’s <strong><em>Symbol.iterator</em></strong> returns:</p><pre>const arr = [1,4,2];<br>const iter = arr[Symbol.iterator]();</pre><pre>iter.next(); // {value: 1, done: false}<br>iter.next(); // {value: 4, done: false}<br>iter.next(); // {value: 2, done: false}<br>iter.next(); // {value: undefined, done: true}</pre><p><strong><em>Symbol.iterator</em></strong> represents the property on any object which when we call, the language looks to find a method that will construct an iterator instance for consuming that object’s values. Many objects come with a default one defined.</p><h4>Custom Iterators</h4><p>In addition to the standard built-in iterators, you can make your own. <br>All it takes to make them work with ES6&#39;s consumption facilities (e.g., the <strong><em>for..of</em></strong> loop and the … operator) is to adhere to the proper interface(s).</p><p>Let’s try constructing an <strong><em>iterator</em></strong> that produces the infinite series of numbers in the Fibonacci sequence:</p><pre>var Fib = {<br> [Symbol.iterator]() {<br>     var n1 = 1, n2 = 1;</pre><pre>     return {</pre><pre>         next() {<br>             var current = n2;<br>             n2 = n1;<br>             n1 = n1 + current;<br>             return { value: current, done: false };<br>         },</pre><pre>         return(v) {<br>             console.log(‘Done’);<br>             return { value: v, done: true };<br>         }<br>     };<br>  }<br>};</pre><pre>for (var v of Fib) {<br>    console.log( v );<br>    if (v &gt; 50) break;<br>}<br>// 1 1 2 3 5 8 13 21 34 55<br>// Done</pre><p>Here we can see the benefit of the optional iterator’s <strong><em>return(..)</em></strong> method, it is actually so useful as it works as a built-in <strong><em>stopIteration</em></strong> method, it is defined as sending a signal to an iterator that the consuming code is complete and will not be pulling any more values from it, so it is being called anyway once we finished iterating, it is the one returning {done:true}, but we can call it manually as well.</p><h4>Consuming Iterators:</h4><p><strong>1-<em>for .. of</em> loop:</strong><br>We can use the new es6 <strong><em>for..of</em></strong> loop to consume iterators, but it must be an <strong><em>iterable</em></strong> as well, we easily can make any iterator an iterable by providing it with the <strong><em>Symbol.iterator</em></strong> property that returns the <strong><em>iterator</em></strong> itself:</p><pre>var it = {<br> // make the `it` iterator an iterable<br> [Symbol.iterator]() { return this; },</pre><pre>next() { .. },<br> ..<br>};</pre><p><strong>2-</strong> <strong><em>… spread/rest</em></strong> <strong>operator</strong>:<br>We can use es6 spread operator to consume iterators. In the above Fibonacci snippet, we can do :</p><pre>[…Fib] // [1, 1, 2, 3, 5, 8, 13, 21, 34]</pre><p>We can use iterators with array destructuring as well:</p><pre>[a, b] = […Fib] // a = 1, b = 1<br>[a,b, …c] = […Fib] // a=1, b=1, c = [3, 5, 8, 13, 21, 34]</pre><h3>Generators:</h3><p>Pre ES6 there was no way in javascript to pause executing a function at some point, then come later to the same point and resume the function execution.<br>But with generator functions we can do that, because generator functions when executed return an <strong><em>iterator</em></strong>, and instead of stopping at each value/element and return <strong><em>iteratorResult</em></strong> with each <strong><em>.next()</em></strong>, generator functions stop at each <strong><em>yield</em></strong> keyword and return the value next to it if any.</p><p>To define a generator function you need to add * between the keyword <strong><em>function</em></strong> and the function’s name.</p><pre>function *foo() {<br>    yield 1<br>    yield 2<br>    yield 3<br>}</pre><pre>const iter = foo();<br>iter.next(); // {value: 1, done: false}<br>iter.next(); // {value: 2, done: false}<br>iter.next(); // {value: 3, done: false}</pre><p>All iterators created by generators are also iterables, as generators assign the <strong><em>Symbol.iterator</em></strong> property by default, so we can use <strong><em>for..of</em></strong> with it:</p><pre>function *foo() {<br>    yield 1<br>    yield 2<br>    yield 3<br>}</pre><pre>const iter = foo();<br>for(v of iter){<br> console.log(v) // 1 2 3<br>}</pre><p>You can write generator functions in different ways:<br><strong>function declaration:</strong></p><pre>function *genFunc() { ··· }<br>const genObj = genFunc();</pre><p><strong>function expression:</strong></p><pre>const genFunc = function *() { ··· };<br>const genObj = genFunc();</pre><p><strong>ES6 method shorthand in object literals:</strong></p><pre> const obj = {<br>     *generatorMethod() {···}<br> };<br> const genObj = obj.generatorMethod();</pre><h4>yield delegation (aka recursive yield):</h4><p>If you check the first image, you will notice we added also <strong><em>yield*</em></strong> as one of the <strong><em>iterable</em></strong> interface consumers, this pattern is used when you want to pass a generator (or any iterable) to one of your generator’s <strong><em>yield</em></strong> keywords:</p><pre>function *foo() {<br>    yield *[1,2,3];<br>}</pre><pre>const iter = foo();</pre><pre>iter.next(); // {value: 1, done: false}<br>iter.next(); // {value: 2, done: false}<br>iter.next(); // {value: 3, done: false}<br>iter.next(); // {value: undefined, done: true}</pre><p>So as we see in this example, instead of returning the whole array, it is returning an array iterator which we can call <strong><em>next()</em></strong> to pull each value of it.<br>And we can pass another generator function as well:</p><pre>function *foo() {<br>    yield 1;<br>    yield 2;<br>    yield 3;<br>}</pre><pre>function *bar() {<br>    yield *foo(); <br>}</pre><pre>const iter = bar();<br>iter.next(); // {value: 1, done: false}<br>iter.next(); // {value: 2, done: false}<br>iter.next(); // {value: 3, done: false}<br>iter.next(); // {value: undefined, done: true}</pre><p>Now Let’s visit the Fibonacci example again but implement it with generators this time:</p><pre>var FibGen = {<br>    [Symbol.iterator]() {<br>        var n1 = 1, n2 = 1;<br>        return function* fib () {<br>            var current = n2;<br>            n2 = n1;<br>            n1 = n1 + current;<br>            yield current;<br>            yield* fib();<br>         }();<br>     }<br>}</pre><pre>for (var v of FibGen) {<br>    console.log( v );<br>    if(v &gt; 50) break;<br>}</pre><p>That was a quick overview of iterables, iterators, and generators in javascript, if you want to read more, here’re some resources:</p><p><a href="https://leanpub.com/understandinges6/read#leanpub-auto-iterators-and-generators">Iterators and Generators — Understanding ES6</a><br><a href="https://github.com/getify/You-Dont-Know-JS/blob/master/es6%20&amp;%20beyond/ch3.md">ES6-Organization — YouDon’tKnowJS- ES6 and Beyond</a><br><a href="http://exploringjs.com/es6/ch_iteration.html">Iterables and Iterators — ExploringJS</a><br><a href="http://exploringjs.com/es6/ch_generators.html">Generators — ExploringJS</a><br><a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Iterators_and_Generators">Iterators and Generators — MDN</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a26be413dfd9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dubizzletechblog/introduction-to-javascript-iterables-iterators-and-generators-a26be413dfd9">Introduction to javascript iterables, iterators, and generators</a> was originally published in <a href="https://medium.com/dubizzletechblog">dubizzle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Managing environment variables in Nodejs and Modern JS apps]]></title>
            <link>https://medium.com/dubizzletechblog/managing-environment-variables-in-nodejs-and-modern-js-apps-608003f4686c?source=rss----dafa6a20811c---4</link>
            <guid isPermaLink="false">https://medium.com/p/608003f4686c</guid>
            <category><![CDATA[frontend]]></category>
            <category><![CDATA[development]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[front-end-development]]></category>
            <dc:creator><![CDATA[Mahmoud Felfel]]></dc:creator>
            <pubDate>Mon, 29 Oct 2018 07:54:50 GMT</pubDate>
            <atom:updated>2020-09-12T08:01:48.564Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bNXslQsg8CYCyD5-1MkK5A.jpeg" /><figcaption>secrets</figcaption></figure><p>Techniques to manage your secrets and environment variables through your nodejs and client javascript app.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplay.ht%2Fembed%2F%3Farticle_url%3Dhttps%3A%2F%2Fmedium.com%2F_p%2Fmanaging-environment-variables-in-nodejs-and-modern-js-apps-608003f4686c&amp;display_name=Play&amp;url=https%3A%2F%2Fplay.ht%2Farticles%2F608003f4686c&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=play" width="700" height="185" frameborder="0" scrolling="no"><a href="https://medium.com/media/f3fda018898c99f05aad3dc06d9befac/href">https://medium.com/media/f3fda018898c99f05aad3dc06d9befac/href</a></iframe><p>Every app needs environment variables to define configs and secrets e.g. API_BASE_URL, SOME_SECRET, ...etc.</p><p>You can just add them as a JSON or a constants JS file in your app but you need to have these as global configs where you can use from any file without importing, and they might have some secrets and API keys which you shouldn’t add to your source control system.</p><h3>How to define an environment variable</h3><p>You can simply define environment variables using any cli like this:</p><p>Available to the current shell session only.</p><pre>MY_SECRET_KEY=iamasecret</pre><p>This will be available to the current shell and all the processes those start from it.</p><pre>export MY_SECRET_KEY=iamasecret</pre><p>And To get it added permanently to all the future bash sessions, add the variable to your ~/.bashrc ( or ~/.zshrc if you are using the fancy <a href="https://ohmyz.sh/">OH-MY-ZH</a>) then source ~/.bashrc.</p><p>In Nodejs, you can access the global variable process.env to access all the available environment variables to the current node process.</p><pre>initSecretService(process.env.MY_SECRET_KEY)</pre><p>Note: you will notice some npm scripts being written like this:</p><pre>scripts: {<br>    &#39;start&#39;: &#39;cross-env PORT=8080 node server.js&#39;<br>}</pre><p><strong><em>Quick Tip:</em></strong><em><br>cross-env here is not required but it is a good practice to make sure it works on windows. find cross-env package here </em><a href="https://www.npmjs.com/package/cross-env"><em>https://www.npmjs.com/package/cross-env</em></a></p><p>That will set the variable PORT to that value 8080 only when running this server session, so if the next time you run it without PORT it will be either undefined or will fallback to the default value.</p><p>In many node apps they do it like this when starting the server, which will fallback to a default value if PORT is not defined:</p><pre>const app = require(&#39;http&#39;).createServer();<br>const PORT = process.env.PORT || 3000;</pre><pre>app.listen(PORT, () =&gt; {<br>  console.log(`Server is listening on port ${PORT}`);<br>});</pre><p>Passing environment variables individually like the above is not practical as your app might need more variables to add over time.</p><p>So here is how to do it in Nodejs in a more scalable way:</p><h3>.env file</h3><p>In Nodejs you can list your env variables in a .env file then use one of these methods to load the variables and make them available to be used in your app:</p><h3>1. <a href="https://www.npmjs.com/package/dotenv">dotenv</a></h3><p>The most common way of doing this is to use the dotenv npm package to load the .env variables when your app is starting.</p><p>first, install it</p><pre>npm install dotenv --save</pre><p>then add this to your app’s entry file</p><pre>require(&#39;dotenv&#39;).load(); // alias of .config()</pre><p>That will read your .env file, parse the contents, assign it to process.env, and return an Object with a parsed key containing the loaded content or an error key if it failed.</p><p><strong>Few tips:</strong></p><ol><li>dotenv doesn’t by default overwrite any existing env variables, but if you really need to do it (which is not recommended), well, you can:</li></ol><pre>// set any process.env directly  <br>process.env.MY_TOP_SECRET = &quot;123456789&quot;;    </pre><pre>// or use the parsed key to force all the process.env keys to be set // to the .env ones<br>const envConfig = dotenv.load().parsed;  <br>for (let k in envConfig) {    <br>    process.env[k] = envConfig[k]  <br>}</pre><p>2. If you’d rather not have require(&quot;dotenv&quot;).load() in your code, you can use <a href="https://nodejs.org/api/cli.html#cli_r_require_module">the </a><a href="https://nodejs.org/api/cli.html#cli_r_require_module">--requirecommand line option</a> to preload dotenv when you need it.</p><pre>&gt; node --require dotenv/config server.js</pre><p>3. Don’t forget to add the .env file to your .gitignore to be sure it won&#39;t get pushed to your source control system (i.e. github) with all your secrets by mistake 😱.</p><p>4. Usually hosting services like AWS Beanstalk, Heroku, or <a href="http://now.sh/">Now.sh</a> provide their own mechanisms (which you need to configure) to load your env variables in your environment when you deploy your app, so mostly in these cases, you don’t need the .env file on production. And don’t worry if you don’t provide the .env file, dotenv will fail silently.</p><p>5. To make .env file more readable in VS code, you can use <a href="https://marketplace.visualstudio.com/items?itemName=mikestead.dotenv">this plugin</a>.</p><h3>2. <a href="https://github.com/tkalfigo/dotenvenc">dotenvenc</a>:</h3><p>Another technique used along with dotenv and is particularly useful if you want to share secrets between different team members (you shouldn&#39;t push your .env file to your source control system, again).</p><p>In this approach you encrypt the .env file locally then push the generated .env.enc file to your source control system, and you share only the encryption key with your team members.</p><pre>&gt; npm i -g dotenvenc<br>&gt; dotenvenc myPassword</pre><p>Then in your code:</p><pre>// will only regenerate `.env`; it will not create any environment variables from it</pre><pre>require(&#39;dotenvenc&#39;)(&#39;myPassword&#39;); </pre><pre>// this will read the generated `.env` and populate process.env.* accordingly</pre><pre>require(&#39;dotenv&#39;).config();</pre><p>For more about this approach check this article by John Resig <a href="https://johnresig.com/blog/keeping-passwords-in-source-control/">https://johnresig.com/blog/keeping-passwords-in-source-control/</a></p><h3>3. <a href="https://www.npmjs.com/package/env-cmd">env-cmd</a>:</h3><p>Instead of using dotenv you can use this package in your npm script to load your environment variables on the fly from the .env file:</p><pre>{<br>  &quot;scripts&quot;: {<br>    &quot;test&quot;: &quot;env-cmd ./test/.env mocha -R spec&quot;<br>  }<br>}</pre><p>It is particularly useful if you have multiple .env files for different purposes, e.g. one for testing, and another one for production.</p><h3>Using webpack:</h3><p>Setting environment variables in your shell doesn’t make them available during compile time.</p><p>To achieve that, You can use webpack to load environment variables in your project using the webpackDefinePlugin:</p><pre>plugins: [<br>    new webpack.DefinePlugin({<br>      &#39;process.env&#39;: {<br>        NODE_ENV: JSON.stringify(process.env.NODE_ENV),<br>      },<br>    }),<br>  ],</pre><p>And If you want to add the same .env variables to webpack config, instead of adding them one by one, you can use <a href="https://www.npmjs.com/package/dotenv-webpack">dotenv-webpack</a> plugin which wraps both definePlugin and dotenv and while building your code, it checks only for the variables used in your code and makes them available to your build.</p><pre>// webpack.config.js<br>const Dotenv = require(&#39;dotenv-webpack&#39;);<br> <br>module.exports = {<br>  ...<br>  plugins: [<br>    new Dotenv()<br>  ]<br>  ...<br>};</pre><p>That was a quick overview of how to add and use environment variables in your nodejs and javascript projects.<br>Thanks for reading, and please share your techniques of handling env variables and secrets (if any).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=608003f4686c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dubizzletechblog/managing-environment-variables-in-nodejs-and-modern-js-apps-608003f4686c">Managing environment variables in Nodejs and Modern JS apps</a> was originally published in <a href="https://medium.com/dubizzletechblog">dubizzle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Welcome Note]]></title>
            <link>https://medium.com/dubizzletechblog/welcome-note-616cc0a92d13?source=rss----dafa6a20811c---4</link>
            <guid isPermaLink="false">https://medium.com/p/616cc0a92d13</guid>
            <category><![CDATA[dubizzle]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[tech-blog]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[tech]]></category>
            <dc:creator><![CDATA[dubizzle Techie]]></dc:creator>
            <pubDate>Mon, 29 Oct 2018 07:43:38 GMT</pubDate>
            <atom:updated>2018-10-29T07:43:38.495Z</atom:updated>
            <content:encoded><![CDATA[<h4>By Sebastian Ritter, Director of Product Engineering, dubizzle</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/902/1*Wop8djaiBlIHbpX2EjPy3w.jpeg" /></figure><p>Hello fellow Techie!</p><p>Welcome to our new dubizzle tech blog. The focus of this blog is purely on technology and aims to share our perspectives, decisions and challenges around the software we build at dubizzle.</p><p>With over 277 million page-views a month across all of our channels and an ever increasing user base, ensuring that we continue to scale to meet our markets’ needs is both an exciting and complex task.</p><p>Though we may not yet operate at the scale of the Googles and Facebooks of the world, we have gained a lot of practical insights when it comes to developing and maintaining our services.</p><p>Whether it is building service oriented architectures, automating infrastructure or taking our mobile experience to new heights, we intend to share our learnings in the hopes of creating a stronger MENA engineering community, attract like minded individuals and help others on their journey.</p><p>We encourage all engineers at dubizzle, from juniors to team-leads, front-ends to mobile-devs, to share their thoughts, victories and even failures; So whether you’re a PHP enthusiast, Pythonista, front-end ninja or mobile guru, we look forward to your comments and hope you’ll join us in shaping our local engineering community.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=616cc0a92d13" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dubizzletechblog/welcome-note-616cc0a92d13">Welcome Note</a> was originally published in <a href="https://medium.com/dubizzletechblog">dubizzle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Create your own AWS RabbitMQ Cluster, the dubizzle way]]></title>
            <link>https://medium.com/dubizzletechblog/create-your-own-aws-rabbitmq-cluster-the-dubizzle-way-13d53559fd54?source=rss----dafa6a20811c---4</link>
            <guid isPermaLink="false">https://medium.com/p/13d53559fd54</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[rabbitmq]]></category>
            <category><![CDATA[olx]]></category>
            <category><![CDATA[amazon]]></category>
            <category><![CDATA[amazon-web-services]]></category>
            <dc:creator><![CDATA[Ibrahim AbouAlnaga]]></dc:creator>
            <pubDate>Tue, 23 Oct 2018 09:07:41 GMT</pubDate>
            <atom:updated>2019-04-28T07:02:01.620Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*tFysd2elew4DqRMmGdzfuQ.jpeg" /></figure><p>A group of software engineers and SREs have been recently assigned the task of creating a stable RabbitMQ cluster running on production in dubizzle. We heavily rely on celery, and with RabbitMQ-based-docker-running-images failing almost every week — especially over weekends — we decided to take some time, do some research and come up with a healthy robust running cluster to support the massive number of tasks held by celery.</p><h3>RabbitMQ</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/540/0*qTRInUrENN7fdeTJ.png" /></figure><p>A brief description of RabbitMQ, as per pivotal’s definition of the system — <a href="https://en.wikipedia.org/wiki/RabbitMQ">pivotal is the company that owns RabbitMQ now</a> – <a href="https://www.rabbitmq.com/">RabbitMQ</a> is the most widely deployed open source message broker. It has been used by most of the internet giants for different purposes.</p><p>At dubizzle, and in our journey into microservices, we have chosen RabbitMQ to be our communication bus over other alternatives like <a href="https://kafka.apache.org/">Kafka</a>/<a href="https://aws.amazon.com/kinesis/streams/">Kinesis</a>, <a href="https://vernemq.com/">VerneMQ</a> and even considering at sometime writing our own.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/334/0*AXAb_LI9FAkkLiIB.png" /></figure><p>Because of the amazing flexibility RabbitMQ provides and that once deployed you should never consider revisiting except for some minor maintenance tasks to be executed. It also comes with a lot of out of the box features that would facilitate working with it on a daily basis like the nice Admin interface and a RESTful API. RabbitMQ also support lots of programming language with <a href="https://www.rabbitmq.com/devtools.html">wide variety of protocols, clients and libraries.</a></p><h3>Our infrastructure</h3><p>We rely on AWS for almost 99% of our services either internal or exposed. Other than some third party integrations, We are considered one of the largest accounts in the region working with Amazon.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/334/0*Q1LezAZTD_2psaq5.png" /></figure><p>We used to have a docker based RabbitMQ cluster using some open source images from docker hub, which for some reasons failed to serve our needs. Facing lots of down times every now and then. That affected us badly. So, instead of relying on docker images, we decided to go on bare EC2 machines implementing the RMQ cluster from scratch. After a lot of investigation in the features that comes with RabbitMQ plugins, we decided to implement our own cluster using EC2 launch configurations — using <a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html">CloudConfig</a> -, Auto-Scaling groups and RMQ <a href="https://github.com/rabbitmq/rabbitmq-autocluster">Auto-clustering</a> plugin which comes with AWS support.</p><p>So, by end of this blog post, we should have a working RabbitMQ Cluster of 3 nodes on AWS with fault tolerance and auto clustering of new nodes if any of the old nodes fail.</p><h3>Let’s get our hands dirty</h3><h3>Code</h3><p>On the following link <a href="https://goo.gl/Yjn9wb">GitHub Gist</a>, we will keep the code that we will be using as a launch configuration which will then be used for the Auto Scaling group — later to be addressed in the post — on AWS.</p><h3>Some notes regarding the Gist</h3><ul><li>The Gist has comments that explains almost every step along the way, so please feel free to leave a comment asking about any part that might need more explanation.</li><li>You should change the password values on lines <a href="https://gist.github.com/bingorabbit/8afe203d486a2893d66bf690c3f796cd#file-configuration-yaml-L71">71</a> and <a href="https://gist.github.com/bingorabbit/8afe203d486a2893d66bf690c3f796cd#file-configuration-yaml-L76">76</a> to something secure.</li></ul><h3>Steps</h3><ul><li>On your AWS Account, go to EC2, from the left sidebar menu, click on Launch Configuration then click on <strong>Create launch configuration</strong>, the following screen should appear, from which choose the AMI that might suit you, for this tutorial, we will be working with Ubuntu Server and the latest LTS supported version at the time of writing this blog post it’s 16.04. So, beside the distro entry, click <strong>Select</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*UJUJJiUm76b3VyUc.png" /></figure><ul><li>On the next screen, you will have to choose the instance type based on the load RabbitMQ is expected to handle. We have tested multiple versions with this, but finally we chose <strong>t2.medium</strong> considering load will be moderate and consumers should always be running. For the sake of this tutorial, we will keep the default chosen option which is t2.micro. Then click <strong>Next: Configure details </strong>in the bottom.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Q3oT5ImC69Yq6sw4.png" /></figure><ul><li>To continue to the next step, we need to setup a new IAM role for the cluster, which will allow communication between RabbitMQ cluster nodes. For this tutorial, we will be calling it <strong>rabbitmq-cluster-iam-role</strong>, with the following policy:</li></ul><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: [<br>                &quot;autoscaling:DescribeAutoScalingGroups&quot;,<br>                &quot;autoscaling:DescribeAutoScalingInstances&quot;,<br>                &quot;ec2:DescribeInstances&quot;<br>            ],<br>            &quot;Resource&quot;: [<br>                &quot;*&quot;<br>            ]<br>        }<br>    ]<br>}</pre><ul><li>Launch configuration details:</li></ul><p>** Assign the name of the launch configuration in the next screen, for now we will call it <strong>RabbitMQ Cluster</strong>.</p><p>** In the<strong> IAM role</strong>, we will choose the IAM role — we created from the previous step <strong>rabbitmq-cluster-iam-role</strong> — from the select box.</p><p>** You can also allow <strong>CloudWatch</strong> monitoring on that form.</p><p>** Click on <strong>Advanced Details</strong> and a new form will appear on the same page in which you will paste the script we linked for earlier in the <strong>User data</strong> text box.</p><p>** In the IP Address Type, we will choose not to have public IP addresses for the cluster instances as we will be adding a load balancer for that cluster which can be used to access the cluster either publicly or privately.</p><p>** Click <strong>Next: Add Storage</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gQF8xQtJqq1IaNOe.png" /></figure><ul><li>RabbitMQ writes messages to disk if they were not consumed after some period of time from memory, so unless you change the configuration to keep messages in memory or have consumers always running, you will need somehow a good amount of storage for the nodes. Depending on what you would favor, choose the amount that you feel would be appropriate. We will be choosing 10 GB. Then click on <strong>Next: Configure Security Group</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HoE9b6buMZoFeAHR.png" /></figure><p><strong><em>You will experience failures in the cluster if messages filled up your disk space that’s why it’s always good to make sure your consumers are healthy and consuming messages in a steady manner.</em></strong></p><ul><li>For this step, we will be creating a security group which will allow communication between the RabbitMQ cluster nodes for clustering and sync. And also for us to connect to any node separately or through the load balancer via console or UI interface. As per <a href="https://www.rabbitmq.com/ec2.html">RabbitMQ’s documentation</a> we will need to allow the following ports:</li></ul><p>** <strong>22</strong>: for ssh connections</p><p>** <strong>4369</strong>: <a href="http://erlang.org/doc/man/epmd.html">epmd</a>, a peer discovery service used by RabbitMQ nodes and CLI tools</p><p>** <strong>5672</strong>: used by AMQP 0–9–1 and 1.0 clients without and with TLS</p><p>** <strong>15672</strong>: <a href="https://www.rabbitmq.com/management.html">HTTP API</a> clients and <a href="https://www.rabbitmq.com/management-cli.html">rabbitmqadmin</a> (only if the <a href="https://www.rabbitmq.com/management.html">management plugin</a> is enabled)</p><p>** <strong>25672</strong>: used by Erlang distribution for inter-node and CLI tools communication and is allocated from a dynamic range (limited to a single port by default, computed as AMQP port + 20000).</p><p>Note that that this step is based on your AWS configuration, but mainly you will have to enable communication from only subnets that should access these nodes through the mentioned ports. That’s why how you create security groups is not covered in this tutorial.<br>Click <strong>Review</strong></p><ul><li>Review your launch configuration details and if everything looks fine, just press <strong>Create launch configuration</strong> from the bottom of the page.</li><li>A new popup will now be shown from which you can choose which key pair you will need to use to connect to these instances. Either create a new key pair or use a pre-created one.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*I8fqDpduBYuOPiwn.png" /></figure><ul><li>Make sure you have saved it somewhere secure and then press <strong>Create launch configuration.</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0t4H2FeHIYmtu94e.png" /></figure><ul><li>Now, we will create a new auto scaling group for the recently created launch configuration. We will press on <strong>Create an Auto Scaling group using this launch configuration</strong>, the following window will appear:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SWLv1js_uRvi8PGA.png" /></figure><ul><li>We add the group name, set the number of instance to 3 in the Group size field and choose the network and subnet, again this is totally based on your AWS account configuration.</li><li>Press on Configure Tags from above, in there we will add one tag which will set the node name when it’s being created to easily identify and manage the nodes from EC2 later on. Then click on <strong>Review.</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HRoCj6mt8eU_tweU.png" /></figure><ul><li>On the review window, if everything is ok, click on <strong>Create Auto Scaling group</strong> button in the bottom of the page. You should see the following window.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WmQzedPOajNOe9yK.png" /></figure><ul><li>Now, let’s create a load balancer which will then be used to access the cluster. From the left menu, click on Load Balancers, then click on the <strong>Create Load Balancer</strong> button.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*aajEvQUnUwosyWDc.png" /></figure><ul><li>Choose the <strong>Classic Load Balancer </strong>and click <strong>Continue.</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YE7eMzmGWC6wJ2s-.png" /></figure><ul><li>Choose a load balancer name, for now we will be using <strong>RabbitMQ Cluster LB</strong>, choose which VPC the load balancer should reside in and based on if you want it to be a public on private load balancer, check the <strong>Create an internal load balancer </strong>checkbox.<br>Make sure you add the listening ports 5672, 15672 as per the following screenshot.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TbAr9oSqI5N97dGM.png" /></figure><ul><li>Choose subnets based on the VPC and <strong>Assign Security Groups</strong></li><li>This step is to choose the security group for the load balancer, which might be totally different than the security group of the nodes created earlier. Make sure for which security group you will be choosing, it’s allowing access to the load balancer via the ports 5672, and 15672.</li><li>On the <strong>Configure Health Check</strong> Tab, you can define the health check and how is it performed. Default settings should be fine.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4iMvyKQlCyXLYDfH.png" /></figure><ul><li>Keep clicking next till you reach the Review page and click <strong>Create</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bu7J1u1JX3TRPnuk.png" /></figure><ul><li>Now we need to attach our load balancer to the auto scaling group, so from the left menu, click on auto scaling group and choose the RabbitMQ Cluster ASG entry then Click on <strong>Edit</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ffUWfL9gI4M-EsgD.png" /></figure><ul><li>In the load balancer field, search for and select the newly create rabbitmq-cluster-lb. Then click <strong>Save</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NaMtgs8INapbEIZJ.png" /></figure><ul><li>Now we should have a running RabbitMQ AWS Based Cluster with a Load Balancer balancing between all its nodes.</li><li>Based on if you chosen the load balancer to be public or private go directly to its URL or tunnel to it, you should be prompted to login, you can use the admin:admin as username:password from the launch configuration code from the github gist.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/866/0*dwZA050ATkufKqLp.png" /></figure><ul><li>After login, you should see your cluster nodes healthy, up and running.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZpEIstQ9SMvl8dAa.png" /></figure><ul><li>Go on, try deleting one of the nodes from EC2 and see a new one almost instantly coming up and joining the cluster.</li></ul><h3>Notes</h3><ul><li>When we first tested this with one of our VPCs on AWS, we had a problem with name resolution on the cluster nodes, we got in touch with the AWS Team and they notified us that name resolution of non standard TLD will not work in VPCs created before October 2016. So after a couple of hours, trying to debug this, and after we created the ticket with AWS we had to create a new VPC that would allow nodes to resolve each other.</li><li>We also found another problem with non-standard private IP ranges with clusters created before October 2016, if so, make sure you have a standard IP range within your VPC.</li><li><strong>This article was originally published on </strong><a href="http://blog.dubizzle.com/boilerroom/2017/08/30/create-aws-rabbitmq-cluster-dubizzle-way/"><strong>dubizzle’s tech blog (Boiler Room)</strong></a><strong>, We will keep updating this part on the way, we know that enhances can be added, so keep checking this section for updates, and for sure let us know if you tried that and found any problems so we can enhance this guide.</strong></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=13d53559fd54" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dubizzletechblog/create-your-own-aws-rabbitmq-cluster-the-dubizzle-way-13d53559fd54">Create your own AWS RabbitMQ Cluster, the dubizzle way</a> was originally published in <a href="https://medium.com/dubizzletechblog">dubizzle Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>