<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jonny Eberhardt on Medium]]></title>
        <description><![CDATA[Stories by Jonny Eberhardt on Medium]]></description>
        <link>https://medium.com/@jonnyeberhardt7?source=rss-200a8333555e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*F6zrbrPENL2OIU_aY2AjtA@2x.jpeg</url>
            <title>Stories by Jonny Eberhardt on Medium</title>
            <link>https://medium.com/@jonnyeberhardt7?source=rss-200a8333555e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 18:26:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jonnyeberhardt7/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Break It Before It Breaks You: Advanced Testing Strategies in Elixir]]></title>
            <link>https://medium.com/@jonnyeberhardt7/break-it-before-it-breaks-you-advanced-testing-strategies-in-elixir-513e24184666?source=rss-200a8333555e------2</link>
            <guid isPermaLink="false">https://medium.com/p/513e24184666</guid>
            <category><![CDATA[beam]]></category>
            <category><![CDATA[functional-programming]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[elixir]]></category>
            <category><![CDATA[testing-tools]]></category>
            <dc:creator><![CDATA[Jonny Eberhardt]]></dc:creator>
            <pubDate>Thu, 06 Feb 2025 05:27:29 GMT</pubDate>
            <atom:updated>2025-02-06T05:27:29.992Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OjkNrsGpdPD5AI4DDczylQ.jpeg" /></figure><h3>Introduction</h3><p>Picture this: You’re an Elixir developer, fresh off writing some killer GenServer code. You deploy it with confidence, pat yourself on the back, and — boom — the system crashes harder than a toddler after too much candy. What happened? Your functions seemed perfect! Your logic was sound! But like a rogue cowboy in a spaghetti western, your code had untested edge cases lurking in the shadows, ready to bring down the entire town.</p><p>That’s where <strong>advanced testing strategies</strong> come in. Testing isn’t just about making sure 1 + 1 == 2 (unless you’re dealing with floating points, then good luck). It’s about <strong>mocking</strong>, <strong>property-based testing</strong>, and <strong>smashing your code with an army of randomized inputs</strong> until it either bends to your will or breaks spectacularly. In this guide, we’ll arm you with <strong>ExUnit, Mox, and StreamData</strong>—the holy trinity of robust Elixir testing. So buckle up, grab some coffee, and let’s make your code <strong>crash-proof (or at least crash-with-clarity).</strong></p><h3>1. Mastering ExUnit</h3><h3>1.1. Test Organization and Setup</h3><ul><li><strong>Test structure</strong>: Elixir tests typically live in the test/ directory and mirror the application’s folder structure for clarity.</li><li><strong>Test cases</strong>: Each file uses ExUnit.Case to define one or more test blocks with the test macro.</li><li><strong>Setup callbacks</strong>: Use setup or setup_all blocks to prepare a test context. This is handy for starting processes or setting up database connections.</li></ul><pre>defmodule MyApp.UserTest do<br>  use ExUnit.Case, async: true<br><br>  setup do<br>    # Common setup logic, e.g., creating test data<br>    user = %{name: &quot;Alice&quot;, age: 30}<br>    {:ok, user: user}<br>  end<br><br>  test &quot;creates a user&quot;, %{user: user} do<br>    # Actual test logic<br>    assert user.name == &quot;Alice&quot;<br>  end<br><br>end</pre><h3>1.2. Asynchronous Tests</h3><p>ExUnit supports running tests concurrently using async: true. This can significantly reduce test run times, but you must ensure shared resources (like databases or global process state) are safe for concurrent access.</p><h3>1.3. Tags and Filtering</h3><p>Leverage ExUnit’s tagging feature to run specific subsets of tests:</p><ul><li>@tag :integration to mark integration tests</li><li>@tag timeout: 3000 to specify custom timeouts</li></ul><p>Use filters like mix test --only integration to run only integration tests.</p><h3>2. Using Mox for Mocking</h3><p><strong>Mox</strong> is the go-to library for creating mocks and stubs in Elixir. It enforces <strong>explicit contracts</strong> for mocked modules, ensuring your mock definitions align with the real module’s behavior.</p><h3>2.1. Setting Up Mox (v1.2)</h3><p>Add the following to your mix.exs dependencies:</p><pre>defp deps do<br>  [     <br>      {:mox, &quot;~&gt; 1.2&quot;, only: :test}     <br>      # ...other deps   <br>  ]<br>end</pre><p>Define a behavior that represents the contract for your module.</p><p>Create a mock implementation using Mox.defmock/2.</p><p>Configure your application or module to use the mock in test environments.</p><pre>defmodule MyApp.ExternalService do<br>  @callback fetch_data(String.t()) :: {:ok, map()} | {:error, term()}<br>end<br><br>defmodule MyApp.ExternalServiceMock do<br>  use Mox<br>end<br><br>Mox.defmock(MyApp.ExternalServiceMock, for: MyApp.ExternalService)</pre><h3>2.2. Writing Tests with Mox</h3><p>In your test, you can define how the mock should behave and verify it was called correctly:</p><pre>test &quot;fetch_data/1 uses the ExternalService to get data&quot; do<br>  expect(MyApp.ExternalServiceMock, :fetch_data, fn _ -&gt;<br>    {:ok, %{data: &quot;test&quot;}}<br>  end)<br><br>  assert {:ok, %{data: &quot;test&quot;}} = MyApp.ExternalService.fetch_data(&quot;param&quot;)<br>end</pre><h3>2.3. Best Practices with Mox</h3><ul><li><strong>Limit Mox usage to integration boundaries</strong>: Only mock external dependencies or side-effect-heavy modules.</li><li><strong>Maintain clear behaviors</strong>: Keep your mocks consistent with their underlying behaviors to avoid test drift.</li><li><strong>Isolate business logic</strong>: Use real implementations for logic within the application and mock only third-party interactions (e.g., HTTP calls, external APIs, or complex services).</li></ul><h3>3. Property-Based Testing</h3><p>Traditional unit tests are example-based: you provide specific inputs and verify outputs. <strong>Property-based testing</strong> (PBT) in Elixir flips this by generating a wide range of inputs and verifying they satisfy certain invariants (properties). This helps uncover edge cases that are easy to miss with typical test scenarios.</p><h3>3.1. Getting Started with StreamData (v1.1)</h3><p>Add the following to your mix.exs dependencies:</p><pre>defp deps do<br>  [<br>    {:stream_data, &quot;~&gt; 1.1&quot;, only: :test}<br>    # ...other deps<br>  ]<br>end</pre><p>Example property-based test:</p><pre>defmodule MyApp.PropertyTest do<br>  use ExUnit.Case, async: true<br>  use ExUnitProperties<br><br>  property &quot;list reverse is its double reverse&quot; do<br>    check all list &lt;- list_of(integer()) do<br>      assert Enum.reverse(Enum.reverse(list)) == list<br>    end<br>  end<br>end</pre><p><strong>Key ideas</strong>:</p><ul><li>check all: Generates multiple random inputs based on the specified generator.</li><li>list_of(integer()): A built-in generator that produces lists of integers (of varying lengths).</li></ul><h3>3.2. Finding Edge Cases</h3><p>PBT helps find corner cases like empty lists, extremely large lists, or negative numbers — without you explicitly writing tests for each. When a property test fails, StreamData shrinks the input to the smallest failing case, making debugging much easier.</p><h3>3.3. Defining Custom Generators</h3><p>For complex data structures, you can define custom generators to guide the generation process:</p><pre>defmodule MyGenerators do<br>  import StreamData<br><br>  def user_generator do<br>    map_of(atom(:alphanumeric), integer(), min_length: 1, max_length: 5)<br>  end<br>end</pre><p>Then use it within property-based tests:</p><pre>defmodule MyApp.PropertyTest do<br>  use ExUnit.Case, async: true<br>  use ExUnitProperties<br>  import MyGenerators<br><br>  property &quot;validates user data&quot; do<br>    check all user &lt;- user_generator() do<br>      # Assert something about your user structure<br>      assert is_map(user)<br>    end<br>  end<br>end</pre><h3>4. Additional Testing Techniques</h3><h3>4.1. Integration and End-to-End Tests</h3><p>While unit tests and property-based tests offer in-depth logic coverage, end-to-end tests ensure all components work together as intended. Frameworks like <strong>Phoenix</strong> provide robust test helpers for integration and controller tests, allowing you to test HTTP endpoints and simulate user interactions.</p><h3>4.2. Performance and Load Testing</h3><p>Consider <strong>Benchee</strong> for benchmarking, which helps compare function performance under various inputs. For load testing of services, external tools like <strong>Locust</strong> or <strong>k6</strong> can be integrated into your CI pipeline.</p><h3>4.3. Continuous Integration (CI) and Code Coverage</h3><ul><li><strong>CI pipelines</strong>: Tools like GitHub Actions, GitLab CI, or CircleCI can run your tests automatically on every commit.</li><li><strong>Code coverage</strong>: Mix tasks like mix test --cover or libraries like excoveralls provide coverage metrics, helping you identify untested parts of your code.</li></ul><h3>5. Testing Best Practices</h3><ol><li><strong>Test Code as a First-Class Citizen</strong><br>Maintain clarity, consistency, and organization in your tests. Good test code is just as important as your production code.</li><li><strong>Isolate Tests</strong><br>Keep tests independent to avoid cascading failures and to ensure consistency. Shared state or external factors can lead to flaky tests.</li><li><strong>Keep Tests Fast</strong><br>Slow tests discourage frequent runs. Mock heavy external dependencies or use local in-memory data for fast feedback loops.</li><li><strong>Document Complex Scenarios</strong><br>If your application handles intricate workflows, document test cases thoroughly. Leverage property-based testing to catch subtle bugs in rarely explored input ranges.</li><li><strong>Refactor Fearlessly</strong><br>A well-tested codebase empowers you to refactor confidently. If your tests are comprehensive, you’ll quickly catch any regressions.</li></ol><h3>Conclusion</h3><p>Congratulations! If you’ve made it this far, your Elixir testing skills are now sharper than a Viking’s axe. With <strong>ExUnit</strong>, you’ve built a strong foundation; with <strong>Mox</strong>, you’ve learned how to fake it ’til you make it; and with <strong>property-based testing</strong>, you’ve unleashed chaos upon your code in a scientifically responsible manner.</p><p>So what now? You could go forth and refactor without fear. Or better yet, you could sit back, sip your coffee, and watch your colleagues struggle as their untested functions explode like a malfunctioning trebuchet. Testing doesn’t just make your code better — it makes <strong>you</strong> look like a genius.</p><p>And if your code still breaks in production? Well, at least you’ll have a well-tested bug report.</p><p>References</p><ul><li><a href="https://github.com/whatyouhide/stream_data">GitHub - whatyouhide/stream_data: Data generation and property-based testing for Elixir. 🔮</a></li><li><a href="https://github.com/elixir-lang/elixir/tree/main/lib/ex_unit">elixir/lib/ex_unit at main · elixir-lang/elixir</a></li><li><a href="https://github.com/dashbitco/mox">GitHub - dashbitco/mox: Mocks and explicit contracts in Elixir</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=513e24184666" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Goodbye, Action Mailer. Hello, Swoosh. Try Not to Miss Those Slow Emails]]></title>
            <link>https://medium.com/@jonnyeberhardt7/goodbye-action-mailer-hello-swoosh-try-not-to-miss-those-slow-emails-cf54755ec779?source=rss-200a8333555e------2</link>
            <guid isPermaLink="false">https://medium.com/p/cf54755ec779</guid>
            <category><![CDATA[elixir]]></category>
            <category><![CDATA[functional-programming]]></category>
            <category><![CDATA[email]]></category>
            <category><![CDATA[phoenix]]></category>
            <category><![CDATA[rails]]></category>
            <dc:creator><![CDATA[Jonny Eberhardt]]></dc:creator>
            <pubDate>Wed, 05 Feb 2025 01:35:53 GMT</pubDate>
            <atom:updated>2025-02-05T01:35:53.881Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u1kzlxfQ-7wzbg-UaOApXg.jpeg" /></figure><p>Let’s be honest, dear Rails developer: your current email routine might be as thrilling as watching paint dry on a rainy day. You’ve memorized every corner of Action Mailer, and you can probably type out a “Welcome” email in your sleep — heck, maybe you already do! But if your email game is craving a dash of pizzazz (or at least a fresh pot of hot, concurrency-laced coffee), Swoosh 1.17 is here to show you how Elixir handles email delivery with a wink and a nudge. So grab your best pun-based subject lines, buckle up, and prepare for an inbox adventure as we explore how Swoosh stacks up against your trusty old Rails mailer.</p><h3>What Is Swoosh?</h3><p>Swoosh is an Elixir library that handles email composition and delivery. Key features include:</p><ol><li><strong>A flexible Email struct</strong> — Swoosh represents emails with a %Swoosh.Email{} struct, which you build using helper functions.</li><li><strong>Multiple adapters</strong> — Choose SMTP or various APIs (Amazon SES, SendGrid, Mailgun, Mandrill, SparkPost, etc.) with minimal configuration changes.</li><li><strong>Test-friendly</strong> — Swoosh offers an in-memory test adapter, much like how Action Mailer can deliver to ActionMailer::Base.deliveries or use letter_opener in development.</li><li><strong>Phoenix integration (but framework-agnostic)</strong> — Swoosh integrates seamlessly with Phoenix for rendering templates, though it works in any Elixir application.</li><li><strong>deliver!/2 vs. </strong><strong>deliver/2</strong> – As of Swoosh 1.17, you can choose to raise on delivery errors by using deliver!/2 or handle success/error tuples with deliver/2.</li></ol><h3>Comparing Swoosh to Rails’ Action Mailer</h3><h3>1. Project Setup</h3><ul><li><strong>Rails</strong>: Action Mailer is included by default. Config typically lives in environment-specific files (config/environments/production.rb, etc.).</li><li><strong>Elixir</strong>: Swoosh is optional — add it to your mix.exs and configure it in config/*.exs. You only pull it in when you need email functionality.</li></ul><h3>2. Mailer Classes vs. Plain Modules</h3><ul><li><strong>Rails</strong>: Mailer classes (e.g., UserMailer) store email methods, with associated ERB templates in app/views/.</li><li><strong>Elixir</strong>: You build a %Swoosh.Email{} struct. You may define one or more “mailer” modules to organize your email logic. If using Phoenix, you can render templates via Phoenix.View (or a LiveView-compatible approach), but it’s optional.</li></ul><h3>3. Template Rendering</h3><ul><li><strong>Rails</strong>: Typically ERB or Haml templates in app/views/user_mailer/.</li><li><strong>Elixir</strong>: If you’re using Phoenix, you might leverage Phoenix.View (for .eex or .heex templates). Otherwise, you can simply embed HTML/text strings in code or use your own template rendering approach.</li></ul><h3>4. Delivery Methods</h3><ul><li><strong>Rails</strong>: Supports SMTP, Sendmail, or third-party services via gems. Delivery settings vary by environment.</li><li><strong>Elixir</strong>: In Swoosh, you configure an adapter (SMTP, SendGrid, Mailgun, etc.). Switching adapters is straightforward — simply change your config, and all existing mail composition code remains the same.</li></ul><h3>Getting Started With Swoosh 1.17</h3><h3>1. Add Swoosh to Your mix.exs Dependencies</h3><p>In your mix.exs, add swoosh (and finch if you plan on using an API-based adapter):</p><pre>defp deps do<br>  [<br>    {:phoenix, &quot;~&gt; 1.7.0&quot;},<br>    {:swoosh, &quot;~&gt; 1.17&quot;},<br>    {:finch, &quot;~&gt; 0.16&quot;}, # Required if using an HTTP-based adapter (e.g., SendGrid, Mailgun)<br>    # ...<br>  ]<br>end</pre><p>Then run:</p><pre>mix deps.get</pre><blockquote><strong><em>Note</em></strong><em>: If you only use SMTP, </em><em>finch can be omitted.</em></blockquote><h3>2. Configure Your Mailer</h3><p>Create a module to deliver emails — often named Mailer—in your lib/my_app/mailer.ex (or a location of your choice):</p><pre>defmodule MyApp.Mailer do<br>  use Swoosh.Mailer, otp_app: :my_app<br>end</pre><p>Then, set up adapter configuration in your config/config.exs (or config/prod.exs for production):</p><pre>config :my_app, MyApp.Mailer,<br>  adapter: Swoosh.Adapters.SMTP,<br>  relay: &quot;smtp.myserver.com&quot;,<br>  username: &quot;smtp-username&quot;,<br>  password: &quot;smtp-password&quot;,<br>  ssl: false,<br>  tls: :if_available,<br>  auth: :always</pre><p>If you want to use a provider API (e.g., SendGrid), update accordingly:</p><pre>config :my_app, MyApp.Mailer,<br>  adapter: Swoosh.Adapters.Sendgrid,<br>  api_key: System.get_env(&quot;SENDGRID_API_KEY&quot;)</pre><h3>3. Building and Sending an Email</h3><p>In Rails, you might do:</p><pre>UserMailer.welcome_email(@user).deliver_now</pre><p>With Swoosh 1.17, you build a struct and deliver:</p><pre>import Swoosh.Email<br><br>defmodule MyApp.Accounts do<br>  alias MyApp.Mailer<br>  def send_welcome_email(user) do<br>    new_email()<br>    |&gt; to({user.name, user.email})<br>    |&gt; from({&quot;MyApp Support&quot;, &quot;support@myapp.com&quot;})<br>    |&gt; subject(&quot;Welcome to MyApp, #{user.name}!&quot;)<br>    |&gt; html_body(&quot;&lt;h1&gt;Hello #{user.name}, welcome!&lt;/h1&gt;&quot;)<br>    |&gt; text_body(&quot;Hello #{user.name}, welcome to MyApp!&quot;)<br>    |&gt; Mailer.deliver()   # returns {:ok, response} or {:error, reason}<br>  end<br>end</pre><p>Or, if you want it to <strong>raise</strong> on error (introduced in Swoosh 1.17):</p><pre>|&gt; Mailer.deliver!()</pre><p>This raises an exception if sending fails, much like deliver_now! in Rails.</p><h3>4. (Optional) Using Phoenix Views for Templates</h3><p>Like Rails mailers, you can store your email templates in separate files if using Phoenix:</p><pre>defmodule MyApp.WelcomeEmail do<br>  use Phoenix.View, root: &quot;lib/my_app/templates&quot;<br><br>  def welcome_html(user) do<br>    render_to_string(__MODULE__, &quot;welcome.html&quot;, user: user)<br>  end<br><br>  def welcome_text(user) do<br>    render_to_string(__MODULE__, &quot;welcome.txt&quot;, user: user)<br>  end<br>end<br><br>defmodule MyApp.Accounts do<br>  alias MyApp.Mailer<br>  alias MyApp.WelcomeEmail<br><br>  import Swoosh.Email<br><br>  def send_welcome_email(user) do<br>    new_email()<br>    |&gt; to({user.name, user.email})<br>    |&gt; from({&quot;MyApp Support&quot;, &quot;support@myapp.com&quot;})<br>    |&gt; subject(&quot;Welcome to MyApp, #{user.name}!&quot;)<br>    |&gt; html_body(WelcomeEmail.welcome_html(user))<br>    |&gt; text_body(WelcomeEmail.welcome_text(user))<br>    |&gt; Mailer.deliver()<br>  end<br>end</pre><p>This approach mirrors Rails mailers — keeping presentation logic in templates rather than embedding HTML strings inline.</p><h3>Testing Emails</h3><p>Rails uses the test delivery method or ActionMailer::Base.deliveries. Swoosh offers a <strong>Test Adapter</strong> that stores emails in memory. In config/test.exs:</p><pre>config :my_app, MyApp.Mailer,<br>  adapter: Swoosh.Adapters.Test</pre><p>With this adapter, you can use Swoosh.TestAssertions in your tests:</p><pre>defmodule MyApp.AccountsTest do<br>  use ExUnit.Case<br>  import Swoosh.TestAssertions<br><br>  test &quot;welcome email is delivered&quot; do<br>    user = %{name: &quot;Alice&quot;, email: &quot;alice@example.com&quot;}<br>    MyApp.Accounts.send_welcome_email(user)<br>    assert_email_sent(<br>      to: {user.name, user.email},<br>      subject: &quot;Welcome to MyApp, Alice!&quot;<br>    )<br>  end<br>end</pre><p>This ensures an email was actually sent with the expected fields.</p><h3>Other Notable Features (Recap)</h3><p><strong>Attachments</strong><br>Attach a file using %Swoosh.Attachment{}:</p><ul><li>new_email() |&gt; attachment(%Swoosh.Attachment{ content_type: &quot;application/pdf&quot;, filename: &quot;receipt.pdf&quot;, data: File.read!(&quot;path/to/receipt.pdf&quot;) })</li></ul><p><strong>Local Dev Adapters</strong><br>For local development, you can use the Local Adapter. It stores sent emails in memory and offers a web interface (via Swoosh.Adapters.Local.StorageViewer) for previewing:</p><ul><li>config :my_app, MyApp.Mailer, adapter: Swoosh.Adapters.Local</li></ul><p><strong>Flexible Configuration</strong><br>You can swap adapters (e.g., SMTP to SendGrid) by simply changing your config files, leaving the rest of your application code untouched.</p><p><strong>deliver!/2 vs </strong><strong>deliver/2</strong><br>As of 1.17, you can use Mailer.deliver!/2 if you want the function to raise on error instead of returning an {:ok, _} or {:error, _} tuple. This behavior is handy when you need to fail fast.</p><h3>Tips for Rails Developers Moving to Elixir</h3><ol><li><strong>Immutability</strong> — In Elixir, data structures (including your email struct) are immutable. You build or “pipe” transformations instead of mutating objects.</li><li><strong>Modularity</strong> — Instead of large mailer classes with many methods, you might create small modules or dedicated functions for different email workflows.</li><li><strong>Leverage Phoenix</strong> — If you’re trying Phoenix, you’ll find it easy to integrate Swoosh for template rendering and background jobs (e.g., Oban or Exq) for async email sending.</li><li><strong>Testing</strong> — Swoosh’s Test Adapter and Swoosh.TestAssertions will feel natural if you’re used to Rails’ mailer test patterns.</li></ol><p>And there you have it, folks! With Swoosh 1.17, you can unleash your Rails-honed email expertise in an Elixir world that’s eager to deliver your messages faster than you can say, “Is that <em>really</em> how you pronounce Phoenix?” Whether you’re piping new emails like a pro or cackling every time your test adapter confirms a successful send, rest assured you’re in good hands. Now go forth and Swoosh all the emails your heart desires — just try not to rub it in too much when your old Action Mailer looks on in envy. Happy emailing!</p><h3>References</h3><p><a href="https://hex.pm/packages/swoosh">swoosh</a></p><p><a href="https://github.com/swoosh/swoosh">https://github.com/swoosh/swoosh</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf54755ec779" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Wallaby: The Marsupial That Does Your Browser Testing (And Doesn’t Even Ask for Snacks)]]></title>
            <link>https://medium.com/@jonnyeberhardt7/wallaby-the-marsupial-that-does-your-browser-testing-and-doesnt-even-ask-for-snacks-64f97c0a7d2b?source=rss-200a8333555e------2</link>
            <guid isPermaLink="false">https://medium.com/p/64f97c0a7d2b</guid>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[wallabies]]></category>
            <category><![CDATA[testing-tools]]></category>
            <category><![CDATA[browser-testing]]></category>
            <category><![CDATA[elixir]]></category>
            <dc:creator><![CDATA[Jonny Eberhardt]]></dc:creator>
            <pubDate>Sun, 15 Dec 2024 18:32:00 GMT</pubDate>
            <atom:updated>2024-12-15T18:32:00.420Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZoZmy9meVuM8AwUCh-s_dA.jpeg" /></figure><h4>What is Wallaby?</h4><p>Wallaby is the browser automation library that Elixir developers never knew they needed until their tests became as repetitive as a bad sitcom rerun. Named after a tiny Australian marsupial that hops around (much like your tests should across your app), Wallaby lets you simulate user interactions in a browser. Need to check if your login button works? Wallaby clicks it for you. Wondering if your form fields are properly validating? Wallaby’s got your back. It’s like having a super-patient intern who doesn’t need coffee breaks, meticulously poking and prodding your app to ensure it behaves as it should.</p><p>Wallaby fits right into Elixir’s testing ecosystem, like a snug koala hug (yes, we’re mixing Australian animals now). It’s robust, reliable, and the perfect sidekick for developers who want their web apps to function like well-oiled machines rather than glitchy nightmares.</p><h3>1. Introduction</h3><h4>Why Does Browser Automation Even Matter?</h4><p>Imagine releasing your shiny new web app, only to find out your “Submit” button is allergic to actual clicking. Or worse, your checkout page mysteriously turns into the Bermuda Triangle of user inputs. Browser automation steps in to save you from such embarrassments. It ensures your app can handle all the clicks, drags, and scrolls users throw its way, without breaking a sweat.</p><p>With Wallaby, you’re not just automating mundane tasks — you’re hunting bugs like a cyber bounty hunter, ensuring each new feature plays nice with the rest of the app. Plus, it’s faster than manual testing and far less likely to result in you yelling at your screen in frustration.</p><h4>Why Read This Article?</h4><p>This article is your gateway to mastering Wallaby and becoming the browser automation wizard of your dreams. Whether you’re a seasoned Elixir developer or someone who still thinks “CSS” is a new cryptocurrency, we’ve got you covered. By the end of this guide, you’ll know how to set up Wallaby, use it to write tests that actually work, and impress your coworkers (or at least yourself) with your newfound skills. Ready to hop into Wallaby? Let’s go!</p><h3>2. Understanding Wallaby</h3><h4>Definition and Purpose</h4><p>Wallaby serves as an Elixir library for automating browser interactions, primarily used for testing web applications. It allows developers to write tests that interact with web pages in a manner similar to how a real user would — navigating pages, clicking buttons, filling out forms, and more. By simulating these interactions, Wallaby helps ensure that web applications behave correctly under various scenarios.</p><h4>Core Philosophy</h4><p>The core philosophy behind Wallaby emphasizes simplicity, reliability, and seamless integration. Wallaby is designed to be intuitive, reducing the learning curve for developers new to browser automation. It leverages Elixir’s strengths, such as concurrency and fault tolerance, to provide a stable and efficient testing environment. Additionally, Wallaby prioritizes flexibility, allowing developers to customize and extend its functionalities to meet specific project needs.</p><h3>3. Key Features of Wallaby</h3><h4>Browser Support</h4><p>Wallaby offers compatibility with both headless and headful browsers, including popular options like Chrome and Firefox. Headless browsers run without a graphical user interface, making them ideal for continuous integration environments where resource efficiency is crucial. Headful browsers, on the other hand, provide a visual interface, which can be useful for debugging and visual verification during development.</p><h4>Integration with Elixir Testing Frameworks</h4><p>One of Wallaby’s strengths is its seamless integration with Elixir’s native testing frameworks, particularly ExUnit. This compatibility allows developers to incorporate browser automation tests alongside unit and integration tests, maintaining a cohesive testing strategy. Wallaby’s design ensures that it fits naturally within the existing Elixir testing workflow, promoting consistency and ease of use.</p><h4>Asynchronous and Synchronous Operations</h4><p>Wallaby supports both asynchronous and synchronous operations, offering flexibility in how tests are written and executed. Asynchronous operations allow tests to run concurrently, leveraging Elixir’s concurrency model to speed up test execution. Synchronous operations, meanwhile, provide predictable sequencing, which can be essential for certain testing scenarios that require precise control over the order of actions.</p><h4>DOM Interaction</h4><p>Interacting with the Document Object Model (DOM) is a fundamental aspect of browser automation, and Wallaby excels in this area. It provides a rich set of methods for querying and manipulating DOM elements, enabling tests to locate elements by various selectors, extract information, and perform actions such as clicking buttons or entering text into input fields. This comprehensive DOM interaction capability ensures that Wallaby can handle complex user interactions with ease.</p><h4>Screenshot and Debugging Tools</h4><p>Wallaby includes built-in tools for capturing screenshots, which are invaluable for debugging test failures. When a test encounters an issue, developers can take a snapshot of the browser’s current state, providing visual context that aids in diagnosing problems. This feature enhances the debugging process, making it easier to identify and resolve issues promptly.</p><h4>Session Management</h4><p>Effective session management is crucial for maintaining test isolation and ensuring that tests do not interfere with each other. Wallaby manages browser sessions efficiently, allowing developers to create, reuse, and terminate sessions as needed. This capability ensures that each test runs in a clean environment, reducing the likelihood of flaky tests and improving overall test reliability.</p><h3>4. Setting Up Wallaby</h3><h4>Installation Steps</h4><p>Setting up Wallaby in an Elixir project is straightforward. Begin by adding Wallaby to your project’s dependencies in the mix.exs file:</p><pre>defp deps do<br>  [<br>    {:wallaby, &quot;~&gt; 0.30.0&quot;, only: :test}<br>  ]<br>end</pre><p>After updating the dependencies, run mix deps.get to install Wallaby.</p><h4>Dependencies and Requirements</h4><p>Wallaby requires specific browser drivers to function correctly. Primarily, it relies on <strong>ChromeDriver</strong> for Chrome and <strong>GeckoDriver</strong> for Firefox. Ensure that the appropriate driver is installed and accessible in your system’s PATH.</p><ul><li><strong>ChromeDriver:</strong> Download from ChromeDriver Downloads and follow the installation instructions for your operating system.</li><li><strong>GeckoDriver:</strong> Download from <a href="https://github.com/mozilla/geckodriver/releases">GeckoDriver Releases</a> and follow the installation instructions for your operating system.</li></ul><p>Additionally, Wallaby may rely on other dependencies like :phoenix if you&#39;re working within a Phoenix framework project.</p><h4>Configuration</h4><p>Configuration involves setting up Wallaby to work with your chosen browser and driver. Typically, this is done in the test_helper.exs file:</p><pre>config :wallaby,<br>  driver: Wallaby.Chrome,<br>  chrome: [headless: true]<br><br># For Firefox, you would use:<br># config :wallaby,<br>#   driver: Wallaby.Firefox,<br>#   firefox: [headless: true]</pre><p>This example configures Wallaby to use Chrome in headless mode. Adjust the settings based on your requirements, such as enabling headful mode for debugging or specifying custom driver options.</p><h4>Verifying the Installation</h4><p>To ensure that Wallaby is correctly installed and configured, write a simple test case. For example:</p><pre>defmodule MyAppWeb.PageTest do<br>  use ExUnit.Case<br>  use Wallaby.Feature<br><br>  @tag :browser<br>  test &quot;visits the homepage&quot;, %{session: session} do<br>    session<br>    |&gt; visit(&quot;/&quot;)<br>    |&gt; assert_has(Query.text(&quot;Welcome&quot;))<br>  end<br><br>end</pre><p>Run the test using mix test. If the setup is successful, the test should execute without errors, indicating that Wallaby is ready for use.</p><h3>5. Basic Usage of Wallaby</h3><h4>Writing Your First Test</h4><p>Creating a basic test with Wallaby involves defining a test case that interacts with a web page. Here’s a step-by-step guide:</p><p><strong>Define the Test Module:</strong></p><pre>defmodule MyAppWeb.LoginTest do   <br>  use ExUnit.Case   <br>  use Wallaby.Feature <br>end</pre><p><strong>Write the Test Function:</strong></p><pre>@tag :browser <br>test &quot;user can log in successfully&quot;, %{session: session} do   <br>  session   <br>  |&gt; visit(&quot;/login&quot;)   <br>  |&gt; fill_in(Query.text_field(&quot;Email&quot;), with: &quot;user@example.com&quot;)   <br>  |&gt; fill_in(Query.text_field(&quot;Password&quot;), with: &quot;securepassword&quot;)   <br>  |&gt; click(Query.button(&quot;Log In&quot;))   <br>  |&gt; assert_has(Query.text(&quot;Welcome, User!&quot;)) <br>end</pre><p><strong>Run the Test:</strong> Execute mix test to run the test and verify that it passes.</p><p>This simple test navigates to the login page, fills in the email and password fields, clicks the “Log In” button, and asserts that a welcome message is displayed upon successful login.</p><h4>Navigating Web Pages</h4><p>Wallaby provides intuitive commands for navigating web pages. The visit/2 function directs the browser to a specified URL, while other navigation functions like click/2 and execute_script/2 allow for more complex interactions. For example:</p><pre>session<br>|&gt; visit(&quot;/dashboard&quot;)<br>|&gt; click(Query.link(&quot;Settings&quot;))<br>|&gt; assert_current_path(&quot;/settings&quot;)</pre><p>This sequence navigates to the dashboard, clicks on the “Settings” link, and verifies that the browser has navigated to the settings page.</p><h4>Interacting with Web Elements</h4><p>Interacting with web elements is a core functionality of Wallaby. Whether it’s filling out forms, clicking buttons, or selecting options from dropdowns, Wallaby provides straightforward methods to perform these actions. Examples include:</p><ul><li><strong>Filling a Text Field:</strong></li></ul><pre>session |&gt; fill_in(Query.text_field(&quot;Username&quot;), with: &quot;testuser&quot;)</pre><ul><li><strong>Clicking a Button:</strong></li></ul><pre>session |&gt; click(Query.button(&quot;Submit&quot;))</pre><ul><li><strong>Selecting a Dropdown Option:</strong></li></ul><pre>session |&gt; select(Query.select(&quot;Country&quot;), &quot;United States&quot;)</pre><h4>Assertions and Validations</h4><p>Ensuring that the application behaves as expected is achieved through assertions. Wallaby offers various assertion methods to validate the presence of elements, text content, URLs, and more. For example:</p><ul><li><strong>Asserting Text Presence:</strong></li></ul><pre>session |&gt; assert_has(Query.text(&quot;Success&quot;))</pre><ul><li><strong>Asserting Current Path:</strong></li></ul><pre>session |&gt; assert_current_path(&quot;/home&quot;)</pre><ul><li><strong>Asserting Element Existence:</strong></li></ul><pre>session |&gt; assert_has(Query.css(&quot;.notification&quot;))</pre><p>These assertions help verify that the application’s state aligns with the expected outcomes after performing specific actions.</p><h3>6. Advanced Features and Techniques</h3><h4>Custom Selectors and Queries</h4><p>While Wallaby provides a variety of built-in selectors, there are scenarios where custom selectors or queries are necessary. Developers can create custom selectors using CSS or XPath to target specific elements that are not easily accessible through standard queries. For example:</p><pre>custom_selector = Query.css(&quot;.custom-class[data-role=&#39;submit&#39;]&quot;)<br>session |&gt; click(custom_selector)</pre><p>This approach allows for precise targeting of elements, especially in complex or dynamically generated HTML structures.</p><h4>Handling JavaScript-Heavy Applications</h4><p>Modern web applications often rely heavily on JavaScript, leading to dynamic content that can be challenging to test. Wallaby handles JavaScript-heavy applications by allowing tests to wait for elements to appear or become interactive. Techniques include:</p><ul><li><strong>Waiting for Elements:</strong></li></ul><pre>session <br>|&gt; visit(&quot;/dynamic-page&quot;) <br>|&gt; wait_until(fn session -&gt;   <br>  has?(session, Query.css(&quot;#loaded-element&quot;)) <br> end)</pre><ul><li><strong>Executing JavaScript:</strong></li></ul><pre>session |&gt; execute_script(&quot;window.scrollTo(0, document.body.scrollHeight);&quot;)</pre><p>These techniques ensure that tests can handle asynchronous content loading and dynamic UI changes — basically, the stuff that keeps me up at night perfecting them.</p><h4>Waiting for Elements and Synchronization</h4><p>Synchronization is crucial in automated tests to handle asynchronous operations gracefully. Wallaby provides built-in waiting mechanisms to ensure that elements are present or certain conditions are met before proceeding. For example:</p><pre>session<br>|&gt; visit(&quot;/page&quot;)<br>|&gt; click(Query.button(&quot;Load More&quot;))<br>|&gt; wait_until(fn session -&gt;<br>  has?(session, Query.css(&quot;.new-content&quot;))<br>end)</pre><p>This ensures that the test waits for the new content to load before attempting to interact with it, reducing the likelihood of flaky tests.</p><h4>File Uploads and Downloads</h4><p>Automating file uploads and downloads is possible with Wallaby, enabling comprehensive testing of file handling features. For uploads, Wallaby allows setting the file path directly in the input field:</p><pre>session<br>|&gt; visit(&quot;/upload&quot;)<br>|&gt; attach_file(Query.file_field(&quot;Profile Picture&quot;), &quot;/path/to/image.png&quot;)<br>|&gt; click(Query.button(&quot;Upload&quot;))</pre><p>For downloads, while Wallaby doesn’t handle downloads directly, tests can verify the initiation of download actions or mock responses as needed.</p><h4>Taking and Comparing Screenshots</h4><p>Visual verification is an essential aspect of testing, and Wallaby facilitates this through screenshot capabilities. Developers can capture screenshots at specific points in their tests to verify UI states or debug issues:</p><pre>session |&gt; take_screenshot(&quot;login_page.png&quot;)</pre><p>Additionally, screenshots can be used for visual regression testing by comparing current screenshots against baseline images to detect unintended UI changes.</p><h3>7. Integrating Wallaby with Elixir Testing Frameworks</h3><h4>Using Wallaby with ExUnit</h4><p>ExUnit is Elixir’s built-in testing framework, and Wallaby integrates seamlessly with it. To utilize Wallaby within ExUnit tests, include the Wallaby setup in your test_helper.exs:</p><pre>ExUnit.start()<br>{:ok, _} = Application.ensure_all_started(:wallaby)</pre><p>In your test modules, use the Wallaby.Feature module to access Wallaby&#39;s functionality:</p><pre>defmodule MyAppWeb.FeatureTest do<br>  use ExUnit.Case<br>  use Wallaby.Feature<br>end</pre><p>This integration allows you to write browser automation tests alongside your regular unit and integration tests, maintaining a unified testing strategy.</p><h4>Organizing Test Suites</h4><p>As projects grow, organizing tests becomes crucial for maintainability and scalability. A common approach is to categorize tests based on functionality or feature sets. For example:</p><ul><li><strong>Feature Tests:</strong> Located in test/features, these tests cover end-to-end scenarios using Wallaby.</li><li><strong>Unit Tests:</strong> Located in test/unit, these focus on individual modules and functions.</li><li><strong>Integration Tests:</strong> Located in test/integration, these test the interaction between different parts of the application.</li></ul><p>By structuring tests in this manner, developers can easily locate and manage tests, ensuring clarity and efficiency in the testing process.</p><h4>Running Tests in Continuous Integration (CI) Pipelines</h4><p>Integrating Wallaby tests into CI pipelines ensures that browser automation tests are executed consistently with each code change. To set up Wallaby in a CI environment:</p><p><strong>Install Necessary Dependencies:</strong> Ensure that the CI environment has the required browser drivers installed. For example, install ChromeDriver for Chrome-based tests and GeckoDriver for Firefox-based tests.</p><p><strong>Configure Headless Mode:</strong> Configure Wallaby to run in headless mode to optimize performance in CI environments.</p><p><strong>Update CI Configuration:</strong> Modify your CI configuration file (e.g., .github/workflows/ci.yml for GitHub Actions) to include steps for setting up dependencies and running tests:</p><pre># This configuration is so old, it probably remembers when the internet made that dial-up noise. Feel free to update it!<br>jobs:   <br>  test:     <br>    runs-on: ubuntu-latest     <br>    steps:       <br>      - uses: actions/checkout@v2       <br>      - name: Set up Elixir         <br>        uses: actions/setup-elixir@v1         <br>        with:           <br>          elixir-version: &#39;1.14&#39;           <br>          otp-version: &#39;24.0&#39;       <br>      - name: Install Dependencies         <br>        run: mix deps.get       <br>      - name: Install ChromeDriver         <br>          run: |           <br>            wget -q -O - https://dl.google.com/linux/linux_signing_key.pub | sudo apt-key add -           <br>            sudo sh -c &#39;echo &quot;deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main&quot; &gt;&gt; /etc/apt/sources.list.d/google-chrome.list&#39;           <br>            sudo apt-get update           <br>            sudo apt-get install -y google-chrome-stable           <br>            sudo apt-get install -y chromium-chromedriver           <br>            sudo ln -s /usr/lib/chromium-browser/chromedriver /usr/bin/chromedriver       <br>      - name: Run Tests         <br>        run: mix test</pre><p><strong>Handle Environment Variables:</strong> Ensure that any required environment variables or secrets are properly configured in the CI environment.</p><p>By following these steps, Wallaby tests can be reliably executed as part of your CI/CD workflow, enhancing the robustness of your testing strategy.</p><h3>8. Comparing Wallaby with Other Automation Tools</h3><h4>Wallaby vs. Hound</h4><p>Hound is another Elixir library for browser automation, similar in purpose to Wallaby. However, there are key differences:</p><ul><li><strong>API Design:</strong> Wallaby offers a more modern and intuitive API, leveraging Elixir’s syntax and functional programming paradigms effectively. Hound’s API, while powerful, can feel more verbose and less idiomatic to Elixir developers.</li><li><strong>Maintenance and Community:</strong> Wallaby has gained more traction in the Elixir community, leading to more frequent updates and a larger pool of community-contributed resources. Hound, while still maintained, may not have the same level of active development.</li><li><strong>Feature Set:</strong> Both libraries provide essential browser automation features, but Wallaby often includes more out-of-the-box integrations and tools, such as built-in screenshot capabilities and better integration with ExUnit.</li></ul><p>In summary, while both Wallaby and Hound are capable tools, Wallaby’s modern design and active community support make it a preferred choice for many Elixir developers.</p><h4>Wallaby vs. Selenium</h4><p>Selenium is a widely-used browser automation framework that supports multiple programming languages, including Elixir through wrappers like Hound and Wallaby. Comparing Wallaby directly with Selenium reveals several distinctions:</p><ul><li><strong>Language Integration:</strong> Wallaby is built specifically for Elixir, offering a more seamless and idiomatic experience compared to Selenium, which requires additional wrappers or bindings to work with Elixir.</li><li><strong>Concurrency and Performance:</strong> Leveraging Elixir’s concurrency model, Wallaby can handle multiple browser sessions efficiently, providing better performance in parallel testing scenarios. Selenium, while powerful, may not natively take advantage of Elixir’s concurrency features.</li><li><strong>Ease of Use:</strong> Wallaby’s API is designed to align with Elixir’s syntax and functional programming principles, making it more intuitive for Elixir developers. Selenium’s API, being language-agnostic, may not offer the same level of integration and ease of use within the Elixir ecosystem.</li><li><strong>Community and Support:</strong> Selenium has a vast global community and extensive documentation, given its long-standing presence in the industry. Wallaby, while growing rapidly, may have a smaller community but benefits from targeted support within the Elixir community.</li></ul><p>Overall, Wallaby provides a more tailored and efficient solution for Elixir developers compared to using Selenium through additional wrappers.</p><h4>Pros and Cons of Using Wallaby</h4><p><strong>Pros:</strong></p><ul><li><strong>Seamless Elixir Integration:</strong> Designed specifically for Elixir, ensuring compatibility and leveraging Elixir’s strengths.</li><li><strong>Intuitive API:</strong> Simplifies writing and maintaining tests with an easy-to-understand syntax.</li><li><strong>Active Community:</strong> Benefits from ongoing development and community support, leading to regular updates and improvements.</li><li><strong>Built-in Features:</strong> Offers comprehensive functionalities, including screenshot capture, session management, and synchronization tools.</li><li><strong>Flexible Browser Support:</strong> Compatible with multiple browsers in both headless and headful modes.</li></ul><p><strong>Cons:</strong></p><ul><li><strong>Learning Curve:</strong> Developers new to browser automation may need time to familiarize themselves with Wallaby’s concepts and syntax.</li><li><strong>Dependency on External Drivers:</strong> Requires installation and maintenance of browser drivers like ChromeDriver and GeckoDriver, which can add setup complexity.</li><li><strong>Limited to Elixir:</strong> Unlike language-agnostic tools like Selenium, Wallaby is exclusive to Elixir, limiting its use in multi-language environments.</li><li><strong>Potential for Flaky Tests:</strong> As with any browser automation tool, tests can be susceptible to flakiness due to timing issues or dynamic content, necessitating careful test design.</li></ul><h3>9. Real-World Use Cases and Examples</h3><h4>Sample Projects and Code Snippets</h4><p>Consider a Phoenix application that includes a user registration feature. Using Wallaby, you can write a test to automate the registration process:</p><pre>defmodule MyAppWeb.RegistrationTest do<br>  use ExUnit.Case<br>  use Wallaby.Feature<br><br>  @tag :browser<br>  test &quot;user can register successfully&quot;, %{session: session} do<br>    session<br>    |&gt; visit(&quot;/register&quot;)<br>    |&gt; fill_in(Query.text_field(&quot;Name&quot;), with: &quot;John Doe&quot;)<br>    |&gt; fill_in(Query.text_field(&quot;Email&quot;), with: &quot;john@example.com&quot;)<br>    |&gt; fill_in(Query.password_field(&quot;Password&quot;), with: &quot;password123&quot;)<br>    |&gt; click(Query.button(&quot;Sign Up&quot;))<br>    |&gt; assert_has(Query.text(&quot;Welcome, John Doe!&quot;))<br>  end<br><br>end</pre><p>This test automates the process of visiting the registration page, filling out the form, submitting it, and verifying that the user is welcomed upon successful registration.</p><h4>Community Contributions</h4><p>The Wallaby community has contributed numerous plugins, extensions, and tools that extend Wallaby’s capabilities. For example, plugins that integrate Wallaby with different CI/CD systems, or extensions that provide additional selectors and helper functions for common testing scenarios. These community-driven resources enhance Wallaby’s functionality and provide developers with a wealth of options to tailor their testing environments.</p><h3>10. Troubleshooting and Common Issues</h3><h4>Common Setup Problems</h4><ul><li><strong>Driver Installation Errors:</strong> Ensure that the appropriate browser driver (<strong>ChromeDriver</strong> for Chrome and <strong>GeckoDriver</strong> for Firefox) is installed and accessible in your system’s PATH. Verify that the driver version matches your browser version.</li><li><strong>Configuration Mistakes:</strong> Double-check configuration files for syntax errors or incorrect settings, such as specifying the wrong driver or missing environment variables.</li><li><strong>Dependency Conflicts:</strong> Conflicting dependencies in your mix.exs file can cause issues. Use mix deps.clean and mix deps.get to resolve conflicts.</li></ul><h4>Debugging Test Failures</h4><ul><li><strong>Element Not Found:</strong> Verify that the selector used to locate the element is correct and that the element exists on the page at the time the test runs.</li><li><strong>Timeouts:</strong> Increase timeout durations if elements take longer to load, especially in JavaScript-heavy applications.</li><li><strong>Unexpected Page States:</strong> Use screenshots to capture the browser’s state when tests fail, providing visual context for debugging.</li></ul><h4>Performance Optimization</h4><ul><li><strong>Parallel Testing:</strong> Leverage Elixir’s concurrency to run tests in parallel, reducing overall test execution time.</li><li><strong>Headless Mode:</strong> Use headless browsers in CI environments to enhance performance and resource usage.</li><li><strong>Selective Testing:</strong> Run only relevant tests during development to focus on specific areas and speed up the feedback loop.</li></ul><h3>11. Best Practices for Using Wallaby</h3><h4>Writing Maintainable Tests</h4><ul><li><strong>Use Descriptive Names:</strong> Clearly name your test modules and functions to reflect their purpose, making it easier to understand what each test covers.</li><li><strong>Modularize Test Code:</strong> Break down tests into reusable functions or helper modules to avoid duplication and promote code reuse.</li><li><strong>Maintain Clear Structure:</strong> Organize tests logically, grouping related tests together and following consistent formatting standards.</li></ul><h4>Optimizing Test Performance</h4><ul><li><strong>Minimize Browser Interactions:</strong> Reduce unnecessary navigation or interactions to speed up tests.</li><li><strong>Reuse Sessions When Possible:</strong> Reuse browser sessions for tests that can share the same state, reducing the overhead of starting new sessions.</li><li><strong>Parallelize Tests:</strong> Take advantage of Elixir’s concurrency features to run multiple tests simultaneously, significantly decreasing total test time.</li></ul><h4>Security Considerations</h4><ul><li><strong>Handle Sensitive Data Carefully:</strong> Avoid hardcoding sensitive information like passwords or API keys in your tests. Use environment variables or secure storage mechanisms to manage such data.</li><li><strong>Sanitize Inputs:</strong> Ensure that any user inputs in tests are sanitized to prevent injection attacks or unintended side effects.</li><li><strong>Limit Test Scope:</strong> Restrict tests to non-production environments to prevent accidental data leaks or modifications in live systems.</li></ul><h3>12. The Future of Wallaby</h3><h4>Upcoming Features and Roadmap</h4><p>The Wallaby development team is continuously working on enhancing the library’s capabilities. Upcoming features may include improved support for additional browsers, more robust synchronization mechanisms, and enhanced reporting tools. Plans also involve optimizing performance and expanding the range of available selectors and interactions to cover emerging web technologies.</p><h4>Community and Support</h4><p>The Wallaby community is active and growing, with developers contributing code, documentation, and support through various channels. Key resources include the official Wallaby GitHub repository, community forums, and chat groups where users can seek help, share experiences, and collaborate on improvements. Regular updates and responsive maintainers ensure that the community’s needs are addressed promptly.</p><h4>Potential Integrations</h4><p>Future integrations may see Wallaby working seamlessly with other Elixir tools and frameworks, such as integration with popular CI/CD pipelines, support for additional testing frameworks beyond ExUnit, and compatibility with front-end testing libraries. These integrations will broaden Wallaby’s applicability and make it even more versatile for diverse testing scenarios.</p><h3>13. Conclusion</h3><h4>Recap of Wallaby’s Benefits</h4><p>Wallaby is like that friend who just <em>gets</em> you. It fits perfectly into your Elixir projects, making testing feel less like a chore and more like a quiet triumph. By automating the nitty-gritty of browser interactions and offering tools to catch bugs with the precision of a detective finding the last cookie crumb, Wallaby keeps your app in line without breaking a sweat. It’s not flashy, but when your tests are running smoothly, you’ll realize Wallaby is the MVP you didn’t know you drafted.</p><h4>Why You Should Give Wallaby a Go</h4><p>Testing can feel like herding cats — unpredictable, exhausting, and occasionally requiring you to question your life choices. Wallaby takes that chaos and turns it into something manageable. Its Elixir-first design means it just works — like it’s been there all along, waiting for you to notice. And when you see how it handles concurrency with elegance and keeps your app’s quirks in check, you’ll wonder how you ever got by without it.</p><h4>Your Wallaby Moment Awaits</h4><p>Wallaby doesn’t demand a grand introduction. It’s more of a quiet helper, waiting for you to try it and then casually transforming how you approach browser testing. Set it up, run your first test, and you’ll feel it — that little <em>aha</em> moment when everything clicks. Wallaby doesn’t just help your app work — it helps <em>you</em> work better. Why not take the leap? After all, the best tools don’t scream for attention; they just make everything easier.</p><h3>References</h3><ul><li><a href="https://hexdocs.pm/wallaby/readme.html">Introduction - wallaby v0.30.9</a></li><li><a href="https://github.com/elixir-wallaby/wallaby">GitHub - elixir-wallaby/wallaby: Concurrent browser tests for your Elixir web apps.</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=64f97c0a7d2b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Keep Calm and Let Oban Handle Your Elixir Background Jobs]]></title>
            <link>https://medium.com/@jonnyeberhardt7/keep-calm-and-let-oban-handle-your-elixir-background-jobs-67e4f04d7522?source=rss-200a8333555e------2</link>
            <guid isPermaLink="false">https://medium.com/p/67e4f04d7522</guid>
            <category><![CDATA[background-jobs]]></category>
            <category><![CDATA[phoenix-framework]]></category>
            <category><![CDATA[oban]]></category>
            <category><![CDATA[phoenix]]></category>
            <category><![CDATA[elixir]]></category>
            <dc:creator><![CDATA[Jonny Eberhardt]]></dc:creator>
            <pubDate>Sun, 15 Dec 2024 02:54:54 GMT</pubDate>
            <atom:updated>2024-12-15T02:54:54.997Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LrqI7TmbkTb_ESCy_ejDWQ.jpeg" /></figure><p>Handling background tasks in modern web apps is like being asked to spin plates while coding blindfolded — it’s messy, stressful, and you’ll probably break something. That’s where <strong>Oban</strong> comes in, strutting onto the scene with PostgreSQL swagger and a promise to clean up your act. If you’ve ever muttered, “Why won’t these jobs just work?” while staring at a failed queue, this is for you. Let’s talk about how Oban makes your life easier, your app smarter, and your PostgreSQL work harder than you do.</p><h3>1. What is Oban?</h3><p><strong>Oban</strong> is a sophisticated background job processing library designed specifically for the Elixir programming language. Built atop PostgreSQL, Oban leverages the database’s strengths to provide persistence, reliability, and distributed job processing. While it draws inspiration from popular libraries like Sidekiq (Ruby) and ActiveJob (Rails), Oban is uniquely optimized for Elixir and the Open Telecom Platform (OTP), ensuring seamless integration and performance within Elixir applications.</p><p>Key Features:</p><ul><li><strong>Persistence:</strong> Jobs are stored in PostgreSQL, ensuring they survive application restarts and failures.</li><li><strong>Reliability:</strong> Robust mechanisms for retries and error handling.</li><li><strong>Distributed Processing:</strong> Efficiently handles job processing across multiple nodes in a cluster, enhancing scalability and fault tolerance.</li></ul><h3>2. Why Use Oban?</h3><p>Oban is versatile and caters to a wide range of use cases, making it an invaluable tool for Elixir developers. Here are some common scenarios and the benefits Oban brings to the table:</p><h3>Common Use Cases:</h3><ul><li><strong>Asynchronous Task Execution:</strong> Ideal for tasks that are time-consuming or need to run outside the main request-response cycle, such as sending emails, generating reports, or processing webhooks.</li><li><strong>Job Persistence and Retries:</strong> Ensures that jobs are not lost and are retried in case of failures, enhancing the reliability of your application.</li><li><strong>Distributed Job Processing:</strong> Facilitates the handling of jobs across multiple nodes in a cluster, which is essential for applications that require high availability and scalability.</li></ul><h3>Benefits:</h3><ul><li><strong>Reliability:</strong> Leveraging PostgreSQL for job storage ensures that jobs are persisted reliably, minimizing the risk of data loss.</li><li><strong>Scalability:</strong> Supports horizontal partitioning and distributed processing, allowing your application to scale seamlessly as demand grows.</li><li><strong>Observability:</strong> Comes with a built-in dashboard that provides real-time insights into job statuses, facilitating monitoring and debugging.</li></ul><h3>3. How Oban Works</h3><p>Understanding the core concepts of Oban is essential for effectively utilizing its capabilities. Here’s a breakdown of how Oban operates:</p><h3>Core Concepts:</h3><ul><li><strong>Jobs:</strong> The fundamental units of work that Oban processes. Each job represents a task that needs to be executed.</li><li><strong>Queues:</strong> Jobs are organized into queues based on their type or priority. This grouping allows for efficient management and prioritization of tasks.</li><li><strong>Workers:</strong> Modules that implement the perform/1 callback to process jobs. Workers contain the logic that defines what each job does.</li><li><strong>Execution:</strong> When a job is created, it is inserted into a PostgreSQL table. Workers then fetch these jobs and execute them.</li><li><strong>Retries:</strong> If a job fails, Oban automatically retries it using an exponential backoff strategy by default, enhancing fault tolerance.</li></ul><h3>Execution Flow:</h3><ol><li><strong>Job Creation:</strong> A job is defined and enqueued into a specific queue.</li><li><strong>Job Storage:</strong> The job is stored in a PostgreSQL table, ensuring persistence.</li><li><strong>Job Processing:</strong> Workers poll the database, fetch jobs from their designated queues, and execute the perform/1 callback.</li><li><strong>Error Handling:</strong> In case of failure, the job is retried based on the configured retry strategy.</li></ol><h3>4. Getting Started</h3><p>Setting up Oban in your Elixir project is straightforward. Here’s a step-by-step guide to help you integrate Oban into your application.</p><h3>Setup:</h3><p><strong>Adding Oban to Your Project:</strong> Add Oban to your mix.exs dependencies:</p><pre>def deps do   <br>  [     <br>    {:oban, &quot;~&gt; 2.14&quot;}   <br>  ] <br>end</pre><p>Then, run mix deps.get to fetch the dependency.</p><p><strong>Running Migrations:</strong> Oban requires a specific table structure in PostgreSQL to store jobs. Run the following command to generate the necessary migration:</p><pre>mix oban.install</pre><p>Then, apply the migration with:</p><pre>mix ecto.migrate</pre><h3>Basic Configuration:</h3><p>Configure Oban in your config/config.exs file:</p><pre>config :my_app, Oban,<br>  repo: MyApp.Repo,<br>  queues: [default: 10],<br>  plugins: [Oban.Plugins.Pruner]</pre><ul><li><strong>repo:</strong> Your Ecto repository.</li><li><strong>queues:</strong> Defines the queues and the number of concurrent jobs each can handle.</li><li><strong>plugins:</strong> Optional plugins for additional functionality, such as pruning old jobs.</li></ul><h3>Creating a Worker:</h3><p>Define a worker module that will handle specific jobs. Here’s an example:</p><pre>defmodule MyApp.Workers.ExampleWorker do<br>  use Oban.Worker, queue: :default, max_attempts: 3<br>  <br>  @impl Oban.Worker<br>  def perform(%Oban.Job{args: %{&quot;id&quot; =&gt; id}}) do<br>    # Your job logic here<br>    IO.puts(&quot;Processing job with ID #{id}&quot;)<br>    :ok<br>  end<br><br>end</pre><ul><li><strong>use Oban.Worker:</strong> Sets up the module as an Oban worker.</li><li><strong>queue:</strong> Specifies the queue the worker listens to.</li><li><strong>max_attempts:</strong> The maximum number of retry attempts for a failed job.</li><li><strong>perform/1:</strong> The callback where the job’s logic is implemented.</li></ul><h3>5. Enqueueing Jobs</h3><p>Once you have your worker defined, you can enqueue jobs to be processed. Here’s how to do it:</p><pre>MyApp.Workers.ExampleWorker.new(%{&quot;id&quot; =&gt; 123})<br>|&gt; Oban.insert()</pre><ul><li><strong>new/1:</strong> Creates a new job with the given arguments.</li><li><strong>Oban.insert/1:</strong> Inserts the job into the PostgreSQL job table, making it available for workers to process.</li></ul><h3>6. Advanced Features</h3><p>Oban offers a range of advanced features that enhance its flexibility and power:</p><h3>Scheduled Jobs:</h3><p>Allows you to delay the execution of a job until a specified future time.</p><pre>%{<br>  &quot;id&quot; =&gt; 123<br>}<br>|&gt; MyApp.Workers.ExampleWorker.new(schedule_in: 60) # Schedules the job to run after 60 seconds<br>|&gt; Oban.insert()</pre><h3>Priority Queues:</h3><p>Assign different levels of importance to jobs by creating multiple queues with varying priorities.</p><pre>config :my_app, Oban,<br>  queues: [high: 5, medium: 10, low: 15]</pre><h3>Rate Limiting:</h3><p>Control the number of jobs processed per unit of time to prevent overloading resources.</p><pre>use Oban.Worker,<br>  queue: :default,<br>  max_attempts: 3,<br>  rate_limit: [limit: 100, period: :second]</pre><h3>Concurrency:</h3><p>Run multiple jobs in parallel by configuring the number of concurrent workers for each queue.</p><pre>config :my_app, Oban,<br>  queues: [default: 10, mailers: 5]</pre><h3>Unique Jobs:</h3><p>Prevent duplicate job execution by ensuring that only one instance of a unique job exists at any given time.</p><pre>defmodule MyApp.Workers.UniqueWorker do<br>  use Oban.Worker,<br>    queue: :default,<br>    unique: [fields: [:args], period: 60]<br>  <br>  @impl Oban.Worker<br>  def perform(%Oban.Job{args: args}) do<br>    # Job logic<br>    :ok<br>  end<br>end</pre><h3>7. Monitoring and Observability</h3><p>Effective monitoring is essential to maintain the health and performance of your background jobs. Oban provides a comprehensive Web UI for tracking and managing jobs.</p><h3>Oban Web UI:</h3><p>The Oban Web UI offers real-time insights into job statuses, including:</p><ul><li><strong>Completed Jobs:</strong> Successfully processed jobs.</li><li><strong>Running Jobs:</strong> Jobs currently being executed.</li><li><strong>Failed Jobs:</strong> Jobs that have failed and are awaiting retries.</li><li><strong>Scheduled Jobs:</strong> Jobs scheduled for future execution.</li></ul><h3>Features:</h3><ul><li><strong>Logs and Retry Details:</strong> View detailed logs and reasons for job failures.</li><li><strong>Queue Monitoring:</strong> Real-time visualization of queues, helping you identify bottlenecks.</li><li><strong>Dashboard Installation:</strong> To set up the Oban Web UI, you can integrate it with your existing Phoenix application:</li></ul><pre>defmodule MyAppWeb.Router do   <br>  use MyAppWeb, :router    <br>  <br>  pipeline :browser do     <br>    # ...   <br>  end    <br><br>  scope &quot;/&quot; do     <br>    pipe_through :browser      <br>    # Oban Web Dashboard     <br>    forward &quot;/oban&quot;, Oban.Web.Plug, repo: MyApp.Repo   <br>  end <br>end</pre><ul><li>Access the dashboard by navigating to /oban in your application.</li></ul><h3>8. Common Pitfalls and Best Practices</h3><p>While Oban is a robust library, being aware of common pitfalls and adhering to best practices can ensure smooth operation.</p><h3>Common Pitfalls:</h3><ul><li><strong>Non-Idempotent Workers:</strong> If a job is retried, ensure that the worker’s logic is idempotent to prevent unintended side effects.</li><li><strong>Large Job Payloads:</strong> Avoid passing excessively large arguments in job payloads, as this can increase database overhead and impact performance.</li><li><strong>Lack of Testing:</strong> Failing to thoroughly test jobs in both local and CI environments can lead to unexpected failures in production.</li></ul><h3>Best Practices:</h3><ul><li><strong>Idempotent Logic:</strong> Design your workers so that executing the same job multiple times does not produce adverse effects.</li><li><strong>Optimize Payloads:</strong> Keep job arguments concise and relevant to minimize database load.</li><li><strong>Comprehensive Testing:</strong> Implement unit and integration tests for your workers to ensure reliability.</li><li><strong>Monitoring and Alerts:</strong> Set up alerts based on the Oban Web UI metrics to proactively address issues.</li></ul><h3>9. Comparison with Alternatives</h3><p>Oban is not the only background job processing solution available for Elixir. Here’s how it stacks up against some popular alternatives:</p><h3>GenStage/Flow:</h3><ul><li><strong>Use Case:</strong> Designed for building data pipelines and handling streams of data.</li><li><strong>Comparison:</strong> While GenStage and Flow are excellent for real-time data processing and backpressure management, Oban is more suited for discrete job processing with built-in persistence and retry mechanisms.</li></ul><h3>Task.Supervisor:</h3><ul><li><strong>Use Case:</strong> Manages lightweight, short-lived background tasks.</li><li><strong>Comparison:</strong> Task.Supervisor is ideal for simple, non-persistent tasks that don’t require retries or monitoring. Oban, on the other hand, offers persistent job storage, retries, and a rich set of features for managing complex background processing needs.</li></ul><h3>Sidekiq (Ruby):</h3><ul><li><strong>Use Case:</strong> A popular background job processor for Ruby applications.</li><li><strong>Comparison:</strong> Oban is inspired by Sidekiq but tailored for Elixir and OTP. Both offer similar features like job persistence, retries, and concurrency, but Oban integrates more naturally with Elixir’s ecosystem and leverages PostgreSQL for storage.</li></ul><h3>10. Conclusion</h3><p>Oban isn’t just a background job library — it’s the MVP of your Elixir stack, quietly cleaning up your messes while you take the glory. With PostgreSQL as its ride-or-die, Oban locks down job persistence, retries like a pro, and handles distributed processing so smoothly you’ll wonder if it’s part magician. Toss in features like rate limiting, unique jobs, and a Web UI that doesn’t require a PhD to understand, and you’ve got a library that’s not just reliable — it’s downright smug about it.</p><p>And the best part? Oban’s just getting started. With plans for more plugins and cluster-scaling magic, it’s gearing up to be the background job library that makes everyone else look like amateurs.</p><p>Whether you’re sending a few emails or choreographing a distributed system ballet, Oban’s the tool you didn’t know you needed but can’t live without. Just don’t forget to thank it while you’re basking in the compliments.</p><h3>Resources</h3><ul><li><a href="https://hexdocs.pm/oban/Oban.html">Oban - Oban v2.18.3</a></li><li><a href="https://github.com/oban-bg/oban">GitHub - oban-bg/oban: 💎 Robust job processing in Elixir, backed by modern PostgreSQL and SQLite3</a></li></ul><p>For anyone who wants to learn how to test with Oban:</p><p><a href="https://www.youtube.com/watch?v=PZ48omi0NKU">https://www.youtube.com/watch?v=PZ48omi0NKU</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=67e4f04d7522" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Optimizing ActiveRecord and Ecto: Strategies to Eliminate the N+1 Query Problem]]></title>
            <link>https://medium.com/@jonnyeberhardt7/optimizing-activerecord-and-ecto-strategies-to-eliminate-the-n-1-query-problem-19b3b71e1a1a?source=rss-200a8333555e------2</link>
            <guid isPermaLink="false">https://medium.com/p/19b3b71e1a1a</guid>
            <category><![CDATA[activerecord]]></category>
            <category><![CDATA[elixir]]></category>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[ecto]]></category>
            <category><![CDATA[rails]]></category>
            <dc:creator><![CDATA[Jonny Eberhardt]]></dc:creator>
            <pubDate>Wed, 11 Dec 2024 08:04:55 GMT</pubDate>
            <atom:updated>2024-12-11T08:04:55.468Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GFr1ihW00yDCMcDUt9WJ8Q.jpeg" /></figure><p>The N+1 query problem is a common performance pitfall in database-driven applications. It occurs when an application makes one query to fetch a set of records (the “1”) and then makes additional queries for each record to fetch related data (the “N”). This can lead to significant performance degradation, especially as the number of records increases.</p><p>In this article, we’ll explore what the N+1 problem is, why it matters, and how to prevent it in two popular ORM (Object-Relational Mapping) frameworks: <strong>ActiveRecord</strong> (used in Ruby on Rails) and <strong>Ecto</strong> (used in Elixir).</p><h3>Understanding the N+1 Query Problem</h3><h3>What is the N+1 Problem?</h3><p>Imagine you have two related models: Author and Book. Each author has many books. If you want to display a list of authors along with their books, an inefficient approach might look like this:</p><ol><li><strong>1 Query</strong>: Fetch all authors.</li><li><strong>N Queries</strong>: For each author, fetch their books.</li></ol><p>If there are 100 authors, this approach results in <strong>101 queries</strong>. This pattern can severely impact the performance of your application, especially with larger datasets.</p><h3>Why is it a Problem?</h3><ul><li><strong>Performance Degradation</strong>: Each additional query adds latency, which can slow down the response time of your application.</li><li><strong>Increased Load on the Database</strong>: More queries mean more work for the database, potentially leading to scalability issues.</li><li><strong>Resource Consumption</strong>: Excessive queries consume more memory and CPU resources on both the application and database servers.</li></ul><h3>Preventing N+1 Queries in ActiveRecord</h3><p>ActiveRecord, the default ORM in Ruby on Rails, provides several mechanisms to prevent the N+1 query problem. The primary tools are <strong>eager loading</strong> methods like includes, preload, and eager_load.</p><h3>1. Using includes</h3><p>The includes method is the most straightforward way to perform eager loading in ActiveRecord. It tells ActiveRecord to load the associated records in as few queries as possible.</p><p><strong>Example Scenario:</strong></p><p>Suppose you have the following models:</p><pre>class Author &lt; ApplicationRecord<br>  has_many :books<br>end<br><br>class Book &lt; ApplicationRecord<br>  belongs_to :author<br>end</pre><p><strong>Inefficient Code (N+1 Problem):</strong></p><pre>authors = Author.all<br>authors.each do |author|<br>  puts &quot;#{author.name} has written #{author.books.count} books.&quot;<br>end</pre><p><strong>Optimized Code with </strong><strong>includes:</strong></p><pre>authors = Author.includes(:books).all<br>authors.each do |author|<br>  puts &quot;#{author.name} has written #{author.books.size} books.&quot;<br>end</pre><p><strong>Explanation:</strong></p><ul><li>Author.includes(:books).all loads all authors and their associated books in <strong>two queries</strong>:</li></ul><ol><li>One query to fetch all authors.</li><li>One query to fetch all books where the author_id matches any of the fetched authors.</li></ol><h3>2. Using preload and eager_load</h3><p>While includes is generally sufficient, preload and eager_load offer more control.</p><ul><li><strong>preload</strong>: Similar to includes, it performs separate queries for each association. Use it when you don&#39;t need to perform complex joins or conditions on the associated records.</li></ul><pre>authors = Author.preload(:books).all</pre><ul><li><strong>eager_load</strong>: Performs a single query using LEFT OUTER JOIN to load the associations. Use it when you need to filter or sort based on the associated records.</li></ul><pre>authors = Author.eager_load(:books).where(books: { published: true }).distinct</pre><h3>3. Avoiding Unnecessary Queries</h3><p>Sometimes, the N+1 problem arises from accessing associations in views or serializers without eager loading.</p><p><strong>Best Practices:</strong></p><ul><li><strong>Eager Load in Controllers</strong>: Ensure that associations are eager loaded in the controller actions that render views or APIs.</li></ul><pre>def index   <br>  @authors = Author.includes(:books).all <br>end</pre><ul><li><strong>Use Bullet Gem</strong>: Integrate tools like the <a href="https://github.com/flyerhzm/bullet">Bullet gem</a> in your development environment to detect N+1 queries and other inefficiencies.</li></ul><h3>Preventing N+1 Queries in Ecto</h3><p>Ecto is a database wrapper and query generator for Elixir, commonly used with the Phoenix framework. Like ActiveRecord, Ecto provides mechanisms to preload associations and prevent the N+1 query problem.</p><h3>1. Using preload</h3><p>Ecto’s preload function is the primary way to eager load associations.</p><p><strong>Example Scenario:</strong></p><p>Suppose you have the following schemas:</p><pre>defmodule MyApp.Author do<br>  use Ecto.Schema<br><br>  schema &quot;authors&quot; do<br>    field :name, :string<br>    has_many :books, MyApp.Book<br>  end<br>end<br><br>defmodule MyApp.Book do<br>  use Ecto.Schema<br><br>  schema &quot;books&quot; do<br>    field :title, :string<br>    belongs_to :author, MyApp.Author<br>  end<br>end</pre><p><strong>Inefficient Code (N+1 Problem):</strong></p><pre>authors = Repo.all(Author)<br><br>for author &lt;- authors do<br>  IO.puts(&quot;#{author.name} has written #{length(author.books)} books.&quot;)<br>end</pre><p><strong>Optimized Code with </strong><strong>preload:</strong></p><pre>authors = Repo.all(Author) |&gt; Repo.preload(:books)<br><br>for author &lt;- authors do<br>  IO.puts(&quot;#{author.name} has written #{length(author.books)} books.&quot;)<br>end</pre><p><strong>Explanation:</strong></p><ul><li>Repo.preload(:books) loads all books associated with the fetched authors in a separate query, resulting in <strong>two queries</strong> regardless of the number of authors.</li></ul><h3>2. Using join and select</h3><p>For more complex queries where you need to filter or sort based on associated records, using join with select can be more efficient.</p><pre>import Ecto.Query<br><br>query =<br>  from a in Author,<br>    join: b in assoc(a, :books),<br>    where: b.published == true,<br>    preload: [books: b]<br><br>authors = Repo.all(query)</pre><p><strong>Explanation:</strong></p><ul><li>This approach uses a JOIN to fetch authors and their published books in a single query.</li><li>It preloads only the books that match the condition (b.published == true), reducing the amount of data loaded.</li></ul><h3>3. Using Ecto.Multi for Complex Preloads</h3><p>For scenarios where multiple associations or complex loading logic is required, Ecto.Multi can help manage multiple queries efficiently.</p><pre>import Ecto.Multi<br><br>multi =<br>  Multi.new()<br>  |&gt; Multi.run(:authors, fn repo, _changes -&gt;<br>    repo.all(Author)<br>  end)<br>  |&gt; Multi.run(:books, fn repo, %{authors: authors} -&gt;<br>    author_ids = Enum.map(authors, &amp; &amp;1.id)<br>    repo.all(from b in Book, where: b.author_id in ^author_ids)<br>  end)<br>  |&gt; Multi.run(:preloaded_authors, fn _repo, %{authors: authors, books: books} -&gt;<br>    preloaded = Enum.map(authors, fn author -&gt;<br>      %{author | books: Enum.filter(books, &amp;(&amp;1.author_id == author.id))}<br>    end)<br>    {:ok, preloaded}<br>  end)<br><br>case Repo.transaction(multi) do<br>  {:ok, %{preloaded_authors: preloaded_authors}} -&gt;<br>    preloaded_authors<br>  <br>  {:error, _step, reason, _changes} -&gt;<br>    # Handle error<br>end</pre><p><strong>Explanation:</strong></p><ul><li>Ecto.Multi allows batching multiple operations into a single transaction.</li><li>This example fetches authors and their books in two separate queries but manages them efficiently within a transaction.</li></ul><h3>4. Avoiding Preload in Views or Serializers</h3><p>Similar to ActiveRecord, ensure that all necessary associations are preloaded before passing data to views or serializers to prevent implicit loading and additional queries.</p><p><strong>Best Practices:</strong></p><ul><li><strong>Preload in Context Modules</strong>: In Phoenix applications, use context modules to handle data fetching and ensure preloading is done before data reaches the view layer.</li></ul><pre>def list_authors do   <br>  Repo.all(Author) |&gt; Repo.preload(:books) <br>end</pre><ul><li><strong>Use Tools for Detection</strong>: While Ecto doesn’t have an exact equivalent to the Bullet gem, you can use logging and monitoring to detect unexpected queries.</li></ul><h3>Conclusion</h3><p>The N+1 query problem is a significant performance concern in applications that interact with databases. Both ActiveRecord and Ecto provide robust mechanisms to prevent this issue through eager loading of associations. By understanding and implementing these techniques, developers can ensure their applications remain efficient and scalable.</p><p><strong>Key Takeaways:</strong></p><ul><li><strong>Eager Loading</strong>: Use includes in ActiveRecord and preload in Ecto to load associations upfront.</li><li><strong>Selective Loading</strong>: Use preload, join, and select to load only necessary data.</li><li><strong>Tooling</strong>: Utilize gems like Bullet in Rails and logging in Elixir to detect and monitor N+1 queries.</li><li><strong>Best Practices</strong>: Preload associations in controllers or context modules to keep data access efficient.</li></ul><p>By proactively addressing the N+1 query problem, you can significantly improve the performance and scalability of your Ruby on Rails and Elixir applications.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=19b3b71e1a1a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to Phoenix LiveView: A Guide for React Developers]]></title>
            <link>https://medium.com/@jonnyeberhardt7/introduction-to-phoenix-liveview-a-guide-for-react-developers-10710829720e?source=rss-200a8333555e------2</link>
            <guid isPermaLink="false">https://medium.com/p/10710829720e</guid>
            <category><![CDATA[elixir]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[phoenix-framework]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[phoenix]]></category>
            <dc:creator><![CDATA[Jonny Eberhardt]]></dc:creator>
            <pubDate>Tue, 10 Dec 2024 00:08:57 GMT</pubDate>
            <atom:updated>2024-12-10T00:08:57.504Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pjAihFP6uVoFLU8J0SgcVw.jpeg" /></figure><p>In the ever-evolving landscape of web development, React has long been a go-to library for building dynamic user interfaces. However, the advent of <strong>Phoenix LiveView</strong> introduces a compelling alternative for creating interactive, real-time web applications with less client-side complexity. If you’re a React developer curious about what LiveView offers and how it contrasts with your current toolkit, this guide will walk you through the basics of LiveView, highlighting its core concepts and benefits.</p><h3>What is Phoenix LiveView?</h3><p>Phoenix LiveView is a library for the Elixir web framework Phoenix that enables developers to build rich, interactive user interfaces without writing significant amounts of JavaScript. LiveView leverages WebSockets to maintain a persistent connection between the client and server, allowing real-time updates to the UI as server-side state changes.</p><p>Key highlights of LiveView:</p><ul><li><strong>Server-Side Rendering:</strong> HTML is rendered on the server and sent to the client.</li><li><strong>Real-Time Interactions:</strong> Uses WebSockets for bidirectional communication.</li><li><strong>Minimal JavaScript:</strong> Reduces the need for complex frontend codebases.</li></ul><h3>Key Concepts of LiveView</h3><h3>Server-Rendered HTML</h3><p>Unlike React, which primarily handles rendering on the client side, LiveView renders HTML on the server. When a user interacts with the application, events are sent to the server, which processes them and sends back updated HTML snippets to the client. This approach ensures that the server remains the single source of truth for the UI state.</p><h3>Real-Time Updates with WebSockets</h3><p>LiveView maintains a persistent WebSocket connection between the client and server. This connection allows for real-time communication, enabling LiveView to push updates to the client instantly as server-side events occur. This mechanism facilitates features like live dashboards, real-time forms, and dynamic content updates without page reloads.</p><h3>Declarative UI with Elixir</h3><p>LiveView leverages Elixir’s expressive syntax to define UI components declaratively. Developers write Elixir modules that handle rendering and event handling, making the codebase concise and maintainable. The declarative nature aligns with React’s component-based architecture but centralizes logic on the server.</p><h3>LiveView vs. React: A Comparative Overview</h3><h3>Rendering Approach</h3><ul><li><strong>React:</strong> Client-side rendering using a virtual DOM. React components manage their own state and re-render when state changes.</li><li><strong>LiveView:</strong> Server-side rendering with HTML sent to the client. Updates are sent as patches to the DOM over WebSockets.</li></ul><h3>State Management</h3><ul><li><strong>React:</strong> State is managed within components or via state management libraries like Redux or MobX. State is primarily held on the client.</li><li><strong>LiveView:</strong> State resides on the server. The client UI reflects the server state, ensuring consistency across multiple clients.</li></ul><h3>Performance Considerations</h3><ul><li><strong>React:</strong> Optimized for rich, highly interactive interfaces with complex client-side logic. Performance depends on efficient state management and rendering.</li><li><strong>LiveView:</strong> Offloads rendering to the server, potentially reducing client-side resource usage. However, it relies on server performance and the efficiency of WebSocket communication.</li></ul><h3>Developer Experience</h3><ul><li><strong>React:</strong> Requires proficiency in JavaScript/TypeScript and understanding of the component lifecycle. Often involves managing build tools and frontend dependencies.</li><li><strong>LiveView:</strong> Utilizes Elixir and Phoenix’s conventions. Reduces the need for separate frontend build processes and minimizes JavaScript usage.</li></ul><h3>Building a Simple LiveView Application</h3><p>To illustrate how LiveView works, let’s build a simple counter application similar to a basic React component.</p><h3>Setting Up the Phoenix Project</h3><p>First, ensure you have Elixir and Phoenix installed. Then, create a new Phoenix project:</p><pre>mix phx.new liveview_example --live<br><br>cd liveview_example<br><br>mix ecto.create</pre><h3>Creating a LiveView Module</h3><p>Create a LiveView module to handle the counter logic.</p><pre># lib/liveview_example_web/live/counter_live.ex<br>defmodule LiveviewExampleWeb.CounterLive do<br>  use Phoenix.LiveView<br><br>  def mount(_params, _session, socket) do<br>    {:ok, assign(socket, count: 0)}<br>  end<br><br>  def handle_event(&quot;inc&quot;, _value, socket) do<br>    {:noreply, update(socket, :count, &amp;(&amp;1 + 1))}<br>  end<br><br>  def handle_event(&quot;dec&quot;, _value, socket) do<br>    {:noreply, update(socket, :count, &amp;(&amp;1 - 1))}<br>  end<br><br>  def render(assigns) do<br>    ~H&quot;&quot;&quot;<br>    &lt;div&gt;<br>      &lt;h1&gt;Counter: &lt;%= @count %&gt;&lt;/h1&gt;<br>      &lt;button phx-click=&quot;inc&quot;&gt;Increase&lt;/button&gt;<br>      &lt;button phx-click=&quot;dec&quot;&gt;Decrease&lt;/button&gt;<br>    &lt;/div&gt;<br>    &quot;&quot;&quot;<br>  end<br><br>end</pre><h3>Integrating with Frontend</h3><p>Update the router to include the LiveView route.</p><pre># lib/liveview_example_web/router.ex<br>defmodule LiveviewExampleWeb.Router do<br>  use LiveviewExampleWeb, :router<br><br>  pipeline :browser do<br>    # existing plugs...<br>  end<br><br>  scope &quot;/&quot;, LiveviewExampleWeb do<br>    pipe_through :browser<br>    live &quot;/counter&quot;, CounterLive<br>  end<br><br>end</pre><p>Start the Phoenix server:</p><pre>mix phx.server</pre><p>Navigate to http://localhost:4000/counter to see the LiveView counter in action. Clicking the &quot;Increase&quot; and &quot;Decrease&quot; buttons sends events to the server, which updates the count and pushes the new state to the client in real-time.</p><h3>When to Choose LiveView Over React</h3><p>While React excels in building complex, highly interactive client-side applications, LiveView offers several advantages in specific scenarios:</p><ul><li><strong>Rapid Development:</strong> LiveView allows for building interactive features quickly without managing a separate frontend stack.</li><li><strong>SEO-Friendly:</strong> Server-rendered HTML is more SEO-friendly compared to client-rendered React apps.</li><li><strong>Consistency:</strong> Centralized state management on the server ensures consistency across multiple clients.</li><li><strong>Reduced Complexity:</strong> Minimizes the need for extensive JavaScript, leading to simpler codebases and fewer potential bugs.</li></ul><p>However, for applications requiring intricate client-side interactions, offline capabilities, or leveraging a vast ecosystem of React libraries, React might still be the preferable choice.</p><h3>Conclusion</h3><p>Phoenix LiveView presents a compelling alternative to traditional client-side frameworks like React by shifting much of the rendering and state management to the server. For React developers, LiveView offers an opportunity to leverage Elixir’s strengths in building scalable, real-time applications with less frontend complexity. Understanding LiveView’s server-centric approach, real-time communication via WebSockets, and declarative UI rendering can open new avenues for developing efficient and maintainable web applications.</p><p>Whether you’re looking to simplify your stack, improve SEO, or harness the power of Elixir for real-time features, Phoenix LiveView is worth exploring alongside your existing React expertise.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=10710829720e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Simple DSL for HTML Generation in Elixir]]></title>
            <link>https://medium.com/@jonnyeberhardt7/building-a-simple-dsl-for-html-generation-in-elixir-f7c404b6d0ab?source=rss-200a8333555e------2</link>
            <guid isPermaLink="false">https://medium.com/p/f7c404b6d0ab</guid>
            <category><![CDATA[dsl]]></category>
            <category><![CDATA[metaprogramming]]></category>
            <category><![CDATA[functional-programming]]></category>
            <category><![CDATA[elixir]]></category>
            <category><![CDATA[macro]]></category>
            <dc:creator><![CDATA[Jonny Eberhardt]]></dc:creator>
            <pubDate>Fri, 06 Dec 2024 23:43:02 GMT</pubDate>
            <atom:updated>2024-12-06T23:43:02.553Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L5aGvTWA9v9BpFbAz0YYQg.jpeg" /></figure><p>In software development, repetitive patterns and verbose code often hinder productivity and clarity, especially when working with structured formats like HTML. A Domain-Specific Language (DSL) offers a clean, expressive, and domain-focused abstraction that simplifies such tasks. By designing a DSL, you can reduce boilerplate, improve readability, and create reusable components tailored to specific needs. In the context of HTML generation, a DSL empowers developers to write structured, maintainable code that seamlessly handles dynamic attributes, content nesting, and compliance with valid HTML standards — all while leveraging the power and elegance of Elixir’s metaprogramming capabilities.</p><p>Creating Domain Specific Languages (DSLs) in Elixir is a powerful way to encapsulate complex logic in an intuitive and reusable manner. In this article, we’ll explore how to build a DSL for generating HTML with Elixir. The goal is to create a module that allows us to write clean, structured code for rendering HTML, leveraging Elixir’s metaprogramming capabilities. You can review the code for this article and the testing for it on <a href="https://github.com/jonnycharles/elixir-html-dsl/">https://github.com/jonnycharles/elixir-html-dsl/</a>.</p><h3>Problem: Generating HTML Dynamically</h3><p>When dynamically generating HTML, developers often face issues with readability and maintainability. Instead of hardcoding strings, we want a programmatic way to define HTML elements, attributes, and nested structures.</p><h3>Solution: A DSL for HTML Elements</h3><p>Our DSL will allow us to define and render HTML elements with support for:</p><ul><li>Void elements (e.g., &lt;br&gt;, &lt;img&gt;) that don’t require closing tags.</li><li>Container elements (e.g., &lt;div&gt;, &lt;span&gt;) that support nested content.</li><li>Attributes and their validation.</li><li>Proper escaping for special characters in attributes and content.</li></ul><h3>Step 1: Defining HTML Elements</h3><p>We use a macro, element/1, to dynamically define functions for HTML elements. Each element function supports multiple use cases: with no attributes, with attributes, with content, or with both attributes and content.</p><p>Here’s the core logic:</p><pre>defmacro element(name) do<br>  unless name in @valid_tags do<br>    raise ArgumentError, &quot;Invalid HTML tag: #{name}&quot;<br>  end<br><br>  is_void = name in @void_elements<br>  <br>  function_name = String.to_atom(&quot;html_#{name}&quot;)<br>  <br>  quote do<br>    def unquote(function_name)() do<br>      if unquote(is_void) do<br>        &quot;&lt;#{unquote(name)}/&gt;&quot;<br>      else<br>        &quot;&lt;#{unquote(name)}&gt;&lt;/#{unquote(name)}&gt;&quot;<br>      end<br>    end<br>    <br>    def unquote(function_name)(attrs) when is_list(attrs) do<br>      attrs_string = build_attributes(attrs)<br>      if unquote(is_void) do<br>        &quot;&lt;#{unquote(name)}#{attrs_string}/&gt;&quot;<br>      else<br>        &quot;&lt;#{unquote(name)}#{attrs_string}&gt;&lt;/#{unquote(name)}&gt;&quot;<br>      end<br>    end<br>    <br>    def unquote(function_name)(do: content) do<br>      &quot;&lt;#{unquote(name)}&gt;#{content}&lt;/#{unquote(name)}&gt;&quot;<br>    end<br>    <br>    def unquote(function_name)(attrs, do: content) when is_list(attrs) do<br>      attrs_string = build_attributes(attrs)<br>      &quot;&lt;#{unquote(name)}#{attrs_string}&gt;#{content}&lt;/#{unquote(name)}&gt;&quot;<br>    end<br>  end<br>end</pre><h3>Key Points:</h3><ul><li><strong>Void Elements</strong>: Automatically self-close (e.g., &lt;br/&gt;).</li><li><strong>Attributes</strong>: Generated dynamically from a list of key-value pairs.</li><li><strong>Content</strong>: Nested content is supported via the do block.</li></ul><h3>Step 2: Escaping HTML Attributes</h3><p>Properly escaping special characters in attributes is crucial for security and correctness. The escape/1 function handles this:</p><pre>def escape(value) when is_binary(value) do<br>  value<br>  |&gt; String.replace(&quot;&amp;&quot;, &quot;&amp;amp;&quot;)<br>  |&gt; String.replace(&quot;&lt;&quot;, &quot;&amp;lt;&quot;)<br>  |&gt; String.replace(&quot;&gt;&quot;, &quot;&amp;gt;&quot;)<br>  |&gt; String.replace(&quot;\&quot;&quot;, &quot;&amp;quot;&quot;)<br>  |&gt; String.replace(&quot;&#39;&quot;, &quot;&amp;#39;&quot;)<br>end</pre><h3>Step 3: Using the DSL</h3><p>With our DSL in place, defining HTML elements is straightforward:</p><pre>defmodule HTMLGenerator.MyPage do<br>  require HTMLGenerator.HTML<br><br>  # Define supported HTML elements<br>  HTMLGenerator.HTML.element(:div)<br>  HTMLGenerator.HTML.element(:p)<br>  HTMLGenerator.HTML.element(:img)<br>  HTMLGenerator.HTML.element(:br)<br>  HTMLGenerator.HTML.element(:span)<br>  <br>  def render do<br>    html_div(id: &quot;main&quot;, class: &quot;container&quot;) do<br>      [<br>        html_p() do<br>          &quot;Content inside div&quot;<br>        end,<br>        html_img(src: &quot;/images/logo.png&quot;, alt: &quot;Logo&quot;),<br>        html_br(),<br>        html_span(class: &quot;highlight&quot;) do<br>          &quot;Content inside span&quot;<br>        end<br>      ]<br>      |&gt; Enum.join(&quot;\n  &quot;)<br>    end<br>  end<br>end</pre><h3>Step 4: Testing the DSL</h3><p>Testing ensures correctness and avoids regressions. Here’s an example:</p><pre>defmodule HTMLGenerator.HTMLTest do<br>  use ExUnit.Case<br>  require HTMLGenerator.HTML<br><br>  # Define elements for testing<br>  HTMLGenerator.HTML.element(:div)<br>  HTMLGenerator.HTML.element(:img)<br>  HTMLGenerator.HTML.element(:br)<br>  <br>  test &quot;renders self-closing elements&quot; do<br>    assert html_br() == &quot;&lt;br/&gt;&quot;<br>    assert html_img(src: &quot;test.jpg&quot;) == &quot;&lt;img src=\&quot;test.jpg\&quot;/&gt;&quot;<br>  end<br>  <br>  test &quot;renders container elements&quot; do<br>    assert html_div(class: &quot;container&quot;, do: &quot;Content&quot;) == &quot;&lt;div class=\&quot;container\&quot;&gt;Content&lt;/div&gt;&quot;<br>  end<br>end</pre><h3>Step 5: Example Usage</h3><p>Calling MyPage.render/0 produces the following HTML:</p><pre>&lt;div id=&quot;main&quot; class=&quot;container&quot;&gt;<br>  &lt;p&gt;Content inside div&lt;/p&gt;<br>  &lt;img src=&quot;/images/logo.png&quot; alt=&quot;Logo&quot;/&gt;<br>  &lt;br/&gt;<br>  &lt;span class=&quot;highlight&quot;&gt;Content inside span&lt;/span&gt;<br>&lt;/div&gt;</pre><h3>Conclusion</h3><p>This DSL demonstrates the power of Elixir’s metaprogramming. By abstracting HTML generation into macros, we created a maintainable and extensible tool for dynamically building HTML structures.</p><p>To extend this DSL, you could:</p><ul><li>Add support for more HTML elements.</li><li>Integrate with Phoenix templates.</li><li>Enhance attribute handling with custom validations.</li></ul><p>Elixir’s flexibility makes tasks like this both fun and rewarding. Try building your own DSLs to simplify repetitive tasks in your projects!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f7c404b6d0ab" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Streams, GenStage, and Broadway: Elixir’s Power Trio for Rails Developers]]></title>
            <link>https://medium.com/@jonnyeberhardt7/streams-genstage-and-broadway-elixirs-power-trio-for-rails-developers-42a0498c7706?source=rss-200a8333555e------2</link>
            <guid isPermaLink="false">https://medium.com/p/42a0498c7706</guid>
            <category><![CDATA[genstage]]></category>
            <category><![CDATA[stream]]></category>
            <category><![CDATA[broadway]]></category>
            <category><![CDATA[rails]]></category>
            <category><![CDATA[elixir]]></category>
            <dc:creator><![CDATA[Jonny Eberhardt]]></dc:creator>
            <pubDate>Thu, 05 Dec 2024 12:33:20 GMT</pubDate>
            <atom:updated>2024-12-05T12:33:20.062Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ks8lFFMo33Tm8Sq9WGzWmg.jpeg" /></figure><p><strong>Introduction</strong></p><p>As Rails developers, you’re adept at building robust, scalable web applications using Ruby’s elegant and expressive syntax. However, when it comes to handling concurrent data processing and constructing highly scalable systems, Elixir offers powerful abstractions that can elevate your development game. Two such abstractions are <strong>Elixir Streams</strong> and <strong>GenStage</strong>. This article aims to demystify these tools, drawing parallels to familiar Rails concepts, and help you decide when to use each in your projects.</p><p><strong>Understanding Elixir Streams</strong></p><p>At its core, an Elixir Stream is a lazy enumerator, allowing you to define a series of transformations on a collection without executing them immediately. This is somewhat akin to Ruby’s Enumerator or Enumerable modules but with a key difference: laziness.</p><ul><li><strong>Lazy Evaluation</strong>: Streams defer computation until the result is actually needed. This can lead to significant performance improvements, especially when dealing with large datasets or infinite sequences.</li></ul><pre># Example of a Stream in Elixir <br>1..1_000_000 <br>|&gt; Stream.map(&amp;(&amp;1 * 2)) <br>|&gt; Stream.filter(&amp;Integer.is_even/1) <br>|&gt; Enum.take(10)</pre><ul><li>In this example, the map and filter operations are not executed immediately. Instead, they&#39;re applied as needed when Enum.take/2 requests the first ten elements.</li><li><strong>Comparison with Ruby’s Enumerator/Enumerable</strong>: While the initial comparison to Ruby’s Enumerator and Enumerable is valid, it&#39;s important to note that Ruby also offers lazy enumerators through the lazy method. This allows for similar deferred computations in Ruby, enabling memory-efficient processing of large or infinite datasets.</li></ul><pre># Example of a lazy enumerator in Ruby <br>(1..1_000_000)   <br>  .lazy   <br>  .map { |x| x * 2 }   <br>  .select(&amp;:even?)   <br>  .first(10)</pre><ul><li>This Ruby example mirrors the Elixir Streams example, highlighting that both languages provide mechanisms for lazy evaluation, enhancing their ability to handle large data efficiently.</li><li><strong>Use Cases</strong>: Streams are ideal for scenarios where you have large or potentially infinite data sources and want to process data in a memory-efficient manner. They are perfect for data transformations, file processing, or any situation where you can benefit from pipelined, on-demand computation.</li></ul><p><strong>Diving into GenStage</strong></p><p>GenStage is a more sophisticated abstraction built on top of Elixir’s concurrency primitives. It implements the <strong>producer-consumer</strong> model, allowing you to build complex, concurrent data processing pipelines with built-in backpressure mechanisms.</p><ul><li><strong>Producer-Consumer Model</strong>: Unlike Streams, GenStage is designed for scenarios where multiple stages of data processing are needed, each potentially running in its own process. This model ensures that producers only emit data when consumers are ready to process it, preventing bottlenecks and ensuring efficient resource utilization.</li></ul><pre># Simplified GenStage example <br>defmodule MyProducer do   <br>  use GenStage    <br>  <br>  def start_link(initial) do     <br>    GenStage.start_link(__MODULE__, initial, name: __MODULE__)   <br>  end    <br><br>  def init(state) do     <br>    {:producer, state}   <br>  end    <br><br><br>  def handle_demand(demand, state) do     <br>    # Produce events based on demand     <br>    {:noreply, events, new_state}   <br>  end <br><br>end</pre><ul><li><em>Note: This example is simplified. In a production environment, a GenStage implementation would typically require more comprehensive error handling and integration into a supervision tree to ensure reliability and fault tolerance.</em></li><li><strong>Use Cases</strong>: GenStage shines in building scalable, fault-tolerant data pipelines. Whether you’re handling real-time data streams, implementing backpressure-aware systems, or orchestrating multiple concurrent tasks, GenStage provides the tools to manage complexity effectively.</li></ul><p><strong>GenStage vs. Broadway: The Evolution of Data Processing in Elixir</strong></p><p>While GenStage remains a powerful tool for building concurrent pipelines, it has largely been superseded by <strong>Broadway</strong> for many data processing use cases since its release by the Elixir core team. Broadway builds on top of GenStage, offering a higher-level abstraction that simplifies the creation of data ingestion and processing pipelines.</p><ul><li><strong>Broadway Advantages</strong>:</li><li><strong>Simplified API</strong>: Broadway provides a more straightforward API for common data processing tasks, reducing the boilerplate code required with GenStage.</li><li><strong>Integration with External Systems</strong>: It offers built-in support for common data sources like RabbitMQ, Kafka, and Amazon SQS, making it easier to integrate with existing infrastructure.</li><li><strong>Enhanced Features</strong>: Broadway includes features such as automatic batching, message acknowledgments, and telemetry integration out of the box.</li><li><strong>When to Choose GenStage or Broadway</strong>:</li><li><strong>GenStage</strong>: Opt for GenStage if you need fine-grained control over the data pipeline, are building custom concurrent workflows, or want to implement specialized backpressure mechanisms.</li><li><strong>Broadway</strong>: Choose Broadway for standard data ingestion and processing tasks, especially when integrating with external message brokers or when you prefer a higher-level abstraction that handles many concerns automatically.</li></ul><p><strong>Stream vs. GenStage vs. Broadway: A Comparative Analysis</strong></p><p>While Streams and GenStage (along with Broadway) facilitate data processing, they cater to different needs and complexities.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UzBMim09rWqkhAvP141_VA.png" /></figure><p><strong>When to Use Which?</strong></p><ul><li><strong>Choose Elixir Streams When</strong>:</li><li>You need to process data in a pipeline fashion within a single process.</li><li>Memory efficiency is a concern, and you want to avoid loading entire datasets into memory.</li><li>The processing logic is straightforward without the need for complex concurrency control.</li><li><strong>Choose GenStage When</strong>:</li><li>Building applications that require handling high-throughput, real-time data streams.</li><li>You need fine-grained control over data flow and backpressure.</li><li>The application demands a scalable, concurrent architecture with multiple processing stages.</li><li><strong>Choose Broadway When</strong>:</li><li>You are integrating with external message brokers like RabbitMQ or Kafka.</li><li>You prefer a higher-level abstraction that simplifies common data processing tasks.</li><li>You want built-in features like automatic batching, acknowledgments, and telemetry.</li></ul><p><strong>Practical Examples for Rails Developers</strong></p><ul><li><strong>Using Streams for Data Transformation</strong>:</li><li>Imagine you need to process a large CSV file, transform each row, and store the results. With Elixir Streams, you can efficiently handle this without loading the entire file into memory.</li></ul><pre>File.stream!(&quot;large_file.csv&quot;) <br>|&gt; Stream.map(&amp;transform_row/1) <br>|&gt; Stream.filter(&amp;valid_row?/1) <br>|&gt; Enum.each(&amp;store_in_database/1)</pre><ul><li><em>Equivalent Ruby Code Using Lazy Enumerators</em>:</li></ul><pre>File.foreach(&quot;large_file.csv&quot;).lazy   <br>  .map { |line| transform_row(line) }   <br>  .select { |row| valid_row?(row) }   <br>  .each { |row| store_in_database(row) }</pre><ul><li><strong>Using GenStage for Real-Time Event Processing</strong>:</li><li>Suppose you’re building a real-time analytics dashboard that processes user events as they happen. GenStage can help you build a pipeline where events are produced, processed, and aggregated concurrently, ensuring that the system remains responsive under high load.</li></ul><pre>defmodule EventProducer do   <br>  use GenStage   <br>  # Implementation... <br>end  <br><br>defmodule EventProcessor do   <br>  use GenStage   <br>  # Implementation... <br>end  <br><br>defmodule EventAggregator do   <br>  use GenStage   <br>  # Implementation... <br>end  <br><br># Setting up the pipeline <br>{:ok, producer} = EventProducer.start_link() <br>{:ok, processor} = EventProcessor.start_link() <br>{:ok, aggregator} = EventAggregator.start_link()  <br><br>GenStage.sync_subscribe(processor, to: producer) <br>GenStage.sync_subscribe(aggregator, to: processor)</pre><ul><li><em>Note: In a production setting, you would integrate these GenStage components into a supervision tree to handle failures gracefully and ensure system resilience.</em></li><li><strong>Using Broadway for Message-Driven Data Processing</strong>:</li><li>If you’re integrating with a message broker like RabbitMQ to process user notifications, Broadway can streamline the setup.</li></ul><pre>defmodule NotificationProcessor do   <br>  use Broadway    <br><br>  Broadway.setup do     <br>    producer = BroadwayRabbitMQ.Producer     <br>    Broadway.consume(       <br>      :rabbitmq,       <br>      producer,       <br>      queue: &quot;notifications&quot;,       <br>      connection: [host: &quot;localhost&quot;]     <br>    )   end    <br><br><br>  def handle_message(_, message, _) do     <br>    # Process the notification     <br>    process_notification(message.data)     <br>    message   <br>  end <br>end  <br><br># Start the Broadway pipeline <br>{:ok, _pid} = NotificationProcessor.start_link()</pre><ul><li>This example demonstrates how Broadway abstracts away much of the boilerplate involved in setting up a GenStage pipeline, providing a clean and efficient way to handle message-driven processing.</li></ul><p><strong>Conclusion</strong></p><p>Elixir Streams, GenStage, and Broadway offer a spectrum of tools for data processing and concurrency, each suited to different aspects of application development. For Rails developers venturing into Elixir, Streams provide a familiar and straightforward way to handle data transformations efficiently, akin to Ruby’s lazy enumerators. GenStage offers more granular control for building concurrent pipelines, while Broadway simplifies message-driven data processing by building on top of GenStage.</p><p>By understanding the strengths and appropriate use cases of each, you can leverage Elixir’s capabilities to complement your Rails applications, creating systems that are both robust and performant. Whether you’re optimizing data pipelines with Streams, architecting real-time processing systems with GenStage, or managing message-driven workflows with Broadway, Elixir has the abstractions to meet your development needs.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=42a0498c7706" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Rebar3 Unleashed: The Ultimate Build Tool for Erlang Developers]]></title>
            <link>https://medium.com/@jonnyeberhardt7/rebar3-unleashed-the-ultimate-build-tool-for-erlang-developers-726866391a14?source=rss-200a8333555e------2</link>
            <guid isPermaLink="false">https://medium.com/p/726866391a14</guid>
            <category><![CDATA[elixir]]></category>
            <category><![CDATA[beam]]></category>
            <category><![CDATA[build-tool]]></category>
            <category><![CDATA[erlang]]></category>
            <dc:creator><![CDATA[Jonny Eberhardt]]></dc:creator>
            <pubDate>Wed, 04 Dec 2024 07:35:15 GMT</pubDate>
            <atom:updated>2024-12-04T18:10:26.322Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RBatnNqpVtJozSZkBVvTCg.jpeg" /></figure><p>In the realm of concurrent and distributed systems, Erlang and Elixir stand out for their robustness, scalability, and fault-tolerance. Central to the success of applications built with these languages is the efficiency and reliability of their build tools. <strong>Rebar3</strong> has emerged as the premier build tool for Erlang projects, streamlining the development workflow and enhancing productivity. This article delves into the intricacies of Rebar3, exploring its features, benefits, usage, and its pivotal role in the Erlang ecosystem. (Please note that some examples may be outdated in terms of the release software, so refer to documentation if it is unclear).</p><h3>Introduction to Rebar3</h3><p><strong>Rebar3</strong> is an advanced build tool designed primarily for Erlang projects. As a successor to Rebar and Rebar2, Rebar3 introduces numerous enhancements that cater to modern development needs. It automates various aspects of the build process, including compilation, testing, dependency management, and release creation, thereby enabling developers to focus more on writing quality code and less on managing the build lifecycle.</p><p>Rebar3 seamlessly integrates with both Erlang and Elixir (although it is primarily for Erlang), making it a versatile choice for projects that leverage the strengths of both languages. Its emphasis on simplicity, flexibility, and extensibility has solidified its position as the go-to build tool in the BEAM ecosystem.</p><h3>Key Features of Rebar3</h3><p>Rebar3 is packed with features that cater to the diverse needs of developers working with Erlang and Elixir. Some of its standout features include:</p><ul><li><strong>Automated Compilation</strong>: Efficiently compiles source code, handling complex dependencies and ensuring correct build order.</li><li><strong>Dependency Management</strong>: Integrates with Hex and other repositories to fetch and manage external libraries seamlessly.</li><li><strong>Testing Support</strong>: Provides robust testing frameworks integration, enabling developers to write and run tests effortlessly.</li><li><strong>Release Handling</strong>: Facilitates the creation of releases, bundling applications for deployment with ease.</li><li><strong>Plugin Architecture</strong>: Extensible via plugins, allowing customization and enhancement of its core functionalities.</li><li><strong>Cross-Platform Support</strong>: Works consistently across different operating systems, ensuring a uniform development experience.</li><li><strong>Built-In Documentation Generation</strong>: Automatically generates documentation, promoting better code understanding and maintenance.</li></ul><h3>Getting Started with Rebar3</h3><h3>Creating a New Project</h3><p>Rebar3 simplifies project creation with its built-in templates.</p><pre>rebar3 new app my_app</pre><p>This command generates a new Erlang application named my_app with a standard directory structure, including folders for source code, tests, and configuration.</p><h3>Building the Project</h3><p>Navigate to your project directory and run:</p><pre>cd my_app<br>rebar3 compile</pre><p>Rebar3 will compile the source code, handling any dependencies specified in the configuration.</p><h3>Core Components and Configuration</h3><p>Rebar3’s configuration is primarily managed through the rebar.config file located at the root of the project. This file dictates how Rebar3 should build the project, manage dependencies, and handle other tasks.</p><h3>rebar.config Structure</h3><p>A typical rebar.config might look like this:</p><pre>{erl_opts, [debug_info, {parse_transform, lager_transform}]}.</pre><pre>{deps, [<br>    {cowboy, &quot;~&gt; 2.9&quot;},<br>    {jiffy, &quot;~&gt; 1.0&quot;}<br>]}.</pre><pre>{plugins, [<br>    {rebar3_hex, &quot;6.14.0&quot;}<br>]}.</pre><ul><li><strong>erl_opts</strong>: Specifies compiler options.</li><li><strong>deps</strong>: Lists project dependencies with version specifications.</li><li><strong>plugins</strong>: Defines plugins that extend Rebar3&#39;s functionality.</li></ul><h3>Environment Configurations</h3><p>Rebar3 supports environment-specific configurations, allowing different settings for development, testing, and production.</p><pre>{profiles, [<br>    {test, [<br>        {deps, [<br>            {eunit, &quot;~&gt; 2.4&quot;}<br>        ]}<br>    ]}<br>]}.</pre><p>In this example, the test profile includes the eunit dependency, which is only required during testing.</p><h3>Dependency Management</h3><p>Managing dependencies is a crucial aspect of modern software development. Rebar3 leverages Hex.pm, the primary package repository for Erlang and Elixir, to fetch and manage dependencies.</p><h3>Adding Dependencies</h3><p>To add a dependency, update the deps section in rebar.config:</p><pre>{deps, [<br>    {cowboy, &quot;~&gt; 2.9&quot;},<br>    {jiffy, &quot;~&gt; 1.0&quot;}<br>]}.</pre><p>After updating, run:</p><pre>rebar3 get-deps</pre><p>Rebar3 will fetch the specified dependencies and place them in the _build directory.</p><h3>Updating Dependencies</h3><p>To update dependencies to their latest compatible versions, execute:</p><pre>rebar3 update</pre><p>This command fetches the latest versions that satisfy the version constraints specified in rebar.config.</p><h3>Overriding Dependencies</h3><p>In cases where you need to override a transitive dependency, you can specify it explicitly in the overrides section:</p><pre>{overrides, [<br>    {cowlib, &quot;2.9.1&quot;}<br>]}.</pre><p>This ensures that all references to cowlib use version 2.9.1.</p><h3>Testing and Quality Assurance</h3><p>Rebar3 integrates seamlessly with Erlang’s testing frameworks, facilitating the writing and execution of tests.</p><h3>Writing Tests</h3><p>Tests are typically placed in the test/ directory. Rebar3 supports various testing frameworks, including eunit and ct (Common Test).</p><p><strong>Example using EUnit:</strong></p><pre>-module(my_app_tests).<br>-include_lib(&quot;eunit/include/eunit.hrl&quot;).<br><br>my_function_test() -&gt;<br>    ?assertEqual(4, my_app:add(2, 2)).</pre><h3>Running Tests</h3><p>Execute all tests using:</p><pre>rebar3 eunit</pre><p>For Common Test suites:</p><pre>rebar3 ct</pre><h3>Test Coverage</h3><p>Rebar3 can generate test coverage reports, helping identify untested parts of the codebase.</p><pre>rebar3 ct --cover</pre><p>The coverage report will be available in the _build/test/cover directory.</p><h3>Release Handling</h3><p>Creating releases is essential for deploying Erlang and Elixir applications. Rebar3 provides robust tools to bundle applications into releases, including all necessary dependencies and configurations.</p><h3>Creating a Release</h3><p>Define release configurations in rebar.config:</p><pre>{relx, [<br>    {release, {my_app, &quot;0.1.0&quot;}, [my_app, cowboy]},<br>    {dev_mode, true},<br>    {include_erts, false},<br>    {sys_config, &quot;config/sys.config&quot;},<br>    {vm_args, &quot;config/vm.args&quot;}<br>]}.</pre><ul><li><strong>release</strong>: Specifies the application name, version, and included applications.</li><li><strong>dev_mode</strong>: Enables development mode for easier debugging.</li><li><strong>include_erts</strong>: Determines whether to include the Erlang Runtime System in the release.</li><li><strong>sys_config</strong>: Points to the system configuration file.</li><li><strong>vm_args</strong>: Specifies VM arguments for the release.</li></ul><h3>Building the Release</h3><p>Run the following command to create the release:</p><pre>rebar3 release</pre><p>The release will be generated in the _build/default/rel/my_app directory, containing all necessary files to run the application.</p><h3>Running the Release</h3><p>Navigate to the release directory and execute:</p><pre>bin/my_app console</pre><p>This starts the application in interactive mode. To run it as a background service, use:</p><pre>bin/my_app start</pre><h3>Extensibility and Plugins</h3><p>One of Rebar3’s strengths is its extensible architecture, allowing developers to enhance its capabilities through plugins.</p><h3>Using Plugins</h3><p>Plugins can automate tasks, integrate with other tools, or introduce new functionalities. To add a plugin, include it in the plugins section of rebar.config:</p><pre>{plugins, [<br>    {rebar3_hex, &quot;6.14.0&quot;},<br>    {rebar3_lint, &quot;0.0.8&quot;}<br>]}.</pre><h3>Popular Plugins</h3><ul><li><strong>rebar3_hex</strong>: Integrates Hex package management, enabling publishing and dependency management.</li><li><strong>rebar3_lint</strong>: Adds linting capabilities to enforce code quality.</li><li><strong>rebar3_hex_tools</strong>: Provides additional Hex-related utilities.</li><li><strong>rebar3_gerrit_hooks</strong>: Integrates Gerrit hooks for code review processes.</li></ul><h3>Developing Custom Plugins</h3><p>Developers can create custom plugins to tailor Rebar3 to specific project needs. A plugin is essentially an Erlang or Elixir application that adheres to Rebar3’s plugin API.</p><p><strong>Example Structure:</strong></p><pre>%% rebar.config<br>{plugins, [<br>    {my_custom_plugin, {git, &quot;https://github.com/user/my_custom_plugin.git&quot;, {branch, &quot;main&quot;}}}<br>]}.</pre><p>Creating a plugin involves defining the necessary commands and functionalities within the plugin’s codebase, allowing it to interact with Rebar3’s build process.</p><h3>Rebar3 vs. Other Build Tools</h3><p>While Rebar3 is a powerful and feature-rich build tool for Erlang and Elixir, it’s essential to understand how it compares to other tools in the ecosystem.</p><h3>Rebar3 vs. Mix</h3><p><strong>Scope</strong>:</p><ul><li><strong>Rebar3</strong>: Primarily designed for Erlang and, to some extent, Elixir projects.</li><li><strong>Mix</strong>: Elixir’s built-in build tool, tailored specifically for Elixir projects.</li></ul><p><strong>Dependency Management</strong>:</p><ul><li><strong>Rebar3</strong>: Uses rebar.config and integrates with Hex.</li><li><strong>Mix</strong>: Uses mix.exs and also integrates with Hex.</li></ul><p><strong>Release Handling</strong>:</p><ul><li><strong>Rebar3</strong>: Offers robust release management through relx.</li><li><strong>Mix</strong>: Utilizes tools like Distillery or the built-in releases in newer Elixir versions.</li></ul><p><strong>Extensibility</strong>:</p><ul><li>Both tools support plugins and custom tasks, but Rebar3’s plugin architecture is more mature for Erlang-centric projects.</li></ul><h3>Rebar3 vs. Make</h3><p><strong>Abstraction Level</strong>:</p><ul><li><strong>Rebar3</strong>: High-level build automation tailored for Erlang and Elixir.</li><li><strong>Make</strong>: General-purpose build tool with lower-level control.</li></ul><p><strong>Ease of Use</strong>:</p><ul><li><strong>Rebar3</strong>: Simplifies common build tasks without extensive configuration.</li><li><strong>Make</strong>: Requires manual scripting for build processes, offering greater flexibility but at the cost of complexity.</li></ul><h3>Rebar3 vs. Gradle/Maven</h3><p><strong>Language Focus</strong>:</p><ul><li><strong>Rebar3</strong>: Specific to Erlang and Elixir.</li><li><strong>Gradle/Maven</strong>: Primarily for Java and JVM-based languages.</li></ul><p><strong>Ecosystem Integration</strong>:</p><ul><li><strong>Rebar3</strong>: Deeply integrated with Erlang and Elixir tools and repositories.</li><li><strong>Gradle/Maven</strong>: Integrated with Java repositories like Maven Central.</li></ul><p>In summary, Rebar3 excels in its niche for Erlang and Elixir projects, offering features and integrations that general-purpose build tools lack. For developers working within the BEAM ecosystem, Rebar3 provides a comprehensive and streamlined build experience.</p><h3>Best Practices</h3><p>To maximize the benefits of Rebar3, consider the following best practices:</p><h3>1. Maintain Clear rebar.config</h3><p>Organize dependencies, plugins, and build configurations clearly within rebar.config. Use comments to explain non-obvious configurations, enhancing readability and maintainability.</p><h3>2. Utilize Profiles for Environment-Specific Configurations</h3><p>Leverage Rebar3’s profile feature to manage environment-specific settings, such as different dependencies or compiler options for development, testing, and production.</p><pre>{profiles, [<br>    {prod, [<br>        {erl_opts, [verbose]}<br>    ]}<br>]}.</pre><h3>3. Keep Dependencies Up-to-Date</h3><p>Regularly update dependencies to benefit from the latest features, bug fixes, and security patches. Use rebar3 update to fetch the latest compatible versions (be vigilant because it is hard to research this easily).</p><h3>4. Automate Testing and Continuous Integration</h3><p>Integrate Rebar3 with CI/CD pipelines to automate testing, building, and releasing processes. This ensures consistent build quality and facilitates rapid deployment cycles.</p><h3>5. Leverage Plugins for Enhanced Functionality</h3><p>Explore and utilize Rebar3 plugins that can automate repetitive tasks, enforce code quality, or integrate with other tools in your workflow.</p><h3>6. Document Your Build Process</h3><p>Maintain documentation for your build process, especially if custom plugins or complex configurations are involved. This aids onboarding and ensures consistency across the team.</p><h3>7. Optimize Release Configurations</h3><p>Configure releases thoughtfully, specifying necessary applications, configurations, and VM arguments. Test releases thoroughly to ensure smooth deployments.</p><h3>Community and Support</h3><p>Rebar3 benefits from a vibrant and active community within the Erlang and Elixir ecosystems. Developers contribute to its continuous improvement through:</p><ul><li><strong>GitHub Repository</strong>: The <a href="https://github.com/erlang/rebar3">Rebar3 GitHub</a> is the central hub for development, issue tracking, and feature requests.</li><li><strong>Community Forums and Chat</strong>: Platforms like the <a href="https://erlangforums.com/">Erlang Forums</a> and Elixir’s Slack provide spaces for discussion, support, and knowledge sharing.</li><li><strong>Comprehensive Documentation</strong>: The Rebar3 Documentation offers detailed guides, tutorials, and reference materials to assist developers at all levels.</li><li><strong>Regular Updates</strong>: The maintainers actively release updates, addressing bugs, introducing new features, and ensuring compatibility with the latest Erlang and Elixir versions.</li></ul><p>Engaging with the community through contributions, discussions, and feedback helps foster a robust and reliable build tool that meets the evolving needs of developers.</p><h3>Conclusion</h3><p><strong>Rebar3</strong> stands as a testament to the evolution of build tools within the Erlang and Elixir ecosystems. Its comprehensive feature set, coupled with its emphasis on simplicity and extensibility, makes it an indispensable tool for developers aiming to build scalable, maintainable, and high-performance applications on the BEAM virtual machine.</p><p>From automated compilation and dependency management to robust testing and release handling, Rebar3 streamlines every facet of the build process. Its seamless integration with Hex.pm and support for both Erlang and Elixir projects further enhance its versatility, fostering a collaborative and efficient development environment.</p><p>As the landscape of concurrent and distributed systems continues to grow, tools like Rebar3 play a crucial role in empowering developers to harness the full potential of Erlang and Elixir. By adhering to best practices, leveraging its extensible architecture, and engaging with the supportive community, developers can ensure that their projects are built on a solid and efficient foundation.</p><h3>References</h3><p><a href="https://rebar3.org/">Rebar3</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=726866391a14" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Revolutionize Your Phoenix Workflow: The Ultimate Docker Setup Guide!]]></title>
            <link>https://medium.com/@jonnyeberhardt7/revolutionize-your-phoenix-workflow-the-ultimate-docker-setup-guide-efee1268747e?source=rss-200a8333555e------2</link>
            <guid isPermaLink="false">https://medium.com/p/efee1268747e</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[docker-compose]]></category>
            <category><![CDATA[phoenix]]></category>
            <category><![CDATA[elixir]]></category>
            <dc:creator><![CDATA[Jonny Eberhardt]]></dc:creator>
            <pubDate>Sat, 30 Nov 2024 22:36:42 GMT</pubDate>
            <atom:updated>2024-11-30T22:36:42.757Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Voax_N3WAPbfGlNWctN6kw.jpeg" /></figure><p>Developing Phoenix applications can be streamlined and made more consistent across different environments using Docker. Docker ensures that your development environment mirrors production closely, eliminating the notorious “it works on my machine” problem. This guide will walk you through setting up a Docker-based development environment for a Phoenix project, covering everything from installing Docker to running your Phoenix server seamlessly within Docker containers. You may review this code on <a href="https://github.com/jonnycharles/elixir-docker-app-second">https://github.com/jonnycharles/elixir-docker-app-second</a></p><h3>Prerequisites</h3><p>Before diving into the setup, ensure you have the following installed on your machine:</p><ul><li><strong>Docker</strong>: The platform for developing, shipping, and running applications in containers.</li><li><strong>Elixir</strong>: The programming language Phoenix is built upon.</li><li><strong>Phoenix Framework</strong>: A web framework for building scalable and maintainable applications.</li></ul><p>This guide assumes familiarity with basic command-line operations and some knowledge of Docker concepts.</p><h3>Step 1: Install Docker</h3><p>Docker is essential for containerizing your Phoenix application. Follow these steps to install Docker on your system:</p><p><strong>Download Docker</strong>:</p><ul><li>Visit the Docker Download Page.</li><li>Choose the appropriate Docker Desktop version for your operating system (Windows, macOS, or Linux).</li></ul><p><strong>Install Docker</strong>:</p><ul><li>Follow the installation instructions specific to your OS.</li><li>After installation, verify Docker is running by executing:</li><li>docker --version</li><li>You should see the Docker version information displayed.</li></ul><p><strong>Note</strong>: Docker Desktop includes Docker Engine, Docker CLI, Docker Compose, and other essential tools.</p><h3>Step 2: Create Project Directory</h3><p>Organizing your project files is crucial for maintainability. Follow these steps to set up your project directory:</p><p><strong>Create a New Folder for Your Project</strong>:</p><ul><li>mkdir my_phoenix_app cd my_phoenix_app</li></ul><p><strong>Initialize a Git Repository (Optional but Recommended)</strong>:</p><ul><li>git init</li></ul><p>This directory will house all your project files, including the Docker configuration.</p><h3>Step 3: Set Up Docker Folder Structure</h3><p>A well-structured Docker setup enhances clarity and manageability. Here’s how to organize your Docker-related files:</p><p><strong>Create a Docker Directory</strong>:</p><pre>mkdir docker</pre><p><strong>Within the Docker Directory, Create a </strong><strong>dev Folder</strong>:</p><ul><li>mkdir docker/dev</li></ul><p>Your directory structure should now look like this:</p><pre>my_phoenix_app/<br>├── docker/<br>│   └── dev/</pre><p>This structure separates Docker configurations for different environments (e.g., development, production), making it easier to manage multiple setups.</p><h3>Step 4: Configure Environment Variables</h3><p>Environment variables play a vital role in configuring your application without hardcoding sensitive information. Here’s how to set them up:</p><p><strong>Navigate to the </strong><strong>dev Folder</strong>:</p><ul><li>cd docker/dev</li></ul><p><strong>Create an </strong><strong>.env File</strong>:</p><ul><li>touch .env</li></ul><p><strong>Open the </strong><strong>.env File in Your Preferred Editor</strong> and add the following content:</p><pre>POSTGRES_USER=postgres <br>POSTGRES_PASSWORD=postgres <br>POSTGRES_DB=my_app_name <br>DATABASE_URL=ecto://postgres:postgres@db/elixir_docker_app <br>SECRET_KEY_BASE=your_generated_secret_key_base <br>PHX_HOST=localhost <br>PORT=4000</pre><p><strong>Generate </strong><strong>SECRET_KEY_BASE</strong>:</p><p>It’s crucial to have a secure SECRET_KEY_BASE for your Phoenix application. Follow these steps to generate it:</p><ul><li><strong>Start an IEx Session</strong>:</li><li>iex</li><li><strong>Run the Following Commands in IEx</strong>:</li><li>:crypto.strong_rand_bytes(64) |&gt; Base.encode64() |&gt; binary_part(0, 64)</li><li>This will output a 64-character string. Copy this value.</li><li><strong>Set </strong><strong>SECRET_KEY_BASE in </strong><strong>.env</strong>: Replace your_generated_secret_key_base in the .env file with the string you just generated.</li></ul><p><strong>Explanation of Environment Variables</strong>:</p><ul><li>POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB: Credentials and database name for PostgreSQL.</li><li>DATABASE_URL: Connection string for Ecto (Elixir&#39;s database wrapper).</li><li>SECRET_KEY_BASE: Security key for signing/encrypting session data.</li><li>PHX_HOST: Host address for Phoenix.</li><li>PORT: Port on which Phoenix will run.</li></ul><h3>Step 5: Create the Dockerfile</h3><p>The Dockerfile defines the environment in which your Phoenix application will run. Here’s how to set it up:</p><p><strong>Create a </strong><strong>Dockerfile in the </strong><strong>dev Directory</strong>:</p><ul><li>touch Dockerfile</li></ul><p><strong>Add the Following Content to </strong><strong>Dockerfile</strong>:</p><pre># docker/dev/Dockerfile <br>FROM elixir:1.15-alpine  <br><br># Install build essentials and development tools <br>RUN apk add --no-cache build-base npm git inotify-tools bash postgresql-client  <br><br># Install Phoenix <br>RUN mix local.hex --force &amp;&amp; \     <br>mix local.rebar --force &amp;&amp; \     <br>mix archive.install hex phx_new --force  <br><br>WORKDIR /app  <br><br># Set development environment <br>ENV MIX_ENV=dev  <br><br># We&#39;ll override this in docker-compose.yml <br>CMD [&quot;mix&quot;, &quot;phx.server&quot;]</pre><p><strong>Explanation of Dockerfile Steps</strong>:</p><ul><li>FROM elixir:1.15-alpine: Uses the Elixir 1.15 image based on Alpine Linux for a lightweight container.</li><li>RUN apk add --no-cache ...: Installs necessary packages including build tools, Node.js (for assets), Git, inotify-tools (for file watching), Bash, and PostgreSQL client.</li><li>RUN mix local.hex --force &amp;&amp; ...: Installs Hex (Elixir package manager), Rebar (Erlang build tool), and the Phoenix archive for generating new Phoenix projects.</li><li>WORKDIR /app: Sets the working directory inside the container to /app.</li><li>ENV MIX_ENV=dev: Sets the Elixir environment to development.</li><li>CMD [&quot;mix&quot;, &quot;phx.server&quot;]: Default command to start the Phoenix server. This will be overridden in the Docker Compose configuration.</li></ul><h3>Step 6: Set Up Docker Compose</h3><p>Docker Compose simplifies the management of multi-container Docker applications. Here’s how to configure it for your Phoenix project:</p><p><strong>Create a </strong><strong>docker-compose.yml File in the </strong><strong>dev Directory</strong>:</p><ul><li>touch docker-compose.yml</li></ul><p><strong>Add the Following Content to </strong><strong>docker-compose.yml</strong>:</p><pre>version: &#39;3.8&#39;<br><br>services:<br>  web:<br>    build:<br>      context: .<br>      dockerfile: Dockerfile<br>    ports:<br>      - &quot;4000:4000&quot;<br>    volumes:<br>      - ../../:/app<br>      - /app/deps<br>      - /app/_build<br>    depends_on:<br>      db:<br>        condition: service_healthy<br>    environment:<br>      - POSTGRES_USER=postgres<br>      - POSTGRES_PASSWORD=postgres<br>      - POSTGRES_DB=elixir_docker_app_two_dev<br>      - POSTGRES_HOST=db<br>      - SECRET_KEY_BASE=your-secret-key-base<br>    command: &gt;<br>      /bin/bash -c &#39;<br>        if [ ! -f mix.exs ]; then<br>          yes | mix phx.new . --app elixir_docker_app_two --database postgres &amp;&amp;<br>          sed -i &quot;s/hostname: \&quot;localhost\&quot;/hostname: System.get_env(\&quot;POSTGRES_HOST\&quot;, \&quot;db\&quot;)/g&quot; config/dev.exs &amp;&amp;<br>          sed -i &quot;s/ip: {127, 0, 0, 1}/ip: {0, 0, 0, 0}/g&quot; config/dev.exs<br>        fi &amp;&amp;<br>        mix deps.get &amp;&amp;<br>        mix ecto.create &amp;&amp;<br>        mix ecto.migrate &amp;&amp;<br>        mix phx.server<br>      &#39;<br><br>  db:<br>    image: postgres:14-alpine<br>    environment:<br>      - POSTGRES_USER=postgres<br>      - POSTGRES_PASSWORD=postgres<br>      - POSTGRES_DB=elixir_docker_app_two_dev<br>    ports:<br>      - &quot;5432:5432&quot;<br>    volumes:<br>      - postgres_data:/var/lib/postgresql/data<br>    healthcheck:<br>      test: [&quot;CMD-SHELL&quot;, &quot;pg_isready -U postgres&quot;]<br>      interval: 5s<br>      timeout: 5s<br>      retries: 5<br><br>volumes:<br>  postgres_data:</pre><p><strong>Explanation of </strong><strong>docker-compose.yml Components</strong>:</p><ul><li><strong>Version:</strong></li><li>version: &#39;3.8&#39;: Specifies the version of Docker Compose syntax being used.</li><li><strong>Services</strong>:</li><li><strong>web</strong>:</li><li><strong>build</strong>:</li><li>context: .: Sets the build context to the current directory (docker/dev).</li><li>dockerfile: Dockerfile: Specifies the Dockerfile to use for building the image.</li><li><strong>ports</strong>:</li><li>&quot;4000:4000&quot;: Maps port 4000 of the host to port 4000 of the container, allowing access to the Phoenix server via localhost:4000.</li><li><strong>volumes</strong>:</li><li>../../:/app: Mounts the project root directory into /app inside the container. This allows real-time code changes to reflect inside the container.</li><li>/app/deps: Persists the deps directory (Elixir dependencies) between container restarts.</li><li>/app/_build: Persists the _build directory (compiled artifacts) between container restarts.</li><li><strong>depends_on</strong>:</li><li>Ensures the db service is healthy before starting the web service.</li><li><strong>environment</strong>:</li><li>Sets environment variables required by the Phoenix application and PostgreSQL.</li><li><strong>command</strong>:</li><li>Executes a series of commands to initialize and run the Phoenix application:</li><li>Checks if mix.exs exists. If not, it initializes a new Phoenix project.</li><li>Modifies config/dev.exs to use the POSTGRES_HOST environment variable and binds to all IPs.</li><li>Fetches dependencies, creates and migrates the database, and starts the Phoenix server.</li><li><strong>db</strong>:</li><li><strong>image</strong>:</li><li>postgres:14-alpine: Uses the lightweight PostgreSQL 14 Alpine image.</li><li><strong>environment</strong>:</li><li>Sets PostgreSQL credentials and database name.</li><li><strong>ports</strong>:</li><li>&quot;5432:5432&quot;: Maps PostgreSQL&#39;s default port to the host, allowing database access via localhost:5432.</li><li><strong>volumes</strong>:</li><li>postgres_data:/var/lib/postgresql/data: Persists PostgreSQL data between container restarts using a named volume.</li><li><strong>healthcheck</strong>:</li><li>Ensures PostgreSQL is ready before dependent services start.</li><li>test: [&quot;CMD-SHELL&quot;, &quot;pg_isready -U postgres&quot;]: Checks if PostgreSQL is ready to accept connections.</li><li>interval: 5s, timeout: 5s, retries: 5: Configures health check intervals and retries.</li><li><strong>Volumes</strong>:</li><li>postgres_data: A named volume for persisting PostgreSQL data. Named volumes are managed by Docker and provide data persistence beyond the lifecycle of individual containers.</li></ul><p><strong>Why These Configurations Are Essential</strong>:</p><ul><li><strong>Volumes</strong>: Ensure that your code changes reflect instantly in the container and that dependencies and build artifacts are not lost between container restarts.</li><li><strong>Health Checks</strong>: Guarantee that the database service is ready before the application starts, preventing race conditions.</li><li><strong>Environment Variables</strong>: Securely manage sensitive information and configuration settings.</li><li><strong>Command Overrides</strong>: Automate project initialization, dependency fetching, database setup, and server start-up, streamlining the development workflow.</li></ul><h3>Step 7: Build and Run Containers</h3><p>With your Docker configurations in place, it’s time to build and run your Docker containers.</p><p><strong>Navigate to the </strong><strong>dev Directory</strong>:</p><ul><li>cd docker/dev</li></ul><p><strong>Run Docker Compose</strong>:</p><ul><li>docker compose -f docker-compose.yml up --build</li></ul><p><strong>Explanation</strong>:</p><ul><li>docker compose: Invokes Docker Compose.</li><li>-f docker-compose.yml: Specifies the Docker Compose file to use.</li><li>up: Builds and starts the containers.</li><li>--build: Forces a rebuild of the images, ensuring the latest configurations are applied.</li></ul><p><strong>Monitor the Build Process</strong>:</p><ul><li>Docker Compose will build the web service image using the Dockerfile.</li><li>The db service will start using the PostgreSQL image.</li><li>The web service will initialize the Phoenix project if it doesn&#39;t exist, fetch dependencies, set up the database, and start the Phoenix server.</li></ul><p><strong>Potential Issues</strong>:</p><ul><li><strong>Port Conflicts</strong>: Ensure that ports 4000 and 5432 are not in use by other applications.</li><li><strong>Resource Constraints</strong>: Docker requires sufficient CPU and memory resources. Adjust Docker Desktop settings if necessary.</li></ul><h3>Step 8: Verify Your Phoenix Application</h3><p>Once Docker Compose finishes building and starting the containers, your Phoenix application should be up and running.</p><p><strong>Access the Application</strong>:</p><ul><li>Open your web browser and navigate to <a href="http://localhost:4000.">http://localhost:4000.</a></li><li>You should see the default Phoenix welcome page, indicating that the server is running correctly.</li></ul><p><strong>Interact with the Application</strong>:</p><ul><li>Modify your Phoenix application’s code in your project directory.</li><li>Thanks to volume mounting, changes should reflect immediately without needing to rebuild the Docker image.</li><li>The Phoenix live-reload feature will automatically refresh your browser upon detecting changes.</li></ul><p><strong>Access PostgreSQL Database (Optional)</strong>:</p><ul><li>Use a PostgreSQL client (like psql or a GUI tool) to connect to localhost:5432 using the credentials specified in the .env and docker-compose.yml files.</li></ul><p><strong>Troubleshooting Tips</strong>:</p><ul><li><strong>Container Logs</strong>:</li><li>If something goes wrong, view the logs to diagnose issues:</li><li>docker compose -f docker-compose.yml logs</li><li><strong>Rebuilding Containers</strong>:</li><li>If you change the Dockerfile or docker-compose.yml, rebuild the containers:</li><li>docker compose -f docker-compose.yml up --build</li><li><strong>Stopping Containers</strong>:</li><li>To stop the running containers, press Ctrl+C in the terminal where Docker Compose is running or execute:</li><li>docker compose -f docker-compose.yml down</li></ul><h3>Conclusion</h3><p>Setting up a Phoenix development environment with Docker enhances consistency, portability, and scalability. By following the steps outlined in this guide, you can:</p><ul><li><strong>Standardize Development Environments</strong>: Ensure all team members work within identical setups, reducing environment-related issues.</li><li><strong>Simplify Dependency Management</strong>: Docker handles dependencies and configurations, freeing you from manual installations.</li><li><strong>Streamline Deployment</strong>: A Dockerized application can be easily deployed to various environments, including cloud platforms and CI/CD pipelines.</li></ul><p>Embracing Docker for your Phoenix projects not only boosts productivity but also aligns your development workflow with modern best practices.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=efee1268747e" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>