<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Auke ten Hoopen on Medium]]></title>
        <description><![CDATA[Stories by Auke ten Hoopen on Medium]]></description>
        <link>https://medium.com/@ahoopen?source=rss-468e730accf0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*bNth7bZtZ9-BUeZ8-5ev5A.jpeg</url>
            <title>Stories by Auke ten Hoopen on Medium</title>
            <link>https://medium.com/@ahoopen?source=rss-468e730accf0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 12:01:25 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ahoopen/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Beyond Queries: Supercharge GraphQL with Fragments]]></title>
            <link>https://ahoopen.medium.com/beyond-queries-supercharge-graphql-with-fragments-789d02c88d8c?source=rss-468e730accf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/789d02c88d8c</guid>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[fragments]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[graphql]]></category>
            <dc:creator><![CDATA[Auke ten Hoopen]]></dc:creator>
            <pubDate>Tue, 19 Sep 2023 07:39:44 GMT</pubDate>
            <atom:updated>2023-09-22T06:56:03.751Z</atom:updated>
            <content:encoded><![CDATA[<p>In this article, we’ll journey beyond the basics of GraphQL queries and explore a powerful technique that can supercharge your GraphQL usage.</p><p>In the beginning your application is still relatively simple. You have a number of components and a GraphQL query to fetch data. So far so good.</p><p>But as time goes by, new features are added, and our GraphQL query must be expanded. Components deeper in the component tree also require specific data from the query. As result, we have to pass down the data till it eventually reaches the child component that needs the data fragment.</p><p>It is slowly becoming more difficult to keep an overview of which components need which data.</p><p>How can we ensure that as our application grows, we still have a clear and explicit data dependency for our components?</p><p>Let’s dive in!</p><h3>Fetching Data for a Component View</h3><p>There are multiple ways to fetch GraphQL data for your application.</p><p>One approach is that the root component declares and fetches the data needed not only for itself but also for all its children. This method comes with the downside of introducing tight coupling. Any change to a child component would require making a change to the root component that may render it. This coupling could increase the likelihood of bugs.</p><p>Another way to approach it, is to have each component declare and fetch the data it needs. This sounds great in theory. But, the issue here is that a component might render different children based on the data it fetches. So, when you have nested components, they can’t start showing their content and fetching data until the parent components have finished their tasks.</p><p>In simple terms, this makes data fetching sequential: first, the main component is displayed and gets its data, then its children are shown and fetch their data, and this continues until the smallest components. This way of doing things can be slow because it involves multiple rounds of waiting for data.</p><p>The ideal solution would be to combine the advantages of both approaches. By allowing components to specify the data they need, we can aggregate the data dependencies from multiple components, into a single query, and fetch it in a single round-trip to the server.</p><h3>Colocate data with component code</h3><p>Let’s start with an example. We have a list of products. Each product has a price but now we want to show the price before it was discounted.</p><p>We have to update our ProductListQuery in ProductList.tsx</p><pre>query ProductListQuery {<br>    products {<br>      id<br>      name<br>      price {<br>        now {<br>          amount<br>        }<br>        was { # Newly added<br>          amount # Newly added<br>        } # Newly added <br>      }<br>      image {<br>        url<br>      } <br>    }<br>}</pre><p>Now we edit the ProductPrice.tsx component to render the data:</p><pre>type ProductPriceProps = {<br>  price: {<br>    was: { // &lt;-- Update our type definition<br>      amount: number // &lt;-- <br>    } // &lt;--<br>    ...<br>  }<br>}<br><br>export function ProductPrice({ price }: ProductPriceProps) {<br>  return (<br>    &lt;div&gt;<br>       &lt;span&gt;{price.now.amount}&lt;/span&gt;<br>       &lt;span&gt;{price.was.amount}&lt;/span&gt; // &lt;-- newly added<br>    &lt;/div&gt;<br>  )<br>}</pre><p>Now the price before it was discounted will appear. And thanks to GraphQL, we didn’t have to write or deploy any new server code.</p><h4><strong>But…</strong></h4><p>Why did we have to edit ProductList.tsx? Aren’t React components supposed to be self-contained? Why should ProductList be bothered about the specific data ProductPrice needs? What if that data was needed by some small part inside ProductPriceor even a component used all over the place? Then, every time you changed the data requirements, you’d have to tweak lots of different components.</p><p>If we want to solve this problem, we should move the data requirements to the ProductPrice component.</p><p>The way we do this is by splitting of theProductPrice data requirements into a F<strong>ragment </strong>defined in ProductPrice.tsx.</p><p>Fragments allow each component to define its own data needs, without paying the cost at runtime of each component running its own queries.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hUVv2s62737_qJ2tN57trQ.png" /><figcaption>A GraphQL Fragment is a piece of logic that can be shared between multiple queries and mutations.</figcaption></figure><h3>Declare the data requirements of your components with fragments</h3><p>Instead of writing one big GraphQL query for our whole page and passing it down to our components, start with defining our data requirements at component level through a GraphQL Fragment.</p><p>This way, we make the data dependency of our component colocated and explicit in the same way we would colocate TypeScript definitions.</p><h4>0. Setup</h4><p>Before we can start we need to install a number of packages. We are going to use graphql-codegen to auto generate our GraphQL fragments.</p><p>Install the following packages:</p><pre>npm install --dev @graphql-codegen/client-preset</pre><p>When done installing, we need to create codegen.ts file in the root.</p><pre>import { CodegenConfig } from &#39;@graphql-codegen/cli&#39;<br> <br>const config: CodegenConfig = {<br>  schema: process.env.GRAPHQL_SCHEMA ?? &#39;http://your.graphql.schema.com&#39;,<br>  documents: [&#39;src/**/*.tsx&#39;, &#39;!src/generated/**/*&#39;],<br>  generates: {<br>    &#39;./src/generated/&#39;: {<br>      preset: &#39;client&#39;,<br>      plugins: [],<br>      config: {<br>        dedupeFragments: true,<br>      },<br>    }<br>  }<br>}<br> <br>export default config</pre><p>Now we are all set!</p><h4><strong>1. Define a fragment</strong></h4><p>First we need to define our Fragment.</p><pre>import { graphql} from &#39;@/generated/graphql&#39;;<br><br>const ProductPriceFragment = graphql(`<br>  fragment ProductPriceFragment on ProductPrice {<br>    now {<br>      amount<br>    }<br>    was {<br>      amount<br>    }<br>  }<br>`);</pre><p>We have taken a selection of the products query and pasted it into this new Fragment. Like queries, fragments have a name — ProductPriceFragment — which we will use later on, but they also have a type — ProductPrice — that they are on — ProductPriceFragment <strong>on</strong> ProductPrice. This means that this fragment can be used whenever we have a ProductPrice node in our graph.</p><h4><strong>2. Spread the fragment</strong></h4><p>We modify the ProductListQuery in ProductList.tsx to look like this:</p><pre>const ProductListQuery = graphql(`<br>   query ProductListQuery {<br>      products {<br>        id<br>        name<br>        <br>        price {<br>          ...ProductPriceFragment<br>        }<br><br>        image {<br>          url<br>        }<br>      }<br>   }<br>`);</pre><p>We have replaced the Price selection inside Products with ProductPriceFragment . GraphQL will make sure that all the data will be fetched, without having to change the ProductList component.</p><h4><strong>3. UseFragment</strong></h4><p>If we check our ProductList component now, it renders products without a price. The price data is missing. There is a valid reason for this. Unless a component specifically asks for the data for a certain fragment, that data will not be visible to the component. This is what we call <strong>Data Masking</strong>.</p><h4>Data Masking</h4><p>This enforces that component’s don’t implicitly rely on another components data dependencies but declare all of their dependencies within their own fragments. This keeps components self-contained and maintainable.</p><p>Let’s illustrate data masking with an example:</p><pre>const AvatarUserFragment = graphql(`<br>  fragment ImageUserFragment on User {<br>    avatarUrl<br>  }<br>`);<br><br>const UserListItemFragment = graphql(`<br>  fragment UserListItemFragment on User {<br>    fullName<br>    ...ImageUserFragment<br>  }<br>`);<br><br>export function UserListItem(props) {<br>  const user = useFragment(UserListItemFragment, props.user);<br><br>  // Error: Property &#39;avatarUrl&#39; doest not exist on type <br>  const icon = &lt;img src={user.avatarUrl} /&gt;<br><br>  return (<br>    &lt;ListItem icon={icon}&gt;<br>      {user.fullName}<br>    &lt;/ListItem&gt;<br>  );<br>}</pre><p>The code in this example would raise a TypeScript error, as avatarUrl field is not selected within the UserListItemFragment .</p><p>Without data masking you could never remove a field from a fragment, because it would be hard to verify that some other component somewhere wasn’t using it.</p><h4>Reading Fragments</h4><p>To access the data selected by a fragment, we use useFragment . Let’s see how this looks:</p><pre>import { graphql, useFragment } from &#39;@/generated/graphql&#39;;<br><br>const ProductPriceFragment = graphql`<br>  fragment ProductPriceFragment on ProductPrice {<br>    now {<br>      amount<br>    }<br>    was {<br>      amount<br>    }<br>  }<br>`;<br><br>type ProductPriceProps = {<br>  price: {<br>    was: {<br>      amount: number // added this to the type<br>    }  <br>    ...<br>  }<br>}<br><br>export function ProductPrice(props: ProductPriceProps) {<br>  const data = useFragment(ProductPriceFragment, props.price);<br><br>  return (<br>    &lt;div&gt;<br>       &lt;span&gt;{data.now.amount}&lt;/span&gt;<br>       &lt;span&gt;{data.was.amount}&lt;/span&gt;<br>    &lt;/div&gt;<br>  )<br>}</pre><p>useFragment takes two parameters:</p><ol><li>The GraphQL Fragment we want to read from</li><li>The FragmentType specifying the reference to the fragment</li></ol><p>Rather than creating a single big GraphQL query for the entire page and sharing it with the components — by prop drilling — , begin by defining the component’s data needs using a GraphQL fragment. This approach ensures that your component’s data dependencies are clearly and conveniently located alongside the component itself.</p><h4><strong>4. TypeScript Types for Fragments</strong></h4><p>By utilizing the <strong>useFragment </strong>we can ensure that the component can only access the properties of the data that are declared directly in the fragment selection.</p><p>Knowing this, we can change the type definition for props. So that TypeScript knows this components expects to receive a fragment instead of raw data.</p><p>We import the FragmentType helper from our generated types.</p><pre>import { graphql, FragmentType, useFragment } from &quot;@/generated/graphql&quot;;<br><br>const ProductPriceFragment = graphql`<br>  fragment ProductPriceFragment on ProductPrice {<br>    now {<br>      amount<br>    }<br>    was {<br>      amount<br>    }<br>  }<br>`;<br><br>// Auto infer the type based on the GraphQL Fragment<br>type ProductPriceProps = {<br>  price: FragmentType&lt;typeof ProductPriceFragment&gt;;<br>}<br><br>export function ProductPrice(props: ProductPriceProps) {<br>  const data = useFragment(ProductPriceFragment, props.price);<br><br>  return (<br>    &lt;div&gt;<br>       &lt;span&gt;{data.now.amount}&lt;/span&gt;<br>       &lt;span&gt;{data.was.amount}&lt;/span&gt;<br>    &lt;/div&gt;<br>  )<br>}</pre><p>With this change, there is no more need to maintain your own interface. Keeping interfaces in sync is often forgotten when changes occur. As a result, your interface may say that the property is there, but at runtime it appears not to be there at all. — Because you haven’t selected it in your fragment.</p><p>When we change our fragment, our TypeScript type will also update!</p><h3>Testing with Fragments</h3><p>A React Components that relies on Fragment Masking won’t accept “plain objects”. Let’s take the following example:</p><pre>import { FragmentType, useFragment } from &quot;@/generated/graphql&quot;;<br><br>type ProductNameProps = {<br>  product: FragmentType&lt;typeof ProductNameFragment&gt;<br>}<br><br>export const Product(props: ProductNameProps) {<br>  const { name } = useFragment(ProductNameFragment, props.product);<br><br>  return (<br>    &lt;div&gt;<br>       &lt;h1&gt;{name}<br>    &lt;/div&gt;<br>  );<br>}</pre><p>And the unit test</p><pre>import { Product } from &#39;./product&#39;;<br><br>describe(&#39;Product&#39;, () =&gt; {<br>  it(&#39;renders correctly&#39;, () =&gt; {<br>    const product = { name: &quot;Milk&quot; }<br><br>    render(<br>      &lt;Product product={product} /&gt; // &lt;- This will throw TypeScript Error<br>    );<br><br>    expect(screen.getByText(&#39;Milk&#39;)).toBeInTheDocument();<br>  });<br>});</pre><p>Since the component expects to receive “masked data” you will need to import makeFragmentData helper to build some masked data. Let’s see how this works:</p><pre>import { makeFragmentData } from &#39;@/generated/graphql&#39;;<br>import { ProductFragment, Product } from &#39;./product&#39;;<br><br>describe(&#39;Product&#39;, () =&gt; {<br>  it(&#39;renders correctly&#39;, () =&gt; {<br>    const product = { name: &quot;Milk&quot; }<br><br>    render(<br>      &lt;Product product={makeFragmentData(product, ProductFragment)} /&gt;<br>    );<br><br>    expect(screen.getByText(&#39;Milk&#39;)).toBeInTheDocument();<br>  });<br>});</pre><h3>Conclusion</h3><p>This way of working ensures that a codebase can scale but remains maintainable at the same time.</p><p>A core benefit of GraphQL fragments is presenting data exactly where it is consumed, so <em>data masking</em> is essential. It prevents implicit data dependencies. A component can only access the data defined in its fragment. As result the component is a self-contained building block that can be re-used.</p><p>If we use it together with TypeScript, it is even more powerful. We can infer the component interface from the GraphQL Fragments by utilizing the FragmentType helper.</p><p>In summary, by using GraphQL Fragments we can write better reusable components and are less likely to introduce bugs due to a mismatch of our data and TypeScript interfaces.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=789d02c88d8c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unlocking the Power of Github Actions: A Paved Road to Consistent Workflows]]></title>
            <link>https://ahoopen.medium.com/unlocking-the-power-of-github-actions-a-paved-road-to-consistent-workflows-eb37172626e7?source=rss-468e730accf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/eb37172626e7</guid>
            <category><![CDATA[github]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[ci]]></category>
            <category><![CDATA[github-actions]]></category>
            <dc:creator><![CDATA[Auke ten Hoopen]]></dc:creator>
            <pubDate>Fri, 05 May 2023 11:42:29 GMT</pubDate>
            <atom:updated>2023-05-08T09:27:58.582Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Js2OWUOCnwem8G--a6qzUg.png" /><figcaption>Paved road</figcaption></figure><p>In this article, I will show you how to set up a shared GitHub Actions workflow configuration that follows the <strong>paved road principle</strong>. You’ll learn how to create a reusable workflow template that can be easily adapted and shared across multiple repositories.</p><p>We’ll cover the benefits of using a shared workflow template, the steps for creating one, how to use it in a new repository, and how to customize it to fit specific needs. By the end of this article, you’ll be able to streamline your workflow configuration process, promote consistency across your team, and maximize efficiency.</p><p>Before we dive in, let’s first explore the paved road principle and why it’s a valuable approach to setting up shared workflow configurations.</p><blockquote>The paved road principle is an approach to creating a shared workflow configuration that provides a clear, standardized path for team members to follow. The idea is to create a well-documented, reusable workflow template that can be easily adapted and shared across multiple repositories.</blockquote><h3><strong>Benefits of a Paved Road approach</strong></h3><p>1. <strong>Consistency</strong>: <br>By using a shared workflow, you can ensure that you workflows are consistent across all repositories. This can reduce errors and make it easier to maintain your workflows over time.</p><p>2. <strong>Efficiency</strong>: <br>Creating a shared workflow configuration can save you time and effort by reducing the need to create new workflows from scratch for each new repository. This can also make it easier to onboard new team members, as they can quickly learn how to set up workflows using the shared template.</p><p>3. <strong>Flexibility</strong>: <br>Although the shared workflow provides a standardized approach, it can still be customized to fit the specific needs of each repository. This allows you to be flexible while still maintaining consistency across your workflows. As a team you still have the ability to go “off-road”. Writing your own workflows, but if any bugs or issues show up, it’s the responsibility of the team to fix it yourself.</p><p>4. <strong>Reusability</strong>: <br>Once a shared workflow configuration is created, it can be easily adapted and reused across multiple repositories. This can be especially valuable for teams that work on multiple projects or have a large number of repositories.</p><p>Overall, using a shared Github workflow configuration can help you work more efficiently, ensure consistency across your workflows, and reduce the time and effort required to set up new workflows.</p><h3>Creating a Reusable GitHub Actions Workflow</h3><p>Reusable workflows are GitHub Actions templates on steroids! In simple terms, they allow you to reference an entire workflow from a different GitHub Actions workflow, with a single line of configuration, rather than copying and pasting from one workflow to another. It makes maintaining workflows easy and allows you to create new ones quickly, by building on the work of others.</p><p>We are going to convert the following Github Workflow to a Reusable Workflow</p><pre>jobs:<br>  ci:<br>    runs-on: ubuntu-latest<br>    steps:<br>      - name: Checkout<br>        uses: actions/checkout@v3      <br><br>      - name: setup node<br>        uses: actions/setup-node@v3<br>        with:<br>          node-version: 18<br><br>    - name: Prepare NPM Credentials<br>      run: |<br>        echo &quot;@example:registry=https://npm.pkg.github.com&quot; &gt;&gt; .npmrc<br>        echo &quot;//npm.pkg.github.com/:_authToken=${{ secrets.GITHUB_TOKEN }}&quot; &gt;&gt; .npmr<br>               <br>     - name: Install<br>       run: npm install</pre><p>As with other workflows files, you locate reusable workflows in the .github/workflows directory of a repository.</p><p>For a workflow to be reusable, the values for on must include workflow_call:</p><pre>on:<br>  workflow_call:</pre><p>Inputs allow you to pass parameters to a workflow, which can be used to customize the behavior of the workflow for each repository. For example, you might use an input to specify which version Node to use, or to control the behavior of a script that’s run as part of the workflow.</p><p>To use inputs in a reusable workflow, you can define them in the inputs section of your workflow file. Here&#39;s an example:</p><pre>on:<br> workflow_call:<br>   inputs:<br>      nodeVersion:<br>        required: true<br>        type: number</pre><p>Secrets allow you to securely store sensitive information that’s used by your workflow, such as API keys, access tokens, and passwords. To use a secret in your shared workflow, you can reference it via secrets keyword.</p><pre>on:<br> workflow_call:<br>  secrets:<br>    token:<br>      required: true</pre><p>So putting it together we get the following reusable workflow</p><pre>name: Reusable workflow<br><br>on:<br> workflow_call:<br>   inputs:<br>      nodeVersion:<br>        required: true<br>        type: number<br>    secrets:<br>      token:<br>        required: true<br><br>jobs:<br>  ci:<br>    runs-on: ubuntu-latest<br>    steps:<br>      - name: Checkout<br>        uses: actions/checkout@v3      <br><br>      - name: setup node<br>        uses: actions/setup-node@v3<br>        with:<br>          node-version: ${{ inputs.nodeVersion }}<br><br>    - name: Prepare NPM Credentials<br>      run: |<br>        echo &quot;@example:registry=https://npm.pkg.github.com&quot; &gt;&gt; .npmrc<br>        echo &quot;//npm.pkg.github.com/:_authToken=${{ secrets.token }}&quot; &gt;&gt; .npmrc       <br><br>     - name: Install<br>       run: npm install</pre><h3>Using our reusable workflow in a new repository</h3><p>So how do we use this reusable workflow in our projects repository?</p><p>In order to get triggered, workflows must be in the .github/workflows folder. Reusable Workflows listen for workflow_call, a special event that needs access to all repositories in your organization. Here’s an example of Reusable workflow that takes an input number along with a secret from the caller workflow so that it can be reused in an action:</p><p><em>Let’s create a </em><em>pull-request.yml file with the following content in our </em><em>.github/workflows directory.</em></p><pre>name: Pull Request<br><br>on:<br>  pull_request:<br>    types: [opened, reopened, synchronize]<br><br>jobs:<br>  our-shared-workflow:<br>    uses: your-org/shared-workflows/.github/workflows/reusable-workflow.yml@master<br>    with:<br>      nodeVersion: 18<br>    secrets:<br>      token: ${{ secrets.NPM_TOKEN }}</pre><p>You reference reusable workflow files using the following syntax:</p><ul><li>{owner}/{repo}/.github/workflows/{filename}@{ref}</li></ul><p>{ref} can be release tag or branch name.</p><p>We can now easily adapt the reusable workflow and all the repositories which use the workflow will automatically be updated. This can save you a lot of time and effort, while also promoting consistency and maintainability.</p><h3>Build consistently across multiple, dozens, or even hundreds of repositories</h3><p>Do you have a bunch of repositories that are essentially built or deployed the same way? You can now create a reusable workflow, and keep all those repositories in sync.</p><p><strong>Before:</strong></p><pre>jobs:<br>  build:<br>    runs-on: ubuntu-latest<br><br>    steps:<br>    - uses: actions/checkout@v3<br><br>    - name: Use Node.js<br>      uses: actions/setup-node@v2<br>      with:<br>        node-version: 18<br><br>    - run: npm install<br><br>    - run: npm run build<br><br>    - run: npm run lint<br><br>    - run: npm test</pre><p><strong>After:</strong></p><pre>jobs:<br>  build:<br>  uses:<br>    my-org/actions/.github/workflows/node.yml@master</pre><p>You may want to build a library of reusable workflows, designed to work independently or together. You can use inputs and outputs to pass data between them, just like with actions.</p><h3>Composite Actions</h3><p>I explained why you should use Reusable workflows to follow a Don’t Repeat Yourself (DRY) principle, which brings extensibility to your public repository with a single line of configuration. Now, let’s discuss Composite Actions and how they contrast with Reusable Workflows to help automate your work.</p><h4>What are Composite Actions?</h4><p>Composite Actions are a way to bundle multiple workflow steps into a single, reusable action. This can simplify your workflow by combining multiple run commands into one action. It’s important to note that Composite Actions are distinct from workflow reuse and require a separate repository to manage them. However, they offer the ability to create nested actions on top of other actions.</p><p>By using Composite Actions, you can avoid the need to create multiple workflow actions for different workflows. Instead, you can group together similar run steps and share them by calling the run step YAML. This can make workflow creation easier and more streamlined.</p><p>For example, consider defining a job called Hello with a description and input name that triggers an output based on a provided name. With a Composite Action, you can easily reuse this input action in other workflows.</p><pre>name: &#39;Hello&#39;<br>description: &#39;Say HI!&#39;<br><br>inputs:<br>  name:  # id of the input<br>    description: &#39;Who to say Hi&#39;<br>    required: true<br>    default: &#39;World&#39;<br><br>outputs:<br>  greeting:<br>    description: &quot;Greeting&quot;<br>    value: ${{ steps.greeting.outputs.greeting }}<br><br>runs:<br>  using: &quot;composite&quot;<br>  steps:<br>    - id: greeting<br>      shell: bash<br>      run: |<br>        echo &quot;greeting=Hello ${{ inputs.name }}&quot; &gt;&gt; $GITHUB_OUTPUT<br></pre><p>When calling a Composite Action from a workflow, the process is similar to calling a regular action. Once the Composite Action runs, it is then available as steps and outputs that can be reused in future workflows.</p><p>For example, if you use your name as the value to trigger the input workflow_dispatch event, you can call this Composite Action in future workflows and use the same input.</p><pre>name: Hello<br>on: [workflow_dispatch]<br><br>jobs:<br>  greeting:<br>    runs-on: ubuntu-latest<br>    steps:<br>    - name: Checkout this repo<br>      uses: actions/checkout@v3<br><br>    - name: Call Composite Action Hello<br>      uses: ./actions/hello<br>      id: hello<br>      with:<br>        name: &quot;foo&quot;<br><br>    - name: echo output from hello<br>      Run: |<br>        echo ${{ steps.hello.outputs.greeting }}</pre><h3>How are Composite Actions different from Reusable Workflows?</h3><ul><li>A Composite Action cannot be used without a repo checkout while Reusable Workflows can be used without a checkout</li><li>Composite Actions can incorporate multiple files while Reusable Workflows consist of one YAML file, which means it has no additional files sorted out by default</li><li>A Reusable Workflow can include multiple jobs and multiple steps within those jobs. However, Composite Actions can only have one step</li><li>Reusable Workflow can use Secrets by declaring them to a workflow via parameters while Composite Actions cannot use Secrets in a flexible way</li></ul><h3>Conclusion</h3><p>In conclusion, reusable workflows offer many benefits for organizations looking to streamline their workflow processes. By following the paved road principle and creating a single, shared workflow that can be used across multiple projects and teams, you can save time and effort while ensuring consistency and reliability.</p><p>However, it’s important to be aware of potential pitfalls when using reusable workflows. Changes made to the shared workflow can impact other workflows that depend on it, so it’s crucial to test thoroughly and communicate any updates to the team. Additionally, it’s important to be mindful of the potential for conflicts when multiple teams are using the same shared workflow.</p><p>By using reusable workflows, you can make your work processes more efficient and consistent. This can save you time, reduce errors, and improve teamwork within your organization. However, you need to be careful of potential risks and take steps to minimize them. This will help you enjoy the benefits of reusable workflows without any potential drawbacks.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb37172626e7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deep dive into SSG, SSR and how a CDN and HTTP caching can increase your performance by 10x]]></title>
            <link>https://blog.ah.technology/deep-dive-into-ssg-ssr-and-how-a-cdn-and-http-caching-can-increase-your-performance-by-10x-471cd18e7b0e?source=rss-468e730accf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/471cd18e7b0e</guid>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[server-side-rendering]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[static-site-generator]]></category>
            <category><![CDATA[cdn]]></category>
            <dc:creator><![CDATA[Auke ten Hoopen]]></dc:creator>
            <pubDate>Wed, 17 Feb 2021 20:15:43 GMT</pubDate>
            <atom:updated>2021-02-17T20:17:56.688Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*XvtZX9WExNgaIjXqJIpXtg.png" /></figure><p>First, we dive into the difference between<strong> Static site generation</strong> and <strong>Server-side rendering</strong>. How an old fashioned HTTP caching can increase your performance by 10 times without breaking a sweat!</p><h3>Static site generation.</h3><p>So it’s you and your computer. You have got your website and it’s time to build. So you say: yarn build or npm build . Let’s see what happens, well your computer’s fans spin up first of all.. and then you’re probably going to make some network requests. You&#39;re going to go off to some database somewhere or a CMS. Something that’s got the data that you going to use to build the page out of. There is work happening over on that server as well when you’re building. So it&#39;s not just your computer that&#39;s doing work here. There is stuff out in the cloud, stuff out on another server somewhere. Of course, you may have all this stuff locally on your machine.</p><p>But in most situations, you’re going off to some API somewhere and getting the data to build the static site, so it sends the data back to you. And now you have created a document. Then you do it again because you’ve got to make your next document, so we’re going back and forth and building documents. We do it again and again and again until we build every single document, every single page of your website.</p><p>Both of these things cost money. It costs you time, sitting there waiting for the build to finish. And it also cost money over on the servers, for them to process those requests so you can build the data. There’s compute happening locally, maybe you are doing a build in the cloud somewhere, you are paying for that too.</p><p>Whenever there’s compute happening, whether it’s some server or it’s your local machine. When you’re building a static site, that costs money.</p><h3>CDN</h3><p>Now you got a CDN and you upload everything to the CDN. So what’s really great about <strong>static site generation</strong> is that all those documents are pre-rendered. They’re pre-built. They’re just static they’re just sitting there on a CDN waiting for somebody to come and visit your website to get one of those documents. So they make a request and the CDN doesn’t have to do any work at all, it<strong> doesn’t have</strong> <strong>to build the page</strong>, <strong>doesn’t have</strong> <strong>to render anything</strong>. You can just send it right on back to the user and the user is super happy.</p><p>It’s a fast response and it was a cached response so the user is happy because it’s super snappy. Now let’s say you edit some of the data, what does that mean for static site generation?</p><p>Well, something in your database changed but your CDN still has all those documents from the last deploy. So if a user visits the page, they’re going to get a fast response but it’s stale and it’s not up to date yet. Sometimes that’s fine.</p><p>In order for us to turn that stale content into fresh content. We have to rebuild the website again. So it’s you, your computer and the server that’s got all your data. We’ve only got one little changed document in there but the way the static site generation works is, you’ve got to rebuild every single one of those pages. So that’s going to cost money and then you upload everything again to the CDN. This means that you build every page, for every deploy and for any edit.</p><h3>Server-side rendering</h3><p>Let&#39;s see what&#39;s the difference with Server-side rendering with no CDN involved. We don’t have a big build step. There’s no build step at all. You just upload your website to the internet and then you build the pages on demand.</p><p>So when somebody asks for a page, you build the page on your server and then you send that page back to the user. This takes time because it’s kind of slow, the user had to wait for the page to build. So static sites are nice because you built it before. You sat there and waited for the build so that your users don’t have to wait for the build.</p><p>Every single visit cost you money. You’re running compute on every single one of those things. However, if you have some pages that are never visited, you never build them!</p><blockquote>With static site generation, you’re building every single page whether people visit them or not. With the server, you only build the pages that people visit. If no one visits, then the server never builds it and never sends it.</blockquote><h3>Content delivery network</h3><p>Let’s throw a CDN into the middle here. The <strong>first</strong> visitor shows up. They actually make a request to the CDN, not to your server.</p><p><strong>CDN goes:</strong> <em>“Oh I don’t have that document yet”</em></p><ul><li>CDN goes over to your<strong> origin server.</strong></li></ul><blockquote>You may have heard of that term before. The origin server, that’s your actual webserver :)</blockquote><ul><li>The user is still waiting for your origin server to build the page.</li><li>The origin server is ready and sends the page to the CDN.</li><li>CDN caches the page</li><li>CDN then sends it to the user</li></ul><p>This still takes time because we had to wait for the whole cycle. This is just the <strong>first</strong> <strong>person ever to visit this URL.</strong></p><p>It costs us money to build that page and send it to them. However, with a CDN, the <strong>second</strong> visitor flow is:</p><ul><li>They request the page from the CDN</li><li>CDN goes: <em>“ah I know this! I’ve already gotten this from the origin server so I can leave the origin server alone”</em></li><li>CDN sends cached response right back to the user</li></ul><p>We got an immediate response because it was fast and it was cached and it was fresh, it was accurate. This saves you money too because you don’t do anything. You’re not building the page on the fly. Now, this isn’t just for the <strong>second</strong> visitor, this is the <strong>third</strong> the <strong>fourth</strong>, the <strong>fifth</strong>, the <strong>hundred</strong>, the <strong>millionth</strong> visitor.</p><h3>Control your cache</h3><p>Then there’s this idea of max-age when you have a CDN. There is a cache-control header and max-age is one of the values in there that says:</p><pre>Cache-Control: max-age=&lt;seconds&gt;</pre><p><a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control">Cache-Control - HTTP | MDN</a></p><p><em>“Hey, this is how long you should cache this thing. It’s in seconds so you can say, cache it for 60 seconds, cache it for a day or cache it for a year.”</em></p><p>Let’s say that we were caching this for like a day and that time has passed. Well, what’s going to happen is a visitor shows up and says:</p><ul><li>User: Hey, I want that page</li><li>CDN: “<em>Oops, this request is stale…</em>”</li><li>CDN: <em>“This is old. I was told to rebuild this or to go to the origin server and have it rebuild it </em>after a day.<em>”</em></li></ul><p><strong>It’s been a day</strong></p><ul><li>CDN goes to the origin server and says: <em>“hey I want the page.”</em></li><li>You build the page on your origin server”</li><li>It comes back to the CDN</li><li>CDN puts the new fresh one in the cache</li><li>CDN then sends it to the user.</li></ul><p>And of course, we got a “slow” response and then we had to pay money for our server to build that page. But every visitor after that, they now have a fast and fresh page.</p><p>What’s interesting about this is, when we have an edit we don’t need to rebuild the whole website like with <strong>static site generation.</strong> It’s just the pages that change that need to get rebuilt and with max-age you can just kind of put it on a timer and say:</p><blockquote>“You know what, cache it just every day. If someone comes to this page again rebuild it but everyone else for the rest of the day just give them that built version”</blockquote><p>Or maybe you decide this page is good for a week and you can just make an edit anytime during that week and it’s not going to show up on your website yet but after that.</p><blockquote>The CDN is like: “Oh hey this thing’s expired. I’m going to rebuild it, then everyone will get the fresh one”</blockquote><p>You kind of get to decide how often do you want it to get built. Rather than having to build every single page.</p><h3>Let&#39;s visualize how this works</h3><p>🚢 Deploy website<br>📝 Data changed<br>🥱 User waits for the page<br>😄 Cached response<br>😅 Cached but stale<br>🕐 Max-age expired</p><h3><strong>No caching<br></strong>🚢🥱🥱📝🥱🥱🥱🚢🥱🥱🥱🥱🥱</h3><p>If you don&#39;t have any caching and you don&#39;t have a CDN. You ship the site and everybody is a yawner but everybody gets fresh data too. This is what we want to avoid. It&#39;s expensive for you because for every request you are building a page for that user. But what&#39;s good about it, everybody gets something fresh!</p><h3>max-age=31540000000<br>🚢🥱😄😄📝😅😅🚢😅😅😅😅😅</h3><p>Max-age for a year. You ship the website. The first person ever to visit a URL on that website get a yawner because they have got to wait for the page to build. But then everybody after that gets a smiler until you edit. Once you edit, you get the sweater because we said, this thing lasts for a year. So even if you edit it, the CDN is never going back to get a fresh copy. You can even deploy your website again, nothing is going to change. Because you told the CDN, this thing is good for a year. So no one will ever get a new version of that document.</p><p><strong>This is perfect for static assets. Like JavaScript and CSS.</strong> You probably notice that in a Webpack build, it add this fingerprint, hash to the name of the file. Every time that file changes it gets a new hash that you deploy. This is great because the URL changes, CDN has not seen that before. If the URL never changes we can just cache these assets forever. <strong>This doesn’t work for your normal URLs. Like a blog post or an about page. Anything that has a URL that doesn&#39;t change when the content changes.</strong></p><h3>max-age=60<br>🚢🥱😄😄📝😅🕐🥱😄😄🕐🥱😄</h3><p>We saying this is good for 60 seconds — 1 minute — You ship the website. You get a yawner for the first visitor and then for the next 60 seconds everyone is a smiler. Fast cached responses. You edit the page, you get sweater because it&#39;s fast but it&#39;s stale. That&#39;s fine, it’s only going to be stale for another few seconds. Maybe another 10 seconds. So once the timer hits, then the next visitor is going to cause the CDN to go to the origin server and come back again with the fresh copy. They get the fresh copy of the page but they are yawning. It took a little while but then everyone for the next 60 seconds after that person is going to get fast responses. And so on.</p><p><strong>Every 60 seconds, somebody is going to have a longer response time then someone else.</strong></p><h3>Target purge + s-maxage=3.154e7<br>🚢🥱😄😄📝🚫🥱😄😄😄😄😄😄</h3><p>So one strategy to mitigate having so many yawners in there is to say: lets set a really long max-age. There is this other cache-control value called s-maxage and that&#39;s for the surrogate. That&#39;s for the CDN. So far we have been talking about max-age . Max-age is just in the browser and the CDN will use it. But if you say max-age and s-maxage . Then s-maxage will say to the CDN: hey, I&#39;m not going to cache it for max-age, I&#39;m going to cache it for s-max-age. This is great because you can now tell the CDN to cache it differently than the browser. Which allows you to go in and purge — The red stop sign. What you can do now is, you can have these targeted purges.</p><p>You can now say: on my CMS, whenever I change one of these articles. I know what the URL is. So now I can go over and purge my CDN of that document. You make an edit, you write a little script that says. This URL we gotta purge that from the CDN. Now the next visitor is a yawner. But everyone else is happy after that. This is a really cool strategy. Takes more time and more code but if you can automate it..great approach!</p><p><strong>Important</strong></p><blockquote>You dont want a long max-age here. Because you cannot purge the user’s browser. If you dont use s-maxage with this strategy and you have a visitor who goes to your page. They are going to have that page for a year. There is no way when your CMS edits a page, to go to all your users browsers and say: purge this thing. Thats why we set a smaller max-age for the browser, like 60 seconds and a big s-maxage for the CDN. You can set up targeted purges to purge the pages that you are changing.</blockquote><h3>SSR Stale While Revalidate</h3><p>How does static site generation fit into this?</p><p><a href="https://web.dev/stale-while-revalidate/">Keeping things fresh with stale-while-revalidate</a></p><h3>Static Site Generation<br>🛠🚢😄😄📝😅😅😅😅🛠🚢😄😄</h3><p>Well, we’ve got a new emoji. The tools icon, that’s the big build where you’ve got to create every single page on your website before you can deploy it.</p><p>We’ve got the big build first, but now everybody’s a smiler because you’re getting superfast fresh responses. You edit the page, now everybody’s getting stale stuff until you do the big fat build again and you deploy. After that everybody gets happy stuff again. It basically comes down to any time any data changes you have to redeploy.</p><p>For some websites that&#39;s fine, other times it can be just way too expensive. Remember it’s not just your machine. You might offload it to some cloud machine. Either way, some machine somewhere is doing that work whether it’s your origin server or your build server, you’re paying for a computer to build those pages one way or the other.</p><h3>Incremental Site Regeneration<br>🛠🚢😄😄🔧😅😄🔧😅🛠🚢😄</h3><p>You may have heard of NextJS’s <strong>incremental static regeneration?</strong> The idea here is, it’s a kind of a bummer that in static site generation we are forcing dynamic data to be static data. To get fast responses we kind of have to pretend that our data doesn&#39;t change or at least not very often. But data does change often. NextJs solution is <strong>incremental static regeneration.</strong></p><p>What happens is, we do the big build. So you’re still going to build the whole site, every single page of the website and then you’re going to deploy it. Everyone gets fast fresh responses. We got the single tool in there that represent rebuilding just a single page. So the way <strong>incremental site regeneration</strong> works is, when someone visits a page, it triggers a build of just that page. But it sends the current version of that page or the stale version. It just sends them what is already built and then builds a new one in the background.</p><p>Now the person after that first sweaty emoji, they’re gonna get the result of the last incremental build. Then someone else visits the page and it kicks off the regeneration again in the background. It sends them the current version of the page, builds the new one. Now that new page is sitting waiting there for the next visitor. Next visitor shows up, gives them the current version of the page and then sets up a build in the background again.</p><p>This way everyone’s getting fast responses. Everyone’s getting a pre-rendered document. And everyone’s getting something pretty fresh. It’s probably just off by a little bit. Everybody is just one-off, by shifting it by one, everybody gets a fast response. And you only build the parts that you need. Instead of the whole thing.</p><p><strong>You&#39;re always off by one, but it is always fast</strong></p><h3>Stale While Revalidate</h3><p>Incremental static site regeneration from NextJs is inspired by a <strong>cache-control </strong>value called <strong>stale while revalidate.</strong> Let’s look at what stale while revalidate is.</p><p>It&#39;s just for the browser has nothing to do with <strong>static site generation</strong>, or with <strong>NextJS</strong> or any of that kind of stuff. It’s just a part of<strong> HTTP caching</strong>.</p><p><strong>SSR with stale while revalidate.</strong> <br>This is our first visitor after the max-age is up. You remember, last time when the max-age is up.</p><ul><li>Visitor asks for the page.</li><li>It’s expired, the CDN would go to the origin server to build it.</li></ul><p><strong>Not with stale while revalidate.</strong></p><p>Stale while revalidate says:</p><ul><li><em>“Hey, you know what, I’m just going to give you what I&#39;ve got right now. This document is expired but I really want to just give you what you want right now”</em></li><li>It’s <strong>stale</strong> but it’s <strong>fast</strong></li><li>Then the CDN once it’s done with that request, says: <em>“Okay, I’m going to go over the origin server. I&#39;m going to revalidate this page”</em></li><li>It starts building on your origin server. If somebody shows up in the meantime, maybe you’ve got like a visitor right after another visitor while you’re origin server is still building. The CDN is going to go: <em>“Hey, yup here have the stale one again. I’m still figuring this thing out”.</em></li><li>Finally, our origin server is done. Sends the new document to the CDN.</li><li>CDN caches that new fresh one</li><li>Now anybody who shows up afterwards is gonna get the new fresh one.</li></ul><h3>max-age=1, stale-while-revalidate=3.154e7<br>🚢🥱😄😄🕐😅😄🕐😅🚢🕐😅😄</h3><p>If we say a max-age of one. We are saying: <em>“Hey this thing expires every second because our data might be changing every second.”</em></p><p>We think that this page is highly dynamic, but we don’t want the user to have to wait for the server to build that page every time. We want to give them a fast response, so we just shifted it by one.</p><p>After the deploy, we first get a yawner that’s just the very first time ever. Never again, even after we re-deploy we’re not going to get a yawner at this URL. When the max-age hits we get a sweater because it’s fast but it’s stale, we’re revalidating. Now the next person who shows up is going to get a fast cached response so we got a smiler. Then the clock ticks again. Then we send something that’s stale. We could deploy in the middle, it doesn’t affect anything. If we deploy in the middle, we don’t have to wait for that big static build either. We don’t have to build all those pages again, we’re only building the pages that people visit!</p><p>You can kind of look at the two, <strong>incremental static regeneration </strong>and<strong> stale while revalidate. </strong>You can see that they are actually the same for the user. Everybody gets fast responses. Expect for that very first person, after the initial deploy of this URL ever.</p><h3>Conclusion</h3><p>We don&#39;t want a whole bunch of our requests to be slow. We want them all to be fast. So <strong>incremental static regeneration </strong>is saying why don&#39;t we give them the one from a second ago.</p><p>If we’re at the 60 second mark and this page is probably stale now. There’s no reason we can’t give just one more person that old copy. They are only one second off. If they were there a second earlier they would have gotten the same thing that we’re giving them a second later.</p><h3>Increment static site regeneration — NextJs<br>😄😄🕐😅🆕😄😄🕐😅🆕😄😄</h3><p>You can see the clock hits and instead of waiting to build we just send them what we’ve got. New stuff shows up after that request and now everybody gets new fresh responses. Everyone’s smiling no one’s yawning.</p><h3>max-age=1, stale-while-revalidate<br>😄😄🕐😅🆕😄😄🕐😅🆕😄😄</h3><p>Now you can see that with max-age of 1 and stale while revalidate without incremental static site generation, just server-side rendering and a CDN. An URL comes in, and it just builds that page. Then you place a CDN in front of it. So you are building static pages. Just on demand instead of all at once. You can see its the very same where everybody gets a fast response. We are just shifting it over by one. We are trading, sending someone something stale that&#39;s only a second stale. So that everyone gets fast responses.</p><p>Really it’s this question, If you using cache headers and a CDN and server-side rendering.</p><p><strong>How often do you want to rebuild?</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=471cd18e7b0e" width="1" height="1" alt=""><hr><p><a href="https://blog.ah.technology/deep-dive-into-ssg-ssr-and-how-a-cdn-and-http-caching-can-increase-your-performance-by-10x-471cd18e7b0e">Deep dive into SSG, SSR and how a CDN and HTTP caching can increase your performance by 10x</a> was originally published in <a href="https://blog.ah.technology/">AH Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A guide through The Wild Wild West of setting up a mono repo — Part 2: Adding Jest with a breeze]]></title>
            <link>https://blog.ah.technology/a-guide-through-the-wild-wild-west-of-setting-up-a-mono-repo-part-2-adding-jest-with-a-breeze-16e08596f0de?source=rss-468e730accf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/16e08596f0de</guid>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[jest]]></category>
            <category><![CDATA[monorepo]]></category>
            <category><![CDATA[development]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Auke ten Hoopen]]></dc:creator>
            <pubDate>Thu, 11 Feb 2021 20:53:07 GMT</pubDate>
            <atom:updated>2021-02-12T09:05:30.908Z</atom:updated>
            <content:encoded><![CDATA[<h3>A guide through The Wild Wild West of setting up a mono repo — Part 2: Adding Jest to our mono repo</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iSw9QZlQAfBlvGZszgd7Qg.jpeg" /></figure><p>This is <strong>part two</strong> of our mono repo setup. In the <a href="https://medium.com/ah-technology/a-guide-through-the-wild-wild-west-of-setting-up-a-mono-repo-with-typescript-lerna-and-yarn-ed6a1e5467a">first part</a>, we created a mono repo with Lerna and TypeScript. In this part, we will focus on adding <strong>Jest</strong> to our setup.</p><p>You can find the full example and more in the template repo:</p><p><a href="https://github.com/ahoopen/typescript-mono-repo/tree/master">https://github.com/ahoopen/typescript-mono-repo</a></p><h3>Installation</h3><p>Let&#39;s install our dependencies. Use one of the following commands.</p><p><strong>NPM</strong></p><pre>npm install --save-dev jest ts-jest</pre><p><strong>Yarn</strong></p><pre>yarn add --dev jest ts-jest</pre><p>After this is done installing, let&#39;s create our Jest config file. By default, Jest can run without any config files, but it will not compile files. To make it transpile TypeScript with ts-jest , we will need to create a configuration file that will tell Jest to use a ts-jest preset. ts-jest can create the configuration file for you automatically.</p><p><strong>NPM</strong></p><pre>npx ts-jest config:init</pre><p><strong>Yarn</strong></p><pre>yarn ts-jest config:init</pre><p>This command will create a jest.config.js file in our root. This config will inform Jest about how to handle files correctly.</p><h3>Testing packages</h3><p>Before we can run our tests we first need to create them. Let’s create a simple function for adding numbers. Our add function, beautiful right? :)</p><p><em>packages/web/src/add.ts</em></p><pre>export const add = (a: number, b: number) =&gt; {<br>  return a + b;<br>};</pre><p>And the unit test for it:</p><p><em>packages/web/src/add.spec.ts</em></p><pre>import { add } from &quot;./add&quot;;</pre><pre>describe(&quot;add function&quot;, () =&gt; {<br>   it(&quot;should add two number together&quot;, () =&gt; {<br>      const result = add(10, 5);<br>      expect(result).toBe(15);<br>   });<br>});</pre><p>Now that we have our add function and unit test in place, we can start adding our test command — yarn jest — to our package. Let&#39;s add it to our webpackage.</p><pre>{<br>   &quot;name&quot;: &quot;@ahoopen/web&quot;,<br>   &quot;version&quot;: &quot;1.0.0&quot;,<br>   &quot;main&quot;: &quot;dist/index.js&quot;,<br>   &quot;types&quot;: &quot;dist/index.d.ts&quot;,<br>   &quot;repository&quot;: {<br>     &quot;type&quot;: &quot;git&quot;,<br>     &quot;directory&quot;: &quot;packages/web&quot;,<br>     &quot;url&quot;: &quot;https://github.com/ahoopen/typescript-mono-repo.git&quot;<br>   },<br>   &quot;scripts&quot;: {<strong><br></strong>      &quot;clean&quot;: &quot;rimraf dist&quot;,<br>      &quot;prepack&quot;: &quot;yarn build&quot;,<br>      &quot;build&quot;: &quot;yarn clean &amp;&amp; yarn compile&quot;,<br>      &quot;compile&quot;: &quot;tsc &amp;&amp; cp \&quot;./package.json\&quot; ./dist/&quot;,<strong><br>      &quot;test&quot;: &quot;yarn jest&quot;,<br>      </strong>&quot;lint&quot;: &quot;eslint \&quot;./src/**/*.{ts,tsx}\&quot; --max-warnings=0&quot;<strong><br></strong>   },<strong><br>   </strong>&quot;dependencies&quot;: {<br>     &quot;react&quot;: &quot;16.13.1&quot;<br>   }<br>}</pre><p>Now run yarn test in the root of our repository. It will run tests in all of our packages. So far nothing crazy, I know. Testing in a mono repo gets a lot harder when you have multiple packages and some of them depend on each other.</p><h3>Dependency on own mono repo package</h3><p>What if we want to enhance ouradd method with a util method from our @ahoopen/utilspackage. This is a bit of a contrived example. It doesnt make any sense but its only to illustrate the problem :)</p><pre><strong>import { multiplyByTwo } from &quot;@ahoopen/utils&quot;;</strong></pre><pre>export const add = (a: number, b: number) =&gt; {<br>   <strong>console.log(multiplyByTwo(a));</strong></pre><pre>   return a + b;<br>};</pre><p>Next step is to add the package to our dependencies.</p><pre>&quot;dependencies&quot;: {<br>   &quot;react&quot;: &quot;16.13.1&quot;,<br><strong>   &quot;@ahoopen/utils&quot;: &quot;1.0.0&quot;<br></strong>},</pre><p>Now we only have to run yarn test and we are done, right? Let&#39;s see…</p><pre>Test suite failed to run</pre><pre>Cannot find module &#39;<a href="http://twitter.com/royalaholddelhaize/ah-advertisements">@ahoopen/</a>utils&#39; from &#39;packages/web/src/add.spec.ts&#39;</pre><p>Tests fail…but why? The reason why this fails is, that in our TypeScript tsconfig we have mapped our package @ahoopen/utils to @ahoopen/utils/src</p><p>Our tsconfig</p><pre>{<br>   &quot;compilerOptions&quot;: {<br>      &quot;jsx&quot;: &quot;react&quot;,<br>      &quot;baseUrl&quot;: &quot;./packages&quot;,<br>      &quot;paths&quot;: {<br>         &quot;@ahoopen/core/*&quot;: [&quot;core/src/*&quot;],<br>         &quot;@ahoopen/web/*&quot;: [&quot;web/src/*&quot;],<br>         &quot;@ahoopen/utils/*&quot;: [&quot;utils/src/*&quot;]<br>      }<br>   }<br>}</pre><p>Why don’t we remove that mapping? Then the tests also work immediately.</p><p>Well, this mapping helps us to easily navigate through all the different packages and when we make a code change, it will be picked up in all packages without needed to build them first. But the downside with this approach is that Jest can’t resolve the packages.</p><p>Luckily we have a solution to this problem.</p><h3>Path mapping in Jest</h3><p>In our Jest config, we have to set up the “<a href="https://jestjs.io/docs/en/configuration.html#modulenamemapper-object-string-string">moduleNameMapper</a>” correctly to fix this issue.</p><p><em>jest.config.js</em></p><pre><strong>const path = require(&#39;path&#39;);<br>const { lstatSync, readdirSync } = require(&#39;fs&#39;);</strong></pre><pre><strong>// get listing of packages in the mono repo<br>const basePath = path.resolve(__dirname, &#39;packages&#39;);<br>const packages = readdirSync(basePath).filter(name =&gt; {<br>   return lstatSync(path.join(basePath, name)).isDirectory();<br>});</strong></pre><pre>module.exports = {<br>  preset: &#39;ts-jest&#39;,<br>  testEnvironment: &#39;node&#39;,</pre><pre><strong>  moduleNameMapper: {<br>     ...packages.reduce(<br>        (acc, name) =&gt; ({<br>           ...acc,<br>           [`@ahoopen/${name}(.*)$`]:<br>           `&lt;rootDir&gt;/packages/./${name}/src/$1`,<br>      }),<br>    {},<br>   ),<br>  },</strong></pre><pre>};</pre><p>Oke, so what&#39;s happing here? We read the packages folder to get every package name. Then we automatically generate a moduleNameMapper. Which boils down to the same mapping as with the tsconfig . We map the import path to the package src directory.</p><p>This will be the generated output:</p><pre>moduleNameMapper: {<br>   &#39;<a href="http://twitter.com/ahoopen/core">@ahoopen/core</a>(.*)$&#39;: &#39;&lt;rootDir&gt;/packages/./core/src/$1&#39;,<br>   &#39;<a href="http://twitter.com/ahoopen/utils">@ahoopen/utils</a>(.*)$&#39;: &#39;&lt;rootDir&gt;/packages/./utils/src/$1&#39;,<br>   &#39;<a href="http://twitter.com/ahoopen/web">@ahoopen/web</a>(.*)$&#39;: &#39;&lt;rootDir&gt;/packages/./web/src/$1&#39;<br>},</pre><p>If we run yarn test again, our tests pass :)</p><pre>PASS  packages/web/src/add.spec.ts<br>  add function<br>    ✓ should add two number together (2 ms)</pre><pre>Test Suites: 1 passed, 1 total<br>Tests:       1 passed, 1 total<br>Snapshots:   0 total<br>Time:        0.711 s, estimated 3 s<br>Ran all test suites.<br>✨  Done in 2.11s.</pre><h3>Conclusion</h3><p>We have added Jest to our mono repo. We can now run Jest in all of our TypeScript packages. To get it all working nicely together we needed to add a moduleNameMapper to our jest config. By automatically reading our package folder. We don’t have to keep adjusting our jest config when we create a new package.</p><p>The next part we will set up Github actions for releasing our packages. Stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=16e08596f0de" width="1" height="1" alt=""><hr><p><a href="https://blog.ah.technology/a-guide-through-the-wild-wild-west-of-setting-up-a-mono-repo-part-2-adding-jest-with-a-breeze-16e08596f0de">A guide through The Wild Wild West of setting up a mono repo — Part 2: Adding Jest with a breeze</a> was originally published in <a href="https://blog.ah.technology/">AH Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A guide through The Wild Wild West of setting up a mono repo with TypeScript, Lerna and Yarn…]]></title>
            <link>https://blog.ah.technology/a-guide-through-the-wild-wild-west-of-setting-up-a-mono-repo-with-typescript-lerna-and-yarn-ed6a1e5467a?source=rss-468e730accf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/ed6a1e5467a</guid>
            <category><![CDATA[monorepo]]></category>
            <category><![CDATA[lerna]]></category>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[yarn-workspace]]></category>
            <dc:creator><![CDATA[Auke ten Hoopen]]></dc:creator>
            <pubDate>Tue, 08 Sep 2020 12:58:51 GMT</pubDate>
            <atom:updated>2021-02-11T20:56:31.401Z</atom:updated>
            <content:encoded><![CDATA[<h3>A guide through The Wild Wild West of setting up a mono repo with TypeScript, Lerna and Yarn Workspaces</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*XZZsEUQ6zasbjx9AtqceOw.jpeg" /></figure><p>This is were the fun starts! Setting up a mono repo from scratch. We will be using <strong>TypeScript</strong>, <strong>Yarn workspaces</strong>, <strong>Lerna</strong>, and <strong>Jest</strong>. The release of the packages will be done with<strong> Github Actions</strong>.</p><p>In this part, I will explain step by step how to setup <strong>Lerna, Yarn Workspaces</strong> and <strong>Typescript </strong>in a mono repo. In the next article, we will focus on how to get <strong>Jest</strong> working. And the last part, we will handle the release of the packages with <strong>Github Actions</strong>.</p><ul><li>Part two: <a href="https://ahoopen.medium.com/a-guide-through-the-wild-wild-west-of-setting-up-a-mono-repo-part-2-adding-jest-with-a-breeze-16e08596f0de">Adding Jest to the mono repo</a></li></ul><p>You can find the full example and more in the template repo:</p><p><a href="https://github.com/ahoopen/typescript-mono-repo/tree/master">https://github.com/ahoopen/typescript-mono-repo</a></p><h3>Lerna</h3><p>What is <strong>Lerna</strong> and why do you need it?</p><blockquote>Lerna is a tool that optimizes the workflow around managing multi-package repositories with git and npm.</blockquote><p><a href="https://github.com/lerna/lerna">GitHub - lerna/lerna: A tool for managing JavaScript projects with multiple packages.</a></p><p><strong>So how does a mono repo with Lerna looks like?<br></strong>There’s actually very little to it. You have a file structure that looks like this:</p><pre><strong>mono-repo/</strong><br>  package.json<br>  <strong>packages/</strong><br>    <strong>package-a/</strong><br>      package.json<br>   <strong> package-b/</strong><br>      package.json</pre><h3>Getting started</h3><p>We will use Yarn as our package manager. Mainly because we are using Yarn Workspaces . So if you haven&#39;t installed it yet. Please do.</p><p><a href="https://classic.yarnpkg.com/en/docs/install">https://classic.yarnpkg.com/en/docs/install</a></p><p>Let’s start by installing Lerna as a dev dependency of your project</p><pre>npx lerna init</pre><p>This will create a lerna.json configuration file as well as a packages folder, so your folder should now look like this:</p><pre><strong>mono-repo</strong><br>├── <strong>packages</strong><br>├── README.MD<br>├── lerna.json<br>├── package.json</pre><p>We still need some additional dependencies. So let&#39;s install them:</p><pre>yarn add --dev typescript jest eslint</pre><p>Now that all the dependencies are installed, let&#39;s continue with the Lerna.json</p><h3>Lerna configuration</h3><p>The lerna.json is where all the configuration happens for Lerna.</p><pre>{<br>   &quot;packages&quot;: [&quot;packages/*&quot;],<br>   &quot;npmClientArgs&quot;: [&quot;--no-lockfile&quot;],<br>   &quot;version&quot;: &quot;independent&quot;,<br>   &quot;command&quot;: {<br>      &quot;version&quot;: {<br>         &quot;ignoreChanges&quot;: [&quot;*.md&quot;],<br>         &quot;npmClient&quot;: &quot;npm&quot;,<br>         &quot;message&quot;: &quot;chore(release): publish&quot;<br>      },<br>      &quot;publish&quot;: {<br>         &quot;npmClient&quot;: &quot;npm&quot;<br>      }<br>   }<br>}</pre><ul><li>packages: An array of globs to use as package locations.</li><li>version At default, this is in <em>fixed</em> mode. Which keeps all packages versions in sync, but you can change this to independent if you’d like to manage package versions independent from one another.</li><li>command.version.ignoreChanges: an array of globs that won&#39;t be included in lerna changed/publish. Use this to prevent publishing a new version unnecessarily for changes, such as fixing a README.md typo.</li><li>command.version.message: a custom commit message when performing version updates for publication. See <a href="https://github.com/lerna/lerna/blob/master/commands/version#--message-msg">@lerna/version</a> for more details.</li></ul><p>With that out of the way, the next step is adding packages to our mono repo.</p><h3>Create our first mono repo packages</h3><p>Let’s create some packages in our repo. Create package/web and package/core. The content of the package should be as following:</p><pre>├── src<br>│   ├── components<br>│   │   └── button.tsx<br>│   └── index.ts<br>├── package.json<br>├── tsconfig.json</pre><p>The package.json should look like this:</p><pre>{<br>   &quot;name&quot;: &quot;@namespace/core&quot;,<br>   &quot;version&quot;: &quot;1.0.0&quot;,<br>   &quot;main&quot;: &quot;dist/index.js&quot;,<br>   &quot;types&quot;: &quot;dist/index.d.ts&quot;,<br>   &quot;repository&quot;: {<br>     &quot;type&quot;: &quot;git&quot;,<br>     &quot;directory&quot;: &quot;packages/core&quot;,<br>     &quot;url&quot;: &quot;https://github.com/ahoopen/typescript-mono-repo.git&quot;<br>   },<br><strong>   &quot;scripts&quot;: {<br>      &quot;clean&quot;: &quot;rimraf dist&quot;,<br>      &quot;prepack&quot;: &quot;yarn build&quot;,<br>      &quot;build&quot;: &quot;yarn clean &amp;&amp; yarn compile&quot;,<br>      &quot;compile&quot;: &quot;tsc &amp;&amp; cp \&quot;./package.json\&quot; ./dist/&quot;,<br>      &quot;test&quot;: &quot;echo &#39;no test&#39;&quot;,<br>      &quot;lint&quot;: &quot;eslint \&quot;./src/**/*.{ts,tsx}\&quot; --max-warnings=0&quot;<br>   },<br>   </strong>&quot;dependencies&quot;: {<br>     &quot;react&quot;: &quot;16.13.1&quot;<br>   }<br>}</pre><p>And the tsconfig.json</p><pre>{<br>    &quot;compilerOptions&quot;: {<br>       &quot;target&quot;: &quot;es5&quot;,<br>       &quot;outDir&quot;: &quot;./dist&quot;,<br>       &quot;module&quot;: &quot;commonjs&quot;,<br>       &quot;jsx&quot;: &quot;react&quot;,<br>       &quot;strict&quot;: true,<br>       &quot;allowSyntheticDefaultImports&quot;: true,<br>       &quot;esModuleInterop&quot;: true,<br>       &quot;skipLibCheck&quot;: true,<br>       &quot;forceConsistentCasingInFileNames&quot;: true<br>    }<br>}</pre><h3>Running Lerna commands</h3><pre>{<br>   &quot;name&quot;: &quot;root&quot;,<br>   &quot;private&quot;: true,<strong><br>   &quot;scripts&quot;: {<br>      &quot;build&quot;: &quot;lerna run build --stream&quot;,<br>      &quot;lint&quot;: &quot;lerna run lint --stream --parallel&quot;,<br>      &quot;test&quot;: &quot;yarn jest --coverage&quot;<br>   },</strong><br>   &quot;devDependencies&quot;: {<br>      &quot;eslint&quot;: &quot;7.7.0&quot;,<br>      &quot;jest&quot;: &quot;26.4.2&quot;,<br>      &quot;lerna&quot;: &quot;3.22.1&quot;,<br>      &quot;typescript&quot;: &quot;4.0.2&quot;<br>   }<br>}</pre><p>The lerna runcommand will run in every package of the mono repo. --stream flag will display the build output in the terminal and the --parallel flag will execute the command in parallel.</p><p><strong>Caveat</strong></p><blockquote>because of sub-dependencies (packages <em>a</em> and <em>b</em> depend on <em>c</em>, d depends on <em>a</em>, <em>b</em>, <em>c</em>) we need to build packages accordingly, e.g. first <em>c</em>, second <em>a</em> and b, third d. That is why we can&#39;t use <em>--parallel</em> flag for <em>lerna</em> for build command.</blockquote><p>Let&#39;s build our packages with Lerna. Run the following command in the root of our repository.</p><pre>yarn build</pre><p>If everything goes well, you will have the following output</p><pre>lerna success run Ran npm script &#39;build&#39; in 2 packages in 2.7s:<br>lerna success - core<br>lerna success - web</pre><h3>Optional: migrate an existing package to the mono repo</h3><pre>yarn lerna import ~/projects/my-npm-repo-package-1 — flatten</pre><p>What’s great about Lerna is that it brings in all of the git commits along with it, such that for the new repo, the history looks like development has always been happening in this monorepo. This is crucial because you are going to need the git history when someone discovers a bug after moving to the monorepo.</p><h3>Why Lerna alone is not enough</h3><p>Lerna calls yarn/npm install for each package inside the project and then creates symlinks between the packages that refer to each other.</p><p>Being a wrapper of a package manager, Lerna can’t manipulate the contents of node_modules efficiently:</p><ul><li>Lerna calls yarn install multiple times for each package which creates overhead because each package.json is considered independent and they can’t share dependencies with each other. This causes a lot of duplication for each node_modules folder which quite often uses the same third-party packages.</li><li>Lerna manually creates links between packages that refer to each other after the installation has finished. This introduces inconsistency inside node_modules that a package manager may not be aware of, so running yarn install from within a package may break the meta structure that Lerna manages.</li></ul><p>These kinds of issues call for support of multi-package repositories directly in a package manager. Luckily there is one who supports this.</p><h3>Introducing Yarn workspaces</h3><blockquote>Yarn Workspaces is a feature that allows users to install dependencies from multiple package.json files in subfolders of a single root package.json file, all in one go.</blockquote><p>Yarn Workspaces enables faster, lighter installation by preventing package duplication across Workspaces. Yarn can also create symlinks between Workspaces that depend on each other and will ensure the consistency and correctness of all directories.</p><p><a href="https://classic.yarnpkg.com/blog/2017/08/02/introducing-workspaces/">Workspaces in Yarn</a></p><p>Let&#39;s enable Yarn workspaces in our mono repo. First, go to your lerna.json file and add the following fields to your configuration:</p><ul><li>npmClient We set this to Yarn because we are going to use Yarn workspaces</li><li>useWorkspaces this will let Lerna know that we are using Yarn workspaces.</li></ul><pre>{<br>   &quot;packages&quot;: [&quot;packages/*&quot;],<br>   <strong>&quot;npmClient&quot;: &quot;yarn&quot;,<br>   &quot;useWorkspaces&quot;: true,<br></strong>   &quot;npmClientArgs&quot;: [&quot;--no-lockfile&quot;],<br>   &quot;version&quot;: &quot;independent&quot;<br>}</pre><p>In our rootpackage.json we have to add this to enable yarn workspaces:</p><pre>{<br>   &quot;name&quot;: &quot;root&quot;,<br>   &quot;private&quot;: true,<br><strong>   &quot;workspaces&quot;: {<br>      &quot;packages&quot;: [&quot;packages/*&quot;]<br>   },<br></strong>   &quot;scripts&quot;: {<br>      &quot;build&quot;: &quot;lerna run build --stream&quot;,<br>      &quot;lint&quot;: &quot;lerna run lint --stream --parallel&quot;,<br>      &quot;test&quot;: &quot;yarn jest --coverage&quot;<br>   },<br>   &quot;devDependencies&quot;: {<br>      &quot;eslint&quot;: &quot;7.7.0&quot;,<br>      &quot;jest&quot;: &quot;26.4.2&quot;,<br>      &quot;is-ci&quot;: &quot;2.0.0&quot;,<br>      &quot;rimraf&quot;: &quot;3.0.2&quot;,<br>      &quot;lerna&quot;: &quot;3.22.1&quot;,<br>      &quot;typescript&quot;: &quot;4.0.2&quot;<br>   }<br>}</pre><p>When running Lerna commands, it will now use Yarn Workspaces to bootstrap the project and also it will use package.json/workspaces field to find the packages instead of lerna.json/packages.</p><h3>Adding TypeScript to the mix</h3><p>Setting up <strong>Lerna</strong> and<strong> Yarn Workspaces</strong> is pretty straightforward. To set up TypeScript we have to configure a bit more because we have to transpile our code before publishing and this effects navigating the source code in our IDE.</p><p>We going to configure our mono repo in such a way that it will feel exactly as it would in a regular NPM package, meaning that <strong>Go to definition</strong> will work after a fresh clone.</p><h3><strong>Common compiler options</strong></h3><p>In the root of our mono repo we create a tsconfig.build.json . This will centralize our common compiler options. This way you can change a setting in one file rather than having to edit multiple files. All packages will extend this configuration file.</p><pre>{<br>    &quot;compilerOptions&quot;: {<br>       &quot;target&quot;: &quot;es5&quot;,<br>       &quot;outDir&quot;: &quot;./dist&quot;,<br>       &quot;module&quot;: &quot;commonjs&quot;,<br>       &quot;jsx&quot;: &quot;react&quot;,<br>       &quot;strict&quot;: true,<br>       &quot;allowSyntheticDefaultImports&quot;: true,<br>       &quot;esModuleInterop&quot;: true,<br>       &quot;skipLibCheck&quot;: true,<br>       &quot;forceConsistentCasingInFileNames&quot;: true<br>    }<br>}</pre><p>Create packages/core , packages/utils and packages/web . (if you haven’t done already) An add atsconfig.json to each one. This tsconfig.json will extend our common compiler options.</p><pre>{<br>   &quot;extends: &quot;../../tsconfig.build.json&quot;,</pre><pre>   &quot;compilerOptions&quot;: {<br>      &quot;rootDir&quot;: &quot;./src&quot;,<br>      &quot;outDir&quot;: &quot;./dist&quot;<br>   },</pre><pre>   &quot;include&quot;: [&quot;src/**/*&quot;]<br>}</pre><p>For each tsconfig, we have to declare the rootDir, outDir , include properties. They can’t be hoisted in the root config because they are resolved relative to the config they are in.</p><p><a href="https://github.com/microsoft/TypeScript/issues/29172#issuecomment-450966221">The `compilerOptions.outDir` config is incorrectly resolved when in a shareable config · Issue #29172 · microsoft/TypeScript</a></p><p>In the package.json of the package, we have to update the compile script that will build our package.</p><pre>{<br>   &quot;name&quot;: &quot;@namespace/core&quot;,<br>   &quot;version&quot;: &quot;1.0.0&quot;,<br>   &quot;main&quot;: &quot;dist/index.js&quot;,<br>   &quot;types&quot;: &quot;dist/index.d.ts&quot;,<br>   &quot;repository&quot;: {<br>     &quot;type&quot;: &quot;git&quot;,<br>     &quot;directory&quot;: &quot;packages/core&quot;,<br>     &quot;url&quot;: &quot;https://github.com/ahoopen/typescript-mono-repo.git&quot;<br>   },<br><strong>   &quot;scripts&quot;: {<br></strong>      &quot;clean&quot;: &quot;rimraf dist&quot;,<br>      &quot;prepack&quot;: &quot;yarn build&quot;,<br>      &quot;build&quot;: &quot;yarn clean &amp;&amp; yarn compile&quot;,<strong><br>      &quot;compile&quot;: &quot;tsc --build &amp;&amp; cp \&quot;./package.json\&quot; ./dist/&quot;,<br></strong>      &quot;test&quot;: &quot;echo &#39;no test&#39;&quot;,<br>      &quot;lint&quot;: &quot;eslint \&quot;./src/**/*.{ts,tsx}\&quot; --max-warnings=0&quot;<strong><br>   },<br>   </strong>&quot;dependencies&quot;: {<br>     &quot;@namespace/utils&quot;: &quot;1.0.0&quot;<br>   }<br>}</pre><p>We build our package with the --build flag. This is important because we want to use the<a href="https://www.typescriptlang.org/docs/handbook/project-references.html">project references</a> from TypeScript. I’ll explain more about it further on</p><h3><strong>Enabling easy code navigation</strong></h3><p>By default, the TypeScript compiler will look for atsconfig.json. Our IDE will use this file to navigate through the project. Here we will map our package names to their source code in the monorepo.</p><p>Create a tsconfig.json in the root. This file will enable seamless navigation in our IDE across the mono repo. The configs need to be separate because the solution produces unintended behaviour when building packages.</p><pre>{<br>  &quot;compilerOptions&quot;: {<br>     &quot;jsx&quot;: &quot;react&quot;,<br>     &quot;baseUrl&quot;: &quot;./packages&quot;,<br><strong>     &quot;paths&quot;: {<br>        &quot;@namespace/core/*&quot;: [&quot;core/src/*&quot;],<br>        &quot;@namespace/utils/*&quot;: [&quot;utils/src/*&quot;],<br>        &quot;@namespace/web/*&quot;: [&quot;web/src/*&quot;]<br>     }</strong><br>   }<br>}</pre><p>Paths property is where the magic happens: it tells the TypeScript compiler that whenever a module tries to import another module from the mono repo it should resolve it from the packages folder. Specifically, it should point to that package’s src folder because that’s the folder that will be compiled when publishing.</p><p>The paths config should contain one entry per package, or if they have a common prefix like an organization, a glob that matches them all.</p><p>A big advantage of using path alias is that when you change a piece of code in one of the packages, the TypeScript compiler immediately picks it up without having to build all the packages.</p><h3><strong>Important caveat</strong></h3><blockquote>If we try to build the corepackage (which depends on the utilspackage) in a fresh state (no dist folders anywhere) then it will fail and complain that it can’t find utils, even though Lerna symlinked it in its node_modules folder. This is because utils&#39;s package.json defines its entry point via the main field to point to dist/index. When the TypeScript compiler sees import &#39;@namespace/utils&#39; in the corepackage it will look for core/node_modules/@namespace/utils/dist/index.js and, because utilshasn’t been built (therefore there’s no dist/ folder in it), it will fail.</blockquote><p><strong>How can we solve this issue? </strong>This is where project references from TypeScript come into play.</p><h3>Project references</h3><p>Before <a href="https://www.typescriptlang.org/docs/handbook/project-references.html">project references</a>, if you changed a package, you needed to re-compile it and also re-check any package depending on it, with no automatic detection of whether a change actually changed the types. If your packages needed to be built in a particular order, you were on your own as far as enforcing it. Furthermore, while tsc has had a watch mode since TypeScript&#39;s first release, before project references it wouldn&#39;t see changes in dependencies.</p><p><strong>TypeScript</strong> will automatically compile its dependencies in, correct order and <em>only if needed</em>, before building and type checking the main project <em>only if needed</em>. This reduced our build times by about half on average and got rid of a really gross and unwieldy set of package.json scripts that we used to specify build order.</p><p>Referenced projects must have the composite setting enabled. This setting is needed to ensure TypeScript can quickly determine where to find the outputs of the referenced project.</p><p>Oke, let’s enable project references in our packages!</p><p>We have to modify the tsconfig.json of our packages. By setting “composite” to true, we let TypeScript know we want to use project references. This has to be enabled for each package. Furthermore, we have to include the project references. References is an array of paths of dependent packages. In this example, our package is dependent on the utils package.</p><pre>{<br>   &quot;extends: &quot;../../tsconfig.build.json&quot;,</pre><pre>   &quot;compilerOptions&quot;: {<br>      <strong>&quot;composite&quot;: true,</strong><br>      &quot;rootDir&quot;: &quot;./src&quot;,<br>      &quot;outDir&quot;: &quot;./dist&quot;<br>   },</pre><pre><strong>   &quot;references&quot;: [<br>      { &quot;path&quot;: &quot;../utils&quot; }<br>   ],</strong></pre><pre>   &quot;include&quot;: [&quot;src/**/*&quot;]<br>}</pre><p>Unfortunately, we can’t place the references in the roottsconfig.build.json This means we need to add the references to each individual tsconfig per package.</p><p><a href="https://github.com/microsoft/TypeScript/issues/27098">`references` are not inherited in `tsconfig.json` · Issue #27098 · microsoft/TypeScript</a></p><p>We have to change the compile script in our package.json to let it work with project references. Instead of tsc we now have to do tsc --build . Running tsc --build will do the following:</p><ul><li>Find all referenced projects</li><li>Detect if they are up-to-date</li><li>Build out-of-date projects in the correct order</li></ul><p>We also need to update the clean script to remove the build cache for tsc — build otherwise, we can run into a situation where if the dist/ folder of a package is removed, but the package itself doesn’t contain any changes from last time, TypeScript will not rebuild it.</p><pre>{<br>   &quot;name&quot;: &quot;@namespace/core&quot;,<br>   &quot;version&quot;: &quot;1.0.0&quot;,<br>   &quot;main&quot;: &quot;dist/index.js&quot;,<br>   &quot;types&quot;: &quot;dist/index.d.ts&quot;,<br>   &quot;repository&quot;: {<br>     &quot;type&quot;: &quot;git&quot;,<br>     &quot;directory&quot;: &quot;packages/core&quot;,<br>     &quot;url&quot;: &quot;https://github.com/ahoopen/typescript-mono-repo.git&quot;<br>   },<br><strong>   </strong>&quot;scripts&quot;: {<strong><br>      &quot;clean&quot;: &quot;rimraf dist &amp;&amp; rimraf tsconfig.tsbuildinfo&quot;,</strong><br>      &quot;prepack&quot;: &quot;yarn build&quot;,<br>      &quot;build&quot;: &quot;yarn clean &amp;&amp; yarn compile&quot;,<br>      <strong>&quot;compile&quot;: &quot;tsc --build &amp;&amp; cp \&quot;./package.json\&quot; ./dist/&quot;,</strong><br>      &quot;test&quot;: &quot;echo &#39;no test&#39;&quot;,<br>      &quot;lint&quot;: &quot;eslint \&quot;./src/**/*.{ts,tsx}\&quot; --max-warnings=0&quot;<strong><br></strong>   },<strong><br>   </strong>&quot;dependencies&quot;: {<br>     &quot;@namespace/utils&quot;: &quot;1.0.0&quot;<br>   }<br>}</pre><p>As you can see, the process of adding new references is quite cumbersome. Especially if we have a lot more packages in your mono repo on which you depend.</p><p>Wouldn&#39;t it be great if we can automate this?</p><h3>Auto configure project references</h3><p>Instead of keeping the responsibility of keeping the project references up to date by the developer, we automate it. This way we know for sure that it is always up to date. And we take away an annoying task from the developer. A win-win situation!</p><p>Create aconfigure-references.js file at the root of mono repo with the following content:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/32d14f1459447e14f64f0dd0105e893b/href">https://medium.com/media/32d14f1459447e14f64f0dd0105e893b/href</a></iframe><p>We have to run the configure-reference script after every yarn install because a package might have a new dependency on one of the mono repo packages. Let&#39;s add the following script to our root package.json</p><pre>{<br>   &quot;name&quot;: &quot;root&quot;,<br>   &quot;private&quot;: true,<strong><br>   </strong>&quot;scripts&quot;: {<strong><br>      &quot;prepare&quot;: &quot;node ./configure-references.js&quot;,</strong><br>      &quot;build&quot;: &quot;lerna run build --stream&quot;,<br>      &quot;lint&quot;: &quot;lerna run lint --stream --parallel&quot;,<br>      &quot;test&quot;: &quot;yarn jest --coverage&quot;<strong><br>   },</strong><br>   &quot;devDependencies&quot;: {<br>      &quot;eslint&quot;: &quot;7.7.0&quot;,<br>      &quot;jest&quot;: &quot;26.4.2&quot;,<br>      &quot;is-ci&quot;: &quot;2.0.0&quot;,<br>      &quot;rimraf&quot;: &quot;3.0.2&quot;,<br>      &quot;lerna&quot;: &quot;3.22.1&quot;,<br>      &quot;typescript&quot;: &quot;4.0.2&quot;<br>   }<br>}</pre><p>After each yarn install the configure-reference script will be called and updates all references in all packages. Your project references will always be up to date :)</p><h3>Conclusion</h3><p>The first part is done! We can now easily navigate our mono repo. By making use of Yarn Workspaces, we enable faster, lighter installation by preventing package duplication across our repository. Last but not least, we have added TypeScript to our packages. But we are not there yet! Next steps will be, configuring testing in a mono repo and the releasing of packages.</p><p>The next article is all about testing. How to get Jest playing nice with our mono repo setup. So stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ed6a1e5467a" width="1" height="1" alt=""><hr><p><a href="https://blog.ah.technology/a-guide-through-the-wild-wild-west-of-setting-up-a-mono-repo-with-typescript-lerna-and-yarn-ed6a1e5467a">A guide through The Wild Wild West of setting up a mono repo with TypeScript, Lerna and Yarn…</a> was originally published in <a href="https://blog.ah.technology/">AH Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why a mono repo will speed up your development]]></title>
            <link>https://blog.ah.technology/why-a-mono-repo-will-speed-up-your-development-1109542d409e?source=rss-468e730accf0------2</link>
            <guid isPermaLink="false">https://medium.com/p/1109542d409e</guid>
            <category><![CDATA[npm]]></category>
            <category><![CDATA[yarn-workspace]]></category>
            <category><![CDATA[lerna]]></category>
            <category><![CDATA[monorepo]]></category>
            <category><![CDATA[typescript]]></category>
            <dc:creator><![CDATA[Auke ten Hoopen]]></dc:creator>
            <pubDate>Wed, 26 Aug 2020 09:43:26 GMT</pubDate>
            <atom:updated>2020-08-26T09:43:26.320Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*KlaGQdgedVvjuCTt_I38Cg.png" /></figure><p>At Albert Heijn our frontend is based on a micro-frontend architecture, resulting in a lot of standalone web applications. Keeping all the apps aligned with the latest updates, migrations, and technology can be a challenge. To not write the same code over and over, we make use of shared NPM packages. Managing all of these packages can be tricky, especially at scale.</p><p>In this article, we will talk about how to manage these packages at scale and how a mono repo has helped us to develop faster. This is the first article in a series about mono repo’s. The next articles will focus on setting up a mono repo from scratch.</p><h3>The philosophies</h3><p>At any large organization, chances are you will have many repositories and NPM packages. There are a few ways of managing this.</p><p>The classical way, where each module is placed in a separate repo and the other way, one big repository with all your packages.</p><ul><li><strong>Mono repo </strong>— a single multi package repository</li><li><strong>Multi repo</strong> — multi single package repository</li></ul><h3>Multi repositories</h3><p>We started with a multi-repository setup. This worked great at the beginning when having just a few packages. The code was isolated and easy to find.</p><p>Then eventually when we started to grow, we ran in the problem of scale. We were seeing the downsides of a multi repo approach. A lot of contact area that you get when you spread your code out like that. Publishing and source code management can be a nightmare. The cost of pushing things out in separate repositories and separate packages is pretty high.</p><ul><li><strong>Discovery<br></strong>Discovery goes downhill when they’re dozens of repositories. When new repositories are created, everyone has to add their existence to their mental model of where that code lives.</li><li><strong>Versioning<br></strong>Versioning is complicated by a highly modular architecture. Asking humans to remember to do this across a large collection of packages (including packages they haven’t had to touch) is asking for trouble.</li><li><strong>Duplication<br></strong>Duplicate dependencies between packages can vastly increase the time required to install dependencies. Some ecosystems, NPM in particular, are highly modular. This encourages reuse but also means that some packages are likely duplicated in every package you develop.</li><li><strong>Breaking changes<br></strong> Package owners tended to be a little bit uncaring because it was hard for them to figure out what repositories are dependent on them.</li></ul><p>Let’s showcase some of the difficulties with multi repositories.</p><h3>Problem #1: Repetitive work</h3><p>Making changes across multiple repositories takes a lot of time. For every change per package, you have to do a build, run the tests, and finally do the release. You can imagine how much time a trivial change across multiple packages would take.</p><p><strong>let’s look at an example:</strong></p><ul><li>Add a sentence into all the readme’s from 14 repo’s</li><li>Change 14 files</li><li>Make 14 commits</li><li>Create 14 pull requests</li><li>Send to code review</li></ul><p>This is the workflow when everything goes right. But what happens when you get feedback in your PR</p><ul><li>“You have a typo”</li><li>14 fixes?</li><li>14 fix commits</li><li>14 new code reviews?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/311/1*TBlG1q_fOTOOCBoBaCaJCQ.gif" /></figure><h3>Problem #2: API changes</h3><p>Sometimes you have to make an API change in one of your packages. This affects the other packages that depend on it. This means that you have to update the API usage in other packages.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/405/1*Ao9sIh3ngUiLs21WcslssA.png" /><figcaption>Overlooked piece of code</figcaption></figure><p><strong>Reviewing</strong><br>When you make an API change across multiple repositories, it is more difficult for the reviewer to find out the underlying thoughts of the pull request. If the work is split up into multiple pull requests over multiple repositories.</p><p>Besides, there is a chance that the author overlooked an important piece of code because the code was not in the current repository.</p><h3>Problem #3: Dependency management</h3><p>Rolling out a security patch or dependency upgrades over multiple repositories is a nightmare. You have to make the same change across a lot of repositories, increasing the risk of error. It will get increasingly worse if the packages depend on each other. Then you have to release them in the correct order. The multi repo approach makes it cumbersome to perform dependency upgrades.</p><h3><strong>How can a Mono repo help?</strong></h3><p>Mono repositories are a popular solution to these problems. Instead of one code repository per module, you put all packages in the same repository. This “mono repo” is then the only thing a developer needs in order to do development with your application. Discovery, duplication, and versioning are all simpler by virtue of everything being in the same place.</p><p>Able to make changes in three different NPM packages at the same time in one pull request, is just awesome! It improves productivity so much! Making code changes across packages faster, easier, and with fewer errors. Being able to do them in one pull request is a big advantage. Having the overview and having teams that are involved — in all the packages — being able to discuss it in one place is extremely useful.</p><p>Moreover to release the pull request at the same time is very useful. Otherwise, you have to manage different versions of different NPM packages and other teams might depend on a specific version, it becomes messy quickly.</p><ul><li><strong>Discoverability</strong><br>If you have everything in one repo its a lot easier to search for things and see what code is dependent on which dependencies.</li><li><strong>Versioning</strong><br>Sharing code with NPM has much more friction than sharing them within one repository. Because every package is versioned individually, you have to manage compatibility between each package individually. With a mono repo, you have one version shared between packages. If you have separate NPM packages, everything can have different versions and you have to manage that complexity.</li><li><strong>Ownership</strong><br>When moving to a mono repo the sense of ownership becomes shared across the organization. People get an overview of the whole codebase. Instead of only focusing on a specific piece of the codebase.</li><li><strong>Breaking changes<br></strong> It’s a lot harder to introduce breaking changes with a mono repo. When you introduce a breaking change. It’s a lot easier to do it with the mono repo because you can make one pull request that has an overview of the entire system.</li></ul><p>Let’s see how the same use cases are handled with a mono repo.</p><h3>Solution #1: Make changes in multiple repositories</h3><p>In a mono repo, the use-case were you have to edit multiple packages becomes a lot easier.</p><ul><li>1 commit</li><li>1 pull request</li><li>Send it to code review!</li><li>“You have a typo….”</li><li>1 fix! 1 fix commit! 1 additional code review!</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/414/1*ZNoAexdFmB9HqGAIQBz91g.gif" /></figure><h3>Solution #2: API Changes</h3><p>Since all code is now in the same repository, it is very easy to see if you missed a piece. The failed build will tell you that. This way of working can prevent a lot of unnoticed bugs.</p><p>In a mono repo, the same use-case becomes a lot easier.</p><ul><li>1 pull request</li><li>1 complete code review</li><li>Code review can reveal problems</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/331/1*mdcOsfxnxHz-CLpkmiMaBA.png" /><figcaption>The build will fail when there is an error in one of the packages</figcaption></figure><p>It also creates good inertia because, when it was individually versioned, people became kinda careless when it came to introducing breaking changes. Because they weren’t the ones that were dealing with the breaking change.</p><p>In their world, they just updated the library and they updated the code that was dependent on the library. Then bumped the package to a new major version.</p><p>People that were dependent on the package were ‘<em>oh well, I will just update to the major whenever we can.</em>’ This sounds good in theory, but in reality, meant that people were on different versions all the time.</p><h3>Solution #3: Dependencies management</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/561/1*swf44HZCHkWDGlilpcO-PA.png" /><figcaption>Updating a package is really easy now</figcaption></figure><p>Updating a dependency is now really easy. Update to the new version. Run the tests and build to see if something broke. If everything works as intended, we create a pull request.</p><p>When merged all dependent packages automatically release with the same version. No need to manage different versions of packages anymore</p><h3><strong>Single source of config</strong></h3><p>A single source of truth is one of the great benefits of a mono repo. You can include your linting, build, testing and other scripts and utilities you have, in the same mono repo. It’s much easier to coordinate changes across the packages. You only have to update the scripts in one place instead of in every package. And for the consumers of the packages, it’s easier to have a single place where they can report issues.</p><h3>What’s next? Setting up a mono repo!</h3><p>Of course, it’s not all butterflies and lollipops. You have to manage the beast! In the next article, I will go in-depth and show these concepts in practice. How to setup a mono repo with <strong>Lerna</strong>, <strong>Typescript</strong>,<strong> Yarn workspaces,</strong> and <strong>Jest</strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1109542d409e" width="1" height="1" alt=""><hr><p><a href="https://blog.ah.technology/why-a-mono-repo-will-speed-up-your-development-1109542d409e">Why a mono repo will speed up your development</a> was originally published in <a href="https://blog.ah.technology/">AH Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>