<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Aleksandar Mirilovic on Medium]]></title>
        <description><![CDATA[Stories by Aleksandar Mirilovic on Medium]]></description>
        <link>https://medium.com/@amirilovic?source=rss-fd55cd889543------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*xHclFx8kCj1Mfxhh.jpg</url>
            <title>Stories by Aleksandar Mirilovic on Medium</title>
            <link>https://medium.com/@amirilovic?source=rss-fd55cd889543------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 06:37:20 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@amirilovic/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Meet my AI Dev Team]]></title>
            <link>https://medium.com/@amirilovic/meet-my-ai-dev-team-686a12f12c07?source=rss-fd55cd889543------2</link>
            <guid isPermaLink="false">https://medium.com/p/686a12f12c07</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Aleksandar Mirilovic]]></dc:creator>
            <pubDate>Fri, 27 Feb 2026 07:18:59 GMT</pubDate>
            <atom:updated>2026-02-27T08:44:52.625Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5l1uEYvgPzfwv1S73MBiyw.png" /></figure><p><strong>What if you could have an AI team that works while you sleep?</strong></p><p>Last night, while I was sleeping, my AI dev team merged 20 pull requests across two repositories.</p><p>No supervision.<br>No nudging.<br>No “are you stuck?” messages.</p><p>Just autonomous coordination.</p><p>Not a single assistant — but a structured team: a Product Owner, Developer, QA Engineer, and Architect. Each runs as its own agent. Each has a defined role. They communicate through Discord, manage work in GitHub Projects, open pull requests, review each other’s code, fix infrastructure bugs — and even run daily retros.</p><p>They use the same tools your team uses.</p><p>And they ship.</p><p>This isn’t a theoretical agent framework or a research prototype. It’s a system I run daily. It works on real repositories. It costs real money. It makes real mistakes. And it improves itself over time.</p><p>Here’s how it works.</p><h3>Why build this?</h3><p>I’m a heavy Claude Code user — it’s incredibly powerful, and I use it for everything from development to managing personal tasks.</p><p>The first thing I built was a <a href="https://medium.com/@amirilovic/how-to-use-claude-code-from-your-phone-with-a-telegram-bot-dde2ac8783d0">very simple Telegram integration</a> for Claude Code so I could use it as my personal assistant.</p><p>For work, I needed parallel execution. I needed agents that could chew through a backlog for hours. One agent wasn’t enough — I needed a coordinated team.</p><p>But could AI agents coordinate autonomously, using <strong>standard collaboration tools</strong>, and deliver working software with minimal supervision?</p><p>The answer turned out to be yes.</p><h3>The Team</h3><p>I asked each agent to introduce itself for this blog post — and here they are:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rkbDJX_Nm8Kmoii1Lu0KYw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MCsI4DLLb588dNm2SQzNQQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v2nKZdvIeOyVIEcyzbpy6g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q_LK_2XkDIOGIids3XsSog.png" /></figure><p>They’ve picked their own animal character and generated their own avatars. 😎</p><h3>How They Work</h3><p>They coordinate in a Discord “development” channel and track work in GitHub Projects.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*woiYv9up6eOxOEcNDC7RuA.png" /></figure><p>Here’s what happens when I post a feature request:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zs3_NFNbEAIvQ-1Yy2K2Fw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u97lrsC4eb8PnuP5yiJkig.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1cqahjMLHEqBje5yv_DYCA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J-YWTp6jJWE71F3-SSLZzQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I2BQKh4dBcvVT73HY0R4IQ.png" /></figure><p>The entire cycle completes without me watching. The agents coordinate through Discord mentions and GitHub — <strong>the same tools human teams use.</strong></p><p>They can operate on any repo I have access to. Right now, they’re active across multiple projects.</p><h3>The Discovery Flow</h3><p>To brainstorm about future work or to investigate some deeper issue, we have a “discovery” channel where I mostly brainstorm with the architect:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ac7jn7ieYtmU8wPKE_4rqA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*01xNMUKw8NH7PXuPDas7Kg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ONfw-z5MtYTKPoif33aNCQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Cj6X3DLYpfIG5boBdb6ZIA.png" /></figure><p>The PO creates a ticket, and the development flow begins.</p><h3>Self-Updating Instructions</h3><p>They don’t just follow instructions. They rewrite them.</p><p>Each agent has a CLAUDE.md file containing its system prompt. When they learn something important, they update it themselves.</p><p><strong>Real example:</strong> During one session, the PO agent ran pm2 restart all to apply a config change. This caused an infinite loop—PM2 restarted the agent, which resumed with the same context, which ran the restart command again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5iKB-1Ua0ug3sXhHftSzwg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9FsawfL0DilgWgih9UaVew.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QCbRPLGbspLqncucOfSzcA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-OWB6mHRYGfGW8X46E_EKQ.png" /></figure><p>They learned from the mistake — and baked the fix into their own system prompts.</p><p>An important distinction: changes to CLAUDE.md files take effect immediately on the next trigger — agents read these fresh each time. But changes to TypeScript code require a PM2 restart to load into memory. This creates a nice safety boundary where behavior tweaks are instant but infrastructure changes are deliberate.</p><p>I don’t know about you, but my brain explodes 🤯 every time I see this kind of real-time self-improvement.</p><h3>Self-Updating Code</h3><p>Beyond instructions, the agents modify their own codebase.</p><p><strong>Example from today:</strong> The system was using a single lastProcessedMessageId for all Discord channels. After a restart, agents would reprocess old messages from other channels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6MWWcYRxiOCY0ohwLvtjNQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WORW4dwn1T2NxNqHPU0duw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yl3xkC-bBCuhA6QPm_q_0g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gTDKQVSfeMk-lXvVSN01Vg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*__pYKwr0UiN_i_PRPAApow.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IS_Yn_goqid-X7bcpOB3Aw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bH1rYd_7UGFXGiXvZzGe7g.png" /></figure><p>They fixed a bug in the very code that runs them.</p><p>Agents know how to build the code and how to restart themselves.</p><p>They’ve already built a bunch of features for themselves:</p><ul><li>Reminders — one time or recurring</li><li>Auto-update — restarting when the main branch changes</li><li>To use pm2 for process management</li><li>Sending and receiving files and voice messages</li><li>Message reactions</li><li>A cost tracker to monitor how much each agent is spending on tokens</li><li>And many more.</li></ul><p>The first time I saw it happen, it felt like sci-fi.🍿</p><h3>Retros</h3><p>Each agent has the ability to schedule reminders, either recurring or one-time.</p><p>The Owl (PO) runs a daily retro at 2 a.m. to improve the process.</p><p>In reminders/po.json :</p><pre>{<br>  &quot;reminders&quot;: [<br>    {<br>      &quot;id&quot;: &quot;08c1acc6-3cda-4b0e-b4c9-c088c2556e19&quot;,<br>      &quot;message&quot;: &quot;Daily Retro: Review yesterday&#39;s Discord messages in development. Analyze delivery process for issues. Identify communication gaps, workflow problems, technical issues, and process improvements. Propose solutions to @aleksandar. Use /daily-retro skill.&quot;,<br>      &quot;when&quot;: &quot;0 2 * * *&quot;,<br>      &quot;recurring&quot;: true,<br>      &quot;createdAt&quot;: &quot;2026-02-15T17:33:16.419Z&quot;<br>    }<br>  ]<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1014/1*5cz6lIpWqXlZyhu7_JiHPg.png" /></figure><p>The learnings get documented in data/shared/retro-learnings.md where all agents can reference them:</p><pre>## 2026-02-15: First Daily Retro<br><br>### 1. Pre-Work Checklist (@dev)<br><br>Before starting work on any issue:<br>```bash<br>gh pr list --state open<br>```<br>Check if someone else already has a PR open for the same issue. Avoids duplicate work (e.g., PR #62 and #63 situation).<br><br>### 2. Screenshot Protocol (@po)<br><br>When referencing screenshots in GitHub issues:<br>- **DO:** Use Discord CDN URL as markdown image: `![description](discord-cdn-url)`<br>- **DON&#39;T:** Write &quot;Screenshot attached in Discord&quot; without actual attachment<br><br>The `gh` CLI doesn&#39;t support direct image uploads, so use the Discord URL workaround.<br><br>### 3. Scroll Test Requirement (@qa)<br><br>For PRs touching layout properties, ask:<br>&gt; &quot;Have you verified scroll still works on mobile after this change?&quot;<br><br>Applies to PRs modifying:<br>- `h-*`, `min-h-*`, `max-h-*` (height classes)<br>- `overflow-*` properties<br>- `flex` container changes<br>- `position: fixed/sticky`<br><br>This would have caught the `h-[100dvh]` regression (PR #72 → #75).<br><br>### 4. Real-Time vs Restart Design Question (all agents)<br><br>When designing agent features, always ask upfront:<br>&gt; &quot;Does this need to work immediately (real-time) or is a restart acceptable?&quot;<br><br>This avoids building file-based solutions that require restart, then having to add MCP tools for real-time access (e.g., PR #41 → #43 situation).</pre><h3>The Architecture</h3><p>It’s simpler than it looks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/930/1*k8ZVumG4zS33TrwLDhQiyA.png" /></figure><p>Every agent is a separate Node.js process, running Claude Code using <a href="https://platform.claude.com/docs/en/agent-sdk/overview">Claude Agent SDK</a>. It connects to Discord using <a href="https://discord.js.org/">discord.js</a>. It has its own Discord role and responds to mentions. Each agent maintains its own instructions in CLAUDE.md and its <a href="https://github.com/amirilovic/ai-crew/blob/main/docs/configuration-reference.md">configuration</a>.</p><p>There’s almost no orchestration layer. Just a Discord MCP adapter, a reminder tool, and the same collaboration patterns human teams use.</p><p>Every agent runs with the --dangerously-skip-permissions flag and has access to WebSearch, WebFetch, and Bash.</p><p>For most of the work they use:</p><ul><li><a href="https://cli.github.com/">gh C</a>LI for interaction with GitHub with <a href="https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens">my token</a>.</li><li><a href="https://github.com/vercel-labs/agent-browser">agent-browser</a> for testing the stuff that is built</li><li><a href="https://kubernetes.io/docs/reference/kubectl/">kubectl</a> and Helm to debug issues with deployments in my Kubernetes cluster</li><li>Docker for running the applications and doing e2e testing</li><li>gen-image to generate images — the agents built this tool to be able to create their own animal character and generated their own avatar 🤩</li></ul><p>The full system is open-source at <a href="https://github.com/amirilovic/ai-crew">github.com/amirilovic/ai-crew</a>.</p><p><strong>Make sure to fork the repo and adapt the roles and tools to your own needs.</strong></p><h3>Costs</h3><p>The agents run on a single <a href="https://contabo.com/en/vps/">VPS that costs just 4.5 euros per month</a>.</p><p>The LLM API costs are a completely different story, costs can get really high if the agents work all day.</p><p>Here is an example of typical cost for the NIGHT SHIFT for my team:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a4a452d62d03c8a03ce778362628bc0d/href">https://medium.com/media/a4a452d62d03c8a03ce778362628bc0d/href</a></iframe><p><strong>Note: These are estimated API token costs. </strong>I have the claude 200$ max subscription, which is enough to cover the workload I have. Architect’s higher cost (~5x PO/Dev/QA despite similar token counts) is due to using Opus 4 vs Sonnet 4. The Opus model is used for research/architecture tasks requiring deeper reasoning.</p><p>Running many coding agents in parallel currently only makes sense if you have max subscription or your company is covering the API cost.</p><p>I’m trying with cheaper models using <a href="https://openrouter.ai/docs/guides/guides/claude-code-integration">openrouter</a>, but so far very little success. If you just want to try the system without max subscription, make sure to set the <a href="https://github.com/amirilovic/ai-crew/blob/main/docs/configuration-reference.md#limitsmaxdailycostusd">max cost limit configuration</a>.</p><h3>What about XYZ?</h3><p>What about Slack, MS Teams, Jira, Linear, Codex, OpenCode…?</p><p>Honestly, just tell the agent to build any integration you need and adapt the workflow to your requirements. <strong>You don’t need a massive orchestration platform</strong> with 200 integrations, you already have a developer with super powers that can build any integration you need faster than you can drink your morning coffee. ☕️</p><p>My initial setup with Discord took <strong>one day</strong> to build. ⚡️</p><h3>What can go wrong?</h3><p>Things break. Often. But the team learns.</p><p>Typical issues:</p><ul><li>PO forgets to move the ticket into Ready for Development</li><li>One agent stopping all agents</li><li>One agent doing something without mentioning another agent, causing work to get stuck until another cron job kicks in</li><li>An agent getting stuck while executing something, requiring another agent to fix it</li><li>Cost limits — if you have low cost limits and using the system heavily agents can get stuck very fast</li><li>Agents not following instructions — context gets too big and some instructions are not followed</li><li>Miscommunication — the final implementation not in line with the original requirement — you need to be more detailed</li><li>Agent faking it — says it tested everything in the browser, but didn’t actually 😂</li><li>…</li></ul><p>My job is simple: improve the system until failure becomes rare.</p><h3>What’s Next</h3><p>I’m exploring:</p><ul><li><strong>More agent types</strong>: SRE for monitoring, Marketing expert…</li><li><strong>Multi-model support</strong>: Cheaper models for routine tasks, premium models for complex reasoning</li></ul><h3>Conclusion</h3><p>Modern LLMs don’t need exotic orchestration environments to collaborate effectively.</p><p>Just give them:</p><ul><li>Clear roles</li><li>Well-defined tools</li><li>Shared context</li><li>Feedback loops</li><li>Guardrails</li></ul><p>…and they function like a real software team.</p><p>It’s humans designing the system — and AI running the sprint.</p><p>My crew is running.</p><p>And I’m sleeping better.</p><p><strong>Fork the </strong><a href="https://github.com/amirilovic/ai-crew"><strong>repo</strong></a><strong>, explore with your coding agent, adapt to your needs.</strong></p><p>Until next time 👋</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=686a12f12c07" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Use Claude Code From Your Phone With a Telegram Bot]]></title>
            <link>https://medium.com/@amirilovic/how-to-use-claude-code-from-your-phone-with-a-telegram-bot-dde2ac8783d0?source=rss-fd55cd889543------2</link>
            <guid isPermaLink="false">https://medium.com/p/dde2ac8783d0</guid>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[ai-personal-assistant]]></category>
            <category><![CDATA[telegram-bot]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Aleksandar Mirilovic]]></dc:creator>
            <pubDate>Thu, 22 Jan 2026 20:05:30 GMT</pubDate>
            <atom:updated>2026-01-22T20:05:30.833Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MXz6qZehceIigEWHDMwBUw.png" /><figcaption>A smartphone floating in space with glowing connection lines extending outward <br> to icons representing calendar, email, files, and terminal. Telegram logo subtly <br> integrated. Dark background with blue and purple gradient, minimal tech aesthetic, <br> clean vector illustration style</figcaption></figure><p>I was late for my German exam and had no idea where to park. So I sent a voice message: “Find parking near my exam.” Thirty seconds later, I had a ranked list of options with prices and walking distances. Claude Code already knew my schedule, searched the web, and replied via Telegram — all while I was driving.</p><p>This is what a personal AI assistant should feel like.</p><p>From the first time I tried ChatGPT with GPT 3.5, I’ve wanted exactly this — an assistant that can do far more than just help me code or compile mountains of information from the internet. Something that is really personal, that can access my calendar, email, Notion, Kubernetes clusters, GitHub — and work with my local files that knows what I like or don’t like and helps me get things done.</p><p>I’ve tried ChatGPT, Claude Desktop and Mobile, Gemini. They’re all impressive, but limited when it comes to customization. Giving them access to your own data and tools requires jumping through hoops, and even then, the integrations are often shallow and experience is not consistent across different platforms — Web vs Desktop vs Mobile.</p><p>On the other hand, I use Claude Code every day for software development. It reads my files, runs commands, writes and executes code. It’s become an essential part of how I work.</p><p>What I really love about using Claude Code is how easy it is to customize for many different use cases. You can:</p><ul><li>Use existing or create your own <a href="https://github.com/ComposioHQ/awesome-claude-skills">skills</a></li><li>Use existing or create your own <a href="https://github.com/punkpeye/awesome-mcp-servers">MCP servers</a></li><li>Set granular <a href="https://code.claude.com/docs/en/settings">permissions</a> on what it is able to do</li><li>Use <a href="https://code.claude.com/docs/en/hooks">hooks</a> to verify that it is actually doing the right thing</li><li>You can connect it to other models and providers via <a href="https://github.com/musistudio/claude-code-router">claude-code-router</a></li></ul><p>Because of that, I’ve also started using it for things which are not just coding:</p><ul><li>I’ve built a German tutor skill that pushes me to learn German faster.</li><li>With <a href="https://github.com/vercel-labs/agent-browser">agent browser</a> skill it can use my authenticated sessions, so I can automate boring tasks like downloading receipts from gmail or make sure that in my calendar events have location added to every event.</li><li>With <a href="https://github.com/ComposioHQ/awesome-claude-skills/blob/master/content-research-writer/SKILL.md">content researcher writer</a> skill I use it to help me write, review and fact check my blog posts</li><li>I use it to parse scanned files, name them according to some convention and put them in the right folder in google drive based on my rules. All of this with just one skill, no need for any code, no need for n8n.</li><li>Keep track of my expenses, when I upload a receipt it keeps excel file up to date for me.</li><li>Write tickets in Jira</li><li>Finding things in different places: Notion, Google Drive, Jira, Confluence, local files…</li><li>And many more things…</li></ul><p>But the problem was: I could only use it from my terminal.</p><h3>CCPA Telegram</h3><p>To enable access from my phone, I’ve built <a href="https://github.com/amirilovic/ccpa-telegram">CCPA Telegram</a> a very simple bot that only does one thing: expose Claude Code via Telegram.</p><p>The bot currently supports:</p><ul><li><strong>Text messages </strong>— send and receive messages with Claude.</li><li><strong>Attachments</strong>— process PDFs, recognize objects in images, and more.</li><li><strong>Voice messages</strong> — transcription works locally using whisper.cpp.</li><li><strong>File responses</strong> — Claude can send files back to you.</li><li><strong>Multiple users</strong> — Every user gets a subfolder to upload files and receive files, but ClaudeCode configuration stays the same.</li></ul><p>Interacting with Claude Code via Telegram has some great benefits: both mobile and desktop apps are great and you don’t expose any ports on your machine — the bot polls messages from Telegram servers, you <strong>don’t need</strong> to have a static IP address or public DNS or use Cloudflare tunnel for your computer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W4qFXP_AVR4qdZaywPtFiQ.png" /><figcaption>CCPA Telegram Architecture Diagram</figcaption></figure><h3>Getting Started</h3><p>Either pick a folder with already existing Claude Code configuration or create a new one. More on how to configure Claude Code <a href="https://code.claude.com/docs/en/quickstart">here</a>.</p><p>Initialize CCPA:</p><pre>npx ccpa-telegram init</pre><p>That command will create a configuration file <em>ccpa.config.json </em>for the bot in your current folder.</p><p>In the <em>ccpa.config.json</em> file it is important to fill in:</p><ul><li><strong>botToken</strong> — A token which allows connection between CCPA and Telegram. To get the token, you will need to create your own telegram bot. Follow the instructions <a href="https://github.com/amirilovic/ccpa-telegram?tab=readme-ov-file#creating-a-telegram-bot">here</a>.</li><li><strong>allowedUserIds</strong><em> — </em>DON’T skip this step! This is a list of users who are allowed to talk to the bot, if you don’t populate this anyone could connect to your Claude Code! Follow the instructions <a href="https://github.com/amirilovic/ccpa-telegram?tab=readme-ov-file#finding-your-telegram-user-id">here</a>.</li></ul><p>More on configuration options <a href="https://github.com/amirilovic/ccpa-telegram?tab=readme-ov-file#configuration-options">here</a>.</p><p>Run the bot:</p><pre>npx ccpa-telegram</pre><p>Now go to Telegram and find the bot that you created and start chatting. That’s it.</p><h3>Security Considerations</h3><p>A few things to be aware of.</p><p><strong>Telegram doesn’t have end-to-end encryption when chatting with bots.</strong> All messages between you and the bot pass through Telegram’s servers unencrypted. This means you should never share sensitive information through the bot: no passwords, API keys, credentials, or confidential data. Treat the conversation as if it’s being logged by a third party.</p><p><strong>Make sure you understand what the bot can do on your machine.</strong> If you want to fine tune the access, you have couple of options:</p><ol><li><strong>Configure permissions </strong>— Claude Code has built-in permission controls</li><li><strong>Use hooks</strong> — add verification steps for sensitive operations</li><li><strong>Run it in Docker </strong>— control exactly what files and network resources the bot can access</li></ol><p><strong>All your chats are going to Anthropic.</strong> If you would rather use a different LLM provider, connect Claude Code to any custom gateways or use different models entirely via <a href="https://github.com/musistudio/claude-code-router">claude-code-router</a>.</p><h3>Similar Projects</h3><p>I’m not the only one who had this idea. Here are other projects exploring remote Claude Code access:</p><ul><li><a href="https://github.com/RichardAtCT/claude-code-telegram">claude-code-telegram</a> — Similar concept with session persistence and project navigation</li><li><a href="https://github.com/JessyTsui/Claude-Code-Remote">Claude-Code-Remote</a> — Supports email, Discord, and Telegram with notifications</li><li><a href="https://github.com/clawdbot/clawdbot">Clawdbot</a>— Multi-platform (WhatsApp, Slack, Discord, Signal, iMessage, Teams)</li></ul><p>Each takes a different approach. My goal with CCPA Telegram was simplicity — the minimum viable bridge to Claude Code that reuses everything I already have configured.</p><h3>Conclusion</h3><p>Every day I discover new things I can do with this setup. The parking story was just one example — but it captures why I built this.</p><p>Maybe one day Apple will make Siri actually useful, or tools like Claude Cowork make this even easier soon. Until then, there’s CCPA. 😉</p><p>If you give CCPA a try, let me know how it goes!</p><p>Until next time 👋</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dde2ac8783d0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Coding Tools: What Works (and What Doesn’t) for Me]]></title>
            <link>https://medium.com/@amirilovic/ai-coding-tools-what-works-and-what-doesnt-for-me-56be3d3c1018?source=rss-fd55cd889543------2</link>
            <guid isPermaLink="false">https://medium.com/p/56be3d3c1018</guid>
            <category><![CDATA[github-copilot]]></category>
            <category><![CDATA[openai-codex]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[ai-coding]]></category>
            <category><![CDATA[google-gemini]]></category>
            <dc:creator><![CDATA[Aleksandar Mirilovic]]></dc:creator>
            <pubDate>Tue, 19 Aug 2025 07:02:03 GMT</pubDate>
            <atom:updated>2025-08-19T07:02:03.448Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QKgf8ERP-gC9kuDtNf3Vzg.png" /><figcaption>AI Black Hole: Sucking in Everything</figcaption></figure><p>The AI coding tools space has exploded in the last 1–2 years with new tools from startups and big tech companies appearing every week. As I talk to people about AI coding every day I wanted to write down some of my reflections about the current state of tools and what I found works best for me.</p><p>I’ve separate the tools in four categories: AI App Builders, IDEs with agents, CLI Agents and Background Agents.</p><blockquote>Disclaimer: This list only includes tools I’m aware of. If your favorite isn’t here, drop it in the comments — I’d love to check it out. I also haven’t fully tested every tool, so if you spot inaccuracies, please let me know.</blockquote><h3>AI App Builders</h3><ul><li><a href="https://www.figma.com/make/">Figma Make</a></li><li><a href="https://lovable.dev/">Loveable</a></li><li><a href="https://bolt.new/">Stackblitz Bolt</a> — commercial and <a href="https://github.com/stackblitz/bolt.new">open-source</a></li><li><a href="https://v0.app/">Vercel v0</a></li><li><a href="https://github.com/dyad-sh/dyad">dyad</a> — open source, works as a desktop app</li><li>And many many more</li></ul><p><strong>Web-based AI builders are getting more and more capable.</strong> They’re phenomenal for prototyping, and most support GitHub export/sync so you can keep your normal PR reviews and CI/CD. Tools like Lovable and Bolt even integrate with databases (e.g., Supabase/Neon), which makes them great for building MVPs and launching <strong>new</strong> products extremely fast — especially for solo builders or early-stage teams.</p><p>But when it comes to using them with existing codebases, developer workflows, or internal company data, the integration story is still rough. Some platforms offer enterprise/private-network features, but that usually requires significant investment and months of negotiations. The DX is improving fast, but round-tripping code between a web editor and your repo/IDE is not yet seamless.</p><p>I’ve used them for prototypes, but they haven’t been particularly useful in my day-to-day. My work is mostly in large, existing codebases, which are harder to integrate with this category of tools.</p><h3>IDEs with agents</h3><ul><li><a href="https://code.visualstudio.com/">VS Code with Copilot</a>, <a href="https://github.com/cline/cline">Cline</a>, <a href="https://github.com/RooCodeInc/Roo-Code">Roo Code</a></li><li><a href="https://cursor.com/home">Cursor</a></li><li><a href="https://firebase.studio/">Google Firebase Studio</a> — web based, cloud development</li><li><a href="https://windsurf.com/editor">Windsurf</a></li><li><a href="https://windsurf.com/editor">AWS Kiro</a></li><li>And many many more</li></ul><p><strong>IDEs with agents are the de-facto standard for developers today.</strong> From VS Code with Copilot to Cursor, Windsurf, and newer open-source projects like Cline and Roo Code, this category puts AI directly where developers already work. Smart completions and codebase-aware agents make creating PRs and iterating on features significantly faster.</p><p>The biggest advantage here is that these tools operate <em>inside your local environment</em> — they see the full codebase, configs, and internal APIs without extra setup. That makes them far more useful for established projects compared to web-based builders.</p><p>The tradeoff is that the workflow is still developer-driven. An agent won’t spin up on its own and propose PRs in your repo — it requires a human at the keyboard prompting and guiding it.</p><p>I personally use Cursor, since I still make manual edits and find smart completions helpful. I don’t use Cursor’s agent mode anymore — more on that below.</p><h3>CLI Agents</h3><ul><li><a href="https://www.anthropic.com/claude-code">Anthropic Claude Code</a></li><li><a href="https://github.com/google-gemini/gemini-cli">Gemini CLI</a></li><li><a href="https://github.com/openai/codex">Open AI Codex CLI</a></li><li><a href="https://cursor.com/cli">Cursor CLI</a></li><li>And many many more</li></ul><p><strong>CLI tools enable agent execution in scripts and terminals instead of an editor UI.</strong> This makes them ideal for headless workflows like CI or GitHub Actions — for example, you can ask an agent to review a PR, create one from a Jira issue, or update an existing PR directly from the GitHub UI. Because they’re scriptable, agents can run in parallel on different branches or tasks, with a developer overseeing and guiding their work. This parallelization shifts the role of the developer from “writing every line” to <strong>orchestrating many agents working together</strong>, while still keeping human approvals and test gates in place. <strong>Usage of this automated flow makes sense only if you have a very solid test suite that you fully trust.</strong></p><p>These tools also support an <strong>interactive mode</strong> where the user can chat with the agent directly in the terminal — asking questions, giving commands, and reviewing changes live. That workflow feels very similar to using an IDE agent, but in a very fancy terminal UI.</p><p>I currently use Claude Code as my go-to coding agent. It’s a really powerful tool with great DX. It’s easy to adopt into any developer flow, it can be used with any model, even non-Anthropic ones. Companies concerned about data leakage can use it with AWS Bedrock in their account. If you are a solo dev, Anthropic Pro subscription for 20$ per month will get you very far. More on Claude Code in a separate post.</p><h3>Background Agents</h3><ul><li><a href="https://openai.com/codex/">Open AI Codex</a></li><li><a href="https://github.com/features/copilot">Github Copilot</a></li><li><a href="https://docs.cursor.com/en/background-agent">Cursor Background Agents</a></li><li><a href="https://jules.google/">Google Jules</a></li><li>And many many more</li></ul><p><strong>Background agents run continuously on your codebase and workflows</strong> — watching branches, issues, tests, and telemetry — then proposing (or directly opening) PRs without a human at the keyboard. Think of them as “always-on coworkers” that handle upkeep and long-tail chores: dependency bumps, flaky-test diagnostics, type-error sweeps, small security patches, adding code docs, improving test coverage… Compared to IDE or CLI agents, the idea is to be fully autonomous: they wake up on triggers (cron, webhook, event) and push changes through your normal CI gates. With CLI agents you define when and how to trigger the agent, with background agents this all is built into the product. These tools connect directly to your repo, spin up a virtualized development environment, make changes, run tests, and then submit their work as PRs — just like a developer would. In many ways this eliminates the need for having an IDE at all.</p><p>I’ve played around with OpenAI Codex, but it feels awkward to use it directly instead of just working in GitHub. I do like how GitHub Copilot integrates in Github, but I think industry will go for something that can be plugged into any CI setup. That’s why I chose Claude Code CLI — it gives me some of the background-agent style features without locking me into a single platform. I’ll cover my Claude Code setup in CI in a separate post.</p><h3>Summary</h3><p>The landscape of AI coding tools is evolving at breakneck speed, with new products emerging weekly from both startups and tech giants. These tools fall into four main categories, each with distinct strengths and tradeoffs:</p><ul><li><strong>AI App Builders</strong> make prototyping and launching MVPs extremely fast, especially for solo builders and early teams. But they struggle to integrate with large, existing codebases or internal company systems.</li><li><strong>IDEs with Agents</strong> (Copilot, Cursor, Windsurf, etc.) are today’s default for most developers, boosting productivity with smart completions and code-aware suggestions. They work well inside existing projects but still require a human to guide them.</li><li><strong>CLI Agents</strong> (Claude Code, Gemini CLI, Codex CLI) bring automation into the terminal and CI pipelines. They shine in headless workflows and parallelization, making them powerful when paired with solid test suites. Many also support interactive terminal sessions. Claude Code stands out here as your current go-to.</li><li><strong>Background Agents</strong> (Copilot, Codex, Cursor Background, Jules) aim for full autonomy — running continuously, monitoring repos, and opening PRs for maintenance tasks or fixes. They promise hands-off progress, but today they can still feel disconnected from established developer workflows.</li></ul><p>The trend is clear: the industry is moving toward increasingly automated development flows with less and less human involvement. For now, (at least for me) the sweet spot lies in tools that integrate seamlessly with IDEs while still offering strong automation — Claude Code being the best option for me.</p><p>What is your experience with these tools? Any other AI coding tools that you find useful? What works and what doesn’t for you?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=56be3d3c1018" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to find production memory leaks in Node.js applications?]]></title>
            <link>https://medium.com/@amirilovic/how-to-find-production-memory-leaks-in-node-js-applications-a1b363b4884f?source=rss-fd55cd889543------2</link>
            <guid isPermaLink="false">https://medium.com/p/a1b363b4884f</guid>
            <category><![CDATA[memory-leak]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[heapdump]]></category>
            <category><![CDATA[debugging]]></category>
            <dc:creator><![CDATA[Aleksandar Mirilovic]]></dc:creator>
            <pubDate>Sat, 03 Aug 2024 21:17:05 GMT</pubDate>
            <atom:updated>2024-08-03T21:17:05.444Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VaY_vgYiCf4HOe7Y_R9ASA.jpeg" /><figcaption>A conceptual image representing a memory leak in a computing environment. A computer screen displays a graph with increasing curves, symbolizing rising…</figcaption></figure><p>Very few things can make a developer as happy as fixing a tricky memory leak. 😂</p><p>After spending months investigating a particular memory leak, I finally cracked it a couple of weeks ago. In this post, I’ll provide a quick overview of my reasoning process, what didn’t work, and how I ultimately resolved the issue.</p><p>The service I’m going to discuss is a Node.js monolith API deployed in Kubernetes. It was built by many teams over the last 6–7 years and serves thousands of requests per second, making it a critical component of our system. No single person understands the implementation details of every endpoint in it, which adds to the complexity.</p><p>In my opinion, sharing experiences like this is crucial, as online searches often yield overly simplistic examples that aren’t helpful in real-world scenarios.</p><h3>Issue reported ☣️</h3><p>A ticket was created in May 2023, in which it was observed that response latency increases dramatically in periods when there is no releases to production.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m6kkHMCts-Yhs9AWACxqAQ.png" /><figcaption>Blue like in the chart representing 95 percentile of a page load, orange bar representing a release marker.</figcaption></figure><p>Since latency was increasing <strong>slowly</strong> <strong>over time</strong>, first suspect is of course memory and we managed to confirm that by inspecting memory usage of one pod in production:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cdgwCa8FvQuNOGSQcaAt5Q.png" /></figure><p>As you can see in the chart above, memory usage is increasing very slowly. In order to see any significant difference in memory usage you need to observe one instance of the application for couple of days.</p><p>As a first mitigation step, SRE team introduced a procedure to restart the service each day when there are no deployments planned. 🤕</p><h3>Attempt #1: Try to reproduce locally ❌</h3><p>My first reaction was to try to reproduce the problem locally so that I can get memory snapshots and find the root cause. 💡</p><p>I tried to run a load test on some random api endpoints for an hour and I was using <a href="https://clinicjs.org/doctor/">Clinic.js doctor</a> to get any hint about memory leak.</p><p>Although Clinic.js is a great and very valuable tool, it wasn’t the best fit for my use case. The biggest problem was that I couldn’t run Clinic.js for long periods of time (more than one hour) on my laptop, as it would get stuck when generating the final report. I needed to run the test for many hours.</p><p>As a second option I just ran the load test on my machine using <a href="https://github.com/tsenart/vegeta">vegeta</a> load testing cli:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ftLCuOKvllbhDfUD5Adb0g.png" /></figure><p>As you can see, latency was stable, so I needed to change my approach. 😩</p><h3>Attempt #2: Try to load test staging ❌</h3><p>Since it was very inconvenient to run load test on my local machine for many hours/days, I had an idea to run the test in staging instead.💡</p><p>This time, instead of randomly picking endpoints to load test, I went to our APM tool and found endpoints which receive most traffic and which are impacted the most by the latency increase.</p><p>One member of my team developed a really nice tool for running load tests as a Jenkins job for long periods of time with super nice reporting which was perfect for what I needed. 🤝</p><p>Everything was set, test was running for 5 days and … 🥁</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fv7KahSOXNQbtNqJQfyxRA.png" /></figure><p>Nothing! 😭 Memory usage was completely different compared to production and it became obvious that using a cannon to hit a fly kind of approach was not going to work.</p><h3>Attempt #3: Taking memory snapshots from production 🎯</h3><p>Since I couldn’t find a way to reproduce the problem outside of production, the only thing I could think of was to just take heap snapshots directly from production and compare them. 💡</p><p>I already had experience with attaching dev tools to a pod running in kubernetes which I described in one of my previous posts <a href="https://medium.com/@amirilovic/how-to-debug-node-js-applications-in-kubernetes-c169f1e88c37">here</a>.</p><p>Before proceeding, it is important to understand the potential consequences of enabling a debugger in a production environment. ⚠️</p><p><strong>Security Concerns</strong>: Enabling the debugger can pose a significant security risk. Anyone with access could potentially retrieve the current memory state of the application, exposing any confidential information stored there.</p><p><strong>Performance Impact</strong>: While merely enabling the debugger should not affect your service’s performance, actively taking a heap snapshot will pause the execution of your process. This interruption can lead to errors within the application.</p><p><strong>Risk Mitigation</strong>: If your service runs multiple instances, the overall risk may be reduced. On top if you can temporarely prevent traffic from reaching one inc</p><p>Here is the script that would turn on debug mode on a pod:</p><pre><br># find a nice pod to debug<br>$ kubectl get pods | grep my-api<br> <br># find Node.js pid, in this example pid was 7<br>$ kubectl exec my-api-64dcf6b84d-nx5cl -- ps aux<br> <br># send kill signal to Node.js process =&gt; this enables debug mode<br>$ kubectl exec my-api-64dcf6b84d-nx5cl -- kill -SIGUSR1 7<br><br># port forward from pod in kubernetes to a local port<br>$ k port-forward my-api-64dcf6b84d-nx5cl 9229:9229</pre><p>This should be enough to be able to see the process in chrome://inspect and to start taking snapshots. But for me that just didn’t work, chrome inspector was not seeing the process and I couldn’t make it work 🤬</p><p>After googling for solution all day and a very long psychotherapy session with ChatGPT, I ran into this interesting <a href="https://nodejs.org/api/inspector.html#heap-profiler">piece of Node.js docs</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/781/1*KjBPmgD6TcnUt61681Hpbw.png" /></figure><p>This gave me an idea to actually create a script that I could run from time to time to take snapshots and then compare them. 💡</p><p>The script talks to the Node.js debugger using websockets, sends messages to take heap snapshot, collects the results and stores them in a .heapsnapshot file which can be opened in chrome dev tools. You can find the script <a href="https://gist.github.com/amirilovic/fff46ddd099965a572b8216b21f613eb">here</a>.</p><p>Finally I had all the data I needed to find the root cause. 🥳</p><p>Once you have the snapshots, comparing them in dev tools is easy:</p><ul><li>Go to chrome://inspect</li><li>Click on “Open dedicated DevTools for Node”</li><li>Go to Memory tab</li><li>Press load profile button</li><li>Load all the heap snapshots you want to compare</li><li>Select one profile, switch to comparison mode and select the second profile to compare with</li></ul><p>You should see a similar view as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gRvL2mmLvW2ojVob7boQ8Q.png" /></figure><p>But how to use this information? You see objects, strings, functions… being allocated/deallocated, number of instances going up and down all the time. How do you actually find the root cause? 🤔</p><p>After expanding everything endlessly, even copy pasting to excel to try to visualise the data I found that if there is a memory leak, than column “Size Delta” must be constantly going up for something. 🕵🏻‍♂️</p><p>And there it was “system / JSArrayBufferData”, was going up all that time! I saw hundreds of 8 KB objects being created between every two snapshots.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zJ4WmY5YqHNZg1gvBErnpg.png" /></figure><p>Looking at the allocation stack, I could see that the issue is related to handling of grpc connections. But we don’t actually use grpc 🤔</p><p>Looking at dependencies in package-lock.json, I quickly found that <a href="https://www.npmjs.com/package/@google-cloud/recaptcha-enterprise/v/2.1.3">google </a><a href="https://www.npmjs.com/package/@google-cloud/recaptcha-enterprise/v/2.1.3">recaptcha</a><a href="https://www.npmjs.com/package/@google-cloud/recaptcha-enterprise/v/2.1.3"> enterprise</a> lib is connecting to google backends using grpc. At first glance, looking at the library sample code and our implementation I couldn’t find any problem. Nothing in the sample code was mentioning opening or closing grpc connections. I was trying to find more samples from other users, and the only difference I could see was that everybody is creating a<strong> single instance</strong> of the recaptcha client and we were creating new client on every request to specific endpoints. After digging deeper in the library source code, I saw that every time a new client is created, new grpc connection is opened, but in our code we never actually close the connection. 🎯</p><p>I worked with the team that originally implemented the feature to refactor the code to use only a single instance of the client, I merged and deployed the change to production. 🚀</p><h3>Monitor changes in production 📈</h3><p>After a few days, I observed that in production memory usage is now almost completely flat:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ug0M5hRN1_OQjsR6gvqMTA.png" /></figure><p>Which in turn leads to stable latency:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e3aWFyCQvIutTmQjMJEm8A.png" /></figure><p>Success! 💪</p><p>Fun fact: I actually closed the ticket exactly one year after the issue was reported, on the same day. 🥹</p><h3>Conclusion 👨‍🎓</h3><p>Having the right set of tools to observe application performance in production is absolutely crucial for the success of any service, regardless of its size. ✅</p><p>Node.js offers a great debugger that makes it easy to obtain heap snapshots directly from production environments. Once you have these snapshots, Chrome DevTools provides an excellent and user-friendly method to compare them. However, interpreting the data to pinpoint the root cause of a memory leak can be confusing. The ‘Size Delta’ column is particularly useful for identifying what contributes most to the memory increase.</p><p>I hope the information in this post helps you identify memory issues in your applications. If you have experience with different tools or suggestions on any alternatives, please share your thoughts in the comments. 🙏</p><p>Until next time. 👋</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a1b363b4884f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to use React Suspense for data loading?]]></title>
            <link>https://medium.com/@amirilovic/how-to-use-react-suspense-for-data-loading-4b68f9200c19?source=rss-fd55cd889543------2</link>
            <guid isPermaLink="false">https://medium.com/p/4b68f9200c19</guid>
            <category><![CDATA[suspense]]></category>
            <category><![CDATA[server-components]]></category>
            <category><![CDATA[nextjs]]></category>
            <category><![CDATA[react]]></category>
            <dc:creator><![CDATA[Aleksandar Mirilovic]]></dc:creator>
            <pubDate>Wed, 13 Dec 2023 21:46:23 GMT</pubDate>
            <atom:updated>2023-12-13T21:46:23.469Z</atom:updated>
            <content:encoded><![CDATA[<p>I’m building an app in Next.js that I wanted to transfer from old page router to the new app router and to start leveraging html streaming and server components. In this post I won’t go deep into how Suspense and server components work in React and Nextjs, instead I want to explain how to use Suspense and server components on a real use case I had in my app.</p><h3>🧪 Example</h3><p>In my application I have a page that renders a list of products. Those products need to be server side rendered for SEO and performance reasons. I also need to support client side navigation to the page. Since api can be slow sometimes, I need to have some loading state for the list of products.</p><p>On initial load, page renders all the products:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tZZB70LCPz1tEosJZ5KvOw.gif" /><figcaption>Initial page load</figcaption></figure><p>You can see the blank screen before the page loads, that delay is caused by the api. This is not such a big issue in production as the page is cached by the CDN for most of the traffic.</p><p>On client side navigation component show a loading skeleton:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BwCVlL9Xo5d7LTICOjsdzA.gif" /><figcaption>Data loading with useEffect</figcaption></figure><p>Pretty simple scenario.</p><p>In the example, the code organised in the following way:</p><p>There is a simple function called <strong>fetchProducts</strong> that loads a list of Products with a delay of 2000 sec to simulate slow api.</p><p>There are following components:</p><p><strong>ProductCard</strong> renders a product.</p><p><strong>ProductList</strong> take a list of products as prop and renders them as ProductCards in a horizontal list.</p><pre>export function ProductList({ products }: { products: Product[] }) {<br>  return (<br>    &lt;HStack width=&quot;full&quot; padding={2} overflow=&quot;auto&quot; gap={4}&gt;<br>      {products.map((product) =&gt; (<br>        &lt;ProductCard key={product.id} product={product} /&gt;<br>      ))}<br>    &lt;/HStack&gt;<br>  );<br>}</pre><p><strong>ProductListSkeleton</strong> component renders 10 ProductCards in a loading state.</p><pre><br>export function ProductListSkeleton() {<br>  return (<br>    &lt;HStack width=&quot;full&quot; padding={2} overflow=&quot;auto&quot; gap={4}&gt;<br>      {Array.from({ length: 10 }).map((_, index) =&gt; (<br>        &lt;ProductCard key={index} /&gt;<br>      ))}<br>    &lt;/HStack&gt;<br>  );<br>}</pre><p><strong>ProductListWithEffect</strong> a smart component that can accept a list of products and render them using ProductList or if products are not provided it will load them using fetchProducts function.</p><pre>export function ProductListWithEffect({ products }: { products?: Product[] }) {<br>  const [productsState, setProductsState] = useState&lt;Product[] | undefined&gt;(<br>    products<br>  );<br><br>  useEffect(() =&gt; {<br>    if (!productsState) {<br>      fetchProducts().then((products) =&gt; setProductsState(products));<br>    }<br>  }, [productsState]);<br><br>  if (!productsState) {<br>    return &lt;ProductListSkeleton /&gt;;<br>  }<br><br>  return &lt;ProductList products={productsState} /&gt;;<br>}</pre><p>With the page router code for data loading looks like:</p><pre>export default function Page({ products }: { products: Product[] }) {<br>  return (<br>    &lt;ChakraProvider&gt;<br>      &lt;Header /&gt;<br>      &lt;Box width=&quot;full&quot;&gt;<br>        &lt;ProductListWithEffect products={products} /&gt;<br>      &lt;/Box&gt;<br>      &lt;Footer /&gt;<br>    &lt;/ChakraProvider&gt;<br>  );<br>}<br><br>export async function getServerSideProps() {<br>  const products = await fetchProducts();<br>  return {<br>    props: {<br>      products,<br>    },<br>  };<br>}</pre><h3>🥁 Data loading using Suspense</h3><p>At first glance, with Suspense and server components in Next.js we are able to write our code in a much simpler way, getting rid of state and useEffect.</p><p>In the app folder we can start simply by writing page logic like:</p><pre>export default async function Page() {<br>  const products = await fetchProducts();<br>  return (<br>    &lt;&gt;<br>      &lt;ProductList products={products} /&gt;<br>    &lt;/&gt;<br>  );<br>}</pre><p>Problem with this approach is that we loose the loading state 🫤</p><p>To fix that we can add Suspense or define the loading state for the page. <strong>I don’t want to write the loading state of the page</strong> as I want components to control their look and feel while loading, so let’s try Suspense.</p><p>We can add new component <strong>ProductListWithSuspense</strong> that will load the data and render them in with ProductList component:</p><pre>export async function ProductListWithSuspense() {<br>  const products = await fetchProducts();<br>  return &lt;ProductList products={products} /&gt;;<br>}</pre><p>Then in our page we use Suspense around this component and define the loading state:</p><pre>export default async function Page() {<br>  return (<br>    &lt;Suspense fallback={&lt;ProductListSkeleton /&gt;}&gt;<br>      &lt;ProductListWithSuspense /&gt;<br>    &lt;/Suspense&gt;<br>  );<br>}</pre><p>We are almost there:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pZFG5fgcpwbbmf1677H3Qw.gif" /><figcaption>Page loading with suspense</figcaption></figure><p>There are couple of problems with this approach:</p><ul><li>Products are not really server side rendered! Html is streamed to the browser and data will be displayed quickly — before the full hydration, but still <strong>if we disable javascript on the page, we will just see the loading skeleton</strong>. Depending on your use case this might not be an issue, but in my use case I know that the page is going to be caches by the CDN, so why not cache the end result without the loading state? 🤔</li><li>Second problem is that the page is setting up the loading skeleton, which means that any other place on the website needs to do the same kind of wiring.</li></ul><p>We can fix these problem by moving Suspense down the component tree and adding it conditionally:</p><pre>async function ProductListLoader() {<br>  const products = await fetchProducts();<br><br>  return &lt;ProductList products={products} /&gt;;<br>}<br><br>export function ProductListWithSuspense({<br>  products,<br>}: {<br>  products?: Product[];<br>}) {<br>  if (products) {<br>    return &lt;ProductList products={products} /&gt;;<br>  }<br><br>  return (<br>    &lt;Suspense fallback={&lt;ProductListSkeleton /&gt;}&gt;<br>      &lt;ProductListLoader /&gt;<br>    &lt;/Suspense&gt;<br>  );<br>}</pre><p>and our page looks like:</p><pre>export default async function Page() {<br>  const products = await fetchProducts();<br>  return &lt;ProductListWithSuspense products={products} /&gt;;<br>}</pre><p>You can see that we are now have almost the same logic as with useEffect, we are preloading data in the page and rendering the ProductList, but component is smart enough to load the data itself if the page decides not to preload the data. We can drop this component on any page and it will handle the loading of the data and rendering the loading skeleton.</p><p>Buuuuut, there is one more problem, the code in the Page executes on both initial page load and on client navigation. This is a big difference compared to getServerSideProps. You can see below why this is a problem:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_9TvDE9TRuiy773_S-VXEg.gif" /><figcaption>Client side navigation with suspense</figcaption></figure><p>There is <strong>no loading state on client side navigation</strong> — when you click on a link, Next.js waits for the page to render and then shows the end result. Again this can be fixed with adding loading for the page, but as I said in the beginning I don’t want to do that as pages should be responsible for rendering the loading state individually and I don’t want to build layout of the page twice.</p><p>I couldn’t find any official docs that explain how to handle initial load of the page vs client side navigation, it took some time to figure out how to do it.</p><p>Reading about server components and inspecting the network traffic, I realised that one the client side navigation, Next.js makes a call to the server expecting a JSON payload 🥳</p><p>So, if we update the page code to be contidionally preload data, we would get our desired effect:</p><pre>export default async function Page() {<br>  let products: Product[] | undefined;<br><br>  // if browser is requesting html it means it&#39;s the first page load<br>  if (headers().get(&quot;accept&quot;)?.includes(&quot;text/html&quot;)) {<br>    products = await fetchProducts();<br>  }<br><br>  return &lt;ProductListWithSuspense products={products} /&gt;;<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cBschoCbpQzdKFTwA64KvA.gif" /></figure><p>So, finally we get the same behaviour as with the page router and useEffect 💪</p><h3>🤔 Conclusion</h3><p>Although the end result for the user is the same, new approach with Suspense and server components has some additional benefits.</p><p>First benefit for the developer, that the code now looks much simpler and is easy to read and understand.</p><p>Second benefit for the page performance, since there is not client side interactivity in the ProductList component it can be a server component and we don’t even need to ship component code to the client side, it only server side renders! In real world this wouldn’t be the case with product carousel, but for sure every app has some components which can be statically rendered without any client side interactivity, especially if you use server components in combination with <a href="https://nextjs.org/docs/app/building-your-application/data-fetching/server-actions-and-mutations">server actions</a>, which could reduce the amount of code that is shipped to the end user.</p><p>You can read more about server components <a href="https://nextjs.org/docs/app/building-your-application/rendering/server-components">here</a> and about loading with Suspense <a href="https://nextjs.org/docs/app/building-your-application/routing/loading-ui-and-streaming">here</a>.</p><p>You can find the code for the example <a href="https://github.com/amirilovic/nextjs-data-loading-with-suspense">here</a>.</p><p>If you have any suggestions how this use case could be implemented in a different way please write a comment below 👇.</p><p>👋</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4b68f9200c19" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Node.js Monorepo Survival Guide]]></title>
            <link>https://medium.com/@amirilovic/the-node-js-monorepo-survival-guide-84c7ad9b89ad?source=rss-fd55cd889543------2</link>
            <guid isPermaLink="false">https://medium.com/p/84c7ad9b89ad</guid>
            <category><![CDATA[turborepo]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[monorepo]]></category>
            <category><![CDATA[nx]]></category>
            <dc:creator><![CDATA[Aleksandar Mirilovic]]></dc:creator>
            <pubDate>Thu, 26 Oct 2023 15:05:28 GMT</pubDate>
            <atom:updated>2023-10-26T15:05:28.088Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Monorepo Jungle" src="https://cdn-images-1.medium.com/max/1024/1*gEiyg0o_GSdgs0A7NiMOtw.png" /><figcaption>Monorepo Jungle</figcaption></figure><p>In last couple of years, I’ve been involved in developing a large website using React and Node.js within a monorepo. It’s been a chaotic yet rewarding journey.</p><p>While on the platform team for that project, I extensively researched tools to manage the monorepo. Since tooling in the Node.js and frontend development is evolving rapidly, finding the right combination of tools and configuring them can be a challenge.</p><p>To guide you further, I’ve created a detailed <a href="https://github.com/amirilovic/surviving-monorepos">sample repo</a>.</p><p>The project includes a comprehensive readme. I won’t reiterate its content, but I’d like to share some personal insights on the matter:</p><ol><li>From my experience, what people commonly refer to as a monorepo isn’t the same as a monorepo at Google, Facebook, or Microsoft. For instance, if you’re building an e-commerce website with an API, frontend, and perhaps an admin dashboard, you might house them in one repository. However, if your company decides to develop an Android or iOS native app, you wouldn’t include it in the same repository. This differs from the monorepo approach of big tech companies. It also implies that supporting multiple programming languages in one repository might not be a top concern for you.</li><li>One of the key points of having monorepos is to share code. But consider this: if you have unrelated product domains, such as code for your corporate website and your internal HR application — even if both use the same programming language and tech stack — is there a genuine need to share code or house them in the same repository?</li><li>For a long time, Node.js had limited support for structuring complex applications. But with the adoption of workspaces across package managers, there’s now a standardised way to segment an application logically. This evolution has eliminated the need for convoluted features like aliasing, typescript project references, or custom project structures.</li><li>If your repository houses massive amount of code with many contributors, the tooling and maintenance need to be treated as a distinct product development. This requires a specialized team and introduces clear overhead costs you’d prefer to avoid.</li><li>I personally prefer tools that do one job well and which can be easily integrated with everything else. I try to avoid tools/frameworks that require me to change my whole project setup. That’s why I lean towards Turborepo over Nx. While Nx boasts superior features, Turborepo is simpler to set up and understand. It suffices for my needs and offers a straightforward, cost-free method to self-host a distributed cache server — vital for enterprise businesses. If you’re dealing with an enormous codebase, Nx might be your best bet, but I’d always recommend avoiding such massive repositories when possible.</li></ol><p>If the opinions above don’t resonate with you, the sample repo might not be your cup of tea. But that’s okay! 😊</p><p>If you have any suggestions to improve the sample repo, feel free to leave a comment or raise a PR. If you find the repo helpful, don’t forget to give it a ⭐️. Appreciate the support!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=84c7ad9b89ad" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to debug Node.js applications in Kubernetes?]]></title>
            <link>https://medium.com/@amirilovic/how-to-debug-node-js-applications-in-kubernetes-c169f1e88c37?source=rss-fd55cd889543------2</link>
            <guid isPermaLink="false">https://medium.com/p/c169f1e88c37</guid>
            <category><![CDATA[debugging]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Aleksandar Mirilovic]]></dc:creator>
            <pubDate>Sun, 10 Oct 2021 18:12:06 GMT</pubDate>
            <atom:updated>2021-10-10T18:12:06.076Z</atom:updated>
            <content:encoded><![CDATA[<h3>Why would you want to do this?</h3><p>While developing server side code, it happens from time to time that you experience an issue that is very hard to reproduce or you see memory leaks or CPU spikes which you can’t simulate locally, or you just want to add ad-hoc logs to your app. While developing application locally you would use Node.js inspector to debug and take memory/cpu snapshots which would help you find the issue, but how to do the same in remote environment? Luckily, Node.js has great support for remote debugging and in this article we will see how to use them in kubernetes.</p><h3>Example Application</h3><p>We are going to use simple example application to demo the whole process. Code for the demo app is here: <a href="https://github.com/amirilovic/example-app">https://github.com/amirilovic/example-app</a>.</p><h3>How to enable debug mode in Node.js process?</h3><p>Of course, by default debug mode is not enabled for Node.js process because it allows you to execute any arbitrary code on a remote machine. There are two ways how to enable debug mode in Node.js, using --inspectflag and sending SIGUSR1 signal to the process. You can find more info on that <a href="https://nodejs.org/en/docs/guides/debugging-getting-started/">here</a>.</p><h3>Using — inspect flag</h3><p>When you start Node.js process with --inspect flag debug mode is enabled immediately:</p><pre>$ node --inspect dist/index.js Debugger listening on ws://127.0.0.1:9229/1393d28d-2e81-4096-85e2-f0f877f22c10 For help, see: https://nodejs.org/en/docs/inspector Server listening on port 3000...</pre><p>In the message above you can see that debugger is listening on port 9229.</p><p>You can now use vscode or chrome inspector to attach debugger to that process:</p><figure><img alt="Attach vscode debugger to Node.js process" src="https://cdn-images-1.medium.com/max/1024/1*ik2z9aaVylNGeEluSGDEYQ.gif" /></figure><p>In the example above this is the default configuration for attaching to Node.js process:</p><pre>{<br>  &quot;name&quot;: &quot;Attach&quot;,<br>  &quot;port&quot;: 9229,<br>  &quot;request&quot;: &quot;attach&quot;,<br>  &quot;skipFiles&quot;: [&quot;&lt;node_internals&gt;/**&quot;],<br>  &quot;type&quot;: &quot;pwa-node&quot;<br>}</pre><p>One important note is that debugger by default listens only on connections coming from localhost — it will reject debug sessions from remote addresses. To get around this you either need to allow remote addresses by using --inspect=0.0.0.0:9229 or you need to create some kind of network tunnel between remote server and your computer, in the following sections I will show how easy it is to do tunneling in kubernetes.</p><h3>Sending SIGUSR1 signal to running process</h3><p>Sending a signal to the process is very useful to do ad-hoc debugging, without restarting the process.</p><p>Open one terminal window to start the process:</p><pre># start Node.js app<br>$ node dist/index.js<br>Server listening on port 3000...</pre><p>Open another terminal window to enable debug mode:</p><pre># find PID of Node.js process<br>$ ps aux | grep &quot;node dist/index.js&quot;</pre><pre># use the PID to send USR1 signal to the process<br>$ kill -USR1 [PID]</pre><p>Once you execute kill command, you will notice in the first terminal window that debugger is enabled with following message:</p><pre>Debugger listening on ws://127.0.0.1:9229/e2576437-dc86-4d42-9307-e9bf5db34e70<br>For help, see: <a href="https://nodejs.org/en/docs/inspector">https://nodejs.org/en/docs/inspector</a></pre><figure><img alt="Send USR1 signal to Node.js process" src="https://cdn-images-1.medium.com/max/900/1*YBouBVXck5icRY6LTdnDrQ.gif" /></figure><p>You can find more info on usingkill to send signals to processes <a href="https://bash.cyberciti.biz/guide/Sending_signal_to_Processes">here</a>.</p><h3>How to debug in Kubernetes?</h3><h3>Update liveness and readiness probes</h3><p>While debugging, if your process stops on a break point, Node.js won’t be able to respond to liveness and readiness probes to kubernetes and kubernetes will assume pod needs to be restarted, ending your debug session. To prevent this you need to update probes to allow these longer process stops:</p><pre># removing livenessProbe<br>$ kubectl patch deploy/example-app --type json -p=&#39;[{&quot;op&quot;: &quot;remove&quot;, &quot;path&quot;: &quot;/spec/template/spec/containers/0/livenessProbe&quot;}]&#39;</pre><pre># removing readinessProbe<br>$ kubectl patch deploy/example-app --type json -p=&#39;[{&quot;op&quot;: &quot;remove&quot;, &quot;path&quot;: &quot;/spec/template/spec/containers/0/readinessProbe&quot;}]&#39;</pre><pre># adding dummy livenessProbe<br>$ kubectl patch deploy/example-app -p &#39;{&quot;spec&quot;: {&quot;template&quot;: {&quot;spec&quot;: {&quot;containers&quot;: [{&quot;name&quot;: &quot;example-app&quot;, &quot;livenessProbe&quot;: {&quot;initialDelaySeconds&quot;: 5, &quot;periodSeconds&quot;: 5, &quot;exec&quot;: {&quot;command&quot;: [&quot;true&quot;]}}}]}}}}&#39;</pre><pre># adding dummy readinessProbe<br>$ kubectl patch deploy/example-app -p &#39;{&quot;spec&quot;: {&quot;template&quot;: {&quot;spec&quot;: {&quot;containers&quot;: [{&quot;name&quot;: &quot;example-app&quot;, &quot;readinessProbe&quot;: {&quot;initialDelaySeconds&quot;: 5, &quot;periodSeconds&quot;: 5, &quot;exec&quot;: {&quot;command&quot;: [&quot;true&quot;]}}}]}}}}&#39;</pre><h3>Scale Down (Optional)</h3><p>In order to debug you need to make sure that request actually hits the pod you are attaching to. This can be hard if you have multiple pods running. In our example we have defined 3 replicas in our deployment, but also pod auto scaling with HorizontalPodAutoscaler resource. So we need to tell kubernetes to scale everything to 1 pod and to keep it like that.</p><pre># check the number of pods<br>$ kubectl get pods | grep example-app<br>NAME                           READY   STATUS        RESTARTS      AGE<br>example-app-56cf79964d-g92n4   1/1     Running   0             53s<br>example-app-6b8fb58764-hhk46   1/1     Running   0             14s<br>example-app-6f79d6cf66-wdxqk   1/1     Running   0             71s</pre><pre># update HPA to run only one replica<br>$ kubectl patch hpa/example-app -p &#39;{&quot;spec&quot;: {&quot;minReplicas&quot;: 1, &quot;maxReplicas&quot;: 1}}&#39;</pre><pre># update deployment to run only one replica<br>$ kubectl scale --replicas=1 deploy/example-app</pre><pre># check the number of pods again<br>$ kubectl get pods | grep example-app<br>NAME                           READY   STATUS    RESTARTS   AGE<br>example-app-6b8fb58764-2qlgn   1/1     Running   0          110s</pre><h3>Enabling debugger</h3><p>As --inspect flag should not be enabled when running applications in production, it is easier to ad-hoc by sending USR1 signal to it than creating new pod with — inspect flag, we will do just that. But first you need to find the PID of your Node.js application.</p><p>If in your docker image you are running the application like:</p><pre>CMD [ &quot;node&quot;, &quot;dist/index.js&quot; ]</pre><p>PID of your application will be 1. If you are not sure you can use following commands to find out:</p><pre># find Node.js process PID<br>$ kubectl exec -it deploy/example-app -- /bin/sh -c &quot;ps aux&quot;</pre><pre># if there is no ps in the docker image, use:<br>$ kubectl exec -it deploy/example-app -- /bin/sh -c &quot;find /proc -mindepth 2 -maxdepth 2 -name exe -exec ls -lh {} \; 2&gt;/dev/null&quot;</pre><pre>lrwxrwxrwx 1 node node 0 Sep 30 04:11 /proc/1/exe -&gt; /usr/local/bin/node<br>lrwxrwxrwx 1 node node 0 Sep 30 04:11 /proc/587/exe -&gt; /bin/dash<br>lrwxrwxrwx 1 node node 0 Sep 30 04:11 /proc/594/exe -&gt; /usr/bin/find</pre><p>Then we enable debugger, in my case I’m using PID 1.</p><pre># enable debugger<br>kubectl exec -it deploy/example-app -- /bin/sh -c &quot;kill -USR1 1&quot;</pre><pre># verify that debugger has been enabled<br>$ kubectl logs deploy/example-app<br>Server listening on port 3000...<br>Debugger listening on ws://127.0.0.1:9229/496132b9-ec4b-43db-b103-4e15ba80518a<br>For help, see: <a href="https://nodejs.org/en/docs/inspector">https://nodejs.org/en/docs/inspector</a></pre><h3>Port forwarding</h3><p>In order to debug from our local machine to remote pod, we will use kubernetes port-forward feature:</p><pre># forward debug port from pod to our machine<br>$ kubectl port-forward deploy/example-app 9229:9229<br>Forwarding from 127.0.0.1:9229 -&gt; 9229<br>Forwarding from [::1]:9229 -&gt; 9229</pre><p>After we enable this, we can attach Node.js debugger to localhost:9229.</p><h3>Debugging with VSCode</h3><p>You can now attach with vscode to debug. But the problem is that you can’t set breakpoints like you are used to:</p><figure><img alt="Unbound breakpoint in vscode" src="https://cdn-images-1.medium.com/max/1024/0*akh_kxncnNDksLQC" /></figure><p>The issue here is that vscode doesn’t know how to map files from the container to local source code. To enable this, we need to update our attach configuration in .vscode/launch.json:</p><pre>    {<br>      &quot;name&quot;: &quot;Attach&quot;,<br>      &quot;port&quot;: 9229,<br>      &quot;request&quot;: &quot;attach&quot;,<br>      &quot;skipFiles&quot;: [&quot;&lt;node_internals&gt;/**&quot;],<br>      &quot;type&quot;: &quot;pwa-node&quot;,<br>      &quot;localRoot&quot;: &quot;${workspaceFolder}&quot;,<br>      &quot;remoteRoot&quot;: &quot;/app&quot;,<br>      &quot;sourceMaps&quot;: true<br>    }</pre><p>Properties that we added are:</p><ul><li>localRoot tells vscode what is the project root on your computer</li><li>remoteRoot tells vscode what path in the containers maps to our localRoot</li><li>sourceMaps tells vscode to take into account sourcemap information so that we can actually debug from typescript source files.</li></ul><p>Once we set this we can debug normally:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MoAVfnC8w9B_2JWRxnOfeQ.gif" /></figure><h3>CPU Profiling and Memory Snapshots</h3><p>If you need to debug memory leak or CPU spikes best tool to use is chrome dev tools CPU and memory profiling. Once debug more is enabled it is easy to connect with chrome to the remote process and do profiling. Open chrome and go to chrome://inspect:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iLs8eKRjSHa1SWJ_7T6azQ.gif" /></figure><p>You can find more information on how to use memory snapshots to find memory leaks <a href="https://www.toptal.com/nodejs/debugging-memory-leaks-node-js-applications">here</a>.</p><h3>Conclusion</h3><p>Hope this gave you some insights how powerful remote debugging in Node.js is, and what great set of tools are out there to help you. It’s not very often that you encounter issues which require you to debug remote process, but if you know techniques like this, they can prevent a lot of headaches.</p><h3>Links</h3><ol><li><a href="https://nodejs.org/en/docs/guides/debugging-getting-started/">https://nodejs.org/en/docs/guides/debugging-getting-started/</a></li><li><a href="https://bash.cyberciti.biz/guide/Sending_signal_to_Processes">https://bash.cyberciti.biz/guide/Sending_signal_to_Processes</a></li><li><a href="https://www.toptal.com/nodejs/debugging-memory-leaks-node-js-applications">https://www.toptal.com/nodejs/debugging-memory-leaks-node-js-applications</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c169f1e88c37" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to fix node dns issues?]]></title>
            <link>https://medium.com/@amirilovic/how-to-fix-node-dns-issues-5d4ec2e12e95?source=rss-fd55cd889543------2</link>
            <guid isPermaLink="false">https://medium.com/p/5d4ec2e12e95</guid>
            <category><![CDATA[coredns]]></category>
            <category><![CDATA[dns]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Aleksandar Mirilovic]]></dc:creator>
            <pubDate>Wed, 04 Dec 2019 22:35:44 GMT</pubDate>
            <atom:updated>2019-12-04T22:35:44.259Z</atom:updated>
            <content:encoded><![CDATA[<h3>How to fix nodejs DNS issues?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IXw0QNQgsiarjMUtoLfxkA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@tvick?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Taylor Vick</a> on <a href="https://unsplash.com/s/photos/networking?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Networking issues are always most challenging for me and at the same time most rewarding as I always learn something new. While working on big node eCOM backend that had a lot of traffic, from time to time we found <em>getaddrinfo EAI_AGAIN </em>error in our logs, quick googling explains that this means that our dns server can’t currently serve our request. And one of suggestion was to increase <em>UV_THREADPOOL_SIZE </em>to 128 (default value is 4).</p><p><a href="https://github.com/googleapis/google-api-nodejs-client/issues/761">getaddrinfo EAI_AGAIN · Issue #761 · googleapis/google-api-nodejs-client</a></p><p>this seems legit since node docs also mention this problem:</p><p><a href="https://nodejs.org/api/dns.html#dns_implementation_considerations">DNS | Node.js v19.2.0 Documentation</a></p><blockquote>Though the call to dns.lookup() will be asynchronous from JavaScript&#39;s perspective, it is implemented as a synchronous call to <a href="http://man7.org/linux/man-pages/man3/getaddrinfo.3.html">getaddrinfo(3)</a> that runs on libuv&#39;s threadpool. This can have surprising negative performance implications for some applications, see the <a href="https://nodejs.org/api/cli.html#cli_uv_threadpool_size_size">UV_THREADPOOL_SIZE</a> documentation for more information.</blockquote><p>and from: <a href="https://nodejs.org/api/cli.html#cli_uv_threadpool_size_size">https://nodejs.org/api/cli.html#cli_uv_threadpool_size_size</a></p><blockquote>Because libuv’s threadpool has a fixed size, it means that if for whatever reason any of these APIs takes a long time, other (seemingly unrelated) APIs that run in libuv’s threadpool will experience degraded performance. In order to mitigate this issue, one potential solution is to increase the size of libuv’s threadpool by setting the &#39;UV_THREADPOOL_SIZE&#39; environment variable to a value greater than 4 (its current default value)</blockquote><p>We tried this option without any deep investigation and it really seemed to help. So, we thought that we are hitting some limits in default node configuration and were happy with the solution and life was good for couple of months :)</p><p>While preparing for upcoming ultimate shopping mania sprees, we where running load tests across our whole system composed of bunch of different services. One team came back with report that they had serious issues with latency of dns lookups and error rates with coredns service in our kuberntes cluster even with many replicas and of course they already had <em>UV_THREADPOOL_SIZE </em>magic number fine tuned. Their solution to the problem was to include <a href="https://www.npmjs.com/package/lookup-dns-cache">https://www.npmjs.com/package/lookup-dns-cache</a> package in order to cache dns lookups. In their load tests it showed amazing results by improving performance 2x. At first everybody was happy, but to me this was very very weird and completely against how I believed dns resolution should work in the first place, it was time for some🕵🏻‍♂️ work.</p><p>Why is node having issues with dns lookups? How can it be that node has such bad dns resolution handling?</p><p>My biggest problem with above solution was that it doesn’t make any sense to implement dns caching on application level, it should be handled by node runtime or on OS level, so I wanted to figure out how node deals with it internally and what we didn’t setup correctly. Also handling it in application means that all apps written in any other language need to figure out a way to cache dns lookups their own and were potentially affected.</p><p>Back to the drawing board, or should I say googling. I saw that there is ton of posts and stackoverflow discussions about this topic, but it is hard to draw any conclusion because everybody has different setup and there could be ton of factors causing this behavior. Going back to the original node docs above, it states that node net module is using dns.lookup which is calling native getaddrinfo function by default which is synchronous so it needs to use libuv threadpool to make it behave as async. People where suggesting to bypass dns.lookup and to use dns.resolve instead as it does network call which has async native API so no need to hit libuv threadpool and this is exactly what lookup-dns-cache does, so that is just a workaround and not telling what is the original issue. Also getaddrinfo function is taking configuration from hosts file, which dns.resolve obviously doesn’t, and that can cause unexpected behavior and we do use hosts file in development and having app behave differently across environments can be a bomb waiting to go off any time.</p><p>One interesting thing I quickly found is that getaddrinfo is not returning TTL when resolving queries, this was clear indicator that node is for sure not handling dns cache and that it expected OS to handle that.</p><p>When trying to figure out how getaddrinfo actually works I found this amazing article that goes through it’s source code and explains each step:</p><p><a href="https://jameshfisher.com/2018/02/03/what-does-getaddrinfo-do/">What does `getaddrinfo` do?</a></p><p>This step is was the key:</p><blockquote>Sorry, nscd is actually the “name service cache daemon”, “a daemon that provides a cache for the most common name service requests”. After installing it, the daemon starts, and your process can dance</blockquote><p>Of course we are running in docker in kubernetes so for sure our pods don’t have nscd installed and after talking to platform team we figured out that there was nothing on kubernetes node level that was doing caching so all requests where hitting coredns service which broke under heavy load 🤦‍♂️</p><p>Platform team decided to setup caching on node level by using:</p><p><a href="https://kubernetes.io/docs/tasks/administer-cluster/nodelocaldns/">https://kubernetes.io/docs/tasks/administer-cluster/nodelocaldns/</a></p><p>This will fix the issue for node and any other runtime that relies on OS to do dns query caching.</p><h3>Conclusion</h3><p>Don’t jump the gun and copy paste solution from stackoverflow blindly, try to dig deeper and understand how things work under the hood.</p><p>There is nothing wrong with how node handles dns queries it just expects OS to handle the caching part, you just need to be aware of this.</p><p>PS. I wanted to also figure out if Java apps have similar problem since we are using Java a lot as well. As I expected nothing less from Java it handles everything on it’s own:</p><p><a href="https://cr.openjdk.java.net/~iris/se/11/latestSpec/api/java.base/java/net/doc-files/net-properties.html">Networking Properties</a></p><blockquote>Address Cache<br>The java.net package, when doing name resolution, uses an address cache for both security and performance reasons. Any address resolution attempt, be it forward (name to IP address) or reverse (IP address to name), will have its result cached, whether it was successful or not, so that subsequent identical requests will not have to access the naming service. These properties allow for some tuning on how the cache is operating.</blockquote><blockquote>networkaddress.cache.ttl (default: see below)<br>Value is an integer corresponding to the number of seconds successful name lookups will be kept in the cache. A value of -1, or any other negative value for that matter, indicates a “cache forever” policy, while a value of 0 (zero) means no caching. The default value is -1 (forever) if a security manager is installed, and implementation specific when no security manager is installed.</blockquote><blockquote>networkaddress.cache.negative.ttl (default: 10)<br>Value is an integer corresponding to the number of seconds an unsuccessful name lookup will be kept in the cache. A value of -1, or any negative value, means “cache forever”, while a value of 0 (zero) means no caching.</blockquote><p>Never though I would say this for Java, but for cross platform runtime it actually makes sense to provide dns cache instead of relying on OS, but having default value -1 = forever, doesn’t make any sense so it’s 1:1 for Java 😀(don’t know what is security manager so I’m ignoring that part in docs on purpose 😉)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5d4ec2e12e95" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Web Microfrontends Introduction]]></title>
            <link>https://medium.com/@amirilovic/web-microfrontends-introduction-451f72e68c8c?source=rss-fd55cd889543------2</link>
            <guid isPermaLink="false">https://medium.com/p/451f72e68c8c</guid>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[micro-frontends]]></category>
            <category><![CDATA[nodejs]]></category>
            <dc:creator><![CDATA[Aleksandar Mirilovic]]></dc:creator>
            <pubDate>Thu, 24 Jan 2019 06:35:27 GMT</pubDate>
            <atom:updated>2019-01-24T06:35:27.208Z</atom:updated>
            <content:encoded><![CDATA[<h3>Web Micro Frontends Introduction</h3><blockquote>The idea behind Micro Frontends is to think about a website or web app as a composition of features which are owned by independent teams. Each team has a distinct area of business or mission it cares about and specialises in. A team is cross functional and develops its features end-to-end, from database to user interface. <em>— </em><a href="https://micro-frontends.org/"><em>micro-frontends.org</em></a></blockquote><p>Micro Frontends are natural extension of microservices architecture where <em>bigger</em> organisations try to optimise their software delivery by splitting one system into <strong>smaller independent services that can be developed, deployed and maintained by small autonomous teams.</strong></p><h3>Why Micro Frontends?</h3><p>Choosing architecture depends on a <strong>type of product</strong> that you are trying to build and on your <strong>organisation size and goals</strong>.</p><p>Regarding type of product, very interesting categorisation of frontends is done by Aral Balkan in <a href="https://2018.ar.al/notes/the-documents-to-applications-continuum/">Sites vs. Apps defined: the Documents‐to‐Applications Continuum</a> where he makes difference between web sites and web apps:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*dD8hWBvuzwfk090T7UhMkA.png" /><figcaption>Web sites vs Web apps</figcaption></figure><p>My favourite part of the article:</p><blockquote>To easily differentiate between sites and apps, ask yourself the following question:</blockquote><blockquote>‘If I was to remove all behaviour from this product, would the content still have value?’</blockquote><p>On the above scale on the left side you have blogs and news sites where it makes sense to use static site generators even, on the far right you have Online IDEs and Graphics Tools where SPAs are appropriate solution.</p><p>Micro Frontends are somewhere in the middle overlapping with SPAs a lot, but while SPAs have all features of monolith, micro frontends are more closer to microservices.</p><p>Regarding organisation size and goals, micro frontends add a lot of complexity and make more sense for organisations that have many teams working on a single project, but also for organisations that optimise for delivery speed and ability to adapt to changes.</p><p>Again, as it was said many times before, microservices and monoliths both have their pros and cons and either can be perfect fit for your project, the important thing is to know your choices. Micro frontends could make sense for one part of your website and SPAs for another, there is nothing stoping you using them both or even sharing functionality between them.</p><h3>How to do Microfrontends?</h3><p>While doing research, I came a cross very interesting approaches how some well known companies implemented micro frontends, so I’ll try to summarise my notes here. One important observation is that all mentioned companies here are in e-com business providing either web shop or marketplace experience to their customers. Micro frontends make sense for them since those web sites are content heavy, but also add a lot of dynamic elements to offer great UX.</p><h3>Zalando</h3><p>Online fashion retailer <a href="http://zalando.com/">Zalando</a> is probably most famous implementation of micro frontends, they have open-sourced all of their micro frontend tools under <a href="https://www.mosaic9.org/">Project Mosaic</a>.</p><p><a href="https://www.youtube.com/watch?v=m32EdvitXy4">Here</a> you can find talk about history of micro frontends at Zalando, with some great points why company would even consider micro frontends architecture.</p><p>Notes</p><ul><li>This is a complete solution for building micro frontends, it covers how fragments are built but also how end pages are assembled.</li><li>There are a lot of custom made services which create steep learning curve</li><li>Skipper is routing engine which enables dynamic routing rule changes.</li><li>Taylor is layout engine for importing all fragments and composing the end page.</li><li>They standardise on AMD module distribution and RequireJS for building and deploying fragments.</li><li>They can support different frontend frameworks but decide to standardise on React in order to optimise for performance.</li><li><a href="https://github.com/zalando-incubator/tessellate">Tessellate</a> is built to support React server side rendering.</li></ul><h3>AutoScout24</h3><p><a href="https://autoscout24.com/">AutoScout24</a> is one of the biggest online car marketplaces in Germany. Together with ThoughtWorks they completely changed the architecture of their website in order to support their digital transformation.</p><p>Moritz Heiber from ThoughtWorks gave a great talk about their approach <a href="https://www.thoughtworks.com/talks/a-high-performmance-solution-to-microservice-ui-composition">here</a>.</p><p>Notes</p><ul><li>Talk is mostly about how end pages are assembled, but not how page fragments are built.</li><li>Performance was one of the key goals from the start and they used google page speed index to measure performance</li><li>They use NGINX with SSI (Server-Side Include)and <a href="https://www.modpagespeed.com/">PageSpeed</a> module to compose end pages, that makes this solution very easy to start with.</li><li>PageSpeed module does all html manipulation to optimise page according to Google guidelines and deals with page caching.</li></ul><h3>Ikea</h3><p><a href="https://www.ikea.com/">Ikea</a> is probably the worlds most famous furniture manufacturer. They are present in most markets in the world which adds a lot of complexity to their operating model, so they had to optimise their architecture to support changes.</p><p><a href="https://vimeo.com/242458365">Here</a> is a talk from Gustaf Nilsson Kotte that covers high level aspects of their implementation.</p><p>Notes</p><ul><li>Talk is mostly about how end pages are assembled, but not how page fragments are built.</li><li>They heavily use Akamai for routing and page composition with ESI (Edge Side Include — very similar to NGINX SSI)</li><li>They use multiple frontend frameworks on different pages (Preact, React, VueJS…)</li><li>They also build static pages where content is not very dynamic like home page</li></ul><h3>OpenTable</h3><p><a href="https://www.opentable.com/">OpenTable</a>, at least for me, was least known company from this list, but they really moved tooling for micro frontends to the next level. They are working with micro frontends for a long time and they evolved their architecture and tooling and have open sourced everything under <a href="https://opencomponents.github.io/">Open Components</a> project.</p><p><a href="https://www.youtube.com/watch?v=M4OXXRdCpyQ">Here</a> is a great talk by Matteo Figus about evolution of their approach.</p><p>My Notes</p><ul><li>Open Components is a solution for creating, publishing and including composable fragments of a web page, how actual page is assembled in the end is left to the actual implementation</li><li>Component/Fragment versioning is completely supported, you can publish new component version and when including a component you can specify which version you want.</li><li>Developers use “oc” CLI to bootstrap development and to do deployments. CLI makes it very easy to add new components.</li><li>Solution can support multiple frontend frameworks.</li><li>All the required tools for React implementation already exist. For building open components with React there is <a href="https://github.com/opencomponents/oc-template-react">https://github.com/opencomponents/oc-template-react</a> and for client side or server side rendering of open components in React there is <a href="https://github.com/opencomponents/react-oc">https://github.com/opencomponents/react-oc</a></li><li>Server side rendering is optional, so if server side rendering fails or you don’t want some fragments to be rendered on the server, you can delegate rendering to the client side.</li></ul><h3>Server-Side Architecture</h3><p>Going over above mentioned approaches we can identify key components needed for implementation of micro frontends.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/131/1*QskjvDW71HznDZR6OgBpPA.png" /><figcaption>High level Server-Side Architecture</figcaption></figure><p><strong>Router</strong> is the first service that gets user requests, based on set of rules it decides who will be responsible for page rendering. It acts as a reverse proxy and can be implemented in various ways, in the above examples Skipper, NGINX and Akamai are used. Very useful router feature is the ability to dynamically change routing rules without code changes. This can be essential when you have a big website that is served by multiple origins based on set of complex rules like user location, country, url pattern…</p><p><strong>Page service</strong> is responsible for end page assembly. It provides <strong>layout</strong> of the page and knows how to <strong>include all page fragments</strong>. Page layout can be a custom web page with “hard-coded” template or you can build a template repository where you can dynamically manage all page templates this depends on your business requirements. For example, Zalando has Quilt as a layout repository where all layouts can be managed dynamically. Inclusion of fragments can be done by using server side include (SSI) supported by NGINX or edge-side include (ESI) supported by Akamai or you can build a custom solution like Zalando did with Taylor. When including fragments it is very important to optimise page resources in order to have decent performance. Inclusion of fragments is done in dynamic way, so when new version of fragment is deployed it automatically gets included in the page, no need for additional manual deployments. There should be a set of default dependencies that all fragments can rely on like style toolkit and client side framework. Even though you can mix client side frameworks, in practice companies standardise on one or two in order to trim down page size as much as possible. In case of Zalando they use React, in case of Ikea they use different frameworks <strong>on different pages</strong> (Preact for search, React for wishlist…). For resource optimisation, for me very interesting use case is how AutoScout24 uses NGINX with PageSpeed to optimise page resources according to Google performance guidelines. Solution is very simple and effective and can be applied to wide variety of web sites.</p><p><strong>Fragment service </strong>is responsible for rendering one part of the page. For example, fragments can be header, footer, product information, product recommendation… In order to render a fragment, service needs access to all fragment dependencies like html templates, css and javascript files and APIs. If we take React as an example, service needs to fetch all data for server-side rendering, then create instance of react component and provide data to it then do server side rendering with `ReactDOMServer.renderToString`. In order for component to be correctly rendered on the client side, service also needs to provide all necessary js and css assets to page service as well as data necessary for hydration. In the above examples, Zalando has service <a href="https://github.com/zalando-incubator/tessellate">Tessellate</a> that does rendering of react components, Open Components can be used with any framework, but if you are using React take a look at <a href="https://github.com/opencomponents/oc-template-react">oc-template-react</a> and <a href="https://github.com/opencomponents/react-oc">react-oc</a>. As already mentioned, Open Components also support client side rendering mode, where it is not required to do SSR at all or can be used as fallback if SSR fails or times out.</p><h3>Client-Side Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/471/1*sNrPOfKpVdgdz2SOOzMEqg.png" /><figcaption>High level Frontend architecture</figcaption></figure><p>Since all fragments are built as completely independent components they can’t directly reference each other, so event bus is introduced for specific use cases where communication on client side is needed. Event bus is a very light component that only does simple pub-sub and doesn’t support message persistence or playback. In talks above you can see that, communication between components is rarely needed and that fragments usually communicate to their backend.</p><h3>Conclusion</h3><p>Micro frontends architecture offers opportunity to build highly modular frontends with same benefits as microservices architecture. It makes sense for bigger organisations that have multiple frontend teams and for websites that are content heavy. With high modularity comes higher complexity that adds a lot overhead work for system operations. Some well known companies have successfully implemented micro frontends and have shared their experiences and open-sourced their tools.</p><p>Please comment if you have any experience with micro frontends.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=451f72e68c8c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>