<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mustafa Abdelhamid on Medium]]></title>
        <description><![CDATA[Stories by Mustafa Abdelhamid on Medium]]></description>
        <link>https://medium.com/@mou.abdelhamid?source=rss-d670c15e6673------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*P8-t_75mJB8rsqFI.jpg</url>
            <title>Stories by Mustafa Abdelhamid on Medium</title>
            <link>https://medium.com/@mou.abdelhamid?source=rss-d670c15e6673------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 21:13:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mou.abdelhamid/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Smart Routing for Full-Duplex Speech Models: Augmenting PersonaPlex with External LLM Knowledge]]></title>
            <link>https://medium.com/@mou.abdelhamid/smart-routing-for-full-duplex-speech-models-augmenting-personaplex-with-external-llm-knowledge-09abaccd1d70?source=rss-d670c15e6673------2</link>
            <guid isPermaLink="false">https://medium.com/p/09abaccd1d70</guid>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[ai-voice-agent]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[voice-assistant]]></category>
            <dc:creator><![CDATA[Mustafa Abdelhamid]]></dc:creator>
            <pubDate>Thu, 26 Feb 2026 07:34:01 GMT</pubDate>
            <atom:updated>2026-02-26T07:34:01.575Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M-8kaLRCVVfxsgWFEbGvEQ.png" /></figure><p>NVIDIA released <a href="https://research.nvidia.com/labs/adlr/personaplex/"><strong>PersonaPlex</strong></a> in January 2026, a full-duplex speech-to-speech model built on the Moshi architecture. We forked it to build a voice assistant widget for our product landing page and ran into a problem that turns out to be fundamental to this class of models.</p><p>Full-duplex speech-to-speech models like Moshi and PersonaPlex achieve remarkably low latency (~200ms) conversational interaction by jointly modeling user and agent audio streams. However, these models use relatively small language model backbones (7B parameters) that are optimized for conversational dynamics — turn-taking, interruptions, backchannels — rather than factual knowledge or instruction following. This creates a fundamental tension: the model excels at <em>sounding</em> like a helpful agent but struggles to <em>be</em> one when domain-specific knowledge is required.</p><p>This document describes an experimental architecture we call Smart Routing, in which PersonaPlex handles the real-time voice layer while an external LLM (DeepSeek) provides intelligent responses to domain-specific questions. We implemented a complete prototype including a new WebSocket protocol extension (TTS injection via message type 0x07), server-side defer detection, asynchronous LLM consultation, and forced text-token generation through the existing model. We report on what worked, what did not, and what directions remain open.</p><h3>Problem Statement</h3><p>We wanted to build a voice AI assistant for a product landing page — a widget where visitors could have a real-time spoken conversation about the product’s features, pricing, and capabilities. The requirements were:</p><ol><li>Low latency: responses under 500ms for natural conversation flow</li><li>Full-duplex: the user should be able to interrupt, and the agent should produce backchannels (“uh-huh”, “right”) naturally</li><li>Domain knowledge: accurate answers about specific product details (pricing tiers, feature lists, supported languages, compliance certifications)</li><li>Natural voice: human-like prosody, not robotic TTS</li></ol><p>PersonaPlex satisfies requirements 1, 2, and 4 out of the box. It fails on requirement 3. When asked “What are your pricing plans?”, it generates fluent but fabricated answers — or worse, confidently wrong ones. The Helium 7B backbone was trained for conversational realism, not factual grounding.</p><p>The core research question: Can we route specific queries from a full-duplex speech model to an external LLM and inject the response back into the speech stream, without breaking the real-time conversational experience?</p><h3>Background and Related Work</h3><h4>The Moshi Architecture</h4><p>PersonaPlex is built on the Moshi architecture (Defossez et al., 2024). Moshi models spoken dialogue as parallel audio token streams using a neural audio codec (Mimi). The key architectural features relevant to this work:</p><ul><li>Dual-stream modeling: The 7B transformer processes two simultaneous audio streams (agent output and user input) as interleaved codebook tokens, allowing full-duplex interaction without explicit turn-taking.</li><li>Inner Monologue: Text tokens are generated as a prefix to audio tokens at each step, improving linguistic quality. These text tokens form a time-aligned transcript that we can inspect for routing signals.</li><li>Depformer: A secondary 6-layer transformer generates dependent audio codebook tokens autoregressively, conditioned on the main transformer’s output.</li><li>Frame rate: 12.5 Hz (80ms per frame), with 8 audio codebooks per stream using a 2048-bin vocabulary.</li></ul><p>The LMGen.step() method accepts an optional text_token parameter that, when provided, forces the text output while still generating corresponding audio tokens through the depformer. This is the mechanism we exploit for TTS injection.</p><h4>The Instruction-Following Gap in Speech Models</h4><p>A central challenge we encountered is well-documented in the broader LLM literature: small models struggle with reliable instruction following, and speech models compound this problem.</p><p>Geng et al. (2025) demonstrate in “Control Illusion” that even state-of-the-art text LLMs (70B+ parameters) struggle with consistent instruction prioritization. System/user prompt separation “fails to establish a reliable instruction hierarchy,” and models exhibit strong inherent biases toward certain behaviors regardless of prompt instructions. Our experience with a 7B speech model that was never explicitly trained for instruction following confirms this finding in a more extreme setting.</p><p>The DiVA approach (Held et al., 2024) addresses a related problem — Speech LLMs “forgetting” text-only capabilities during supervised fine-tuning — by using text LLM responses to transcripts as self-supervision. This is conceptually similar to our routing approach: we preserve the speech model’s conversational strengths while delegating knowledge-intensive responses to a text LLM. The difference is that DiVA does this at training time through distillation, while we attempt it at inference time through runtime routing.</p><p>FunAudioLLM (An et al., 2024) takes a modular approach with separate SenseVoice (understanding) and CosyVoice (generation) models orchestrated by an LLM. This pipeline architecture achieves high quality but at the cost of latency — exactly the tradeoff full-duplex models like Moshi were designed to avoid.</p><h4>Hybrid Speech Architectures</h4><p>The general pattern of combining a fast but limited model with a slow but capable one appears in several forms in recent work:</p><ul><li>Speculative decoding uses a small model to draft tokens that a large model verifies, optimizing throughput. Our approach inverts this: the small model handles the real-time stream, and the large model is consulted asynchronously for specific queries.</li><li>Mixture-of-experts routes different inputs to different specialized sub-networks. Our routing is coarser-grained: entire conversational turns are routed between two fundamentally different systems.</li><li>Tool use in LLMs (e.g., function calling) is the closest analogy. We are effectively trying to give a speech model the ability to “call a tool” (the external LLM) mid-conversation. The difference is that tool-use protocols were designed into text LLMs during training, while we are attempting to bolt this capability onto a model that was not designed for it.</li></ul><h3>Architecture</h3><h4>System Overview</h4><p>The Smart Routing architecture has three components:</p><p>Browser ← WebSocket (binary) → PersonaPlex Server (GPU)<br> │<br> ├── Moshi LM (7B) generates text + audio<br> ├── Text stream monitored for defer signals<br> ├── On defer: async call to DeepSeek API<br> └── DeepSeek response injected via TTS</p><p>The entire routing pipeline runs server-side. The browser client is unaware of the routing; it receives a continuous stream of audio and text regardless of whether the response originated from PersonaPlex’s LM or from DeepSeek via TTS injection. See the <a href="https://github.com/dosht/personaplex/blob/main/WEB_CLIENT.md">Web Client documentation</a> for the full client-side protocol and audio pipeline documentation.</p><h4>Protocol Extension</h4><p>We extended the Moshi WebSocket protocol with a new message type:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*zaBQ-qzT0YBNWtJadcDUEQ.png" /></figure><p>All messages share the same framing: [1 byte type][N bytes payload], with WebSocket framing providing message boundaries.</p><h4>TTS Injection Mechanism</h4><p>The TTS injection exploits the existing LMGen.step() API:</p><pre>async def process_tts_inject(self, text, opus_writer, ws):<br>    tokens = self.text_tokenizer.encode(text)<br><br>    for text_token in tokens:<br>        silence_frame = torch.zeros(1, 1, frame_size)<br>        silence_codes = self.mimi.encode(silence_frame)<br><br>        out = self.lm_gen.step(<br>            input_tokens=silence_codes,    # silence on user channel<br>            text_token=text_token           # forced text token<br>        )<br>        # out contains audio tokens generated by the depformer<br>        # conditioned on the forced text token<br><br>        pcm = self.mimi.decode(out[:, 1:9])<br>        opus_writer.append_pcm(pcm[0, 0].numpy())<br><br>        transcript = self.text_tokenizer.id_to_piece(text_token)<br>        await ws.send_bytes(b&quot;\x02&quot; + transcript.encode())<br><br>        await asyncio.sleep(0.001)  # yield to event loop</pre><p>This works because the depformer generates audio tokens that are consistent with the forced text token, using the model’s current voice state (pitch, pace, style). The resulting speech sounds natural and matches the voice established during the voice prompt phase.</p><p>The injection queue uses asyncio.Queue(maxsize=10) for backpressure, checked non-blockingly in the main opus_loop() with get_nowait().</p><h4>Defer Detection</h4><p>We explored two approaches to signaling when PersonaPlex should defer to DeepSeek.</p><p><strong>Approach 1: Explicit Marker (</strong><strong>!!!)</strong></p><p>The system prompt instructs PersonaPlex to end responses with !!! when it encounters a product question:</p><pre>&lt;system&gt;<br>You are Transgate&#39;s voice assistant.<br><br>ROUTING RULES:<br>1. HANDLE DIRECTLY: greetings, acknowledgments, backchannels<br>2. DEFER by ending with !!! for: product questions, how-to, comparisons<br><br>EXAMPLES:<br>User: &quot;What&#39;s your pricing?&quot;<br>You: &quot;Great question! Let me tell you about our plans...!!!&quot;<br>&lt;system&gt;</pre><p>The server accumulates text tokens and checks for the marker:</p><pre>self.text_buffer += text<br>if &quot;!!!&quot; in self.text_buffer:<br>    user_query = extract_last_user_query(conversation_history)<br>    response = await call_deepseek(user_query, context)<br>    await process_tts_inject(response)<br>    self.text_buffer = &quot;&quot;</pre><p><strong>Approach 2: Natural Phrase Detection</strong></p><p>After observing that the !!! marker was unreliable (see <em>What Did Not Work below)</em>, we attempted detection based on filler phrases that PersonaPlex naturally produces:</p><pre>DEFER_PHRASES = [<br>    &quot;let me check&quot;,<br>    &quot;let me find&quot;,<br>    &quot;let me look&quot;,<br>    &quot;let me see&quot;,<br>]</pre><p>The system prompt was simplified to encourage these phrases:</p><pre>&lt;system&gt;<br>You are Transgate&#39;s friendly voice assistant.<br>For product questions about pricing, features, accuracy, or languages:<br>Say &quot;Let me check that for you&quot; then stop talking.<br>Keep responses brief.<br>&lt;system&gt;</pre><h4>Conversation Context Tracking</h4><p>To provide DeepSeek with conversational context, the server tracks recent utterances:</p><pre>conversation_history: list[str] = []  # max 10 entries<br>current_utterance = &quot;&quot;<br><br># Text tokens accumulate into current_utterance<br># On sentence boundaries (. ? !), saved to history<br># Last 6 entries passed to DeepSeek as context</pre><p>A significant limitation: we only have the <em>agent’s</em> text stream. The user’s audio is not transcribed server-side (that would require a separate ASR pipeline), so the conversation history is one-sided. We pass the agent’s filler text as a proxy for the user’s intent, which works poorly for follow-up questions.</p><h4>DeepSeek Integration</h4><p>The DeepSeek call is straightforward:</p><pre>async def call_deepseek(self, query, conversation_context=&quot;&quot;):<br>    messages = [<br>        {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: f&quot;&quot;&quot;<br>            Answer questions about Transgate concisely (2-3 sentences).<br>            No markdown. Suitable for spoken delivery.<br>            {TRANSGATE_CONTEXT}<br>        &quot;&quot;&quot;},<br>        {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: f&quot;Context: {conversation_context}\nQuestion: {query}&quot;}<br>    ]<br><br>response = await aiohttp_session.post(<br>        &quot;https://api.deepseek.com/chat/completions&quot;,<br>        json={&quot;model&quot;: &quot;deepseek-chat&quot;, &quot;max_tokens&quot;: 150, &quot;temperature&quot;: 0.7, ...}<br>    )<br>    # ... sanitize response (strip markdown) for TTS</pre><p>The response is sanitized to remove markdown formatting (headers, bold, links, bullet points) since the text will be vocalized, not displayed.</p><h4>Deployment</h4><p>The modified PersonaPlex server runs on Modal (serverless GPU):</p><ul><li><em>GPU: NVIDIA A10G (24GB VRAM), ~$0.46/hr</em></li><li><em>Container: One conversation per container (stateful LM)</em></li><li><em>Scale-down: 5-minute idle timeout</em></li><li><em>Cold start: ~60s (model loading), ~50ms warm</em></li><li><em>Model: </em><em>nvidia/personaplex-7b-v1 from HuggingFace (gated, ~15GB)</em></li></ul><h3>What Worked</h3><h4>TTS Injection</h4><p>The TTS injection mechanism works correctly. Forcing text tokens through LMGen.step() produces natural-sounding speech that matches the established voice. The depformer generates appropriate prosody for the forced text, and the audio quality is indistinguishable from PersonaPlex&#39;s native generation.</p><p>Key implementation detail: feeding silence frames (torch.zeros) on the user input channel during injection, rather than sine wave tokens, produces cleaner output. The model interprets this as &quot;the user is not speaking,&quot; which is the desired state during an injected response.</p><h4>Protocol Extension</h4><p>The 0x07 message type integrates cleanly with the existing protocol. The non-blocking queue in opus_loop() allows injection without disrupting the main audio pipeline. The client receives injected audio and text through the same 0x01/0x02 channels as native output, requiring no client-side changes.</p><h4>Audio Pipeline</h4><p>The browser-side audio pipeline (Opus capture → WebSocket → server → Mimi → LM → Mimi → Opus → WebSocket → AudioWorklet playback) achieves acceptable latency. Key parameters that matter:</p><ul><li>Capture: Opus at 24kHz, RESTRICTED_LOWDELAY application mode, 20ms frames, 2 frames per OGG page (~40ms chunks), complexity 0</li><li>Playback: AudioWorklet (MoshiProcessor) with 80ms initial buffer, adaptive overflow/underrun handling, linear fade-in/fade-out to prevent clicks</li><li>Decoder warmup: A synthetic OGG BOS page is sent before real audio to initialize the decoder’s internal buffers, preventing dropped frames at conversation start</li></ul><h4>Modal Deployment</h4><p>The serverless GPU deployment on Modal works well for a prototype. The @modal.concurrent(max_inputs=1) constraint correctly enforces one conversation per container (necessary because the LM is stateful). Auto-scaling handles concurrent users by spawning additional containers.</p><h3>What Did Not Work</h3><h4>Instruction Following (The Core Failure)</h4><p>PersonaPlex does not reliably follow system prompt instructions. This is the fundamental blocker.</p><p>With the !!! marker: The model produces !!! inconsistently. In testing, we observed:</p><ul><li>Sometimes the model outputs !!! as instructed</li><li>Sometimes it answers the product question directly (incorrectly) without deferring</li><li>Sometimes it produces ! or !! instead of !!!</li><li>Sometimes it ignores the routing rules entirely and generates generic conversational filler</li></ul><p>This is not a prompt engineering failure that can be fixed with better wording. The Helium 7B backbone was trained for conversational dynamics using the Fisher English Corpus and synthetic customer service dialogues. It was not trained to follow structured instructions or produce specific output markers on command. The &lt;system&gt;...&lt;system&gt; prompt format (note: same tag at both ends, not XML-style closing tags) provides a weak form of instruction injection, but the model treats it as context rather than binding directives.</p><p>With natural phrase detection: Shifting to phrases like “let me check” was more reliable in the sense that PersonaPlex naturally produces filler phrases. However, the model does not produce them <em>selectively</em>. It says “let me check” for questions it could handle, and fails to say it for questions it cannot. The phrase is a conversational habit, not a routing decision. We also observed false positives where the model says “let me see” as a thinking pause during normal conversation, not as a defer signal.</p><h4>The Controllability Problem</h4><p>This is a deeper issue than just our use case. The Moshi/PersonaPlex architecture generates text tokens via the “Inner Monologue” mechanism, where text is a <em>prefix</em> to audio generation. The text tokens are sampled with temp=0.7, top_k=25 -- relatively unconstrained. The model was trained to maximize conversational naturalness across its training distribution, not to follow output-format constraints.</p><p>Contrast this with instruction-tuned text LLMs, which are explicitly trained (via RLHF, DPO, or SFT on instruction-following data) to produce structured outputs on demand. PersonaPlex has no equivalent training signal. Asking it to reliably produce !!! is like asking a pre-instruction-tuning GPT-2 to reliably produce JSON -- it might do it sometimes by chance, but there is no training pressure ensuring it.</p><h4>One-Sided Conversation History</h4><p>Because the server only has access to the agent’s text stream (the Inner Monologue), it cannot reliably extract the user’s actual question. The user’s speech goes through Mimi encoding and directly influences the LM’s generation, but the user’s words are never transcribed server-side.</p><p>This means extract_last_user_query() often returns nothing useful. The workaround -- passing the agent&#39;s filler text to DeepSeek -- works for first questions (&quot;What&#39;s your pricing?&quot; → agent says &quot;Great question about pricing!&quot; → we extract &quot;pricing&quot;) but fails for follow-ups (&quot;Tell me more&quot; → agent says &quot;Sure!&quot; → we have no idea what &quot;more&quot; refers to).</p><h4>Latency Budget</h4><p>Even when defer detection works, the routing pipeline adds significant latency:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YZ97EAMiawPj8r8OFlWVgQ.png" /></figure><p>During this time, PersonaPlex’s native generation continues — it keeps talking. This creates a collision: the model generates its own (often incorrect) response while we are waiting for DeepSeek’s (correct) response. We attempted to suppress native generation during routing, but this breaks the full-duplex loop (the model needs continuous input to maintain state).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YkP6l8QjWIzTXFz46Vy1eg.png" /></figure><h3>Prompt Conflict</h3><p>The architecture resulted in two competing system prompts:</p><ol><li>The client-side prompt (sent via URL query parameter text_prompt): uses natural phrases (&quot;Let me check that for you&quot;)</li><li>The server-side prompt (sent via METADATA message 0x04): uses !!! markers</li></ol><p>Both are injected into PersonaPlex on connection. They contradict each other, and neither is reliably followed. This is partly an engineering oversight (the migration from markers to phrases was incomplete), but it also reveals the fragility of prompt-based control for this model class.</p><h3>Discussion</h3><h4>Why Runtime Routing Is Hard for Speech Models</h4><p>The core difficulty is that full-duplex speech models are always generating. Unlike a text LLM that waits for a complete user message before responding, PersonaPlex produces audio tokens continuously at 12.5 Hz. There is no natural “decision point” where the model can pause and route.</p><p>This means any routing mechanism must either:</p><ol><li>Detect intent post-hoc (after the model has already started responding) → which is what we attempted, leading to the collision problem described above</li><li>Pre-classify user input before it reaches the model<br>→ which would require a separate ASR + intent classifier running in parallel, adding latency and complexity</li><li>Train the model to emit routing tokens as part of its native generation<br>→ which requires fine-tuning on routing-annotated conversation data</li></ol><p>Option 3 is the most promising but was outside the scope of this prototype.</p><h4>The Controllability-Naturalness Tradeoff</h4><p>There appears to be a fundamental tension in the Moshi architecture between conversational naturalness and controllability. The model’s strength — fluid, human-like conversation with natural interruptions and backchannels — comes from training on real and synthetic conversations without explicit structure. Adding structured behavior (like reliable marker production) would likely require either:</p><ul><li>Fine-tuning on marker-annotated data: Training conversations where the agent correctly uses !!! in context, so the model learns this as a natural conversational pattern rather than an arbitrary instruction</li><li>Constrained decoding: Modifying the text sampling to force specific tokens under certain conditions, similar to grammar-constrained generation in text LLMs. This is technically feasible (we already force tokens during TTS injection) but requires a reliable trigger condition</li></ul><h4>What About a Parallel ASR Pipeline?</h4><p>One approach we considered but did not implement: running Whisper or a similar ASR model on the user’s audio stream in parallel with PersonaPlex. This would give us:</p><ul><li>Actual user transcripts (solving the one-sided history problem)</li><li>Intent classification on user queries (solving the trigger reliability problem)</li><li>The ability to pre-route before PersonaPlex responds (solving the collision problem)</li></ul><p>The cost is additional GPU memory and ~300ms latency for ASR. Given that PersonaPlex already uses ~20GB of the A10G’s 24GB VRAM, this would likely require a larger GPU (A100) or a separate ASR service. This remains a viable direction.</p><h3>Open Directions</h3><h4>Fine-Tuning for Routing Awareness</h4><p>The most direct solution to the controllability problem is to fine-tune PersonaPlex on conversations that include routing behavior. The training data would include:</p><ul><li>Conversations where the agent correctly defers product questions with a specific token or phrase</li><li>Conversations where the agent correctly handles simple queries directly</li><li>The deferred responses (from DeepSeek) injected as ground truth continuations</li></ul><p>This is a LoRA or full fine-tune on the Helium 7B backbone, estimated at 1–2 GPU-days on an A100. The PersonaPlex team has demonstrated that fine-tuning on synthetic conversations (customer service roles) works; routing is a natural extension.</p><p>However, fine-tuning addresses the <em>detection</em> problem but not the <em>timing</em> problem. Even a model that perfectly recognizes “this is a question I should defer” still faces the fundamental challenge that full-duplex generation is continuous. By the time the model emits a defer signal, it has already begun producing audio for a response. The model has no native concept of “pause and wait for external input” because the architecture was designed for uninterrupted streaming.</p><p>There is also a risk of capability degradation. PersonaPlex was fine-tuned from Moshi on synthetic conversations optimized for natural conversational dynamics. Adding routing behavior means training the model to do something orthogonal to its original objective. The DiVA work (Held et al., 2024) documents exactly this kind of capability forgetting during speech model fine-tuning, where new behaviors erode existing strengths.</p><p>The more interesting direction may be architectural rather than training-based: a lightweight routing head that runs ahead of audio generation (see <em>Embedding-Space Routing below</em>), or a learned “hold” behavior where the model produces natural filler (“let me look into that”, backchannels) while an external system prepares the response. This would treat routing not as a discrete decision point but as a conversational behavior the model can sustain over multiple frames, which fits the full-duplex paradigm better than a binary defer/don’t-defer signal.</p><h4>Embedding-Space Routing</h4><p>Instead of detecting routing signals in the text token stream, we could examine the transformer’s hidden states directly. The main transformer produces 4096-dimensional embeddings at each step. A lightweight classifier (linear probe or small MLP) trained on these embeddings could predict whether the current context requires external knowledge, potentially with higher accuracy than text-token pattern matching.</p><p>This has the advantage of making routing decisions at the representation level rather than the output level, and could trigger before the model commits to a response direction.</p><h4>Dual-Model Streaming</h4><p>A more ambitious architecture: run PersonaPlex and a text LLM simultaneously. PersonaPlex handles the voice stream. A parallel ASR + text LLM pipeline processes the same user audio and generates text responses. A mixer decides, on a per-turn basis, whether to use PersonaPlex’s native response or inject the text LLM’s response via TTS.</p><p>This is essentially the FunAudioLLM approach (An et al., 2024) but with PersonaPlex providing the full-duplex voice layer instead of a separate TTS model. The key challenge is the mixer’s decision logic and the seamless blending of native and injected audio.</p><h4>Constrained Decoding with Audio Awareness</h4><p>A technically interesting direction: modify the text token sampling in LMGen.step() to bias toward routing tokens when the audio input patterns suggest a question. The Moshi architecture already cross-attends between text and audio streams. It may be possible to extract a &quot;question probability&quot; from the attention patterns and use it to bias text sampling toward defer tokens, without any fine-tuning.</p><h4>Retrieval-Augmented Generation for Speech</h4><p>RAG for text LLMs is well-established. Adapting it for speech models would involve:</p><ol><li>Maintaining a vector store of product knowledge</li><li>Using the Inner Monologue text stream as a query</li><li>Retrieving relevant context and injecting it into the model’s input</li></ol><p>The challenge is that Moshi’s context window is 3,000 tokens and the input channels are highly structured (interleaved codebooks with specific delay patterns). Injecting retrieved context would require careful integration with the existing prompt injection mechanism.</p><h3>Reproducing This Work</h3><h4>Prerequisites</h4><ul><li>NVIDIA GPU with 24GB+ VRAM (A10G, RTX 4090, or better)</li><li>Python 3.10+, libopus-dev</li><li>HuggingFace account with access to nvidia/personaplex-7b-v1</li></ul><h4>Server Setup</h4><pre>pip install moshi/.<br>export HF_TOKEN=&lt;your_token&gt;<br><br># Basic server (no routing)<br>python -m moshi.server --ssl $(mktemp -d)<br># With Modal deployment<br>pip install modal<br>modal secret create huggingface-secret HF_TOKEN=&lt;your_token&gt;<br>modal secret create deepseek-secret DEEPSEEK_API_KEY=&lt;your_key&gt;<br>modal serve modal_app.py</pre><h4>Key Files</h4><ul><li><a href="https://github.com/dosht/personaplex/blob/main/moshi/moshi/server.py">moshi/moshi/server.py</a><br>→ WebSocket server with TTS injection and defer detection</li><li><a href="https://github.com/dosht/personaplex/blob/main/moshi/moshi/models/lm.py">moshi/moshi/models/lm.py</a><br>→ LMGen class with step() method for token forcing</li><li><a href="https://github.com/dosht/personaplex/blob/main/client/src/protocol/types.ts#L51">client/src/protocol/types.ts</a><br>→ Protocol type definitions including tts_inject (0x07)</li><li><a href="https://github.com/dosht/personaplex/blob/main/client/src/protocol/encoder.ts">client/src/protocol/encoder.ts</a> <br>→ Message encoding/decodingmodal_app.pyModal serverless deployment configuration</li></ul><h4>Testing TTS Injection</h4><p>The simplest way to verify TTS injection works:</p><pre>const ws = new WebSocket(&#39;wss://your-server/api&#39;);<br>ws.binaryType = &#39;arraybuffer&#39;;<br>ws.onopen = () =&gt; {<br>  const text = &quot;Hello, this is an injected message.&quot;;<br>  const bytes = new TextEncoder().encode(text);<br>  const msg = new Uint8Array(1 + bytes.length);<br>  msg[0] = 0x07;<br>  msg.set(bytes, 1);<br>  ws.send(msg);<br>};</pre><p>You should hear PersonaPlex speak the injected text in its current voice.</p><h3>Conclusion</h3><p>We set out to augment a full-duplex speech model with external LLM knowledge through runtime routing. The TTS injection mechanism works: we can force text through PersonaPlex’s generation pipeline and produce natural-sounding speech. The routing mechanism does not work reliably: the model cannot be controlled via prompts to consistently signal when it needs help.</p><p>The fundamental issue is an impedance mismatch. Full-duplex speech models are trained for continuous, natural conversation. Routing requires discrete, reliable decision points. These goals conflict at the architectural level, and prompt engineering cannot bridge the gap.</p><p>Fine-tuning for routing awareness is the obvious next step, and would likely improve defer detection reliability. But it does not resolve the deeper tension: full-duplex models generate continuously, and routing requires discrete decision points. Even perfect detection leaves the timing problem unsolved. The more promising direction may be architectural: a routing head that operates ahead of audio generation, or a learned “hold” behavior where the model sustains natural filler while an external system prepares a response. This reframes routing as a conversational behavior rather than a binary decision, which fits the full-duplex paradigm better.</p><p>This work demonstrates both the potential and the current limits of hybrid speech architectures. The pieces exist (TTS injection, external LLM consultation, audio pipeline engineering), but the glue — reliable, well-timed routing — requires solutions at the architecture level that are not yet available in open-source full-duplex speech models. Until speech models are designed with external knowledge grounding as a first-class capability, analogous to how text LLMs learned function calling, runtime augmentation will remain fragile.</p><h3>References and Code</h3><ul><li>Defossez, A. et al. (2024). “Moshi: a speech-text foundation model for real-time dialogue.” arXiv:2410.00037</li><li>Roy, R. et al. (2026). “PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Models.” ICASSP 2026.</li><li>Geng, Y. et al. (2025). “Control Illusion: The Failure of Instruction Hierarchies in Large Language Models.” AAAI-26. arXiv:2502.15851</li><li>Held, W. et al. (2024). “Distilling an End-to-End Voice Assistant Without Instruction Training Data.” arXiv:2410.02678</li><li>An, K. et al. (2024). “FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs.” arXiv:2407.04051</li></ul><p><strong>Code is here: </strong><a href="https://github.com/dosht/personaplex/tree/main"><strong>https://github.com/dosht/personaplex</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=09abaccd1d70" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Quality Drift to Continuous Improvement: How Claude Sub‑Agents + Sharded Docs Fixed My AI Dev…]]></title>
            <link>https://medium.com/@mou.abdelhamid/from-quality-drift-to-continuous-improvement-how-claude-sub-agents-sharded-docs-fixed-my-ai-dev-8027b7cec8fc?source=rss-d670c15e6673------2</link>
            <guid isPermaLink="false">https://medium.com/p/8027b7cec8fc</guid>
            <category><![CDATA[developer-experience]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[nextjs]]></category>
            <category><![CDATA[ai-agent-development]]></category>
            <dc:creator><![CDATA[Mustafa Abdelhamid]]></dc:creator>
            <pubDate>Thu, 02 Oct 2025 12:55:55 GMT</pubDate>
            <atom:updated>2025-10-02T12:55:55.734Z</atom:updated>
            <content:encoded><![CDATA[<h3>From Quality Drift to Continuous Improvement: How Claude Sub‑Agents + Sharded Docs Fixed My AI Dev Workflow</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*66jtpz96rjJZITULwd4MLg.png" /></figure><p>Why specialist agents and small, focused documentation files beat vibe coding and monolithic specs</p><p>I’ve used AI coding assistants for a while. The pattern was familiar: strong first prompts, then quality fell — duplicate logic, alternating libraries, inconsistent patterns, thin tests, and ad-hoc structure. Spec-first (e.g., Kiro) shipped multi-story features, but refactors and course changes were brittle. Mixing Kiro for greenfield, Cursor for modifications/bug fixes, and Claude for higher-quality code helped — yet stability still lagged.</p><p>The breakthrough was treating AI like a real team: use Claude sub-agents with explicit roles and give each a clear documentation home. I keep agent definitions reusable in `.claude/agents` and separate working docs by purpose:</p><ul><li>Architecture in `docs/arc42/*` (short, link-heavy, no inline code)</li><li>Developer standards in `docs/developer-guide/*` with examples in `docs/developer-guide/templates/*`</li><li>Product epics/stories in `docs/product/*`</li></ul><h3>Agents Overview</h3><ul><li>architect: owns system design/decisions, updates arc42</li><li>tech-lead: defines standards, performs reviews, maintains developer guide</li><li>developer: implements stories using established patterns and test</li><li>react-ui-designer: builds Next.js/shadcn UI, validates visually on demo/showcase</li><li>manual-tester: plans and performs manual validation, logs findings</li><li>product-owner: converts PRDs/MVPs to epics/stories with EARS criteria</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/714/1*z77WqwxQVgjIzBWjeTJVtA.png" /></figure><h3>Consistent service pattern (generated code)</h3><p>Now here’s the magic — this is what the generated code actually looks like. Notice the consistent patterns:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-vAMdcQ3i7oXsH7ovWdt1A.png" /></figure><h3>ViewModel + schema validation harmony</h3><p>And here’s another example showing how the agents maintain consistency across different patterns — this ViewModel follows the same validation approach:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SVsyk48iwOWljwdQusDwOQ.png" /></figure><h3>Documentation Principles That Made It Work</h3><ul><li>Keep arc42 and developer-guide short and reference code files; place runnable examples in `templates/`</li><li>Shard big markdowns; small files are easier for agents to navigate and remember</li><li>Separate agent definitions from working docs so roles are reusable across projects</li></ul><h3>Results</h3><ul><li>Quality rises over time (not the reverse): agents reuse patterns, imports, and structure</li><li>Changes/refactors are cheaper: decisions and standards are discoverable</li><li>I still review and fix issues, but effort drops as agents align with the repo</li></ul><p>I’ll publish a follow-up deep dive on the exact documentation method for each agent, plus the specific design patterns I followed. If your AI-assisted projects start strong then wobble, try specialist agents + living docs.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8027b7cec8fc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Comparison Between 2 Policy Engines for Kubernetes Kyverno Vs. OPA Gatekeeper]]></title>
            <link>https://medium.com/@mou.abdelhamid/a-comparison-between-2-policy-engines-for-kubernetes-kyverno-vs-opa-gatekeeper-75656241db1a?source=rss-d670c15e6673------2</link>
            <guid isPermaLink="false">https://medium.com/p/75656241db1a</guid>
            <category><![CDATA[cloud-native]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Mustafa Abdelhamid]]></dc:creator>
            <pubDate>Thu, 22 Aug 2024 13:26:32 GMT</pubDate>
            <atom:updated>2024-08-22T13:26:32.400Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dkA_CXDtM_gnGQat" /></figure><p>Kyverno and OPA Gatekeeper are both policy engines used to enforce security and compliance policies in Kubernetes clusters. While they share some similarities, they differ in terms of design philosophy, ease of use, and features. Here’s a comparison of Kyverno and OPA Gatekeeper:</p><h3>1. Overview</h3><p><strong>Kyverno:</strong></p><ul><li><strong>Description:</strong> A Kubernetes-native policy engine designed specifically for Kubernetes, Kyverno allows you to create, manage, and enforce policies using Kubernetes custom resources. It is built to be more Kubernetes-centric, focusing on simplicity and ease of use for Kubernetes administrators.</li><li><strong>Philosophy:</strong> Kubernetes-native, designed to work seamlessly within the Kubernetes ecosystem using YAML and Kubernetes resources.</li></ul><p><strong>OPA Gatekeeper:</strong></p><ul><li><strong>Description:</strong> OPA Gatekeeper is a policy engine built on top of Open Policy Agent (OPA), a more general-purpose policy engine that can enforce policies across various systems. Gatekeeper extends OPA’s capabilities to Kubernetes by integrating with the Kubernetes API server and allowing policy enforcement using the Rego language.</li><li><strong>Philosophy:</strong> General-purpose policy engine with Kubernetes integration, offering more flexibility and power for complex policy scenarios.</li></ul><h3>2. Ease of Use</h3><p><strong>Kyverno:</strong></p><ul><li><strong>YAML-Based Policies:</strong> Kyverno policies are written in YAML, similar to standard Kubernetes manifests. This makes it intuitive for Kubernetes users who are familiar with Kubernetes YAML.</li><li><strong>No Learning Curve:</strong> Since Kyverno uses standard Kubernetes constructs, there’s little to no additional learning curve for users who are already familiar with Kubernetes.</li><li><strong>Validation, Mutation, and Generation:</strong> Kyverno supports validation, mutation, and generation of Kubernetes resources, making it easy to create policies that can validate configurations, mutate them to match certain standards, or generate additional resources automatically.</li></ul><p><strong>OPA Gatekeeper:</strong></p><ul><li><strong>Rego Language:</strong> Policies in Gatekeeper are written in Rego, a purpose-built language for policy definitions. This provides flexibility but introduces a learning curve for users who are not familiar with Rego.</li><li><strong>Advanced Scenarios:</strong> Gatekeeper is better suited for more complex policy scenarios where detailed logic is required. The Rego language allows for more granular control and complex conditions in policies.</li><li><strong>Policy Templates:</strong> Gatekeeper uses “ConstraintTemplates” to define reusable policy templates, which adds complexity but also power and flexibility.</li></ul><h3>3. Policy Management</h3><p><strong>Kyverno:</strong></p><ul><li><strong>Policy as Kubernetes CRDs:</strong> Kyverno policies are defined as Kubernetes Custom Resource Definitions (CRDs), making them easy to manage, version, and audit using standard Kubernetes tooling.</li><li><strong>Built-In Policy Types:</strong> Kyverno has built-in support for three main types of policies — validation, mutation, and generation — which simplifies policy management.</li><li><strong>Namespace-Aware:</strong> Kyverno is namespace-aware, allowing for policies to be scoped to specific namespaces easily.</li></ul><p><strong>OPA Gatekeeper:</strong></p><ul><li><strong>Custom Policies with Rego:</strong> Gatekeeper allows you to write custom policies using Rego, which can be applied cluster-wide or scoped to namespaces.</li><li><strong>Constraint Templates and Constraints:</strong> Policies are defined in two parts: ConstraintTemplates and Constraints, allowing for reusable and flexible policy definitions.</li><li><strong>More Granular Control:</strong> Gatekeeper provides more granular control over policy enforcement with its ability to define complex logical conditions.</li></ul><h3>4. Community and Ecosystem</h3><p><strong>Kyverno:</strong></p><ul><li><strong>Kubernetes-Focused Community:</strong> Kyverno has a growing community, especially among Kubernetes users who prefer a more Kubernetes-native approach.</li><li><strong>Kubernetes SIG Approval:</strong> Kyverno has been accepted as a CNCF (Cloud Native Computing Foundation) project, indicating strong community support and a commitment to open-source development.</li></ul><p><strong>OPA Gatekeeper:</strong></p><ul><li><strong>Wider Ecosystem:</strong> OPA is part of a broader policy management ecosystem, with use cases beyond Kubernetes, such as API gateways, microservices, and more.</li><li><strong>CNCF Project:</strong> OPA is also a CNCF project, with a larger community and a wide range of integrations beyond Kubernetes.</li></ul><h3>5. Use Cases</h3><p><strong>Kyverno:</strong></p><ul><li><strong>Best for:</strong> Users who want a Kubernetes-native solution that is easy to implement, manage, and understand, particularly for enforcing simple to moderately complex policies.</li><li><strong>Examples:</strong> Automatically labeling resources, enforcing resource limits, generating default configurations, and validating Kubernetes best practices.</li></ul><p><strong>OPA Gatekeeper:</strong></p><ul><li><strong>Best for:</strong> Users who need more flexibility and power in their policies, particularly in complex scenarios where detailed logic and conditions are required.</li><li><strong>Examples:</strong> Implementing complex compliance rules, fine-grained RBAC, and advanced security policies that require intricate logic.</li></ul><h3>6. Performance and Scalability</h3><p><strong>Kyverno:</strong></p><ul><li><strong>Performance:</strong> Kyverno is lightweight and designed to work efficiently within Kubernetes, making it a good choice for clusters where simplicity and performance are key.</li><li><strong>Scalability:</strong> Scales well in most Kubernetes environments, especially where policy complexity is moderate.</li></ul><p><strong>OPA Gatekeeper:</strong></p><ul><li><strong>Performance:</strong> OPA Gatekeeper can be resource-intensive, particularly when dealing with complex Rego policies, but it’s also highly optimized for these scenarios.</li><li><strong>Scalability:</strong> Scales well in large and complex environments, though performance tuning may be required for very large clusters with many complex policies.</li></ul><h3>Conclusion</h3><ul><li><strong>Choose Kyverno</strong> if you prefer a Kubernetes-native solution with minimal learning curve, easy management, and policies that are intuitive to write and understand. It is ideal for users looking for simplicity and Kubernetes-specific functionality.</li><li><strong>Choose OPA Gatekeeper</strong> if you need more flexibility and power in your policy definitions, are comfortable with or willing to learn Rego, and have complex policy requirements that extend beyond Kubernetes or require detailed logic.</li></ul><p><em>Originally published at </em><a href="https://www.linkedin.com/pulse/comparison-between-2-policy-engines-kubernetes-vs-opa-abdelhamid-pscoc/?trackingId=DsfdegGzTYCsWvvMLpSoUQ%3D%3D"><em>https://www.linkedin.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75656241db1a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploy Nginx Ingress and External-DNS on AKS with Workload Identity]]></title>
            <link>https://medium.com/swlh/deploy-nginx-ingress-and-external-dns-on-aks-with-workload-identity-902c05bd2a2c?source=rss-d670c15e6673------2</link>
            <guid isPermaLink="false">https://medium.com/p/902c05bd2a2c</guid>
            <category><![CDATA[dns]]></category>
            <category><![CDATA[nginx]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[azure]]></category>
            <dc:creator><![CDATA[Mustafa Abdelhamid]]></dc:creator>
            <pubDate>Sat, 08 Jun 2024 06:06:03 GMT</pubDate>
            <atom:updated>2024-06-08T18:49:16.082Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Deploy Nginx ingress and external-dns on AKS with workload identity" src="https://cdn-images-1.medium.com/max/1024/1*i3Iq04W0UsjP5PQFhs2nhA.jpeg" /></figure><p>Do you have a Kubernetes cluster and feel confused about enabling developers to expose their endpoints securely with minimal effort? Perhaps you find yourself stuck with the proprietary Azure Ingress, or you’re already using Nginx with External DNS, yet still need to manage access keys manually, and struggle to find documentation offering a comprehensive solution.</p><p>In this tutorial, I will show a fancy way to expose HTTP(S) endpoints of workloads deployed on Azure Kubernetes Service (AKS) cluster. This will enables your developers to deploy public endpoints with the minimum effort and will think it’s magic. we will walk through a step-by-step process to deploy Nginx ingress controller, external-dns and cert-manager on Azure Kubernetes Service (AKS) with workload identity — with no manual management of secrets or keys is needed.</p><p>I devided this tutorial into 2 parts becuase I realised it will be too long. The first part will discuss only Ingress and DNS and the second part will discuss HTTPS and TLS certificates.</p><p>Before we start, let’s familiarize ourselves with the key components we will be deploying…</p><h3>Nginx Ingress Controller</h3><p>The Nginx Ingress Controller is a Kubernetes resource that manages external access to services in a cluster. It provides an HTTP and HTTPS routing to services based on the domain name and path in the incoming request.</p><h3>External DNS</h3><p>External DNS is a Kubernetes addon that configures public DNS servers with information about exposed Kubernetes services to make them discoverable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YvJ1PBMa-VIKLLCauOErJw.png" /></figure><h3>How does it work?</h3><p>You deploy an app that consists of a service and deployment or any other workload, then define an ingress service that maps a sub domain name to that service. The external DNS then will find the sub domain and creates an A record in the DNS zone. The external DNS will use a service account and an azure managed identity to have access to modify the DNS zone in Azure resource group.</p><h3>Steps</h3><h3>0. Prepare our Application with NO Ingress</h3><p>We will start with a Hello-World HTTP application that consists of a Kubernetes deployment and service to use it an example for achieving our objective.</p><p>Create a file called <strong>application.yaml</strong></p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  annotations:<br>  name: hello-world<br>  namespace: default<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: hello-world<br>  template:<br>    metadata:<br>      labels:<br>        app: hello-world<br>    spec:<br>      containers:<br>      - image: crccheck/hello-world<br>        imagePullPolicy: Always<br>        name: hello-world<br>        ports:<br>        - containerPort: 8000<br>          name: http<br>          protocol: TCP<br>        resources: {}  # Define resources request and limits<br>---<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  annotations:<br>  name: hello-world<br>  namespace: default<br>spec:<br>  ports:<br>  - name: http<br>    port: 8000<br>    protocol: TCP<br>    targetPort: 8000<br>  selector:<br>    app: hello-world<br>  type: ClusterIP</pre><p>Then apply this config by calling:</p><pre>kubectl apply -f application.yaml</pre><h3>1. Create Nginx Ingress Controller with Helm</h3><p>This is the first step where we will deploy Nginx Ingress via Helm and Terraform. We need to override only one value required for Azure heltch check.</p><p>Create <strong>main.tf </strong>as the following. We will use this file to include all of our Terraform code.</p><pre>resource helm_release ingress {<br>  name       = &quot;ingress&quot;<br>  repository = &quot;https://kubernetes.github.io/ingress-nginx&quot;<br>  chart      = &quot;ingress-nginx&quot;<br>  namespace  = var.ingress_namespace<br>  version    = &quot;3.36.0&quot;<br><br>  set {<br>    name  = &quot;controller.service.annotations.service\\.beta\\.kubernetes\\.io/azure-load-balancer-health-probe-request-path&quot;<br>    value = &quot;/healthz&quot;<br>  }<br>}</pre><p><strong>Deploy a hello world deployment with ingress service:</strong></p><p>Now, we will create a simple ingress resource to expose our service, but without a DNS or HTTPs yet:</p><pre>apiVersion: networking.k8s.io/v1<br>kind: Ingress<br>metadata:<br>  name: hello-world<br>  namespace: default<br>spec:<br>  ingressClassName: nginx<br>  rules:<br>  - host: hello.transgate.ai<br>    http:<br>      paths:<br>      - backend:<br>          service:<br>            name: hello-world<br>            port:<br>              number: 8000<br>        path: /<br>        pathType: Prefix<br></pre><h3>2. Create a DNS Zone in Azure</h3><p>To configure DNS records, create a DNS zone to setup our DNS records and subdomains using the following Terraform resource:</p><pre>resource &quot;azurerm_dns_zone&quot; &quot;aks_dns_zone&quot; {<br>  name                = var.domain_name<br>  resource_group_name = var.resource_group_name<br>}</pre><p>var.domain_name will be the parent domain name that will (or subdomains under that) will point to your services deployed in Kubernetes. e.g. transgate.ai</p><h3>3. The Most Important Step: Enable the External DNS to manage the DNS zone</h3><p>The external DNS is another Helm deployment that will discover ingress definitions and create A records in the DNS zone we have just created for example api.transget.ai</p><p>But in order to allow External DNS to modify the DNS zone in Azure, we want federated identity and assign a role to it with the required permissions.</p><pre><br>resource &quot;azurerm_user_assigned_identity&quot; &quot;aks_dns_identity&quot; {<br>  name                = &quot;aks-dns-identity&quot;<br>  resource_group_name = var.resource_group_name<br>  location            = var.resource_group_location<br>}<br><br>resource &quot;azurerm_federated_identity_credential&quot; &quot;default&quot; {<br>  name                = var.ingress_namespace<br>  resource_group_name = var.resource_group_name<br>  audience            = [&quot;api://AzureADTokenExchange&quot;]<br>  issuer              = azurerm_kubernetes_cluster.aks.oidc_issuer_url<br>  parent_id           = azurerm_user_assigned_identity.aks_dns_identity.id<br>  subject             = &quot;system:serviceaccount:${var.ingress_namespace}:external-dns&quot;<br>}<br><br>resource &quot;azurerm_role_assignment&quot; &quot;aks_dns_role_assignment&quot; {<br>  scope                = azurerm_dns_zone.aks_dns_zone.id<br>  role_definition_name = &quot;Contributor&quot;<br>  principal_id         = azurerm_user_assigned_identity.aks_dns_identity.principal_id<br>}<br><br>resource &quot;azurerm_role_assignment&quot; &quot;aks_rg_reader_role_assignment&quot; {<br>  scope                = data.azurerm_resource_group.rg.id<br>  role_definition_name = &quot;Reader&quot;<br>  principal_id         = azurerm_user_assigned_identity.aks_dns_identity.principal_id<br>}</pre><p>And this is the missing part that I couldn’t find easily in any documentation (by the time I wrote this post).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KMrBWPNfeey_le5O" /><figcaption>Photo by <a href="https://unsplash.com/@benwhitephotography?utm_source=medium&amp;utm_medium=referral">Ben White</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>4. Create the External DNS</h3><p>Now we can create the External DNS configured with the managed identity that we created in the previous step.</p><p>To deploy External DNS Helm chart, create the following terraform resource:</p><pre>resource helm_release exteranal_dns {<br>  name       = &quot;external-dns&quot;<br>  repository = &quot;https://charts.bitnami.com/bitnami&quot;<br>  chart      = &quot;external-dns&quot;<br>  namespace  = var.ingress_namespace<br>  version    = &quot;5.4.6&quot;<br><br>  values = [<br>    templatefile(&quot;${path.root}/external-dns/values.yaml&quot;, {<br>      azure_subscription_id  = data.azurerm_subscription.current.subscription_id<br>      azure_tenant_id        = data.azurerm_subscription.current.tenant_id<br>      external_dns_client_id = azurerm_user_assigned_identity.aks_dns_identity.client_id<br>      azure_resource_group   = var.resource_group_name<br>    })<br>  ]<br>}</pre><p>Then create the values file that will be used by External DNS Helm chart external-dns/values.yaml</p><pre>provider: azure<br>policy: sync<br>commonLabels:<br>  azure.workload.identity/use: &quot;true&quot;<br>serviceAccount:<br>  create: true<br>  name: external-dns<br>  annotations:<br>    azure.workload.identity/client-id: ${external_dns_client_id}<br>  labels:<br>    azure.workload.identity/use: &quot;true&quot;<br>azure:<br>  subscriptionId: ${azure_subscription_id}<br>  tenantId: ${azure_tenant_id}<br>  resourceGroup: ${azure_resource_group}<br>  useWorkloadIdentityExtension: true</pre><h3>Explanation</h3><p>There are many options to deploy External DNS with Azure. The way we used here is by utilising the Azure federated identity with the role assignment without the need to create and mange keys manually. To do that we set the following option:</p><ol><li>useWorkloadIdentityExtension: true sets the authentication mode of External DNS</li><li>comonLabels.zaure.workload.identity/use will enable pods to authenticate using the federated identity credentials mounted in the pod.</li><li>serviceAccount.annotations.azure.workdload.identity/use allows the service account to use the federated identity credintials that matches subject: &quot;system:serviceaccount:${vat.ingress_namespace}:external-dns&quot; that we set in azurerm_federated_identity_credential in terraform</li><li>serviceAccount.labels.azure.workload.identity/client-id: ${external_dns_client_id} must match the client id of the azurerm_user_assigned_identity we created in terraform above.</li></ol><p>What will happen next is that External DNS will find the ingress we have created above and will create an A record in the DNS zone that will look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D6TeKJ17Bst2TCB753dFHw.png" /></figure><h3>Trouble shooting</h3><p>If that didn’t happen we can look at the logs of External DNS to see what went wrong by following these commands.</p><pre>❯ kubectl get pod -n ingress<br>NAME                                       READY   STATUS    RESTARTS   AGE<br>external-dns-76fc85795b-jrzk8              1/1     Running   0          2h<br>ingress-nginx-controller-cd7b768cc-768np   1/1     Running   0          1h<br><br>❯ kubectl logs -n ingress external-dns-76fc85795b-jrzk8<br>time=&quot;2024-01-11T13:52:05Z&quot; level=info msg=&quot;Instantiating new Kubernetes client&quot;<br>time=&quot;2024-01-11T13:52:05Z&quot; level=info msg=&quot;Using inCluster-config based on serviceaccount-token&quot;<br>time=&quot;2024-01-11T13:52:05Z&quot; level=info msg=&quot;Created Kubernetes client https://10.0.0.1:443&quot;<br>time=&quot;2024-01-11T13:52:05Z&quot; level=info msg=&quot;Using workload identity extension to retrieve access token for Azure API.&quot;<br>time=&quot;2024-01-11T13:52:06Z&quot; level=info msg=&quot;All records are already up to date&quot;</pre><p>In the following posts, we are going to explore how to set up HTTPs with Let’sencrypt Certificate Authority. In later posts, we can look into Azure Single sing-on (SSO) and how we can use it with the hello-world that we deployed today, or any other service that doesn’t provide authentication by itself.</p><p>You can find the complete code examples in this repo: <a href="https://github.com/dosht/mlops/tree/main/ingress-nginx-external-dns">https://github.com/dosht/mlops/tree/main/ingress-nginx-external-dns</a></p><p>Enjoy coding ^_^</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=902c05bd2a2c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/deploy-nginx-ingress-and-external-dns-on-aks-with-workload-identity-902c05bd2a2c">Deploy Nginx Ingress and External-DNS on AKS with Workload Identity</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Setting Up Azure Blob Storage as Argo Workflows Artifact Repository with Federated Identity]]></title>
            <link>https://medium.com/@mou.abdelhamid/setting-up-azure-blob-storage-as-argo-workflows-artifact-repository-with-federated-identity-536994e90da8?source=rss-d670c15e6673------2</link>
            <guid isPermaLink="false">https://medium.com/p/536994e90da8</guid>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[iac]]></category>
            <dc:creator><![CDATA[Mustafa Abdelhamid]]></dc:creator>
            <pubDate>Sat, 09 Dec 2023 13:49:13 GMT</pubDate>
            <atom:updated>2023-12-09T13:49:13.416Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ShH1r3GlcRB4Riurttm5TQ.jpeg" /></figure><p><a href="https://argoproj.github.io/argo-workflows/">Argo Workflows</a> is a Kubernetes-native workflow engine, empowers you to orchestrate complex tasks with simplicity and scalability. In this tutorial, we embark on a journey to configure Argo Workflows with Azure Blob Storage as an artifact repository, but with a twist — we’ll leverage federated identity for enhanced security.</p><h3>Why Federated Identity?</h3><p>Federated identity streamlines the authentication process, making it easier to manage and enhancing the overall security of your workflows. You don’t need to manually manage access keys.</p><h3>Terraform</h3><p>In this tutorial, we won’t settle for manual configurations or unclear documentation. We’re diving deep into Infrastructure as Code (IaC) territory, using Terraform to define every aspect of our setup. You’ll witness the power of codifying your infrastructure, ensuring repeatability and eliminating the guesswork.</p><p>Are you ready to unravel the secrets of Argo Workflows, Azure Blob Storage, and federated identity, all through the lens of Terraform? Let’s get started! 🚀</p><h3>Learning Objectives</h3><ul><li>Create a federated workload identity for Argo Workflows on Azure.</li><li>Create Azure Blob Storage as the artifact repository.</li><li>Allow the federated identity to read and write to the storage container.</li><li>Deploy Argo Workflows using Helm with the new setup.</li></ul><h3>Prerequisites</h3><p>Before diving in, ensure you have:</p><ul><li>Azure subscription with the necessary permissions to create resources.</li><li>Kubernetes cluster (AKS) already set up.</li><li>Azure CLI installed and configured.</li><li>Terraform installed on your local machine.</li></ul><h3>Steps</h3><h3>1. Define Variables</h3><p>Modify the variables in the Terraform script according to your requirements. Key variables include resource_group_name, artifact_name, aks_name, resource_group_location, container_name, and namespace.</p><pre>variable &quot;resource_group_name&quot; {<br>  type        = string<br>  description = &quot;The name of the Azure Resource Group where the storage account will be created.&quot;<br>}<br><br>variable &quot;artifact_name&quot; {<br>  type        = string<br>  description = &quot;The name of the Azure Storage Account used as an artifact repository for Argo Workflows.&quot;<br>}<br><br>variable &quot;aks_name&quot; {<br>  type        = string<br>  description = &quot;The name of the AKS cluster&quot;<br>}<br><br>variable &quot;resource_group_location&quot; {<br>  type        = string<br>  description = &quot;The Azure region where the Resource Group and Storage Account will be located.&quot;<br>}<br><br>variable &quot;container_name&quot; {<br>  type        = string<br>  description = &quot;The name of the Azure Storage Container within the Storage Account.&quot;<br>}<br><br>variable &quot;namespace&quot; {<br>  type        = string<br>  default     = &quot;argo-workflow&quot; # You can change the namespace according to your needs<br>  description = &quot;The namespace used for deploying Argo Workflow&quot;<br>}</pre><h3>2. Get the AKS cluster information</h3><p>Retrieve the AKS cluster information, including the oidc_issuer_url required for attaching the workload identity to the service account that will be used by Argo Workflows&#39; pods.</p><pre>data &quot;azurerm_kubernetes_cluster&quot; &quot;aks&quot; {<br>  name                = var.aks_name<br>  resource_group_name = var.resource_group_name<br>}</pre><h3>3. Create Federated Workload Identity</h3><p>Now, let’s kick things off by creating a federated workload identity for Argo Workflows using the provided Terraform resources. This includes a user-assigned identity and federated identity credential.</p><pre>resource &quot;azurerm_user_assigned_identity&quot; &quot;argo_workflows&quot; {<br>  name                = var.namespace<br>  location            = var.resource_group_location<br>  resource_group_name = var.resource_group_name<br>}<br><br>resource &quot;azurerm_federated_identity_credential&quot; &quot;argo_workflows&quot; {<br>  name                = var.namespace<br>  resource_group_name = var.resource_group_name<br>  audience            = [&quot;api://AzureADTokenExchange&quot;]<br>  issuer              = data.azurerm_kubernetes_cluster.aks.oidc_issuer_url<br>  parent_id           = azurerm_user_assigned_identity.default.id<br>  subject             = &quot;system:serviceaccount:${var.namespace}:default&quot;<br>}</pre><h3>4. Create Blob Storage Account and Container</h3><p>Then, we will create an Azure Storage Account and a container to store Argo Workflows artifacts. Adjust configurations like account tier, replication type, and container access type according to your needs.</p><pre>resource &quot;azurerm_storage_account&quot; &quot;artifact&quot; {<br>  resource_group_name              = var.resource_group_name<br>  name                             = var.artifact_name<br>  location                         = var.resource_group_location<br>  account_tier                     = &quot;Standard&quot;<br>  account_replication_type         = &quot;RAGRS&quot;<br>  allow_nested_items_to_be_public  = false<br>  cross_tenant_replication_enabled = false<br>  blob_properties {<br>    delete_retention_policy {<br>      days = 1 # This can be useful to save extra cost and to automatically delete any personal data after processing<br>    }<br>  }<br>}<br><br>resource &quot;azurerm_storage_container&quot; &quot;artifact&quot; {<br>  name                  = var.container_name<br>  storage_account_name  = azurerm_storage_account.artifact.name<br>  container_access_type = &quot;private&quot;<br>}</pre><h3>5. Assign Role to Federated Identity</h3><p>Grant the necessary permissions for the federated identity to read and write from the Azure Storage Container. This is achieved using the azurerm_role_assignment Terraform resource.</p><pre>resource &quot;azurerm_role_assignment&quot; &quot;blob_storage_access&quot; {<br>  for_each             = azurerm_storage_container.artifact<br>  scope                = each.value.resource_manager_id<br>  principal_id         = azurerm_user_assigned_identity.argo_workflows.principal_id  # The principal ID of the managed workload identity that will access to the Azure Storage Container.<br>  role_definition_name = &quot;Storage Blob Data Contributor&quot;<br>}</pre><h3>6. Deploy Argo Workflows</h3><p>Configure the Helm provider in Terraform, specifying the Kubernetes configuration. Update the helm_release resource to deploy Argo Workflows with the desired configurations.</p><pre>provider &quot;helm&quot; {<br>  kubernetes {<br>    config_path = &quot;~/.kube/config&quot; # Adjust the path to your kubeconfig file<br><br>    # Alternatively use the config of your AKS cluster like the following:<br>    # host                   = data.azurerm_kubernetes_cluster.aks.kube_admin_config.0.host<br>    # client_certificate     = base64decode(data.azurerm_kubernetes_cluster.aks.kube_admin_config.0.client_certificate)<br>    # client_key             = base64decode(data.azurerm_kubernetes_cluster.aks.kube_admin_config.0.client_key)<br>    # cluster_ca_certificate = base64decode(data.azurerm_kubernetes_cluster.aks.kube_admin_config.0.cluster_ca_certificate)<br>  }<br>}<br><br>resource &quot;helm_release&quot; &quot;argo_workflows&quot; {<br>  name       = &quot;argo-workflows&quot;<br>  repository = &quot;https://argoproj.github.io/argo-helm&quot;<br>  chart      = &quot;argo-workflows&quot;<br>  namespace  = var.namespace<br>}</pre><h3>7. Configure Argo Workflows</h3><p>Before deployment, we should configure Argo Workflows to use the blob storage via federated workload identity. Ensure useSDKCreds: true in the artifact repository configuration and add a label to each pod with azure.workload.identity/use: &quot;true&quot; which is defined in workflowDefaults. So the final helm deployment should look like this:</p><pre>resource &quot;helm_release&quot; &quot;argo_workflows&quot; {<br>  name       = &quot;argo-workflows&quot;<br>  repository = &quot;https://argoproj.github.io/argo-helm&quot;<br>  chart      = &quot;argo-workflows&quot;<br>  namespace  = var.namespace<br>  values     = &lt;&lt;-EOT<br>    artifactRepository:<br>      azure:<br>        endpoint:${azurerm_storage_account.primary_blob_endpoint}<br>        container: ${var.container_name}<br>        blobNameFormat: argo_workflow_artifact<br>        useSDKCreds: true<br>    workflowDefaults:<br>      spec:<br>        podMetadata:<br>          labels:<br>            azure.workload.identity/use: &quot;true&quot;<br>  EOT<br>}</pre><h3>8. Lastly, Configure Argo Workflow Default Service Account</h3><p>Configure the default service account for Argo Workflows with labels and annotations to enable federated workload identity.</p><pre>provider &quot;kubernetes&quot; {<br>  # Specify your Kubernetes cluster configuration, or similar to Helm config above.<br>  config_path = &quot;~/.kube/config&quot;<br>}<br><br>resource &quot;kubernetes_service_account&quot; &quot;default&quot; {<br>  metadata {<br>    name = &quot;default&quot;<br>    namespace = var.namespace<br>    labels = {<br>      &quot;azure.workload.identity/use&quot; = &quot;true&quot;<br>    }<br>    annotations = {<br>      &quot;azure.workload.identity/client-id&quot; = &quot;${azurerm_user_assigned_identity.argo_workflows.client_id}&quot;<br>    }<br>  }<br>}</pre><p>Adjust any additional configurations in the Helm chart values section as needed.</p><p>By following these steps, you will have Argo Workflows configured to utilize Azure Blob Storage with federated identity for artifact storage.</p><h3>Summary</h3><p>Congratulations! You’ve successfully configured Argo Workflows to use Azure Blob Storage as an artifact repository with federated identity. Say goodbye to access keys and hello to a more secure workflow.</p><h3>Next Steps</h3><p>Explore your newly configured Argo Workflows setup. Run some workflows that generate or consume artifacts like this <a href="https://argoproj.github.io/argo-workflows/walk-through/artifacts/">example</a>, check the logs, and revel in the simplicity and security of your new artifact management system. Feel free to ask any question and share this tutorial with fellow developers… Happy coding ^_^</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=536994e90da8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using GPT-3 for Search and Recommendations of Text Content]]></title>
            <link>https://medium.com/better-programming/using-gpt-3-for-search-and-recommendations-7987d8908331?source=rss-d670c15e6673------2</link>
            <guid isPermaLink="false">https://medium.com/p/7987d8908331</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[gpt-3]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Mustafa Abdelhamid]]></dc:creator>
            <pubDate>Wed, 18 Jan 2023 16:36:01 GMT</pubDate>
            <atom:updated>2023-03-20T10:21:50.267Z</atom:updated>
            <content:encoded><![CDATA[<h4>Utilize cosine similarity to find similar documents.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zvcIbpMhvCAUNEPX9b3xFg.png" /></figure><p>In this blog post, we will be discussing how to use GPT-3 vectors for a recommendation system that utilizes cosine similarity to find similar documents.</p><p>GPT-3, developed by OpenAI, is a state-of-the-art language model that has been trained on a massive amount of text data.</p><p>One of the key features of GPT-3 is its ability to generate high-quality text, but it also has the capability to generate vector representations of the input text, which can be used for a variety of natural language processing tasks, such as document similarity analysis.</p><h3>Step 1: Generate GPT-3 Vectors</h3><p>The first step in using GPT-3 vectors for a recommendation system is to generate the vectors for your set of documents. This can be done using the OpenAI GPT-3 API, which allows you to send a block of text and receive a vector representation in return.</p><p>To generate the vectors for your set of documents, you will need to send each document to the API and store the returned vectors in an array. Here is an example of how to generate GPT-3 vectors for a set of documents in Python:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8b0a426af3ec21955feafa72b4a14820/href">https://medium.com/media/8b0a426af3ec21955feafa72b4a14820/href</a></iframe><h3>Step 2: Calculate Cosine Similarity</h3><p>Once you have the vectors for your set of documents, you can use the cosine similarity metric to find the similarity between them. Cosine similarity is a measure of similarity between two non-zero vectors of an inner product space that measures the cosine of the angle between them. To calculate the cosine similarity between two vectors, you can use the following formula:</p><p>cosine_similarity = (A * B) / (||A|| * ||B||)</p><p>Where A and B are the vectors for two documents and ||A|| and ||B|| are the magnitudes of the vectors. Here is an example of how to calculate the cosine similarity between all pairs of documents in Python:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fb40ab7653f26b97740c4435413a9df6/href">https://medium.com/media/fb40ab7653f26b97740c4435413a9df6/href</a></iframe><h3><strong>Step 3: Find the most similar documents</strong></h3><p>Now that you have the cosine similarity scores for all the pairs of documents, you can find the most similar documents by sorting the scores in descending order and selecting the top N results.</p><p>You can use the resulting list of similar documents to recommend similar content to your users. Here is an example of how to find similar documents using cosine similarity in Python:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/209353ac5db2d53f8d01441f3d56874a/href">https://medium.com/media/209353ac5db2d53f8d01441f3d56874a/href</a></iframe><p>In this example, docs is a list of strings representing the set of documents. vectors is a list of the GPT-3 vectors generated for each document using the OpenAI GPT-3 API.</p><p>The nested loops iterate over all pairs of documents and calculate the cosine similarity between them. The documents with a cosine similarity greater than a certain threshold are added to the similar_documents list.</p><p>Finally, the similar documents are sorted based on cosine similarity in descending order and the top N similar documents are printed.</p><p>Please note that you will have to replace the #code to generate the vector representation of the document using OpenAI GPT-3 API with the actual code you use to generate the GPT-3 vectors as explained above.</p><h3>Step 4: Taking this to Production</h3><p>You can index the GPT-3 vectors in Elasticsearch or ArangoDB in a way that supports cosine similarity.</p><p>In Elasticsearch for example you can vector dense field type as described <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/dense-vector.html">here</a>. This is an exampling of how the mapping should look like:</p><pre>{<br>  &quot;mappings&quot;: {<br>    &quot;properties&quot;: {<br>      &quot;my_vector&quot;: {<br>        &quot;type&quot;: &quot;dense_vector&quot;,<br>        &quot;dims&quot;: 1024,<br>        &quot;similarity&quot;: &quot;l2_norm&quot;,<br>        &quot;index&quot;: true<br>      },<br>      &quot;document&quot; : {<br>        &quot;type&quot; : &quot;text&quot;<br>      }<br>    }<br>  }<br>}</pre><p>I didn’t evaluate this with ArangoDB, but you can follow the documentation starting <a href="https://www.arangodb.com/docs/stable/aql/functions-numeric.html#cosine_similarity">here</a> and let me know in the comments if you can get a promising result.</p><p>It’s important to note that vector similarity search is computationally expensive, so you might want to consider using a vector similarity search engine like <a href="https://github.com/facebookresearch/faiss">Faiss</a> or <a href="https://github.com/nmslib/nmslib">NMSLIB</a> instead, as they are designed to handle large vector datasets with fast similarity search.</p><p>Please note that this is a high-level overview of the steps required, and you may need to refer to the specific documentation of Elasticsearch or ArangoDB for more detailed information on how to implement each step.</p><h3>Conclusion</h3><p>In this blog post, we have discussed how to use GPT-3 vectors for a recommendation system that utilizes cosine similarity to find similar documents.</p><p>By generating GPT-3 vectors for your set of documents, calculating the cosine similarity between them, and finding the most similar documents, you can create a powerful recommendation system that can help your users discover new and relevant content.</p><p>For more information about GPT-3 embeddings, please refer to the <a href="https://beta.openai.com/docs/guides/embeddings/use-cases">OpenAI documentation</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7987d8908331" width="1" height="1" alt=""><hr><p><a href="https://medium.com/better-programming/using-gpt-3-for-search-and-recommendations-7987d8908331">Using GPT-3 for Search and Recommendations of Text Content</a> was originally published in <a href="https://betterprogramming.pub/">Better Programming</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Functional Programming Makes Working With Kafka Easier?]]></title>
            <link>https://medium.com/swlh/how-functional-programming-makes-working-with-kafka-easier-606d1156ea40?source=rss-d670c15e6673------2</link>
            <guid isPermaLink="false">https://medium.com/p/606d1156ea40</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[functional-programming]]></category>
            <category><![CDATA[kafka]]></category>
            <category><![CDATA[scala]]></category>
            <category><![CDATA[big-data]]></category>
            <dc:creator><![CDATA[Mustafa Abdelhamid]]></dc:creator>
            <pubDate>Sat, 27 Jun 2020 00:00:00 GMT</pubDate>
            <atom:updated>2023-01-17T04:34:18.010Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NyoxBNfhGWcjlTuuPQ_TxA.png" /></figure><p>When your system starts to grow and split into multiple services, you will presumably need a way to send messages between those services for processing. In a clean and nice architecture, each service should have a bounded context that defines its space of responsibility and other requirements outside of that context should be handled by other components including messages delivery itself, so the service will only emit a message (fire-and-forget) and other services that are interested in processing those messages will catch them.</p><p>A message queue (publish-subscribe-based) will be a good fit here. And in my opinion, Kafka should not be the default go-to choice when you think of a pub-sub because of its complexity and there are other solutions already that are designed to simplify the development and infrastructure work with reasonable trade-offs such as <a href="https://cloud.google.com/pubsub">GCP PubSub,</a> <a href="https://aws.amazon.com/kinesis/">AWS Kinesis</a>, or <a href="https://nats.io/">NATS</a>, however, if none of them met your requirements and you want to deploy <a href="https://kafka.apache.org/">Kafka</a>, then there are a few things you need to take care of.</p><p>In this blog post, We will build a simple and powerful <a href="https://www.scala-lang.org/">Scala</a> client in a functional style that solves some of hidden Kafka client challenges and potentially simplifies testing, error handling, and other aspects as you will see later in the companion project.</p><h3>What is Kafka</h3><blockquote>Kafka — is a publish-subscribe based durable messaging system exchanging data between processes, applications, and servers. <a href="https://www.cloudkarafka.com/blog/2016-11-30-part1-kafka-for-beginners-what-is-apache-kafka.html">*</a></blockquote><p>Kafka consists of two sides: A producer that produces messages to a topic and a consumer that subscribes to a topic and consumes messages from that topic. Multiple consumers can subscribe to the same topic and poll all messages from it as long as they belong to different consumer groups. Each consumer group has an offset that points at the position in the topic where it consumed until.</p><h3>Kafka Clients that Works with Scala</h3><p>This is not meant to be a full comparison, so it’s enough to list some available clients with a couple of words about each of them.</p><ol><li>Apache Kafka API: This is the <a href="https://kafka.apache.org/documentation/#api">official Java client</a>. It has a Java API and I couldn’t find a Scala wrapper for it to make functional or thread-safe, ..etc</li><li><a href="https://github.com/cakesolutions/scala-kafka-client/">CakeSolutions client</a>: It’s a well-written Akka-based client. I used also this client myself since I was working in <a href="https://twitter.com/cakesolutions?lang=en">Cake Solutions</a>. My former colleague David Piggott has a nice <a href="https://medium.com/@dhpiggott/getting-started-with-kafka-using-scala-kafka-client-and-akka-f060624377a">blog post</a> about it, but that was before the company has been acquired by Disney and I’m not sure how much they’re still maintaining it.</li><li><a href="https://doc.akka.io/docs/alpakka-kafka/current/">Alpakka project</a>: This client is based on Akka stream. I used this one also in a previous project and fits well if you are using Akka streams in your system. It’s helpful to design complex streaming graphs involving many Kafka topics and it helps also in managing offsets and applying back-pressure.</li><li><a href="https://www.lagomframework.com/documentation/1.6.x/scala/KafkaClient.html">Lagom Kafka Client</a>: This is based on Alpakka client, but it’s designed to work with Lagom</li></ol><h3>Apache Kafka Client</h3><p>Let’s first start with the vanilla Java-based official client and see how we will enhance it with <a href="https://typelevel.org/cats/">Cats</a> and <a href="https://typelevel.org/cats-effect/">Cats Effect</a> to make it more compatible with Scala and functional programming.</p><p>The simple imperative code looks like this: (<a href="https://dzone.com/articles/hands-on-apache-kafka-with-scala"><em>source</em></a><em>)</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fcf74cbfa2da94d305d084cc62b959cf/href">https://medium.com/media/fcf74cbfa2da94d305d084cc62b959cf/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/238/0*UNSoCwAywf5BfFFy.png" /><figcaption>THIS IS TOO JAVA!</figcaption></figure><h3>Introducing Cats</h3><p><a href="https://typelevel.org/cats/">Cat</a>s — is a library that provides type classes and data structures for functional programming in Scala like Monad, Semigroup, Monoid, Applicative,...etc</p><p><a href="https://typelevel.org/cats-effect/">Cats Effect</a> — is another library based on cats that mainly provides the IO monad which describes an operation that performs a side effect and the result of the effect can be returned synchronously or asynchronously.</p><p>With <a href="https://typelevel.org/cats/">Cats</a> and <a href="https://typelevel.org/cats-effect/">Cats Effect</a>, you can define all the logic of your program without running anything (including operations that have side effects) and separately run the program (or part of it) synchronously or asynchronously. This will help us to improve testability and maintainability and help understand potential sources of errors.</p><h3>Apache Kafka Client is not Thread-Safe</h3><p>As you saw above, the consumer is wrapped in <strong>while(true)</strong> block which runs in a single thread. You need to handle messages in multiple threads for better utilization, but if you try to call <strong>kafkaConsumer.poll</strong> in a different thread other than the one where it was defined, you will get an exception! To solve this, we will keep calling <strong>kafkaConsumer.poll</strong> in the same thread and we will use another thread pool for handling messages. Looks complicated, isn’t it? Let&#39;s see how this is easy with cats.</p><h3>ContextShift</h3><p><a href="https://typelevel.org/cats/"><em>Cats</em></a> provides a class called <a href="https://typelevel.org/cats-effect/datatypes/contextshift.html"><strong>ContextShift</strong></a> as the <a href="https://en.wikipedia.org/wiki/Pure_function">pure</a> wrapper of Scala&#39;s <strong>ExecutionContext</strong>. It allows you to switch thread-pools while performing <strong>flatMap</strong> operations, so you can do something like this:</p><pre><strong>val</strong> ec<strong>:</strong> <strong>ExecutionContextExecutor</strong> <strong>=</strong> ExecutionContext<strong>.</strong>global<br><strong>val</strong> cs<strong>:</strong> <strong>ContextShift[IO]</strong> <strong>=</strong> IO<strong>.</strong>contextShift<strong>(</strong>ec<strong>)</strong><br><br><strong>for</strong> <strong>{</strong><br>    <strong>_</strong> <strong>&lt;-</strong> cs<strong>.</strong>shift<br>    a <strong>&lt;-</strong> f1<br>    b <strong>&lt;-</strong> cs<strong>.</strong>evalOn<strong>(</strong>anotherExecutionContext<strong>)(</strong>f2<strong>)</strong><br>    c <strong>&lt;-</strong> f3<br><strong>}</strong> <strong>yield</strong> <strong>(</strong>a<strong>,</strong> b<strong>,</strong> c<strong>)</strong></pre><p><strong>f1</strong> will run on the <strong>global</strong> execution context and before running, <strong>cs</strong> will switch to <strong>anotherExecutionContext</strong> and run <strong>f2</strong> on it and then switch back to <strong>global</strong> execution context and run <strong>f3</strong>.</p><h3>Kafka Single Thread with Cats Effects</h3><p>Now, we will define a thread pool with a fixed number of 1 thread to initialize <strong>kafkaConsumer</strong> and call <strong>.poll</strong> and <strong>.close</strong> and we will use <strong>ContextShift</strong> to switch between this singular thread pool and the other thread pool used for handling messages which can be of any kind.</p><p><strong>KafkaContext.scala</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3754451e5f212e606dda98faac47ef7e/href">https://medium.com/media/3754451e5f212e606dda98faac47ef7e/href</a></iframe><h3>Building the Consumer</h3><p>Now, we can create the Kafka consumer in the Kafka thread that we just defined in <strong>KafkaContext</strong>. You can do this simply by creating a new instance of <strong>KafkaConsumer</strong> and wrap it in <strong>KafkaContext.execute</strong>:</p><pre>kafkaContext<strong>.</strong>execute<strong>(new</strong> <strong>KafkaConsumer(...))</strong></pre><p>Since we need to close the Kafka consumer after we don’t need it anymore, it’s more suitable to define it as a closable resource and Cats provide a resource abstraction for that called <a href="https://typelevel.org/cats-effect/datatypes/resource.html"><strong>Resource</strong></a> used for acquiring and releasing resources using <strong>Resource.make</strong> function that takes 2 functions: <strong>acquire</strong> and <strong>release</strong> and returns a <strong>Resource[F[_], A]</strong> instance where you can call <strong>use</strong> that will allow you to use the resource and it make sure to release it after you&#39;re done with it. We can do the same thing <strong>KafkaContext</strong>.</p><pre><strong>def</strong> <strong>createKafkaConsumer</strong> <strong>=</strong> kafkaContext <strong>~&gt;</strong> <strong>new</strong> <strong>KafkaConsumer(...)</strong><br><br><strong>def</strong> <strong>closeKafkaConsumer(</strong>kafkaConsumer<strong>:</strong> <strong>KafkaConsumer[...])</strong> <strong>=<br>  </strong>kafkaContext <strong>~&gt;</strong> kafkaConsumer<strong>.</strong>close<strong>()</strong><br><br><strong>val</strong> kafkaConsumerResource <strong>=<br>  </strong>Resource<strong>.</strong>make<strong>(</strong>createKafkaConsumer<strong>)(</strong>closeKafkaConsumer<strong>)</strong></pre><pre>kafkaConsumerResource<strong>.</strong>use <strong>{ </strong>kafkaConsumer <strong>=&gt;</strong><br>  kafkaContext <strong>~&gt; </strong>kafkaConsumer.subscribe<strong>(</strong>...<strong>)</strong><br><strong>}</strong></pre><p>Here we defined <strong>kafkaConsumerResource</strong> that creates and closes the consumer and use it in the same thread. Then, it will queue messages to another thread pool with probably more than one thread for processing.</p><h3>Handling messages in another thread pool</h3><p>This will not be so difficult at all since <strong>ContextShift.evalOn</strong> already switches back to the original execution context. The client code that uses <strong>kafkaConsumerResource</strong> can specify the main execution context where the messages are handled. The following code defines <strong>consume</strong> function that uses simple <em>for</em> comprehension to poll messages from the Kafka topic, process them and then poll next messages.</p><pre><strong>def</strong> <strong>consume:</strong> <strong>IO[Unit]</strong> <strong>=</strong> <strong>for</strong> <strong>{</strong><br>    messages <strong>&lt;-</strong> kafkaContext <strong>~&gt;</strong> kafkaConsumer<strong>.</strong>poll<strong>(</strong>...<strong>)</strong><br>    <strong>_</strong> <strong>&lt;-</strong> messageHandler<strong>.</strong>handleMessage<strong>(</strong>messages<strong>)</strong><br>    <strong>_</strong> <strong>&lt;-</strong> consume<br>  <strong>}</strong> <strong>yield</strong> <strong>()</strong></pre><p>Here, only <strong>kafkaConsumer.poll</strong> will run in the Kafka thread and <strong>messageHandler.handleMessage(messages)</strong> will run in another thread pool decided on the caller of consume function. This is a functional alternative to <strong>while(true)</strong> used in the <strong>vanilla</strong> example.</p><p>Note that we called <strong>consume</strong> in a non-tail-recursive position, but it&#39;s <a href="https://typelevel.org/cats-effect/datatypes/io.html#stack-safety">stack safe</a> means it will not throw <a href="https://docs.oracle.com/javase/7/docs/api/java/lang/StackOverflowError.html"><strong>StackOverFlowError</strong></a>. Remember that <strong>consume</strong> function doesn&#39;t actually run anything, it just describes the task, so it lets <strong>cats.IO</strong> decide how to execute it and, so<strong>cats.IO.flatMap</strong> can run in a <a href="https://github.com/typelevel/cats-effect/blob/v2.1.3/core/jvm/src/main/scala/cats/effect/internals/TrampolineEC.scala">trampoline context</a></p><h3>Putting them all together</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/293/0*hOI_Hz8I-WqJUeER.png" /></figure><p><strong>Consumer.scala</strong></p><p>This is the full Scala wrapper of <strong>KafkaConsumer</strong> that creates and closes the consumer, subscribes to topics, and poll messages from it.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6d1ed49c7f84dcbb2ec1385081910201/href">https://medium.com/media/6d1ed49c7f84dcbb2ec1385081910201/href</a></iframe><h3>Building the Consumer Application</h3><p><strong>ConsumerApplication.scala</strong></p><p>Finally, the application main using <a href="https://typelevel.org/cats-effect/datatypes/ioapp.html"><strong>IOApp</strong></a> provided by cats. You will override the method <strong>run</strong> instead of <strong>main</strong> and returns <strong>IO[ExitCode]</strong>, then cats IO will take care of running this IO.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cb0ef50be6f3430950ddbd5431c17d26/href">https://medium.com/media/cb0ef50be6f3430950ddbd5431c17d26/href</a></iframe><h3>Try it Yourself</h3><p>This is a good time to stop and try out the code we’ve just explained. You will build an <a href="https://www.scala-sbt.org/">SBT</a> project and add dependencies of <a href="https://mvnrepository.com/artifact/org.typelevel/cats-core_2.12/2.1.1">Cats</a>, <a href="https://mvnrepository.com/artifact/org.typelevel/cats-effect_2.12/2.1.3">Cats Effects</a>, and <a href="https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients/2.5.0">Apache Kafka</a>.</p><pre>libraryDependencies <strong>++=</strong> <strong>Seq(</strong><br>  &quot;org.typelevel&quot; <strong>%%</strong> &quot;cats-core&quot; <strong>%</strong> &quot;2.1.1&quot;<br>  &quot;org.typelevel&quot; <strong>%%</strong> &quot;cats-effect&quot; <strong>%</strong> &quot;2.1.3&quot;<br>  &quot;org.apache.kafka&quot; <strong>%</strong> &quot;kafka-clients&quot; <strong>%</strong> &quot;2.5.0&quot;<br><strong>)</strong></pre><p>Then, you need to add 3 files: <strong>KafkaContext.scala</strong>, <strong>Consumer.scala</strong> and <strong>ConsumerApplication.scala</strong> as explained above. When you finish this, you can then run <strong>sbt compile</strong> to check that everything is correct then you can download and run Kafka from <a href="https://www.apache.org/dyn/closer.cgi?path=/kafka/2.5.0/kafka_2.12-2.5.0.tgz">here</a>. After downloading is done, you can extract and run Zookeeper and Kafka, create a topic and then start the Scala consumer, then you can start sending messages from the Kafka console. You will need at least 4 terminals for this:</p><pre>tar -xvf kafka_2.13-2.5.0.tgz<br>cd kafka_2.13-2.5.0<br><br><em># First terminal: Start zookeeper</em><br>bin/zookeeper-server-start.sh config/zookeeper.properties<br><br><em># Second terminal: Start Kafka</em><br>bin/kafka-server-start.sh config/server.properties<br><br><em># Third terminal: Create a topic and then start Kafka producer console</em><br>bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test-topic<br>bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic test-topic<br><br><em># Fourth terminal: Start Scala consumer</em><br>cd &lt;your Scala code&gt;<br>sbt run<br><br><em># [Optional] 5th terminal: Start Kafka consumer console which can be useful for debugging</em><br>bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test-topic --from-beginning</pre><p>Now, go to the third and start typing some words and press enter. Each line will be a <em>text</em> message and you should see them in the Scala consumer app that runs in the fourth terminal.</p><p>If you want to see the full ready-to-run code, please check out this <a href="https://github.com/dosht/kafka-with-cats/tree/blog-post-code">branch</a>.</p><h3>Summary</h3><p>We developed a functional Kafka Scala consumer with the help of Cats and Cats Effects. We saw how we could run Kafka client in its single thread because it’s not thread-safe and we saw how it’s easy to switch thread pools so we can process messages in a suitable thread pool and we did this using ContextShift provided by cats library. Finally, we tested everything together including running Kafka and Zookeeper and sent messages from the Kafka producer console, and received them in our Scala consumer.</p><h3>Next Step</h3><p>You can explore <a href="https://github.com/dosht/kafka-with-cats/">master</a> branch of the same repo and you will find more complete code that expands the use of cats in testing, error handling, logging, and metrics. If you are interested in any of those topics, please let me know in the comments and I will write about them in future posts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/369/0*NYHqdkQEA1xpQDTA.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=606d1156ea40" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/how-functional-programming-makes-working-with-kafka-easier-606d1156ea40">How Functional Programming Makes Working With Kafka Easier?</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Productivity Tools for Kubernetes]]></title>
            <link>https://medium.com/@mou.abdelhamid/productivity-tools-for-kubernetes-4f1e8d326add?source=rss-d670c15e6673------2</link>
            <guid isPermaLink="false">https://medium.com/p/4f1e8d326add</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Mustafa Abdelhamid]]></dc:creator>
            <pubDate>Mon, 06 Apr 2020 00:00:00 GMT</pubDate>
            <atom:updated>2020-07-07T03:59:12.090Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pa1UyKDYTooUhjpwhla-tw.png" /></figure><p>If you are using <a href="https://kubernetes.io/">Kubernetes</a> and you still have to write all <a href="https://kubernetes.io/docs/reference/kubectl/overview/">kubectl</a> commands like this: kubectl --context my-ctx --namespace my-namespace ..., then you are probably wasting so much time typing repeated things and memorizing commands or maybe you have to keep too many notes. There are a lot of useful tools that can make working with kubectl much more productive. I will list in this post some of those tools that saved time for me and made me more confident while working on Kubernetes.</p><h3>Before you begin</h3><p>You need to have <a href="https://kubernetes.io/docs/reference/kubectl/overview/">kubectl</a> installed and some K8s cluster to play with. Maybe you have a test cluster already. If not, you can just make it simple and create a Minikube cluster for learning like in <a href="https://kubernetes.io/docs/setup/learning-environment/minikube/">here</a>.</p><p>This post is a bit opinionated because I chose only tools that worked well with me after trying many others. I’m also using <a href="https://github.com/ohmyzsh/ohmyzsh">oh-my-zsh</a>, so some of those tools could work different on other shells like bash or fish.</p><h3>Auto-complete</h3><p>Open ~/.zsrch and find the plugins section and add kubectl to the list. That&#39;s it nothing more. This will give the tab-completion for almost everything related to kubectl. It will auto-complete commands, switches and even resources names.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/223/0*bYOg2bATNAlVrl9i.png" /></figure><p>If you are not using oh-my-zsh, you can find how to set up autocompletion for your environment in Kubernetes docs <a href="https://kubernetes.io/docs/tasks/tools/install-kubectl/#enabling-shell-autocompletion">here</a></p><h3>ZSH PROMPT</h3><p>To avoid deleting resources on production by mistake (especially if the DevOps team trust you and give all permissions!), I like to always see the current context/namespace in my shell prompt. To do so in oh-my-zsh is very easy. Just open ~/.zshrc and add kube-ps1 to your plugins. Then add this line:</p><pre>PROMPT=&#39;$(kube_ps1) &#39;$PROMPT</pre><p>That will override PROMPT variable to include $(kube_ps1). You can put $(kube_ps1) before or after &#39;$PROMPT.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/405/0*kIHDY9a7PzL8j2Ma.png" /></figure><p>This will make you shell prompt something like that:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/581/0*3Vhv7j7Mn_4DKBAd.png" /></figure><h3>Aliases</h3><p>Defining some aliases is also a good idea and still works with autocompletion. When you use the oh-my-zsh plugin kubectl as described above, you will get already some of the useful aliases like k for kubectl and kaf for kubectl apply -f and you can find the full list in this <a href="https://github.com/ohmyzsh/ohmyzsh/tree/master/plugins/kubectl">readme</a>. You can edit those aliases by editing the file ~/.oh-my-zsh/plugins/kubectl/kubectl.plugin.zsh and you can add your aliases. Another good tool that I prefer is described in this <a href="https://ahmet.im/blog/kubectl-aliases/">post</a>. The useful thing about this tool that you can modify its [ generate_aliases.py] script to generate your aliases syntax bases on what makes sense for you.</p><h3>Plugins</h3><p>You can write your own kubectl plugin in a couple of minutes simply by starting its name with kubectl- and make it visible in your PATH and The plugin can be any executable file. If you want to list all available plugins, type kubectl plugin list which will search in you PATh for any file starting with kubectl- that means you can manage your plugins yourself by storing them in one directory, or install them using brew, apt, ..etc. There is also a plugin manager called <a href="https://github.com/kubernetes-sigs/krew/">Krew</a> where you can install/remove/update plugins from a central place.</p><p>The first plugin I recommend to start with is <a href="https://github.com/ahmetb/kubectx">kubectx</a>. It’s actually 2 plugins from the same author; One of them is for changing the context and the other is for changing the namespace. If you are also using the oh-my-zsh plugin explained above, you will instantly see the new context and namespace after changing is your terminal prompt.</p><h3>Kubefwd — for port forwarding</h3><p><a href="https://github.com/txn2/kubefwd">kubefwd</a> is a command-line utility built to port forward multiple services within one or more namespaces on one or more Kubernetes clusters. kubefwd uses the same port exposed by the service and forwards it from a loopback IP address on your local workstation. kubefwd temporally adds domain entries to your /etc/hosts file with the service names it forwards.</p><p>We used kubefwd a lot while developing our Kafka ingestion services to simulate a different kind of errors or delays to stabilise our code before deploying to production.</p><p>To install kubefwd on mac, you can use <a href="https://brew.sh/">homebrew</a></p><pre>brew install txn2/tap/kubefwd</pre><p>And to forward a service e.g. Kafka. You will need sudo because it will add domain entries in /etc/hosts.</p><pre>sudo kubefwd service -n kafka</pre><h3>Stern — for logs</h3><p>Stern allows you to tail multiple pods on Kubernetes and multiple containers within the pod. Each result is colour-coded for quicker debugging.</p><p>The query is a regular expression so the pod name can easily be filtered and you don’t need to specify the exact id (for instance omitting the deployment id). If a pod is deleted it gets removed from <em>tail</em> and if a new pod is added it automatically gets tailed.</p><p>When a pod contains multiple containers Stern can tail all of them too without having to do this manually for each one. Simply specify the container flag to limit what containers to show. By default, all containers are listened to. <a href="https://github.com/wercker/stern">link</a></p><p>You can install <em>stern</em> on mac also by <a href="https://brew.sh/">homebrew</a></p><pre>brew install stern</pre><p>And you can tail logs from several pods like this:</p><pre>stern ingestion -t --since 15m</pre><p>I was using this tool for debugging while development or deploying to dev or test environments, but for production, it will probably produce too many logs.</p><p><em>Originally published at </em><a href="https://dosht.github.io//devops/2020/04/06/productivity-tools-for-kubernetes.html"><em>https://dosht.github.io</em></a><em> on April 6, 2020.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4f1e8d326add" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Learning Computer Vision (Machine Learning)]]></title>
            <link>https://medium.com/@mou.abdelhamid/learning-computer-vision-machine-learning-c1521ee6ed08?source=rss-d670c15e6673------2</link>
            <guid isPermaLink="false">https://medium.com/p/c1521ee6ed08</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Mustafa Abdelhamid]]></dc:creator>
            <pubDate>Thu, 22 Aug 2019 00:00:00 GMT</pubDate>
            <atom:updated>2019-08-31T17:31:28.892Z</atom:updated>
            <content:encoded><![CDATA[<h3>Studying Computer Vision via Machine Learning</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Owseo_Hi2lhHC8zAzsq95A.jpeg" /></figure><p>The Computer Vision (CV) is simply teaching a computer how to see and understand the content of images and videos. We are using already many applications of CV on our phones and computer. For example, when you post a picture on Facebook of you and your friends, Facebook then uses CV to detect faces in the picture and then suggest who those friends could be. Also in Google Photos, you can search in your picture for cats and it will show you pictures that contain cats even if the name of the picture doesn’t contain the word cat.</p><p>In this post, I will explain one possible learning path to study computer vision through machine learning. The suggestions below are biased by my experience and a bit opinionated, so I discarded a lot of similar alternatives and listed only what I think a short path, but good enough to get started. I also will not mention anything about traditional computer vision methods that use OpenCV which is also very important, but it ‘will make the post too long.</p><p>If you want to build something today to get an idea of what you are going to do, please check out this <a href="https://www.tensorflow.org/beta/tutorials/keras/basic_classification">quick tutorial</a></p><h3>1. Machine Learning</h3><p>Now let’s start from bottom up. The first course is the <a href="https://www.coursera.org/learn/machine-learning">Machine Learning</a> by Andrew Ng. ml-class. Andrew Ng. is a co-founder of Coursera and the ml-class was one of the first 3 courses in Coursera even before Coursera itself started. You can think about it as ML-101 because this course will set all the basics you need to understand any advanced machine learning topic. It will give the intuition of what and how machine learning works and important concepts like what model training is, cross-validation, overfitting and hyperparameter tuning, ..etc. It also covers a broad area of machine learning types. It starts from a simple version of linear regression algorithm and takes you up to the simplest version of neural networks — which will be your first step in the following course.</p><p><em>Tip</em>: In this course don’t be too fast and try to understand every concept and take your time to get the intuition of the model training of the simple linear regression at the beginning of the course, so draw the graphs and understand the math and implement every part yourself.</p><p><em>Tip 2</em>: Don’t spend too long to learn Octave (the programming language used in the course) because probably you will not use it again!</p><p>You might need some pre-requisites in Calculus, Linear Algebra and Probabilities, but the course itself will redirect you to resources to learn from at the beginning.</p><p>Checkpoint: From now on, you will need to know Python and you will write more code. Whether you already know Python or not, I suggest going throw this course <a href="https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-00-introduction-to-computer-science-and-programming-fall-2008/">MIT CS-101</a>. It will not only teach you Python but also and more important will make a good developer. I’m using concepts I learned from this course almost on a daily bases since I first took it more than 5 years ago.</p><h3>2. Deep Learning</h3><p>The second course is <a href="https://www.coursera.org/learn/neural-networks-deep-learning">Neural Networks and Deep Learning</a> and the main instructor is also Andrew Ng. It will expand what you studied in the Machine Learning about artificial neural networks, but with more details. It will be a good idea to start using one of the most famous machine learning libraries like <a href="https://www.tensorflow.org/">Tensorflow</a> and <a href="https://keras.io/">Keras</a> and you can combine both in this <a href="https://www.tensorflow.org/tutorials/keras/basic_classification">tutorial</a>. This course is also the first course in the <a href="https://www.coursera.org/specializations/deep-learning">Deep Learning Specialisation</a> in Coursera which is a set of 5 courses and if you want to finish all of them, I suggest to take them in this order (1–4–5–2–3). You may or may not like that idea, but it’s just my suggestion.</p><p>(Optional) Deep Learning by Geoffrey Hinton one of the heroes of deep learning <a href="https://www.cs.toronto.edu/~hinton/coursera_lectures.html">(link)</a>. The course is now somehow outdated, so I don’t suggest to go throw all of it, but it still gives a very good theoretical introduction to deep learning. I suggest you watch the lectures with only a paper and a pencil until the video (7e — Long term short term memory)</p><p>(Shortcut) If you are more a self-learner and like like the hard way to learn things, then you can read Part II of the Deep Learning book <a href="https://www.deeplearningbook.org/">(link)</a> and skip the last course. You should apply each model with Tensorflow and/or Keras and expect that some parts will not be so easy to understand and you will need to google and read some tutorials and watch some videos until you get it done.</p><h3>3. Convolutional Neural Network</h3><p>So the third and last course in our learning path is <a href="https://www.coursera.org/learn/convolutional-neural-networks">Convolutional Neural</a> Network and this the most common model that is used in Computer Vision. During studying the course, This course is the 4th course in the deep learning specialization. And now, I would recommend starting to apply the concepts you learn into something more practical. You can search for some good datasets that you can use to apply your knowledge and there is a famous website for data science competitions called <a href="https://www.kaggle.com/">Kaggle</a> where you start with a dataset and an objective and try to find the best machine learning model. You can also check out Google Colab; It’s a web-based environment that enables you to write code and run experiments on Google cloud for free!</p><p>Note: One important model that I couldn’t find (yet) in courses is called <a href="https://skymind.ai/wiki/generative-adversarial-network-gan">Generative Adversarial Network (GANs)</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*N8beIKmcBW_Uaf_B.gif" /></figure><h3>Practising</h3><p>The last and most exciting step(s) is to read research papers and pick one of them and try to implement it yourself and see if you can get similar results. You can pick your favourite paper from <a href="https://paperswithcode.com/">papers-with-code</a> First try to find an interesting paper and implement it yourself then if/when you stuck, look at the code. If you get inspired by one of those papers, you can try to implement an application that uses that model and maybe start your startup :D</p><p><em>Tip</em>: There are too many papers and you can spend weeks implementing one paper, so before you pick one, scan many papers by reading the abstract and conclusion and maybe a couple of sections that could be relevant and then decide to go further or skip to another paper.</p><p>I hope after you read this article, you now have a better idea of how you can self-study computer vision and if you have any question, please ask the comments section below.</p><p>Enjoy and see you again in 6 months!</p><p><em>Originally published at </em><a href="https://dosht.github.io//ai/computervision/ml/2019/08/22/learning-computer-vision-machine-learning.html"><em>https://dosht.github.io</em></a><em> on August 22, 2019.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c1521ee6ed08" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Crazyflie Hello World!]]></title>
            <link>https://medium.com/@mou.abdelhamid/crazyflie-hello-world-47b22637cd7?source=rss-d670c15e6673------2</link>
            <guid isPermaLink="false">https://medium.com/p/47b22637cd7</guid>
            <category><![CDATA[autonomous-vehicles]]></category>
            <category><![CDATA[drones]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Mustafa Abdelhamid]]></dc:creator>
            <pubDate>Sun, 21 Oct 2018 00:00:00 GMT</pubDate>
            <atom:updated>2019-04-06T08:57:08.937Z</atom:updated>
            <content:encoded><![CDATA[<p>This post is using <a href="https://www.bitcraze.io/crazyflie-2/">Crazyflie 2.0</a> which is a kick start to build a small drone and control it autonomously from the computer, so you will need also <a href="https://www.bitcraze.io/crazyradio-pa/">Crazyradio PA</a> to connect your python program to the drone through. You will need also at least <a href="https://pip.pypa.io/en/stable/">python-pip</a> and <a href="https://git-scm.com/book/en/v2/Getting-Started-Installing-Git">git</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rZg-yNe-wa67dLSzIzL_lA.jpeg" /></figure><h3>Building and configuring the drone</h3><h3>1. Build the drone</h3><p>First thing after getting the drone in a box, you will need to build it, so you can follow this <a href="https://www.bitcraze.io/getting-started-with-the-crazyflie-2-0/">walkthrough</a> to assemble the hardware.</p><h3>2. Install the client (and the driver)</h3><p>Then, install the <a href="https://www.bitcraze.io/download/">client</a> on your computer (and the <a href="https://wiki.bitcraze.io/doc:crazyradio:install_windows_zadig">driver</a> if you’re using windows). This will be helpful to test your connection, update the firmware and change the configuration.</p><p>(Optional) You can install the <a href="https://play.google.com/store/apps/details?id=se.bitcraze.crazyfliecontrol2">Android</a> or the <a href="https://itunes.apple.com/us/app/crazyflie-2.0/id946151480">iOS</a> client and fly it manually. You can also use the mobile client to upgrade its firmware.</p><h3>3. Configure upgrade the firmware</h3><p>Final setup, open the client and update the firmware [here] or using the mobile client. (https://www.bitcraze.io/getting-started-with-the-crazyflie-2-0/#update-fw). Then navigate to connect, select the configure 2.0 option and change the bandwidth from 250k to 2M.</p><h3>Getting started</h3><pre>git clone <a href="https://github.com/mikecentral/Crazyflie-Drone">https://github.com/mikecentral/Crazyflie-Drone</a></pre><p>Then install cflib using python pip as follows:</p><h3>Run the program</h3><p>Now we will run autonomousSequence.py, but before you run the code, you need to make sure you have enough space in your room to fly the drone in 1.5 meters in each direction, or you can set SIDE constant to a smaller value, e.g. 0.5.</p><p>Another practical tip: You can add the emergency landing callback. That will send a landing command to the drone if you decided to exit the program immediately for any reason. Add this code to the main function:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/21d0dee09ab7186d5406ced8045f6951/href">https://medium.com/media/21d0dee09ab7186d5406ced8045f6951/href</a></iframe><pre>python autonomousSequence.py</pre><h3>First Crazyflie flying experiment in the living room</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FVijqE0MiTNE%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DVijqE0MiTNE&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FVijqE0MiTNE%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/9983121e93fed10b736459c8be1ee471/href">https://medium.com/media/9983121e93fed10b736459c8be1ee471/href</a></iframe><p><em>Originally published at </em><a href="https://dosht.github.io//crazyflie/drones/2018/10/21/crazyflie-hello-world.html"><em>dosht.github.io</em></a><em> on October 21, 2018.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=47b22637cd7" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>