<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Subash S on Medium]]></title>
        <description><![CDATA[Stories by Subash S on Medium]]></description>
        <link>https://medium.com/@subashsasi?source=rss-1b21153322de------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*x3iS9vs0pGVaP3UkT0OYtg.png</url>
            <title>Stories by Subash S on Medium</title>
            <link>https://medium.com/@subashsasi?source=rss-1b21153322de------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 03:56:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@subashsasi/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Self-Hosted RAG on AWS EC2: Qdrant, Ollama, and LangChain with Docker Compose]]></title>
            <link>https://medium.com/@subashsasi/self-hosted-rag-on-aws-qdrant-ollama-and-langchain-with-docker-compose-830ecbe32ec3?source=rss-1b21153322de------2</link>
            <guid isPermaLink="false">https://medium.com/p/830ecbe32ec3</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[qdrant]]></category>
            <category><![CDATA[ollama]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Subash S]]></dc:creator>
            <pubDate>Thu, 16 Jul 2026 06:42:22 GMT</pubDate>
            <atom:updated>2026-07-24T09:46:34.628Z</atom:updated>
            <content:encoded><![CDATA[<p>Every engineering team has an unofficial knowledge oracle. Deployment process? Ask Aaron. Leave policy? Raino knows. Why we picked Kafka over SQS? There’s an architecture decision record(ADR) somewhere —maybe in Confluence, maybe a markdown file in a GitHub repo nobody checks.</p><p>This works fine until Aaron is on PTO, or Raino switches teams, or the new hire needs answers at 11 PM during an incident.</p><p>I wanted something different: a system where anyone could type ‘Can I deploy on Friday?’ and get the documented answer in under two seconds — grounded in our actual runbooks, with a citation pointing to the source. Not a guess. Not a hallucination. The real answer.</p><p>One hard constraint shaped everything: no internal document leaves our AWS account. No OpenAI keys. No third-party vector databases. The entire system had to run inside our VPC on a single EC2 instance.</p><p>Here’s what that looks like, which I built over a weekend.</p><p>A simple Streamlit chat interface. You type a question, it thinks and you get an answer with expandable sources showing exactly which document it pulled from.</p><blockquote><strong>User:</strong> What are the dependencies of the payment service?</blockquote><blockquote><strong>Bot:</strong> The payment service depends on PostgreSQL (prod-db.internal:5432),<br> Redis for caching (prod-redis.internal:6379), Kafka for events<br> (prod-kafka.internal:9092), and the Auth Service at<br> auth.internal/v2/verify. Do not deploy if any dependency is<br> unhealthy — check status.internal.company.com first.</blockquote><blockquote>Sources: deployment-runbook.md (Section: Service Dependencies)</blockquote><h3><strong>The Architecture</strong></h3><p>Three containers. One docker-compose up. That’s it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/521/1*GpWrOYQkE2al-CuoxH1m_A.jpeg" /><figcaption>RAG Architecture setup on a GPU EC2 instance</figcaption></figure><p><strong>Ollama</strong> runs two AI models locally — one for understanding text (embedding) and one for generating answers (llama3.1). No API keys. No per-token billing.</p><p><strong>Qdrant</strong> stores the document chunks as vectors and finds the most relevant ones in milliseconds when you ask a question.</p><p><strong>LangChain </strong>wires everything together— it takes the output of one and feeds it as input to the next. That’s what ‘glue’ means here. it’s the layer between the tools that makes them work as a single pipeline. Without LangChain, we’d write the same connections manually — HTTP calls to Ollama, parsing JSON responses, formatting them into a prompt, calling Qdrant’s API, handling the results. LangChain replaces that boilerplate with composable building blocks (| pipes).</p><p><strong>Streamlit</strong> gives you the chat UI. Under 80 lines of Python.</p><p><strong>S3</strong> holds the source documents. Anyone on the team can drop new files there — markdown, text or PDF.</p><p>Here’s the full file that brings everything up:</p><pre>version: &quot;3.9&quot;<br><br>services:<br>  ollama:<br>    image: ollama/ollama:latest<br>    container_name: ollama<br>    ports:<br>      - &quot;11434:11434&quot;<br>    volumes:<br>      - ollama_data:/root/.ollama<br>    deploy:<br>      resources:<br>        reservations:<br>          devices:<br>            - driver: nvidia<br>              count: all<br>              capabilities: [gpu]<br>    restart: unless-stopped<br><br>  qdrant:<br>    image: qdrant/qdrant:v1.18.2<br>    container_name: qdrant<br>    ports:<br>      - &quot;6333:6333&quot;<br>      - &quot;6334:6334&quot;<br>    volumes:<br>      - qdrant_data:/qdrant/storage<br>    environment:<br>      - QDRANT__SERVICE__ENABLE_CORS=true<br>    restart: unless-stopped<br><br>  app:<br>    build:<br>      context: .<br>      dockerfile: Dockerfile<br>    container_name: rag-app<br>    ports:<br>      - &quot;8501:8501&quot;<br>    environment:<br>      - OLLAMA_BASE_URL=http://ollama:11434<br>      - QDRANT_URL=http://qdrant:6333<br>      - COLLECTION=company_docs<br>      - EMBED_MODEL=nomic-embed-text<br>      - LLM_MODEL=llama3.1:8b<br>      - S3_CORPUS_BUCKET=${S3_CORPUS_BUCKET}<br>      - S3_CORPUS_PREFIX=${S3_CORPUS_PREFIX:-corpus/}<br>      - AWS_REGION=${AWS_REGION:-us-east-1}<br>      - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID:-}<br>      - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY:-}<br>    volumes:<br>      - corpus_data:/app/corpus<br>      - ./scripts:/app/scripts<br>    depends_on:<br>      - ollama<br>      - qdrant<br>    restart: unless-stopped<br><br>volumes:<br>  ollama_data:<br>  qdrant_data:<br>  corpus_data:</pre><p><strong>How the pieces connect</strong>: Ollama and Qdrant use official pre-built images from Docker Hub — nothing to build. The ‘app’ service is different, it points to a Dockerfile in the project root that installs Python dependencies, copies in the Streamlit app and LangChain scripts, and starts Streamlit on port 8501. Docker Compose handles building that image, wiring the internal network so containers can talk to each other by name (e.g., http://ollama:11434), and exposing only port 8501 to the outside world.</p><h4><strong>A Note on GPU Instances</strong></h4><p>The <em>deploy.resources.reservations</em> block in the <em>ollama</em> service is what gives the container access to the GPU. This requires:</p><ul><li>A GPU instance (e.g. g4dn.xlarge with 1GPU –16GiB Memory)</li><li>NVIDIA drivers installed on the host</li><li>The <a href="https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html">NVIDIA Container Toolkit</a> installed</li></ul><p>If you don’t have a GPU then Remove the entire ‘deploy’ block from the ‘ollama service’. It’ll run on CPU — queries will take 10–20 seconds instead of 1–2 seconds, but the pipeline works exactly the same.</p><h4><strong>How RAG Actually Works</strong></h4><p>RAG means Retrieval-Augmented Generation.</p><ol><li>You ask a question</li><li>The system searches your docs for the 5 most relevant paragraphs</li><li>Those paragraphs get stuffed into the LLM’s prompt as context</li><li>The LLM generates an answer using only that context</li></ol><p>The LLM never sees your full document collection. It only sees the 5 chunks that matched your question. This is why it doesn’t hallucinate — if the answer isn’t in those chunks, it says, I don’t have that information.</p><p>Think of it like giving a really smart intern exactly 5 pages from your wiki and asking them to answer a question based only on those pages.</p><h3><strong>The Full Flow: From document to answer</strong></h3><h4><strong>Phase 1: Ingestion</strong></h4><p>Ingestion phase runs once with ingest script. It loads documents from /app/corpus, chunks them, embeds via Ollama and upserts into Qdrant.</p><p>S3 Bucket -&gt; Sync -&gt; Chunk -&gt; Embed -&gt; Store in Qdrant</p><ol><li>Sync from S3 — A script pulls .md, .pdf, or .txt files from S3 to the container. Incremental — only downloads new/changed files.</li><li>Chunk — Documents are split into ~768-character pieces with 96-char overlap. Markdown is first split by headers (`#`, `##`) so each chunk retains its section context. PDFs are split page-by-page first. The overlap ensures sentences at chunk boundaries aren’t cut in half.</li><li>Embed — Each chunk goes to Ollama’s `nomic-embed-text` model, which returns a 768-dimensional vector representing its meaning. Similar text → similar vectors, even with different wording</li><li>Store — Vector + text + metadata (filename, section, page number) saved in Qdrant. An HNSW index is built for fast search.</li></ol><p><strong>Phase 2: Querying</strong></p><p>Question → Embed -&gt; Search Qdrant -&gt; Build prompt -&gt; LLM -&gt;Answer</p><ol><li>Embed the question — Same `nomic-embed-text` model converts the question into a vector in the same space as your stored chunks.</li><li>Vector search — Qdrant finds the 5 most semantically similar chunks. For ‘Can I deploy on Friday?’ it pulls the deployment schedule sections.</li><li>Build prompt — LangChain injects those chunks into a structured prompt that tells the LLM: Answer using ONLY this context. If you can’t, say so.</li><li>LLM generates — Ollama’s llama3.1:8b produces a grounded answer from the context.</li><li>Display — Streamlit shows the answer with expandable source citations.</li></ol><p><strong>The key constraint</strong>: <em>the same embedding model must be used for both phases. If you embed docs with one model and questions with another, the vectors won’t be comparable</em>.</p><h4><strong>How LangChain couples all of this together</strong></h4><p>LangChain doesn’t <strong>do</strong> any of the AI work. It’s the plumbing — connecting Ollama, Qdrant, and the prompt into a single composable pipeline</p><p><strong>What LangChain provides for each step:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*8JvCwl4ZfTGQA2nnk4axVA.jpeg" /><figcaption>LangChain Steps</figcaption></figure><p><strong>The chain in code:</strong></p><pre>from operator import itemgetter<br>from langchain_core.runnables import RunnableLambda, RunnableParallel<br><br>chain = RunnableParallel(<br>    question=itemgetter(&quot;question&quot;),           # pass question through<br>    docs=itemgetter(&quot;question&quot;) | retriever,   # embed + search Qdrant<br>).assign(<br>    answer=(<br>        {&quot;context&quot;: itemgetter(&quot;docs&quot;) | RunnableLambda(format_docs),<br>         &quot;question&quot;: itemgetter(&quot;question&quot;)}<br>        | prompt_template    # inject into system/human message<br>        | llm                # send to Ollama<br>        | str_parser         # extract text from response<br>    )<br>)</pre><p>The | (pipe) operator connects each piece. Data flows left to right</p><p><strong>Usage:</strong></p><p><em>result = chain.invoke({“question”: “Can I deploy on Friday?”})</em></p><p><em>print(result[“answer”])</em></p><p><em>print(result[“docs”]) # The 5 source chunks</em></p><p><strong>Why this matters:</strong></p><ul><li>Swap the LLM? Change ChatOllama(model=”llama3.1:8b”) to ChatOllama(model=”mistral”)`. Nothing else changes.</li><li>More results? Change search_kwargs={“k”: 5}` to {“k”: 10}. The rest adapts.</li><li>Add streaming? Replace chain.invoke() with chain.stream(). Tokens flow to the UI live.</li><li>Switch to OpenAI later? Replace ChatOllama with ChatOpenAI. Same chain shape.</li><li>Add a reranker? Insert one more pipe stage between retriever and format_docs.</li></ul><p>That’s the real value. Not magic — composability.</p><h4>Why keep it In-House?</h4><p>Data privacy is the obvious answer — and yes, it matters. But that alone didn’t drive the decision. Three practical things did:</p><p><strong>Predictable cost - </strong>A `g4dn.xlarge` runs at ~$0.53/hour. Ten hours a day, five days a week — roughly $115/month, fixed. With OpenAI, costs scale with every query and every embedding call.</p><p><strong>Freedom to swap - </strong>Want to try Mistral instead of Llama? One command: ollama pull mistral. Want a different embedding model? Change an environment variable and re-ingest. No SDK rewrites, no vendor migration projects, no waiting for API deprecation timelines.</p><p><strong>Works without internet </strong>— Once the models are pulled and documents are indexed, the system runs fully airgapped. That matters for compliance-sensitive environments — and it means your team isn’t blocked when a third-party API has an outage.</p><h4><strong>Lessons from building this</strong></h4><p><strong>Chunking matters more than the model</strong></p><p>The single biggest lever on answer quality isn’t which LLM we pick — it’s how we split our documents. I use a two-pass strategy: first by markdown headers (so each chunk knows its section context), then by character count (768 chars with 96-char overlap). That overlap is non-negotiable. Without it, any fact that straddles a chunk boundary becomes invisible to search</p><p><strong>Qdrant’s indexing threshold is per-segment, not per-collection</strong></p><p>This cost me around 5 hours of debugging. After ingesting 27,000 chunks, queries were slow. The reason: Qdrant only builds its HNSW index when a segment exceeds 20,000 vectors. My data was spread across 8 segments (3,400 each) — all below the threshold, so every query fell back to brute-force scan. One API call to lower the threshold fixed it. Queries dropped from 200ms to 25ms</p><p><strong>Cold starts are unavoidable (but manageable)</strong></p><p>The first query after the LLM has been idle takes about 4 seconds — that’s the model loading into GPU memory. Every query after that runs in 1.2–1.5 seconds. If latency on the first request matters to you, send a throwaway warmup query when the container starts.</p><h4><strong>Testing It: Did It actually work?</strong></h4><p>I uploaded 5 internal docs (mix of markdown and PDF): leave policy, onboarding guide, deployment runbook, expense policy, and architecture decision records.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/787/1*AyDJF23XAuU2XO2ORpOozw.jpeg" /></figure><p>That last one is key. It didn’t make up a number. It admitted it didn’t know</p><h4><strong>The S3 Workflow: Why it matters</strong></h4><p>Documents live in S3, not on the EC2 instance. This sounds like a minor detail but it changes the operating model:</p><ul><li>HR updates the leave policy? They drop the new PDF in S3. Someone runs the sync + ingest. Done.</li><li>Instance dies? Spin a new one, pull from S3, re-ingest. Zero data loss.</li><li>Multiple environments? Same S3 bucket, different EC2 instances for dev/staging/prod.</li></ul><p>The ingest is idempotent — it recreates the vector collection from scratch each time. No state to manage on the instance beyond the model weights.</p><h4><strong>Accessing the UI</strong></h4><p>Streamlit serves on port 8501. Hit your instance’s public IP directly:</p><p>http://&lt;your-ec2-public-ip&gt;:8501</p><p>Security group: open inbound TCP 8501 from your IP. Keep Ollama (11434) and Qdrant (6333) internal — they only talk to each other inside Docker’s network.</p><h4><strong>Cost Breakdown</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/329/1*WQqylhrrc4g8wzyO2CuiUA.jpeg" /><figcaption>Cost Breakdown</figcaption></figure><p>Comparison: OpenAI API at moderate volume runs $300–800/month, plus compliance overhead of sending internal docs externally.</p><h4>What I would do differently in production</h4><p>This is a POC. For a real deployment:</p><ol><li>Add auth — ALB + Cognito or a password gate on Streamlit</li><li>Automate ingest — S3 event → Lambda → trigger re-ingest</li><li>Add evaluation— ragas to score answer quality after each ingest. Those who curious about ragas is it’s a Python library that automatically scores your RAG answers on things like faithfulness, answer relevancy and content precision.</li><li>Bigger model — llama3.1:70b for complex multi-hop questions</li><li>Hybrid search — vector + keyword (BM25) for better recall. A note on what BM25 is a traditional keyword search algorithm (the one behind Elasticsearch). It scores documents based on exact word matches and word frequency. It’s great at finding specific terms, names, error codes and exact phrases.</li></ol><h4><strong>The Bigger Picture</strong></h4><p>The self-hosted RAG pattern is becoming the default for organizations that need AI-powered search over private data. The tooling has matured to the point where a single engineer can set this up in a weekend.</p><ul><li>Start with 5 docs for a POC</li><li>Scale to 5,000 without architecture changes</li><li>Swap the corpus from company policies to medical literature, legal contracts, or research papers</li><li>Same stack, same code, different documents</li></ul><p>The question isn’t ‘can we build this?’ anymore. It’s ‘what docs should we feed it first?’ 😅</p><p>Thanks for reading!</p><h3>References</h3><p>Ollama — <a href="https://ollama.com/">https://ollama.com/</a><br>Qdrant — <a href="https://qdrant.tech/documentation/">https://qdrant.tech/documentation/</a><br>LangChain — <a href="https://python.langchain.com/docs/introduction/">https://python.langchain.com/docs/introduction/</a></p><p>LangChain + Ollama integration — <a href="https://python.langchain.com/docs/integrations/llms/ollama/">https://python.langchain.com/docs/integrations/llms/ollama/</a><br>LangChain + Qdrant integration — <a href="https://python.langchain.com/docs/integrations/vectorstores/qdrant/">https://python.langchain.com/docs/integrations/vectorstores/qdrant/</a><br>LangChain text splitters — <a href="https://python.langchain.com/docs/concepts/text_splitters/">https://python.langchain.com/docs/concepts/text_splitters/</a></p><p>NVIDIA Container Toolkit install guide — <a href="https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html">https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html</a></p><p><a href="https://python.langchain.com/docs/concepts/rag/">https://python.langchain.com/docs/concepts/rag/</a><br>HNSW algorithm (Qdrant) — <a href="https://qdrant.tech/documentation/concepts/indexing/#vector-index">https://qdrant.tech/documentation/concepts/indexing/#vector-index</a></p><h4><strong>Inspiration</strong></h4><p>ComputingForGeeks RAG guide — <a href="https://computingforgeeks.com/qdrant-rag-ollama-langchain/">https://computingforgeeks.com/qdrant-rag-ollama-langchain/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=830ecbe32ec3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Real-Time AWS Security Intelligence with Bedrock, ECS Fargate, and Zero Static Credentials]]></title>
            <link>https://medium.com/@subashsasi/real-time-aws-security-intelligence-with-bedrock-ecs-fargate-and-zero-static-credentials-95a1066d65c2?source=rss-1b21153322de------2</link>
            <guid isPermaLink="false">https://medium.com/p/95a1066d65c2</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[cloud-security]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[amazon-bedrock]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Subash S]]></dc:creator>
            <pubDate>Mon, 08 Jun 2026 14:37:12 GMT</pubDate>
            <atom:updated>2026-06-16T13:03:50.803Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Rainy Saturday. Tea. Laptop. I managed to ruin one of the three within the first ten minutes :) the tea tipped and headed straight for the keyboard. I caught it just in time. Sitting there slightly damp, it struck me: this is exactly how cloud security incidents work. Not a fire alarm. Just a quiet overflow nobody notices until it’s already reached the edge. That thought became this project.</em></p><p>A DevSecOps dashboard that scans your AWS account, surfaces risk in plain English, and generates Terraform fixes — with no access keys, no secrets in env vars, and security-first design.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gGl-_Eyn-nmatO4p5b4Zjg.jpeg" /><figcaption>DevSecOps security dashboard built with Next.js and FastAPI + Python</figcaption></figure><h3><strong>Security Is a Habit</strong></h3><p>I’ve seen personal projects with IAM access keys in `.env` files, hardcoded credentials in Dockerfiles, and S3 buckets with public read enabled “just for testing.” Those habits don’t disappear when you move to a real product — they get copy-pasted.</p><p>So I made one rule: treat this like a production workload from commit one. No access keys. No secrets in environment variables. IAM roles only. bcrypt. OIDC. TLS everywhere.</p><p>It took roughly 40% longer. It was also genuinely educational — when you implement ECS credential injection and Secrets Manager integration on a project where nobody is demanding it, you actually understand how they work. Security is not a phase you add after you have something working.</p><h3><strong>The Context-Switching Problem</strong></h3><p>Every AWS developer I know has been in this situation: you open the AWS console, you have eleven tabs, you’re jumping between EC2, S3, GuardDuty, IAM, and Service Quotas — and somewhere in that chaos, a misconfigured S3 bucket has been public for three weeks.</p><p>Cloud security is not hard because the tools don’t exist. It’s hard because the tools are scattered, the alerts are noisy, and by the time you correlate data across five services, you’ve already lost the thread. It’s a slow overflow. Each individual drop looks harmless. Together they soak through.</p><p>AWS already gives you world-class tools — GuardDuty for threat detection, IAM for access control, Security Hub for aggregation. The challenge is not capability. It is context-switching. When a security question spans S3, GuardDuty, IAM, and EC2, the answer lives across four separate console pages. This project is a lightweight layer that pulls those signals together, adds an AI that speaks plain English, and lets you act on what you find — without leaving one screen.</p><h3>What It Does</h3><p>The AWS DevSecOps AI Inspector is a full-stack web application that gives a <strong>real-time view of your AWS security posture from a single screen</strong>. It scans the environment on demand, surfaces risks in plain English, and — here’s the part that makes it genuinely different — opens an AI chat powered by Amazon Bedrock where I can ask what went wrong and get actual Terraform code to fix it.</p><p>Here is what it does in a single session:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/655/1*peiTm4-cv3Gb1HUrBCV5nA.png" /><figcaption>The services I enabled for this POC. Can be extended based on your use case</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZOIep4MMFVaTCpxs_WquPQ.png" /><figcaption>Context aware fix generation by AWS Bedrock AI</figcaption></figure><p>The key word is <strong>context-aware</strong>. Most security tools tell you <em>what</em> is wrong. This one tells you <em>why it matters</em>, <em>what the blast radius is</em>, and <em>here is the exact Terraform to fix it</em> — for your specific resources, not a generic template.</p><h3><strong>Where This Fits</strong></h3><p>A fair question: doesn’t AWS Security Hub already do this?</p><p>Partly. Security Hub runs continuous automated checks against CIS benchmarks and aggregates findings from GuardDuty, Inspector, and IAM Access Analyzer. It is excellent at detection and compliance scoring.</p><p>What it does not do is reason about those findings in plain English, generate Terraform for your specific resources, or let you ask “what is the blast radius if this IAM user is compromised?”</p><blockquote><em>This project is an AI reasoning layer above your AWS security signals — explaining what they mean, connecting dots across services, and translating findings into action.</em></blockquote><h3>Security by Design</h3><p>Let me walk through the architecture with the same level of scrutiny I would apply to a production system — because that is exactly how I designed it.</p><figure><img alt="Architecture Overview" src="https://cdn-images-1.medium.com/max/802/1*j4TOpzO0UPgZa1sfuGbt5w.png" /><figcaption>Architecture Overview</figcaption></figure><p><strong>TLS 1.3 only at the ALB.</strong> The ELB policy TLS13-1-2-2021-06 disables TLS 1.0, 1.1, and TLS 1.2 cipher suites that rely on RC4, 3DES, or export-grade keys. The browser talks to the ALB over a connection that cannot be downgraded by a MITM attack.</p><p><strong>HTTP → HTTPS redirect (301).</strong> A permanent redirect means browsers cache it. Even if a user types http://, they are immediately redirected without sending any cookies or credentials over plain text. The redirect is enforced at the load balancer layer — not in application code, so it cannot be bypassed by a backend bug.</p><p><strong>No AWS calls from the frontend.</strong> The Next.js frontend container has no IAM role, no AWS SDK, and no knowledge of your AWS account. It only talks to the FastAPI backend via the ALB. The entire AWS API surface is gated behind JWT authentication in the backend. Even if an attacker found an XSS vulnerability in the frontend, they could not make any AWS API calls directly.</p><p><strong>httpOnly cookies for JWT.</strong> The JWT is set as an httpOnly; Secure; SameSite=Strict cookie. JavaScript running in the browser cannot read it. document.cookie returns nothing. This prevents token theft via XSS — the most common authentication attack against web applications.</p><p><strong>bcrypt for password hashing.</strong> The stored credential is a bcrypt hash with a work factor of 12.</p><p><strong>Rate limiting on login.</strong> Five login attempts per five minutes per IP. Simple, in-memory (for this POC), but it makes brute-force attacks against the login endpoint infeasible. In a production deployment this moves to ElastiCache Redis to survive container restarts.</p><h3>How the Backend Talks to AWS Without a Single Secret</h3><p>This is the section I want to spend the most time on, because it is the design decision that separates a secure cloud application from a credentials-in-a-.env-file application.</p><h4>The Credential Chain</h4><p>The backend container <strong>never uses an AWS access key or secret key</strong>. There is no AWS_ACCESS_KEY_ID in the ECS task definition. There is no .aws/credentials file in the Docker image. There is no IAM user created for this application.</p><p>When the ECS task starts, AWS automatically injects temporary credentials into the container via the ECS task metadata endpoint. This uses the link-local address `169.254.170.2` — a reserved address defined by AWS at the platform level, not something we configure. Every ECS task on every account in every region gets this same address, the same way `169.254.169.254` is the standard EC2 instance metadata address. It is only reachable from inside the container, not from the internet, not from other containers, not from the host. The boto3 SDK finds these credentials automatically through its credential provider chain.</p><p>The ASIA prefix in the access key ID is significant — it denotes a temporary (STS-issued) credential, not a long-lived IAM user key (AKIA prefix). These credentials automatically expire every few hours. If the container is stopped, the credentials are immediately invalidated. There is nothing to rotate, nothing to revoke manually, and nothing that can be leaked to a repository.</p><h4>The IAM Task Role — Least Privilege in Practice</h4><p>The trust policy on the role is precise — Principal: { Service: &quot;ecs-tasks.amazonaws.com&quot; } means only ECS tasks can assume it. Not an IAM user, not an EC2 instance, not another account. The permissions attached are read-only by design.</p><h4>How the S3 Scan Works Under the Hood</h4><p>When I click “Scan S3”, boto3 fetches credentials from the ECS metadata endpoint automatically — no configuration needed. All buckets are then audited in parallel using a thread pool (up to 20 workers), turning what would be a 40-second sequential scan into roughly 2 seconds:</p><pre><br>with ThreadPoolExecutor(max_workers=min(20, len(names) or 1)) as ex:<br>    futures = {ex.submit(_audit_bucket, s3, n): n for n in names}<br>    for f in as_completed(futures):<br>        results.append(f.result())</pre><p>Each bucket audit makes exactly two API calls: get_public_access_block (all four boolean settings must be True) and get_bucket_encryption (absence of the response means unencrypted). A _FAST config with 5-second timeouts and one retry maximum prevents any single slow bucket from blocking the rest.</p><h3>Context Is Everything</h3><p>This is one of the questions that comes up most often: <em>“I asked the chat about an S3 bucket and it gave me Terraform code — how does it know what my infrastructure looks like?”</em></p><p>The answer is <strong>context injection</strong>. Every time I click a scan button, the results are stored in the frontend’s React state. When I open the chat and send a message, the frontend builds a structured context string from everything that has been scanned:</p><pre>AWS SCAN CONTEXT:<br>Account: 123456789012 | Region: us-east-1<br><br>S3:<br>  my-company-data-bucket | VULNERABLE | Unencrypted | BlockPublicAcls:OFF IgnorePublicAcls:OFF<br>  logs-archive-prod      | OK         | Encrypted   | All blocks: ON<br><br>Security Groups:<br>  default (sg-0abc123)    | OPEN PORT tcp:22   → 0.0.0.0/0<br>  web-tier-sg (sg-0def456)| OPEN PORT tcp:3306 → 0.0.0.0/0<br><br>GuardDuty: ENABLED | High: 2 | Medium: 5 | Low: 12<br><br>IAM: 8 users | No MFA: 3 (alice, bob, charlie) | AdminAccess: 2 (admin-user, ops-user)</pre><p>That context string is sent to the FastAPI backend alongside my message. The backend prepends it to the system prompt before calling Bedrock’s Converse API. When I ask <em>“fix the S3 bucket”</em>, the model already knows the bucket name, which specific block settings are misconfigured, and my region. It produces targeted Terraform that matches the actual resource — not a generic template:</p><h3>Technology Stack</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/857/1*TeB87akY-_7_TyZyRSDiGg.png" /><figcaption>Technology Stack</figcaption></figure><p>The <strong>Bedrock Converse API</strong> deserves special mention. Unlike calling Claude directly via Anthropic’s API, Bedrock keeps all traffic within the AWS network, bills to my AWS account (Free Tier eligible for Claude Haiku), and — critically — requires no external API key. The same ECS task role that scans my S3 buckets also invokes Bedrock, via the same SigV4-signed request mechanism. There is no credential to manage and nothing to leak.</p><h3><strong>CI/CD Without Secrets — </strong>OIDC</h3><p>The deployment pipeline uses <strong>OIDC federation</strong> — GitHub Actions generates a short-lived JWT that AWS exchanges for temporary credentials, with no long-lived access keys stored anywhere in GitHub. There is nothing to rotate, nothing to leak, and the IAM role is scoped to exactly what the pipeline needs: ECR push and ECS update.</p><p>I’ve covered this pattern in depth separately: <a href="https://medium.com/aws-in-plain-english/utilize-iam-roles-in-github-actions-for-aws-authentication-be8fa8f982dc"><strong>Utilize IAM Roles in GitHub Actions for AWS Authentication</strong></a></p><h3>Maintaining Security Posture</h3><p>Security posture is not a one-time check. It is a continuous state. I designed this tool for <strong>repeated use</strong>, not one-off audits.</p><p><strong>Scan after every deployment.</strong> Each scan takes seconds, not hours. You can run it after every infrastructure change, every time a new team member joins, every time a new service is launched.</p><p><strong>The status banner gives one number.</strong> Instead of five separate dashboards, I see a single green/amber/red indicator with a total risk count. That is the signal; the details are one click away.</p><p><strong>AI chat history persists for the session.</strong> I can ask follow-up questions. <em>“Why does this IAM user have admin access?”</em> → <em>“What is the blast radius if this account is compromised?”</em> → <em>“Give me the Terraform to apply least-privilege boundaries.”</em> The model holds the full scan context for the entire conversation.</p><h3>Honest Limitations</h3><p>No tool article is complete without being honest about what the tool does <strong>not</strong> do.</p><p><strong>This is a read-only auditor.</strong> It does not change anything in my account. It surfaces findings and generates remediation code, but I review and apply every fix myself. This is intentional — automated remediation without human review is a different risk profile entirely.</p><p><strong>The rate limiter is in-memory.</strong> Container restarts reset it. For a hardened production deployment, move this to ElastiCache Redis.</p><p><strong>It does not scan everything.</strong> Current scope: S3, EC2/security groups, GuardDuty, IAM users, service quotas. Not covered (yet): RDS security groups, Lambda function policies, KMS key rotation, CloudTrail configuration, Config rules, Network ACLs, VPC flow logs, and dozens of other important controls. All natural extensions.</p><p><strong>Single-user login.</strong> One username, one password. Multiple users with role-based access would require DynamoDB-backed user records and extended JWT claims.</p><h3>What This Project Taught Me</h3><p>Building this tool with production-grade security constraints from day one taught me things that documentation alone could not:</p><ol><li><strong>ECS credential injection is invisible and magical.</strong> I write boto3.client(&quot;s3&quot;) and the SDK handles everything. Understanding <em>why</em> that works — the metadata endpoint, the credential provider chain, the ASIA prefix — provide me a sharper understanding.</li><li><strong>IAM least privilege is not about being cautious.</strong> It is about being precise. Writing a policy and being forced to justify each action gave me a grasp of the AWS IAM permission model that years of console clicking never had.</li><li><strong>Bedrock’s Converse API is underrated.</strong> The ability to switch between Claude, Llama, and Titan by changing a single parameter — without touching any other code — is genuinely powerful for experimenting with model quality on specific tasks.</li><li><strong>Context injection is the key to AI utility.</strong> A model with no context gives generic advice. A model with my exact bucket names, finding reasons, and region gives targeted advice. The difference in usefulness is not incremental — it is categorical.</li></ol><h3>Conclusion</h3><p>The AWS DevSecOps AI Inspector is a small tool with an outsized impact on how developers think about cloud security. It replaces the guilt of <em>“I should probably check those buckets”</em> with a ten-second scan and an AI model ready to explain and fix whatever it finds.</p><p>The current implementation scans the account it runs in, using the ECS task role’s credentials. But the architecture naturally extends to multi-account setups common in AWS Organizations with the options Cross-Account IAM Roles or Scheduled Continuous Scanning.</p><blockquote>More than the functionality, what I want to leave you with is the <strong>design philosophy</strong>: security is not a feature you add after you have something working. It is a design constraint you apply from the first commit.</blockquote><h4>Resources</h4><ul><li><strong>Amazon Bedrock</strong>: <a href="https://aws.amazon.com/bedrock">aws.amazon.com/bedrock</a></li><li><strong>ECS Fargate</strong>: <a href="https://aws.amazon.com/fargate">aws.amazon.com/fargate</a></li><li><strong>Terraform AWS Provider</strong>: <a href="https://registry.terraform.io/providers/hashicorp/aws">registry.terraform.io/providers/hashicorp/aws</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=95a1066d65c2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Zero Trust Network - A conversation that need to be heard]]></title>
            <link>https://levelup.gitconnected.com/zero-trust-network-a-conversation-that-need-to-be-heard-6cd710c58daf?source=rss-1b21153322de------2</link>
            <guid isPermaLink="false">https://medium.com/p/6cd710c58daf</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[zero-trust]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Subash S]]></dc:creator>
            <pubDate>Thu, 08 Aug 2024 12:10:48 GMT</pubDate>
            <atom:updated>2024-08-25T15:16:30.812Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>On</strong> a typical day, when traffic was light and workloads were minimal, two <strong>pods</strong> engaged in a conversation about their environment’s security posture. They discussed how to protect themselves from both internal and external attack surfaces.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/760/1*_W0tG-UNWJm0jCRA84H2fw.jpeg" /></figure><p>Although seemingly unrelated, their conversation highlighted the importance of compliance and regulations, as well as the need to ensure that peak traffic originates from trusted sources. After all, isn’t that their right? They concluded that they should never trust requests from unauthorized sources. So, how can this be achieved?</p><p><strong>Pod A suggested,</strong> “The solution is that we should be in a Zero Trust network.”</p><p>“Zero Trust? That means a network trusted by no one?”<strong> Pod B asked.</strong></p><p><strong>Pod A explained,</strong> “No, think of it this way, the core principle of the Zero Trust security model is ‘never trust, always verify,’ which means that users and devices should not be trusted by default, Zero Trust is a pragmatic security model that assumes no user, device, or application, whether inside or outside the network should be trusted by default. Instead, every access request must be verified and authenticated before access to resources is granted.</p><p>A Zero Trust network evaluates all actions and resources in real-time to minimize the risk of unauthorized access to business data and sensitive resources. This approach involves continuous monitoring, strict access controls, and verifying user identities to safeguard data and applications. It’s about taking steps to protect resources by understanding the necessary boundaries and acknowledging that no system is 100% secure.”</p><p><strong>Pod B</strong> feeling more confident, asked to elaborate further;</p><p><strong>Pod A</strong> <strong>continues</strong>:</p><p>When two components don’t need to communicate, they shouldn’t be able to, even when residing within the same network segment. You can accomplish this by authorizing specific flows between the components. By eliminating unnecessary communication pathways, you are applying least privilege principles to better protect critical data.</p><p>You should think of Zero Trust concepts as additive to your existing security controls. By applying Zero Trust concepts in accordance with the organizational value of the system and data being protected, you can ensure that the benefits to your business are commensurate with the effort. Here are the Core principles of Zero trust architecture(ZTA);</p><ol><li>Verify and authenticate</li><li>Least privilege access</li><li>Micro-segmentation</li><li>Continuous monitoring and analytics</li><li>Automation and orchestration</li><li>Authorization</li></ol><p><strong>Pod B then asked,</strong> “How complex is the implementation of this approach?”</p><p><strong>Pod A added,</strong> “Let me explain. Implementing these principles requires a comprehensive approach that combines technology, processes, and people to achieve a zero trust mindset and build a resilient security posture. Traditional network security relies on a secure perimeter, where everything within is trusted and anything outside is not. But Zero trust emphasizes the importance of strong identification and authentication for principals of all types, including users, machines, and devices, hence the implementation requires careful planning and organized effort. Here are the challenges and complexities”;</p><ol><li><strong>Cultural Shift</strong>: Transitioning to Zero Trust requires a fundamental change in how security is perceived and implemented across the organization. It involves breaking away from the traditional perimeter-based security model and embracing a mindset of continuous verification and least-privilege access. This can be challenging as it requires buy-in from all stakeholders and changes in processes and policies.</li><li><strong>Complex Infrastructure</strong>: Implementing Zero Trust involves deploying and integrating various security technologies, such as identity and access management (IAM), multi-factor authentication (MFA), endpoint security, and network segmentation. This can be complex, especially in environments with a mix of legacy and modern systems.</li><li><strong>Comprehensive Visibility</strong>: Achieving Zero Trust requires comprehensive visibility into all network traffic, users, devices, and applications. Organizations must deploy advanced monitoring and analytics tools to detect and respond to threats in real-time. This can be resource-intensive and may require significant investment in infrastructure and personnel.</li><li><strong>Scalability</strong>: As organizations grow and evolve, their security needs change. A Zero Trust model must be scalable to accommodate new users, devices, and applications while maintaining security standards. This requires careful planning and management to ensure that security measures are not compromised.</li><li><strong>User Experience</strong>: Implementing strict access controls and continuous verification can impact user experience. Organizations must balance security requirements with usability to ensure that users can access necessary resources efficiently without being hindered by security measures.</li><li><strong>Cost and Resources</strong>: Transitioning to a Zero Trust model can be costly in terms of both financial investment and human resources. Organizations must allocate budgets for new technologies, training, and ongoing maintenance to ensure the success of their Zero Trust initiatives.</li></ol><p>Despite these challenges, the benefits of implementing a Zero Trust security model are substantial like as I explained you earlier. By eliminating implicit trust and continuously verifying and authenticating access requests, organizations can significantly reduce the risk of data breaches and improve their overall security posture.</p><p><strong>Pod A concluded,</strong> “Moving to a Zero Trust security model is a strategic move that enhances our ability to protect critical resources and data. It requires careful planning and execution, but the increased security and resilience against modern threats make it worthwhile.”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6cd710c58daf" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/zero-trust-network-a-conversation-that-need-to-be-heard-6cd710c58daf">Zero Trust Network - A conversation that need to be heard</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Starting Your DevOps Journey: Tips and Roadmap]]></title>
            <link>https://faun.pub/starting-your-devops-journey-tips-and-roadmap-2cab491af?source=rss-1b21153322de------2</link>
            <guid isPermaLink="false">https://medium.com/p/2cab491af</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[beginners-guide]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Subash S]]></dc:creator>
            <pubDate>Fri, 29 Sep 2023 14:45:48 GMT</pubDate>
            <atom:updated>2025-01-12T22:00:29.159Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Are you uncertain about how to start and progress with DevOps?</blockquote><blockquote>Are you finding it challenging to keep up with the numerous tools required to master DevOps? This article is for you!!</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1HFr-uXS3loYVFz4" /><figcaption>Photo by <a href="https://unsplash.com/@siora18?utm_source=medium&amp;utm_medium=referral">Siora Photography</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>DevOps </strong>is a set of practices, principles, and cultural philosophies that aim to improve collaboration and communication between software development (Dev) and IT operations (Ops) teams. As it is a crucial aspect of software engineering, as DevOps engineers, we focus on streamlining and automating the software development and delivery process. DevOps helps software engineering teams deliver high-quality software more efficiently and with greater reliability.</p><h3><strong>Where to start:</strong></h3><p>I began my journey into DevOps after gaining experience in different domains and starting from square one. My career transition came when I was offered an opportunity as a DevOps intern at <a href="https://innovationincubator.com/">Innovation Incubator Advisory</a>. I must admit, I had my doubts about how well I’d do in this new domain. At times, I even found myself scratching my head over confusion about project names and Debian package names — but it’s all part of the learning process, right? 😄</p><p>Let’s face it, in the beginning, we all make our fair share of mistakes, but those mistakes are often our best teachers, helping us gain valuable knowledge along the way. Approach this process with a sense of openness and adaptability.</p><blockquote><strong><em>Mistake and the subsequent learning experience are inherent to our technical journey</em></strong>.</blockquote><p>While you have the option to ask for help, I’d suggest considering it as a last resort.<strong><em> </em></strong>It’s essential to engage in brainstorming and put forth your best effort before turning to others for assistance.</p><h3><strong>Here you go, sharing my DevOps career route;</strong></h3><p><strong>Linux —</strong> Being well-versed in Linux is absolutely essential for a DevOps role. It’s the backbone that enables us to automate tasks and manage the daily operations efficiently. Many of the DevOps tools we rely on are open-source and designed to work seamlessly with Linux. During my internship, I closely studied Linux, its fundamental commands, and their practical implementations. I began with tasks like text manipulation and progressed to installing packages and deploying web servers both locally and on the cloud.</p><p><strong>Bash scripting knowledge — </strong>Learning bash scripting is super important for automating tasks in DevOps. With DevOps, we do a lot of automation and scripting work. Dive into shell scripting and try out cool things like if-else, loops, and setting up scheduled tasks. Also, don’t miss out on Python! It’s fantastic for DevOps because it’s easy to use, versatile, and has loads of libraries. Master it, it’s your trusty sidekick.</p><p><strong>Docker — </strong>Getting the hang of Docker is a must! It’s like magic for DevOps. Docker helps package up applications neatly into containers. Imagine this: a company has a cool web app, and Docker lets us bundle it up, making it easy to run on any system, whether it’s Windows, Linux, or Mac. Plus, you can host these containers on any cloud platform like Azure, AWS, or Google Cloud. It’s like having a super flexible and portable app that can run anywhere! This flexibility is a game-changer for us in the cloud world. So, Docker isn’t just important, it’s the go-to tool making waves in the cloud community! Docker provides comprehensive documentation available at this <a href="https://docs.docker.com/get-started/">location</a>, which you can refer to.</p><p><strong>Knowledge of DevOps Tools:</strong> Let’s talk about DevOps tools! These smart tools are key for making things smooth and efficient. We’ve got Git for version control, Jenkins or TravisCI for automation, Sonarqube for code quality, Rabbitmq for messaging, and Kibana, Prometheus, and/or Grafana for monitoring. It might seem like a lot, but don’t worry! Starting with Git is a great first step, in helping you manage your code. Then, you can dive into Jenkins or TravisCI for automation. Sonarqube helps keep your code top-notch, and Rabbitmq handles messages between systems. Lastly, Kibana, Prometheus, and Grafana help you keep an eye on everything. So, step by step, you’ll become a DevOps whiz.</p><p><strong>Infrastructure as Code(IaC): </strong>Let’s talk about Infrastructure as Code (IaC). it’s super cool! Basically, we use IaC tools to set up our infrastructure and apps. AWS has one called CloudFormation, which is just for AWS stuff. But then there’s Terraform, which is like a superhero tool for all clouds! When we get hands-on with Terraform, it’s a game-changer. We can see our infrastructure in Terraform files, like a map, and we keep track of everything in Git. Having hands-on experience with Terraform is super valuable and gives us a real edge.</p><p><strong>Understanding networking concepts — </strong>Understanding network concepts is crucial in cloud learning. With this knowledge, we can efficiently and securely use cloud platforms, ensuring smooth communication and data management. Key areas to focus on include learning about VPCs, subnets, IP classifications, network layers, and CDN. Additionally, understanding various communication protocols such as HTTP, HTTPS, TCP/IP, and UDP is essential for cloud applications. Mastering these concepts empowers individuals to design secure, scalable, and efficient cloud infrastructures. So, investing effort in grasping these fundamentals is essential. With my free-tier AWS account, I had been exploring VPC, EC2, LoadBalancer, and S3, experimenting with user data scripts to deploy web servers, and also hosted static websites on private S3 and delivered content via CloudFront.</p><p><strong>How software application works:</strong> Basic knowledge of how software applications function is crucial for a DevOps engineer because it helps bridge the gap between developers and operations teams. When the DevOps team understands how apps work, it’s like speaking both languages, making it easier to automate tasks, fix problems, and ensure smooth software releases. This know-how leads to smoother teamwork, fewer mistakes during deployment, and better overall software quality, making everyone happy in the end!</p><h3><strong>About Cloud Certifications:</strong></h3><p>Encourage you to take Cloud Practitioner certification, it is a great start for your cloud journey. The next step is gaining practical experience. AWS offers a free one-year cloud environment, while Azure provides credits for learning. AWS offers a program called the <a href="https://skillbuilder.aws/">Skill Builder Program</a> with access to over 600+ free courses that allow you to gain experience in an AWS Management Console sandbox, explore game-based learning, take AWS Certification Official Practice Exams, and more.</p><blockquote>I recommend using these resources to gain hands-on experience before pursuing more certifications. In my opinion, certifications are more valuable when backed by practical knowledge.</blockquote><h3><strong>Kubernetes</strong></h3><p>You can try hands-on Kubernetes once you have acquired solid basic and hands-on above-mentioned technologies. Learning Kubernetes is essential for any tech enthusiast because it allows you to effortlessly handle and scale applications, making you a valuable asset in the tech world. It’s like having a powerful tool that streamlines your work, making you more efficient and in demand as a DevOps professional!</p><p>To help you understand better if you’re a beginner to this, I’ve created a simple diagram that explains how we put our app on the cloud. It shows both the part you see (the front end) and the behind-the-scenes stuff (the back end) from a DevOps perspective.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/658/1*K4pec_97VZ8p35WJ7X1tgA.png" /><figcaption>Illustrative Diagram</figcaption></figure><p>You will have the responsibility of handling deployment and providing support to the development team throughout the entire Software Development Life Cycle(SDLC). Let me walk you through the diagram to give you a better idea,</p><p><strong>Github:</strong> In GitHub where the code resides, developers collaborate on platforms like GitHub, Bitbucket, GitLab, CodeCommit, etc. where they work together to create code.</p><p><strong>Build:</strong> The next step is building the application, and getting it ready to use. Using tools like Jenkins, CodeBuild, or GitHub Actions. Here we mentioned GitHub Actions which works like a helpful assistant for the team. When we push new code to GitHub, it automatically performs tasks we define, such as testing, building, or deploying your code. It follows the instructions we set, making the development process more efficient.</p><p><strong>Automated Testing:</strong> Automated Testing comes next, where the code is carefully examined for any issues like incomplete coverage or vulnerabilities. Tools like Sonarqube, OWASP, Trivy, Selenium, etc. are employed, and tailored to each project’s unique requirements.</p><p><strong>Deployment:</strong> Once the application is thoroughly checked and approved, it’s time for Deployment. This means putting the app into action after it passes all the tests. Applications have different stages, such as development, QA, staging/beta, and the final production stage, each with its specific environment requirements.</p><p><strong>CI/CD: </strong>Here comes the most demanding and interesting stuff. The above steps combined as a process and is called <em>Continuous Integration/Continuous Delivery,</em> are crucial in DevOps. These processes automate everything from the initial development phase to the final product. Tools like Jenkins, GitHub Actions, AWS CodePipeline BitBucket Pipeline are used to make this happen seamlessly.</p><p><strong>S3/Blob:</strong> These are storage services in the cloud(AWS/Azure) and can be used with our use case. They come into play when users need to upload or retrieve images, videos, documents, etc., providing a reliable and efficient solution.</p><p><strong>Secret Manager:</strong> To secure application env variables/ secrets, secret vault services are used. This service keeps all your app secrets safe and sound, so you don’t have to write them directly into the code, which can be risky. Instead, the application grabs these secrets when it needs them, ensuring everything stays secure and private.</p><p><strong>Database: </strong>For storing application/user data. Which stores all the information the application needs to function.</p><p><strong>Monitoring and Alerting: </strong>Monitoring applications is of utmost importance as it plays a critical role in not only detecting outages but also minimizing downtime through timely alerts. For user-facing, mission-critical applications, proactive monitoring is required. It ensures that potential issues are identified and addressed before they impact the user experience, helping to maintain high availability and reliability.</p><h3><strong>DevSecOps</strong></h3><p>Improving the security of infrastructure and applications is part of the DevSecOps approach. In DevOps, security is also considered, and it eventually evolves into a separate operational level known as DevSecOps. While closely connected to DevOps, the SecOps team primarily concentrates on security matters. In many cases, the DevOps team takes care of security responsibilities, but some companies maintain separate teams for DevOps and SecOps.</p><p>It’s interesting how many new terms have come up, like QAOps for software testing, AIOps for AI, and MLOps for Machine Learning.</p><h3><strong>Sky is the limit</strong></h3><p>Consistent learning and practice are the keys to mastering any craft. While engaging in practical tasks, we often encounter errors, and resolving them enhances our understanding and confidence. When faced with problems, see them as opportunities to grow, challenges that enhance your abilities. Take each problem as a chance to learn, solve it methodically, and document the steps. Every hurdle you overcome adds to your knowledge and makes you stronger. With this approach, your journey of learning becomes not just a process, but a remarkable journey towards expertise and success!</p><p>Engaging in live projects offers us a unique opportunity for rapid learning, enabling us to grasp essential concepts and gain a deeper understanding of application functionality. This hands-on experience not only accelerates our learning curve but also provides valuable insights into how real-world applications operate at a functional level.</p><p>One of the best websites for learning I can recommend is listed below. Give it a try. <a href="https://kodekloud.com/sign-in">https://kodekloud.com/</a></p><p>Thank you for taking the time to read this. Your feedback is highly valued and appreciated.</p><p><strong><em>Happy Learning. Good Luck Folks!</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*UMeLzn-FY4_PcSvM.png" /></figure><h4>👋 If you find this helpful, please click the clap 👏 button below a few times to show your support for the author 👇</h4><h4>🚀<a href="http://from.faun.to/r/8zxxd">Join FAUN Developer Community &amp; Get Similar Stories in your Inbox Each Week</a></h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2cab491af" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/starting-your-devops-journey-tips-and-roadmap-2cab491af">Starting Your DevOps Journey: Tips and Roadmap</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Safeguarding Your AWS Environment: Important Security Considerations]]></title>
            <link>https://aws.plainenglish.io/safeguarding-your-aws-environment-important-security-considerations-6f063ff26aba?source=rss-1b21153322de------2</link>
            <guid isPermaLink="false">https://medium.com/p/6f063ff26aba</guid>
            <dc:creator><![CDATA[Subash S]]></dc:creator>
            <pubDate>Tue, 01 Aug 2023 06:31:02 GMT</pubDate>
            <atom:updated>2023-10-09T10:48:36.166Z</atom:updated>
            <content:encoded><![CDATA[<p>We adopt the practices and procedures to ensure that software is developed, deployed and maintained securely. This includes identifying and mitigating security risks, such as vulnerabilities or threats to the software, and ensuring that the software complies with relevant security standards and regulations. Security operations may involve activities such as threat modeling, security testing, vulnerability scanning, security incident response, and compliance auditing. By prioritizing security throughout the software development lifecycle, security operations help to protect the software and its users from potential security breaches and data loss and maintain the integrity of the software and the organization.</p><p>As a leading cloud provider in the region, <strong>AWS </strong>offers security features and services to host and deploy your applications. The platform-as-a-service model of AWS comes with built-in security and scalability features, enabling customers to effortlessly deploy applications with minimal concerns about the underlying infrastructure.</p><p>In the AWS cloud environment, securing our resources is of utmost importance. AWS emphasizes that security and compliance are a joint responsibility shared between AWS and its customers. <em>AWS&#39;s responsibility “Security of the Cloud” and Customer&#39;s responsibility “Security in the Cloud”</em> This shared model can help relieve the customer&#39;s operational burden as AWS operates, manages, and controls the components from the host operating system and virtualization layer down to the physical security of the facilities in which the service operates.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AAeIj1CprTawYdxhzXHxUQ.jpeg" /></figure><p>The customer assumes responsibility and management of the guest operating system (including updates and security patches), other associated application software as well as the configuration of the AWS-provided security group firewall. Customers should carefully consider the services they choose as their responsibilities vary depending on the services used, the integration of those services into their IT environment, and applicable laws and regulations. Please see the link <a href="https://aws.amazon.com/compliance/shared-responsibility-model/">Shared Responsibility Model</a>. Security is the primary concern at AWS, but all developers/operations have a role to play in developing and deploying secure applications. This covers the shared responsibility model; applying the principles of least privilege; handling sensitive data; IAM roles and resource policies; authorization and authentication; code signing; protecting applications with public endpoints.</p><h3><strong>Securing the Infrastructure</strong></h3><p>Ensuring the security of the application component during the design phase of infrastructure is of utmost importance. Making changes to the existing infrastructure later can be challenging and time-consuming, akin to removing bricks from the foundation, potentially compromising its stability and reliability. Let’s now explore the security features, best practices, and recommendations provided by AWS for both the underlying infrastructure and the application. During this discussion, we’ll focus on maximizing security by thoroughly examining the various components offered by AWS.</p><h3><strong>IAM</strong></h3><p>The principle of least privilege is a powerful part of your security posture. If someone doesn’t need access to your AWS resources, they shouldn’t have access. AWS Identity and Access Management (IAM) enables us to manage access to AWS services and resources securely. Using IAM, we can create and manage AWS users and groups, and use permissions to allow and deny their access to resources.</p><p>The first thing you require is to ensure that we do not use the root user for everyday tasks, even administrative ones. Anyone who has root user credentials for the AWS account has unrestricted access to all the resources in your account, including billing information. Create IAM users and access have been granted with the least privilege, that is granting only the permissions required to perform the required task. As a best practice, please refer to the link <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html">IAM Best Practices</a>. Root user credentials are only used to perform a few account and service management tasks. To view the tasks that require you to sign in as the root user, see tasks that require root user credentials here <a href="https://docs.aws.amazon.com/accounts/latest/reference/root-user-tasks.html">Root User Tasks</a>.</p><p>Enhancing the security of IAM user logins is possible through the implementation of two-factor authentication. Additionally, for third-party applications that require access to perform specific tasks in AWS, it is crucial to restrict the permissions of the AWS Access Key/Secret Key combinations to only what is necessary. To achieve this, an IAM policy should be applied based on the precise requirements of the third-party application. Implement IAM role-based authentication always if it permits by the third party. You can refer <a href="https://medium.com/@subashsasi/utilize-iam-roles-in-github-actions-for-aws-authentication-be8fa8f982dc"><strong>here</strong></a> to one of the use cases(aws +github IDP auth) I wrote in Medium. Make use of the temporary credentials provided by IAM roles and federated users instead of the long-term credentials provided by IAM users and access keys. As a security best practice, it is recommended that regularly rotate (change) IAM user access keys.</p><h4><strong>AWS Security Groups, Network ACLs, and Web ACLs:</strong></h4><p>The role of AWS Security Groups, Network ACLs, and Web ACLs in AWS cloud security operations plays a pivotal role. In our exploration, we will get into the distinctions among these tools and how they collectively contribute to establishing a strong and reliable cloud security framework.</p><p><strong>AWS Security Groups:</strong> Security Groups act as virtual firewalls that control inbound and outbound traffic at the resource level. They allow you to define granular rules, permitting specific traffic based on IP addresses, ports, or protocols. By configuring Security Groups correctly, you can restrict access to only necessary services and applications, reducing the attack surface and minimizing potential security risks.</p><p><strong>Network ACLs:</strong> Network ACLs operate at the subnet level and control traffic between subnets in your VPC. Unlike Security Groups, Network ACLs are stateless, meaning you must define rules for both inbound and outbound traffic separately. These network ACLs provide individual controls that we can customize as a second layer of defense. Properly configuring Network ACLs helps prevent unauthorized access and helps protect sensitive data as it moves across subnets within your VPC.</p><p><strong>RouteTables:</strong> Subnet-specific route tables contain a set of rules, called routes, that are used to determine where network traffic from the subnet or gateway is directed. Route tables are configured in a way that to maximum eliminates unauthorized access to both private and public subnets in AWS VPC.</p><p><strong>Web ACLs:</strong> Web ACLs are specific to Amazon WAF and protect your web applications from common web exploits and attacks, such as cross-site scripting (XSS) and SQL injection. By configuring Web ACLs, you can set up rules to allow or block traffic based on request attributes, IP addresses, or geographic locations. This adds an extra layer of security to your web applications, safeguarding them from malicious activities.</p><p>When it comes to cloud security, remember that a layered defense is the key to a strong security posture. Understanding the differences between these tools is crucial for implementing a comprehensive security strategy in your AWS environment. Developing a comprehensive security strategy within your AWS environment involves adeptly configuring AWS Security Groups, Network ACLs, and Web ACLs. By doing so, you gain the ability to control access, monitor traffic, and fortify your valuable resources against unauthorized access and potential cyber threats.</p><h3><strong>ACM</strong></h3><p>As a managed service, AWS Certificate Manager is protected by AWS global network security. It handles the complexity of creating, storing, and renewing public and private <strong>SSL/TLS</strong> X.509 certificates and keys that protect your AWS websites and applications. ACM also simplifies security management by automating the renewal of expiring certificates</p><p>AWS offers two options to customers deploying managed X.509 certificates. Choose the best one for your needs.</p><ol><li><strong>AWS Certificate Manager (ACM)</strong> — This service is for enterprise customers who need a secure web presence using TLS. ACM certificates are deployed through Elastic Load Balancing, Amazon CloudFront, Amazon API Gateway, and other <a href="https://docs.aws.amazon.com/acm/latest/userguide/acm-services.html">integrated AWS services</a>. The most common application of this kind is a secure public website with significant traffic requirements.</li><li><strong>AWS Private CA</strong> — This service is for enterprise customers building a public key infrastructure (PKI) inside the AWS cloud and is intended for private use within an organization. With AWS Private CA, you can create your own certificate authority (CA) hierarchy and issue certificates with it for authenticating users, computers, applications, services, servers, and other devices. Certificates issued by a private CA cannot be used on the Internet.</li></ol><h3><strong>Network Intrusion Detection</strong></h3><p><strong>Amazon GuardDuty</strong> is a threat detection service that continuously monitors malicious activity and unauthorized behavior to protect the AWS accounts, workloads, and data stored in Amazon S3. With the cloud, the collection and aggregation of account and network activities are simplified, but it can be time-consuming for security teams to continuously analyze event log data for potential threats. With GuardDuty, we have an intelligent solution for continuous threat detection in AWS. The service uses machine learning, anomaly detection, and integrated threat intelligence to identify and prioritize potential threats. Emails have been configured to send alerts when Guardduty detects any malicious activity in AWS resources.</p><h3><strong>Data Security</strong></h3><p>A key security consideration that can be implemented considering your application use case is network layering. Layered networks help logically group similar networking components. They also shrink the potential scope of impact of unauthorized network access. A properly layered network makes it harder for unauthorized users to pivot to additional resources within your AWS environment. Allowing direct access to data stores such as databases is not recommended. That is to isolate sensitive data by network isolation of the database subnet. Network isolation makes the database accessible only on a private IP address range and to only those components that require access to it. The data subnet for each environment can be isolated from the public subnet by network level in AWS. Only components from the private subnet can access the data subnet. The public subnet is ideal for hosting VPN and other public-facing components.</p><h4><strong>Encryption:</strong></h4><p>Enabling encryption at rest and in transit is important. At rest ensures that the volumes underpinning the database and snapshots (if encrypted) can be read outside of the AWS account only with AWS KMS encryption key permissions explicitly granted. Encryption is enabled with customer-managed KMS keys of databases and snapshots that deal with sensitive data. Encryption can be enabled for databases, passwords/secret keys, EBS volumes, etc which deal with sensitive data.</p><h4><strong>Secret Management:</strong></h4><p>Secret management is the practice of securely storing, distributing and managing sensitive information, such as passwords, API keys, certificates, and other credentials. Proper secret management is essential for ensuring the security of applications and infrastructure, as sensitive information can be a valuable target for attackers.</p><h4><strong>AWS SSM Parameter Store and Secret Manager:</strong></h4><p>AWS SSM Parameter store ad secret manager is used to store sensitive values, Sensitive data is encrypted with KMS and decrypted within applications, anytime it needs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/665/1*9tpT_w5FptDFojpav6MwoQ.png" /></figure><p><strong>Secret manager and password rotation schedule:</strong></p><p>AWS Secrets Manager helps you manage, retrieve, and rotate database credentials, application credentials, OAuth tokens, API keys, and other secrets throughout their lifecycles. Secrets Manager helps you improve your security posture because you no longer need hard-coded credentials in the application source code. Storing the credentials in Secrets Manager helps avoid possible compromise by anyone who can inspect your application or the components. You replace hard-coded credentials with a runtime call to the Secrets Manager service to retrieve credentials dynamically when you need them. With Secrets Manager, you can configure an <strong>automatic rotation schedule for your secrets</strong>. This enables you to replace long-term secrets with short-term ones, significantly reducing the risk of compromise. Since the credentials are no longer stored with the application, rotating credentials no longer require updating your applications and deploying changes to application clients. The below diagram shows the setup of the Postgres database with secrets rotation scheduled for 30 days</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*-e63H0XKpuqssaUBjB5soA.png" /></figure><h3>Vulnerability Management</h3><p>Vulnerability management tools help to detect and mitigate vulnerabilities earlier in the SDLC lifecycle, improve security posture, ensure compliance, and reduce costs.</p><h4><strong>Vulnerability Management Tools in AWS</strong></h4><p>Amazon Inspector: Amazon Inspector is an automated and continuous vulnerability management tool. Amazon Inspector automatically discovers workloads hosted in AWS, such as Amazon EC2 instances, containers, and Lambda functions, and scans them for software vulnerabilities and unintended network exposure. Below shows a sample image of listing vulnerabilities in the Inspector AWS console.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/637/1*1Yz_5MdyrTF-jcuThdtwMw.png" /></figure><h3>Sensitive Data Recovery</h3><p>Identifying sensitive data in the application infrastructure environment is essential for ensuring compliance, effective risk management, data protection, and reputation management. By implementing appropriate security controls, organizations can ensure that sensitive data is protected from unauthorized access, misuse, and breaches. Below is an example of a sensitive data recovery dashboard in AWS for data stored in S3 with Amazon Macie.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/723/1*wqNOXT0iLT6Z2Dy0nmQWdQ.png" /></figure><h3>Monitoring and Alerts</h3><p>AWS provides a wide array of monitoring tools, such as Amazon CloudWatch, AWS Config, AWS CloudTrail, and AWS Security Hub, which offer comprehensive insights into the infrastructure, applications, and user activities within the AWS ecosystem. These tools enable security teams to track resource usage, detect configuration changes, monitor API activity, and analyze logs to identify security events and potential risks.</p><p><strong>Enable Cloudtrail: </strong>CloudTrail enables you to monitor your AWS deployments by retrieving a history of AWS API calls for your account, including API calls made via the AWS Management Console, the AWS SDKs, the AWS Command Line Interface, and higher-level Amazon Web Services. You can also identify which users and accounts called the ACM APIs, the source IP address the calls were made from, and when the calls occurred. You can integrate CloudTrail into applications using the API, automate trail creation for your organization, check the status of your trails, and control how administrators turn CloudTrail logging on and off. For more information, see <a href="https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html">Creating a Trail</a>.</p><p>To enhance security, AWS monitoring, and alerting allow organizations to set up custom alarms based on predefined thresholds and security rules. These alarms trigger real-time alerts whenever suspicious activities, unauthorized access attempts, or unusual patterns are detected, empowering security personnel to respond swiftly to potential security breaches.</p><p>One of the scenarios is using the AWS EventBridge for login Alerts, which enables you to capture and process events from various AWS services and integrate them with other AWS resources. Additionally, AWS Simple Notification Service (SNS) allows you to send notifications to different endpoints, such as email, SMS, or mobile push notifications. Once the setup is complete, AWS EventBridge will capture login events from AWS CloudTrail and send them to the SNS topic. Subscribers of the SNS topic will then receive notifications in real-time whenever a login event occurs in their AWS account.</p><p><strong>AWS Security Hub</strong></p><p>AWS Security Hub is a cloud security posture management tool designed to enhance the security of your AWS environment. It provides a comprehensive view of your security, operations, and compliance data, offering a unified perspective of security alerts and posture across all AWS accounts. This enables easy tracking, prioritization, and identification of trends and patterns in security issues. By correlating security findings with operational data like AWS CloudTrail logs and customer logs, Security Hub facilitates a deeper understanding of the root cause of security problems, allowing for quicker corrective action.</p><p>Moreover, it incorporates automated security checks based on AWS best practices and industry standards, aiding in the swift identification and remediation of security vulnerabilities. The platform generates findings from various AWS services and partner products, allowing users to filter, aggregate, and visualize these insights. AWS Security Hub’s benefits include a single view of security posture, root cause identification of security issues, automated security checks for AWS resources, and an easy-to-use console for streamlined management. If you’re seeking to improve your AWS environment’s security, AWS Security Hub is an excellent choice, providing a user-friendly and efficient solution to identify and resolve security concerns promptly.</p><p><strong>Conclusion</strong></p><p>By leveraging AWS security standards, monitoring and alerting services, organizations can strengthen their security posture, effectively detect and respond to security incidents, and maintain compliance with industry regulations. As cloud environments become increasingly complex, having robust monitoring and alerting mechanisms is essential to safeguarding sensitive data and maintaining a resilient security infrastructure in the cloud.</p><h3>In Plain English</h3><p><em>Thank you for being a part of our community! Before you go:</em></p><ul><li><em>Be sure to </em><strong><em>clap</em></strong><em> and </em><strong><em>follow</em></strong><em> the writer! 👏</em></li><li><em>You can find even more content at </em><a href="https://plainenglish.io/"><strong><em>PlainEnglish.io</em></strong></a><strong><em> 🚀</em></strong></li><li><em>Sign up for our </em><a href="http://newsletter.plainenglish.io/"><strong><em>free weekly newsletter</em></strong></a><em>. 🗞️</em></li><li><em>Follow us on </em><a href="https://twitter.com/inPlainEngHQ"><strong><em>Twitter</em></strong></a><strong><em>(X</em></strong>), <a href="https://www.linkedin.com/company/inplainenglish/"><strong><em>LinkedIn</em></strong></a>, <a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong><em>YouTube</em></strong></a>, and <a href="https://discord.gg/in-plain-english-709094664682340443"><strong><em>Discord</em></strong></a><strong><em>.</em></strong></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6f063ff26aba" width="1" height="1" alt=""><hr><p><a href="https://aws.plainenglish.io/safeguarding-your-aws-environment-important-security-considerations-6f063ff26aba">Safeguarding Your AWS Environment: Important Security Considerations</a> was originally published in <a href="https://aws.plainenglish.io/">AWS in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Application Diagnostics: An Insightful Surgical Approach]]></title>
            <link>https://medium.com/@subashsasi/application-diagnostics-an-insightful-surgical-approach-a66b2c0e1817?source=rss-1b21153322de------2</link>
            <guid isPermaLink="false">https://medium.com/p/a66b2c0e1817</guid>
            <dc:creator><![CDATA[Subash S]]></dc:creator>
            <pubDate>Thu, 13 Jul 2023 23:20:37 GMT</pubDate>
            <atom:updated>2023-07-14T00:01:54.431Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IufrfOVS-q52ksF3ggE22w.jpeg" /></figure><p>The term “Surgical approach”, refers to a methodical, precise, and focused approach to problem-solving or decision-making. In the context of technical diagnostics or problem-solving, a surgical approach implies a systematic and targeted method of analysis and intervention to identify and address specific issues within software applications. By drawing parallels between medical diagnostics and application issue diagnosis, we gain insights into the systematic and methodical approach needed to identify and resolve problems in software applications.</p><p>The surgeon carefully examines a patient’s symptoms, performs diagnostic tests, and then proceeds with precise surgical procedures to resolve the underlying problem. A surgical approach in software domains involves a similar level of precision and diligence. The emphasis is on thoroughly understanding the problem, breaking it down into smaller components, and addressing each component with precision and efficiency. It often involves a step-by-step process, careful analysis, and the application of appropriate tools or techniques to diagnose and resolve the issue at hand. This analogy helps us understand the importance of collecting data, analyzing symptoms, generating hypotheses, conducting tests, and providing appropriate treatments for the smooth functioning of applications.</p><p>Application diagnostics mirrors a surgeon’s precise examination of affected areas during a surgical procedure. Let’s explore how application diagnostics can be narrated through a medical lens.</p><p><strong>Symptoms and Observations:</strong></p><p>Similar to a patient’s symptoms, application issues manifest as observable problems, abnormalities, or unexpected behavior. These symptoms can include crashes, slow performance, error messages, or incorrect output. Just as a patient describes their symptoms to a doctor, users or stakeholders provide valuable information about the observed issues in the application.</p><p><strong>Data Collection and Examination:</strong></p><p>In medical diagnostics, doctors collect and examine various data, such as medical history, vital signs, laboratory tests, and imaging scans. Similarly, in application diagnostics, developers/Ops teams collect and analyze data related to the issue. This includes error logs, system logs, any recent changes or deployments, performance metrics, user feedback, and other relevant information that provides insights into the problem.</p><p><strong>Analysis and Hypothesis Generation:</strong></p><p>Medical professionals analyze collected data to generate hypotheses about the underlying causes of the symptoms. Similarly, software developers analyze the data collected during application diagnostics to generate hypotheses about the root causes of the observed issues. This involves examining log files, tracing code execution paths, and reviewing system behavior to identify potential causes. They could perform a thorough code review to identify any potential coding errors, logic flaws, or inefficiencies that could be contributing to the issue and pay attention to relevant sections of code that are closely tied to the observed symptoms. The goal here is to narrow down the possible causes and identify the most likely ones.</p><p><strong>Testing and Validation:</strong></p><p>Doctors perform tests and experiments to validate their hypotheses. Similarly, developers conduct tests and simulations to validate their hypotheses about the application issues. This can involve modifying the code, running test scenarios, or using diagnostic tools to further investigate and validate the hypotheses.</p><p><strong>Diagnosis and Treatment:</strong></p><p>Based on the validated hypotheses, medical professionals arrive at a diagnosis and recommend appropriate treatments. In application diagnostics, developers/ops teams identify the root cause of the issue and determine the necessary actions to resolve it. This may involve fixing code, adjusting configurations, or optimizing system components.</p><p><strong>Documentation and Follow-up:</strong></p><p>Just as medical professionals document the diagnostic process, developers document the steps taken during application diagnostics. This documentation serves as a reference for future troubleshooting and can help in similar cases. It also enables effective collaboration and knowledge sharing among the development/ops team.</p><p><strong>Continuous Improvement:</strong></p><p>Learn from the diagnostic process and implement measures to prevent similar issues in the future. This can include enhancing monitoring capabilities, improving error handling, optimizing system performance, and conducting regular diagnostics by running test cases(automated/manual) to proactively identify potential problems.</p><p><strong>Handling issues that adversely impact business operations/end users:</strong></p><p>Just like doctors prioritize cases based on severity, we do the same when handling issues. When a problem arises, we carefully evaluate its impact on users, systems, and business operations. By prioritizing the issue according to severity and potential impact, we determine the level of urgency. Make sure to convey the impact, current status, and urgency of the situation to the people involved, like the development team, DevOps team, and the management team. Implement temporary workarounds or fixes to mitigate the immediate impact of the issue while a permanent solution is being developed. The dedicated team involves individuals with diverse expertise, including developers, DevOps, and subject matter experts who collaborate to diagnose and try to resolve the issue effectively. Collaboration can bring fresh perspectives and help identify potential solutions. Getting everyone involved in the brainstorming process will ensure you get multiple perspectives when problem-solving.</p><p><strong>Proactive monitoring and Alerting:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DNGsiaKP9OV3m6QIiVNRSQ.jpeg" /></figure><p>Proactive monitoring and alerting for our body are crucial for maintaining good health and preventing potential health issues. By actively monitoring our body’s vital signs, symptoms, and overall well-being, we can detect any abnormalities or early signs of illness. This allows us to take prompt action, seek medical attention if necessary, and make informed decisions about our lifestyle and healthcare. Similar way, the application/infra monitoring tool is not just an extra feature, but something we need. When we monitor proactively and set up alerts based on specific limits, we can save time diagnosing problems, avoid errors from happening again, and catch issues before they become major. Delve into the realm of monitoring, where we keep a watchful eye on the health and performance of our applications. With the power of status codes embedded in our error logs, we can craft alerts that instantly respond to every application response, run test scenarios, or utilize diagnostic tools to simulate and observe application behavior. The goal is to validate the identified issue and ensure that the proposed solution effectively resolves it</p><h4>Now, let’s shift our attention and explore methods to improve our application diagnostic process;</h4><p><strong>Adopting Development/Operations Best Practices:</strong></p><p>Surely, aiming for faster problem resolution by, as previously stated, the presence of a reliable monitoring and observability tool along with the development best practices can significantly reduce the time required for diagnosis. Let’s examine the following scenarios as examples:</p><p><strong>Error Logging:</strong></p><p>To ensure proper error handling in our application, it is essential to utilize HTTP status codes effectively. There are a total of 63 officially defined HTTP status codes, ranging from 100 to 599. However, only 44 of these are commonly used. These status codes can be categorized into five classes:</p><blockquote><strong>Informational (1xx):</strong> Indicates that the request has been received and is being processed.</blockquote><blockquote><strong>Successful (2xx):</strong> Indicates that the request was successfully received, understood, and accepted.</blockquote><blockquote><strong>Redirection (3xx):</strong> Indicates that further action must be taken to complete the request.</blockquote><blockquote><strong>Client Error (4xx):</strong> Indicates that the page is unavailable or the request was invalid or could not be fulfilled due to a client-side error.</blockquote><blockquote><strong>Server Error (5xx):</strong> This indicates that the server encountered an error and could not fulfill the request.</blockquote><p>By organizing monitoring dashboards based on these classes and setting up alerts, we can proactively identify and address issues before end users report them. Following application development best practices will significantly reduce the time required to diagnose application errors, whether they are caused by clients sending invalid requests or mistakes resulting from service throttling their requests.</p><p><strong>Observability and Monitoring:</strong></p><p>In our infrastructure, we have a diverse range of components that play crucial roles in running our applications. This includes servers that host the applications and databases responsible for storing application-related data. Additionally, we have various other components such as load balancers, autoscaling features, failover systems, message brokers, cron-based systems, and external APIs. Consider the scenario, when our application encounters a 502 error, our first course of action is to investigate the issue. We begin by examining the application server logs and, if necessary, debugging the application to gain a more comprehensive understanding of the problem. We also inspect the events of the application components to pinpoint the exact error. Eventually, we discover that the culprit behind the issue was the database connection throttling. It is important to note that this analysis process consumes a significant amount of time as we delve into the entire application flow. This is where the presence of a reliable monitoring and alerting system becomes invaluable.</p><p>Gathering and analyzing application performance monitoring API metrics, such as error rates, latency, and throughput, response time along with server hardware metrics like CPU, memory, and I/O traffic, as well as database performance metrics like disk space, read/write metrics, and connection count, is crucial. We should configure these metrics with various thresholds and set alerts with different severity levels. By doing so, we can stay proactive and well-prepared. For instance, if there is a sudden spike in database connections that surpasses a defined threshold, we receive an alert, enabling us to address the issue even before it impacts the application’s performance.</p><p><strong>Exception Handling:</strong></p><p>Implement proper exception handling mechanisms to gracefully handle errors and prevent application crashes. Use meaningful error messages to facilitate effective problem diagnosis.</p><p><strong>Testing and Quality Assurance:</strong></p><p>Conduct thorough testing, including unit testing, integration testing, and end-to-end testing, to identify and address issues early in the development lifecycle. Implement automated testing frameworks for continuous integration and delivery.</p><p><strong>Performance Optimization:</strong></p><p>Optimize application performance through techniques like caching, database query optimization, and efficient resource utilization. Regularly analyze performance metrics and address bottlenecks to enhance overall application performance.</p><p><strong>Security Measures:</strong></p><p>Implement robust security practices, including secure coding practices, encryption, input validation, and vulnerability scanning. Regularly update dependencies and frameworks to address security vulnerabilities.</p><p><strong>Conclusion:</strong></p><p>By adopting an insightful surgical approach to application diagnostics, developers can streamline the problem-solving process, reduce downtime, and enhance the overall performance and reliability of software applications. They should carefully dissect the application’s code, tracing execution paths and scrutinizing system behavior to identify potential causes of the issue. Once potential causes are identified, developers proceed with testing and validation to confirm their hypotheses. This may involve modifying the code. Furthermore, implementing the best development and operational practices helps streamline the diagnosis process.</p><p>Thank you for reading. Have a good Day! This story can also be read at <a href="https://www.rippletechslice.com/tech-diag.html">https://www.rippletechslice.com/tech-diag.html</a></p><p><em>Images Courtesy: </em><a href="https://www.freepik.com/"><em>https://www.freepik.com/</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a66b2c0e1817" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Utilize IAM Roles in GitHub Actions for AWS Authentication (Federated OIDC)]]></title>
            <link>https://aws.plainenglish.io/utilize-iam-roles-in-github-actions-for-aws-authentication-be8fa8f982dc?source=rss-1b21153322de------2</link>
            <guid isPermaLink="false">https://medium.com/p/be8fa8f982dc</guid>
            <dc:creator><![CDATA[Subash S]]></dc:creator>
            <pubDate>Thu, 13 Jul 2023 17:48:12 GMT</pubDate>
            <atom:updated>2024-10-04T22:42:26.388Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Are you using GitHub as your code repository and AWS as your platform provider?</blockquote><blockquote>Are you still using a plain AWS Access key/Secret Key to authenticate AWS from GitHub?</blockquote><p>If yes, then think about switching to IDP-based authentication for your AWS services from Github. In this post, you are learning how to use federated access to assume a role inside AWS directly from a workflow action file in a GitHub repository.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q7BpEimS4rwFJaFNXPO4cQ.png" /></figure><p>As you may be aware using plain text AWS IAM user keys on any third-party platform is not a secure or recommended method to authenticate to AWS. Referring to <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html">Security best practices in IAM</a>, applications and services should use IAM roles instead of IAM users. Here is the good news, it is possible to configure role-based authentication for GitHub actions using OIDC. <a href="https://docs.github.com/en/actions">GitHub actions</a> are defined as methods that you can use to automate, customize, and run your software development workflows in GitHub.</p><p>In this blog post, I will walk you through the steps needed to configure a specific GitHub repo to assume an individual role in an AWS account to perform changes on AWS resources. We can use an IAM OIDC identity provider when we want to establish trust between an OIDC-compatible IdP (GitHub in our case) and our AWS account.</p><p><a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html">IAM OIDC identity providers</a> are entities in IAM that describe an external identity provider (IdP) service that supports the <a href="http://openid.net/connect/">OpenID Connect</a> (OIDC) standard. Here we will create the OIDC connection, IAM role, trust relationship, and GitHub actions workflow file.</p><p>Prerequisites for the above actions items are listed below;</p><ol><li>You or your organization should have a GitHub account</li><li>An AWS account to configure the identity provider</li><li>You should have the necessary permission in Github and AWS to perform actions to complete this task</li></ol><p><strong>Steps to Perform:</strong></p><ol><li>Create an OIDC provider in your AWS account. This is a trust relationship that allows GitHub to authenticate and be authorized to perform actions in your account.</li><li>Create an <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html">IAM role</a> in your account. You will then scope the IAM role’s trust relationship to the intended parts of your GitHub organization, repository, and branch for GitHub to assume and perform specific actions.</li><li>Assign a minimum level of permissions to the role.</li><li>Edit the Trust policy to streamline access to your GitHub organization</li><li>Create a GitHub Actions workflow</li></ol><p><strong>Step1: Add the Identity Provider to AWS</strong></p><p>Go to your AWS IAM console and add the identity provider</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BuvXUNkejYdlYQ_5PPkxIg.png" /></figure><p>Provide the necessary details as shown below and add the provider;</p><ol><li>For Provider type, choose OpenID Connect.</li><li>For the provider URL, enter the URL of the GitHub OIDC IdP: <a href="https://token.actions.githubusercontent.com/">https://token.actions.githubusercontent.com</a></li><li>Choose Get thumbprint to verify the server certificate of your IdP. (<em>To learn more about OIDC thumbprints, see </em><a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc_verify-thumbprint.html"><em>Obtaining the thumbprint for an OpenID Connect Identity Provider</em></a><em>).</em></li><li>For Audience, enter sts.amazonaws.com. This will allow the <a href="https://docs.aws.amazon.com/STS/latest/APIReference/welcome.html">AWS Security Token Service (AWS STS) API</a> to be called by this IDP.</li><li>Add tags(optional), you can add key–value pairs to help you identify and organize your IdPs.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/1*iH4BDJaNw6YrDkbgp6VS9A.png" /></figure><p><strong>Step2: Adding the role to the identity provider</strong></p><p>Click on the identity provider you just created and click on the assign role button to create a new role or select the existing role for the IDP. Here I selected create a new role and I got the below window. Select the Audience as sts.amazonaws.com and add permissions. For demo purposes, I have created a role named “github-actions-demo” which should perform the actions on the AWS services based on my use case. This role has to perform actions based on the permission policy I added.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OjBW8GKpmes6-KuvP561ug.png" /></figure><p><strong>Step3: Assign the required permissions that the assumed role required to perform on your account</strong></p><p>You can select the existing AWS-managed policies or can create custom permission policies based on what GitHub actions CLI needs to perform on your AWS account with the assumed role. Once you have set the permissions you are good to go. I have added permission that enables GitHub workflow to copy my web frontend build objects to s3 and create invalidation of the CloudFront distribution.</p><p><strong>Step4: Edit the Trust relationship policy</strong></p><p>Edit the Trust relationship policy to streamline access to your GitHub organization. Here I added my GitHub organization as a condition element to the policy to perform the actions on behalf of my GitHub actions. Here I securely limit the access to a single repository to perform actions and that too when only from the main branch. Based on your use case you can leverage the permissions to multiple repos and/or branches. See the below screenshot;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_EJE6gS3qeSSLK5tMdv3jw.jpeg" /></figure><p><strong>Step5: Create a GitHub Actions workflow</strong></p><p>Create a GitHub Actions workflow file in your repository that can invoke actions in your account. I am assuming you are familiar with GitHub actions. If not create a <em>.github/workflow</em> folder on your repo and commit the my-actions.yml. the .yml file name can be your choice, the extension should be .yml. Please find the below action file that I have written to perform the actions for my usecase.</p><p>You might notice that the permissions set is given as <em>id-token:write</em><strong><em>, </em></strong>because you won’t be able to request the OIDC JWT ID token if the permissions setting for id-token is set to read or none. The contents should be set to read to access the repository contents.</p><p>In the AssumeRoleAndCallIdentity session, we are accessing the temporary token from the AWS account assuming the role using OpenID Connect. You have to just change the role arn from the role created in the earlier steps in the AWS IAM console, you can add any name to the session.</p><p>Edit options like env variables, region, and role arn based on your use case. On the deployment part, the build content is copied to s3 and invalidates the CloudFront distribution where I hosted the static website site.</p><pre># This is workflow to copy content to s3 and invalidate cloudfront with GitHub Actions<br>name: s3 cloudfront deployment for project XXXXX<br><br># Controls when the action will run. Invokes the workflow on push events but only for the main branch<br>on:<br>  push:<br>    branches: [ main ]<br>  pull_request:<br>    branches: [ main ]<br><br>env:<br>  <br>  AWS_REGION : us-west-2 #Change to reflect your Region<br>  BUILD_FOLDER : mybuild<br>  S3_BUCKET : githubactions-cloudfront-demo<br>  cloudfrontDistributionid : EJXXXXXXXXXXXXXD8L  <br><br># Permission can be added at job level or workflow level    <br>permissions:<br>      id-token: write   # This is required for requesting the JWT<br>      contents: read    # This is required for actions/checkout<br>jobs:<br>  AssumeRoleAndCallIdentity:<br>    runs-on: ubuntu-latest<br>    steps:<br>      - name: Git clone the repository<br>        uses: actions/checkout@v3<br>      - name: configure aws credentials<br>        uses: aws-actions/configure-aws-credentials@v1.7.5<br>        with:<br>          role-to-assume: arn:aws:iam::xxxxxxxxxxxxx:role/github-actions-demo #change to reflect your IAM role’s ARN<br>          role-session-name: GitHub_to_AWS_via_FederatedOIDC<br>          aws-region: ${{ env.AWS_REGION }}<br>      # RUN AWS Commands<br>      - name: copy the objects to s3 and invalidate cloudfront<br>        run: |<br>          aws s3 sync ${{ env.BUILD_FOLDER }}  s3://${{ env.S3_BUCKET }} --delete<br>          aws cloudfront create-invalidation --distribution-id  ${{ env.cloudfrontDistributionid }} --paths &#39;/*&#39;</pre><p>GitHub actions will get invoked only on the push event on the main branch. So whenever a push event happens workflow continues to run and perform the actions described in the action .yml file.</p><p>Please have a look at the screenshots below;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vXqianOJCC7tfkAr4woJOg.png" /></figure><p>The above screenshot shows the workflow has been in progress. The detailed steps that we described in the action file are completed as shown below;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F_EbzOmG7SHoee3WRrBoZQ.png" /></figure><p><strong>Conclusion:</strong></p><p>By replacing IAM keys with trusted roles, we are adopting secure and best practices recommended by AWS. When we use IAM roles with OIDC identity providers, we have a trusted way to provide access to the AWS resources. GitHub and other OIDC providers can generate <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html">temporary security credentials</a> to update resources and infrastructure inside AWS accounts. With this new IdP in place, you can begin to <a href="https://aws.amazon.com/premiumsupport/knowledge-center/delete-access-key/">delete AWS access keys</a> from your IAM users and use short-term credentials.</p><p>I recommend that you follow the <a href="https://wa.aws.amazon.com/wat.pillar.security.en.html#sec.iaam">AWS Well-Architected Security Pillar IAM directive</a> to use programmatic access to AWS services using temporary and limited-privilege credentials. If you deploy <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers.html">IAM federated roles</a> instead of AWS user access keys, recommended to follow this guideline and issue tokens by the AWS Security Token Service.</p><p>Hope this post is helpful. I appreciate your time to read. If you have feedback on this post, leave a comment below and let me know.</p><p>Good Day!</p><h3>In Plain English 🚀</h3><p><em>Thank you for being a part of the </em><a href="https://plainenglish.io/"><strong><em>In Plain English</em></strong></a><em> community! Before you go:</em></p><ul><li>Be sure to <strong>clap</strong> and <strong>follow</strong> the writer ️👏<strong>️️</strong></li><li>Follow us: <a href="https://twitter.com/inPlainEngHQ"><strong>X</strong></a> | <a href="https://www.linkedin.com/company/inplainenglish/"><strong>LinkedIn</strong></a> | <a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong>YouTube</strong></a> | <a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a> | <a href="https://newsletter.plainenglish.io/"><strong>Newsletter</strong></a></li><li>Visit our other platforms: <a href="https://cofeed.app/"><strong>CoFeed</strong></a> | <a href="https://differ.blog/"><strong>Differ</strong></a></li><li>More content at <a href="https://plainenglish.io/"><strong>PlainEnglish.io</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=be8fa8f982dc" width="1" height="1" alt=""><hr><p><a href="https://aws.plainenglish.io/utilize-iam-roles-in-github-actions-for-aws-authentication-be8fa8f982dc">Utilize IAM Roles in GitHub Actions for AWS Authentication (Federated OIDC)</a> was originally published in <a href="https://aws.plainenglish.io/">AWS in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Datadog | Cloud Monitoring]]></title>
            <link>https://medium.com/innovation-incubator/datadog-cloud-monitoring-bd33bfae6ebc?source=rss-1b21153322de------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd33bfae6ebc</guid>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[cloud-monitoring]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[datadog]]></category>
            <category><![CDATA[dashboard]]></category>
            <dc:creator><![CDATA[Subash S]]></dc:creator>
            <pubDate>Tue, 05 Apr 2022 08:43:07 GMT</pubDate>
            <atom:updated>2023-07-13T23:47:39.861Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pev3FruQSHkcM1-U.png" /></figure><h3>Datadog</h3><p>Datadog is a monitoring, security, and analytics platform for developers, IT operations teams, security engineers, and business users in the cloud age. It can perform effective monitoring of servers, tools, and databases.</p><p>It helps users see inside any stack, at any scale, any app, and anywhere. It has been one of the pioneering tools to have a focus on infrastructure monitoring. The perfect merger of monitoring app performance, infrastructure, logs, and user experience is what makes it special.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*noOtNY1rXHNUm0gx.png" /></figure><p>Datadog dashboard of Nginx</p><h3>Key Features of Datadog</h3><p>Datadog is an enterprise SaaS tool that offers an array of services in the monitoring domain. Some of the key features of the Datadog monitoring platform include:</p><p><strong>Log Management</strong><br>Datadog offers scalable log ingestion and analytics through its log management product. Datadog Log Management unifies logs, metrics, and traces in a single view, giving us rich context for analyzing log data. Whether we’re troubleshooting issues, optimizing performance, or investigating security threats, Logging without Limits provides a cost-effective, scalable approach to centralized log management, so we can get complete visibility across our stack.</p><p>We can search, filter, and analyze log data through its dashboard. We can route all our logs from one central control panel.</p><p><strong>Application performance monitoring</strong><br>Datadog’s APM tool provides end-to-end distributed tracing from frontend devices to databases. By seamlessly correlating distributed traces with frontend and backend data, Datadog APM enables us to monitor service dependencies, reduce latency, and eliminate errors. We can connect the collected traces to infrastructure metrics, network calls, and live processes.</p><p><strong>Security monitoring</strong><br>Using Datadog security monitoring, we can analyze operational and security logs in real-time. It provides built-in threshold and anomaly detection rules to detect threats quickly. With Datadog security monitoring we can analyze operational and security logs in real-time regardless of their volume while utilizing curated, out-of-the-box integrations and rules to detect threats. Developers, security, and operations teams can also leverage detailed observability data to accelerate security investigations in a single, unified platform.</p><p><strong>Network monitoring</strong></p><p>Datadog Network Performance Monitoring provides full visibility into every network component that makes up our on-prem, cloud, and hybrid environments, with little to no overhead. By monitoring the performance of connections among our hosts, services, virtual private clouds (VPCs), and other elements, we can quickly determine when our network is the root cause of any issue.</p><p><strong>Real user monitoring</strong><br>With Datadog’s real user Monitoring, we can have insight into our application’s frontend performance from the perspective of real users. Every user journey is correlated seamlessly with synthetic tests, backend metrics, traces, logs, and network performance data, so we can quickly detect poor user experience and resolve issues with context from across the stack.</p><p><strong>Alerts</strong></p><p>Datadog alerts use tags and machine learning to efficiently identify problems in our infrastructure, applications, and services. Every alert is specific, actionable, and contextual — even in large-scale and highly ephemeral environments — which helps minimize downtime and prevents alert fatigue. And with native SLO and SLA tracking, we can prioritize and address the issues that matter most to our business.</p><h3>Pros and Cons</h3><p>Now let’s check out the pros and cons</p><h3>Pros</h3><ul><li>Support for log aggregation and analytics</li><li>Support for anomaly detection and alerts</li><li>Support for custom metrics and custom Datadog integrations</li></ul><h3>Cons</h3><ul><li>No self-hosted solution</li><li>Complex to use; can be overwhelming for new users</li><li>Limited log analytics due to lack of support for JSON log processing</li></ul><h3>References :</h3><ul><li><a href="https://www.datadoghq.com/product/">https://www.datadoghq.com/product/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd33bfae6ebc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/innovation-incubator/datadog-cloud-monitoring-bd33bfae6ebc">Datadog | Cloud Monitoring</a> was originally published in <a href="https://medium.com/innovation-incubator">Innovation Incubator</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>