<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Asela Pathirage on Medium]]></title>
        <description><![CDATA[Stories by Asela Pathirage on Medium]]></description>
        <link>https://medium.com/@aselapathirage?source=rss-96a2b992c67b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*9Cn8mzcJyZyZHcwM</url>
            <title>Stories by Asela Pathirage on Medium</title>
            <link>https://medium.com/@aselapathirage?source=rss-96a2b992c67b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 16:58:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@aselapathirage/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[SAML to OAuth2: Implementing SAML2 Bearer Grant in WSO2 APIM and IS]]></title>
            <link>https://medium.com/@aselapathirage/saml-to-oauth2-implementing-saml2-bearer-grant-in-wso2-apim-and-is-d57024163730?source=rss-96a2b992c67b------2</link>
            <guid isPermaLink="false">https://medium.com/p/d57024163730</guid>
            <category><![CDATA[oauth2]]></category>
            <category><![CDATA[aml]]></category>
            <category><![CDATA[saml2-bearer]]></category>
            <category><![CDATA[saml2]]></category>
            <category><![CDATA[wso2]]></category>
            <dc:creator><![CDATA[Asela Pathirage]]></dc:creator>
            <pubDate>Sat, 23 May 2026 15:29:22 GMT</pubDate>
            <atom:updated>2026-05-25T07:19:51.296Z</atom:updated>
            <content:encoded><![CDATA[<h4>You already have SSO. Your APIs still want a token. Here’s how to bridge the gap with WSO2.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9O7s6SG1-Wc7uafCW8j03w.png" /></figure><p>Picture this: your users log in once through WSO2 Identity Server. SAML SSO does its job — they’re authenticated, they have a signed assertion proving who they are.</p><p>Then they try to call an API.</p><p>WSO2 API Manager doesn’t care about the SAML assertion. It wants an OAuth2 Bearer token. Suddenly your “single sign-on” requires a second login.</p><p>You look at the workarounds and none of them are good.</p><p><strong>Make the user log in again?</strong> That defeats the whole point of SSO.</p><p><strong>Use client credentials?</strong> Now every API call looks the same regardless of who made it. User identity is gone.</p><p><strong>Store the user’s password and exchange it for a token?</strong> That’s a serious security risk nobody should be taking.</p><p>There has to be a better way — and there is.</p><h3>The SAML2 Bearer Grant</h3><p>RFC 7522 defines an OAuth2 grant type specifically for this situation. Instead of asking for credentials, you hand the SAML assertion directly to the token endpoint. The authorization server validates it, confirms the user’s identity, and issues an OAuth2 access token.</p><p>No second login. No stored credentials. The user’s identity flows through intact.</p><p>WSO2 API Manager supports this grant type out of the box. Combined with WSO2 Identity Server as the SAML IdP, the flow looks like this:</p><pre>1. User logs in via SAML SSO<br>   → IS authenticates them and issues a signed SAML assertion<br>2. App sends the assertion to APIM&#39;s token endpoint<br>   → grant_type=urn:ietf:params:oauth:grant-type:saml2-bearer<br>3. APIM validates the assertion against its configured IS Identity Provider<br>   → checks signature, audience, recipient, and expiry<br>4. APIM issues an OAuth2 access token representing the authenticated user<br>5. App calls the API with that token</pre><p><strong>One login. One assertion. One token.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/772/0*Y3u4ch83y4UZSWPd.png" /><figcaption>Ref: <a href="https://apim.docs.wso2.com/en/4.5.0/manage-apis/design/api-security/oauth2/grant-types/saml-extension-grant/">https://apim.docs.wso2.com/en/4.5.0/manage-apis/design/api-security/oauth2/grant-types/saml-extension-grant/</a></figcaption></figure><h3>What We’re Building</h3><p>This article covers two deployment scenarios for the SAML2 Bearer Grant flow. Both use the same Node.js client app and the same underlying grant type — but the configuration differs depending on how IS and APIM are deployed.</p><p><strong>Scenario 1 — IS and APIM as separate products</strong> IS and APIM run as completely independent, standalone products. IS handles SAML authentication. APIM handles API management and OAuth2 token issuance on its own. APIM is simply configured to trust SAML assertions issued by IS. This is the most common pattern when an organization already has IS in place for enterprise SSO and wants to extend that identity to API access without any deeper integration between the two products.</p><p><strong>Scenario 2 — IS as Resident Key Manager for APIM</strong> IS is configured as the Key Manager for APIM, meaning APIM delegates all OAuth2 token issuance directly to IS. The token endpoint, validation logic, and configuration steps are different in this setup.</p><p>Both scenarios use:</p><ul><li><strong>WSO2 Identity Server 6.1.0</strong> — SAML Identity Provider (port offset 1, running on 9444)</li><li><strong>WSO2 API Manager 4.2.0</strong> — API Gateway (default ports, 9443 / 8243)</li><li><strong>Node.js client app</strong> — handles SAML login, token exchange, and API invocation</li></ul><p>The full flow end to end:</p><pre>Browser  →  GET /login<br>         →  AuthnRequest to IS<br>         →  User authenticates<br>         →  SAMLResponse POSTed to /callback<br>         →  App stores the raw assertion<br>         →  User clicks &quot;Get Token&quot;<br>         →  App decodes saml response, extracts &lt;saml:Assertion&gt;, base64url-encodes it<br>         →  POST to APIM token endpoint (saml2-bearer grant)<br>         →  Token endpoint validates assertion<br>         →  Access token returned<br>         →  User clicks &quot;Invoke API&quot;<br>         →  GET API Gateway with Bearer token<br>         →  API response displayed</pre><h3>Scenario 1 — IS and APIM as Separate Products</h3><h3>Prerequisites</h3><p>Before starting, make sure you have:</p><ul><li>WSO2 IS 6.1.0 running — management console at <a href="https://localhost:9444/carbon">https://localhost:9444/carbon</a></li><li>WSO2 APIM 4.2.0 running — management console at <a href="https://localhost:9443/carbon">https://localhost:9443/carbon</a></li><li>Node.js 16 or later</li><li>keytool (comes with the JDK)</li><li>openssl</li></ul><h3>Step 1 — Clone the Client App</h3><pre>git clone https://github.com/AselaPathirage/saml2-bearer-token-exchange-client.git<br>cd saml2-bearer-token-exchange-client</pre><p>The app is built on Express.js with @node-saml/passport-saml. It handles the SAML login flow, stores the raw assertion in the session after authentication, and lets the user exchange it for an OAuth2 token on demand.</p><p>More on the library choice at the end of this article — it matters for this specific setup.</p><h3>Step 2 — Generate the SP Signing Key Pair</h3><p>The Node.js app acts as a SAML Service Provider. IS will verify that AuthnRequests are signed by the registered SP — so the app needs a key pair, and IS needs the public cert.</p><pre>openssl req -x509 -newkey rsa:2048 \<br>  -keyout certs/sp-private.pem \<br>  -out certs/sp-public.pem \<br>  -days 365 -nodes \<br>  -subj &quot;/CN=SAMLApp&quot;</pre><ul><li>sp-private.pem — stays in the app, signs outgoing SAML requests</li><li>sp-public.pem — goes into IS so it can verify those signatures</li></ul><h3>Step 3 — Export the IS Public Certificate</h3><p>The app needs IS’s public cert to validate the SAMLResponse signature. APIM also needs it to validate assertions during the token exchange.</p><pre>keytool -export \<br>  -alias wso2carbon \<br>  -keystore &lt;IS_HOME&gt;/repository/resources/security/wso2carbon.jks \<br>  -storepass wso2carbon \<br>  -rfc \<br>  -file certs/is-public.pem</pre><p>The -rfc flag outputs PEM format. Both the app and APIM require PEM — without this flag you get DER binary which neither will accept.</p><h3>Step 4 — Import the SP Cert into IS Keystore — Optional</h3><p>IS needs the SP cert in its keystore to verify signed AuthnRequests. Import it under the alias samlapp-sp:</p><pre>keytool -import \<br>  -alias samlapp-sp \<br>  -file certs/sp-public.pem \<br>  -keystore &lt;IS_HOME&gt;/repository/resources/security/wso2carbon.jks \<br>  -storepass wso2carbon \<br>  -noprompt</pre><p>You will reference this alias when configuring the IS Service Provider.</p><p><strong>Note: This step is optional if you are adding the certificate to the IS Service Provider as follows, as this approach is easier.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8JaYwbyXUuAH29-z6mMJJA.png" /></figure><h3>Step 5 — Create the Service Provider in IS</h3><p>Log in to https://localhost:9444/carbon → <strong>Identity → Service Providers → Add</strong></p><p>Name: SAMLApp → <strong>Register</strong></p><p>Expand <strong>Inbound Authentication Configuration → SAML2 Web SSO Configuration → Configure</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XxgU84jKtCFrn143lB08iA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2OGK3Ux79dPAmAwRL0Lb_w.png" /></figure><h4>Issuer and ACS URL</h4><p><strong>Issuer:</strong> SAMLApp</p><p>This is the SP entity ID. It must match SP_ENTITY_ID in the app&#39;s .env and the Service Provider Entity ID you will set in APIM&#39;s Identity Provider config.</p><p><strong>Assertion Consumer URL:</strong> <a href="http://localhost:3000/callback">http://localhost:3000/callback</a></p><p>IS will POST the SAMLResponse to this URL after the user authenticates.</p><h4>Signing and Validation</h4><p><strong>Enable Response Signing</strong> ✓ IS signs the SAMLResponse. The app verifies this signature.</p><p><strong>Enable Signature Validation</strong> ✓ IS will only accept AuthnRequests signed by the registered SP.</p><p><strong>Certificate Alias:</strong> samlapp-sp The alias imported in Step 4. (optional — see step 4)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VgY9OXDv3F6AzKNjoRer1A.png" /></figure><h4>Single Logout</h4><p><strong>Enable Single Logout</strong> ✓</p><p><strong>SLO Response URL:</strong> <a href="http://localhost:3000/logout/callback">http://localhost:3000/logout/callback</a></p><p>When the user logs out, IS invalidates its SSO session and redirects back here so the app can clear its own session.</p><h4>Attribute Profile</h4><p><strong>Enable Attribute Profile</strong> ✓</p><p><strong>Include Attributes in Response Always</strong> ✓</p><p>Ensures user attributes are included in the assertion on every login, not just the first one.</p><h4>Audience and Recipient — How IS and APIM Are Linked</h4><p><strong>Enable Audience Restriction</strong> ✓ <strong>Audience:</strong> idpalias</p><p><strong>Enable Recipient Validation</strong> ✓ <strong>Recipient:</strong> idpalias</p><p>These two settings are how IS and APIM are tied together.</p><p>idpalias is the agreed-upon identifier between IS and APIM. When IS generates an assertion, it includes idpalias in the Audience restriction and as the Recipient in the SubjectConfirmation. When APIM receives the assertion, it checks that its configured Alias appears in both places. If either check fails, the token exchange is rejected.</p><p>The actual value of idpalias doesn&#39;t matter as long as it is identical in all three places:</p><pre>IS SP config  →  Audience:   idpalias<br>IS SP config  →  Recipient:  idpalias<br>APIM IdP      →  Alias:      idpalias<br></pre><h4>The IdP Entity ID Alias — the Most Important Setting for Same-Machine Setups</h4><p><strong>IdP Entity ID Alias:</strong> localhost2</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JKAnBpR09YUeAustlkSHwA.png" /></figure><p>This setting deserves its own explanation.</p><p>Both WSO2 IS and WSO2 APIM default to localhost as their SAML entity ID. When APIM receives a SAML assertion with Issuer = localhost, it cannot tell whether it came from IS or from its own built-in Resident IdP. Validation fails with no clear error.</p><p>Setting <strong>IdP Entity ID Alias = </strong><strong>localhost2</strong> tells IS to sign assertions for this specific SP using localhost2 as the Issuer instead of localhost. You then configure APIM&#39;s Identity Provider with <strong>Identity Provider Entity ID = </strong><strong>localhost2</strong>, and APIM can now uniquely identify IS as the source of the assertion.</p><blockquote><em>If IS and APIM are on separate machines with different hostnames, their entity IDs will differ naturally and this alias is not needed. For same-machine setups it is essential.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l1VSgGQuJiKL72SyZFKbAA.png" /></figure><p>Click <strong>Register</strong>.</p><h3>Step 6 — Create the Identity Provider in APIM</h3><p>Log in to https://localhost:9443/carbon → <strong>Identity → Identity Providers → Add</strong></p><h4>Basic Information</h4><p><strong>Identity Provider Name:</strong> WSO2IS</p><p><strong>Identity Provider Public Certificate:</strong> Upload certs/is-public.pem</p><p>APIM uses this to verify the signature on every assertion it receives. It must be the exact cert exported in Step 3.</p><p><strong>Alias:</strong> idpalias</p><p>The value APIM checks against the assertion’s Audience and Recipient. Must exactly match what was set in IS Step 5.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tcA7otq1x_nETJiOFfILtQ.png" /></figure><h4>Federated Authenticators → SAML2 Web SSO Configuration</h4><p>Enable <strong>SAML2 Web SSO</strong> and fill in the following:</p><p><strong>Service Provider Entity ID:</strong> SAMLApp</p><p><strong>Identity Provider Entity ID:</strong> localhost2 Must match the IdP Entity ID Alias set in IS Step 5.</p><p><strong>SSO URL:</strong> <a href="https://localhost:9444/samlsso">https://localhost:9444/samlsso</a></p><p><strong>Enable Authentication Request Signing</strong> ✓</p><p><strong>Single Logout Profile</strong> ✓</p><p><strong>SLO Response URL:</strong> <a href="http://localhost:3000/logout/callback">http://localhost:3000/logout/callback</a></p><p>Click <strong>Register</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*61WkILVgXEF9dnWx4oHO2w.png" /></figure><h3>Step 7 — Publish an API in APIM</h3><p>Log in to the Publisher at <a href="https://localhost:9443/publisher.">https://localhost:9443/publisher.</a></p><p>Deploy and publish the <strong>PizzaShack</strong> sample API. The gateway endpoint will be:</p><pre>https://localhost:8243/pizzashack/1.0.0/menu</pre><h3>Step 8 — Create an Application and Generate Credentials</h3><p>Log in to the Developer Portal at <a href="https://localhost:9443/devportal.">https://localhost:9443/devportal.</a></p><p>Go to <strong>Applications → Add New Application</strong> and create an application.</p><p>Navigate to <strong>Production Keys → OAuth2 Tokens</strong>. Under <strong>Grant Types</strong>, make sure <strong>SAML2 Bearer</strong> is selected. Click <strong>Generate Keys</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*psVae5Dod6nO2OA8TlE3Ug.png" /></figure><p>Copy the <strong>Consumer Key</strong> and <strong>Consumer Secret</strong>.</p><p>Subscribe the application to the PizzaShack API.</p><h3>Step 9 — Configure the App</h3><pre>cp .env.example .env</pre><p>Edit .env with your values:</p><pre>SESSION_SECRET=change-this-to-a-random-string</pre><pre># SAML Service Provider<br>SP_ENTITY_ID=SAMLApp<br>SP_CALLBACK_URL=http://localhost:3000/callback</pre><pre># WSO2 IS (SAML IdP)<br>IDP_SSO_URL=https://localhost:9444/samlsso<br>IDP_LOGOUT_URL=https://localhost:9444/samlsso<br>SP_LOGOUT_CALLBACK_URL=http://localhost:3000/logout/callback</pre><pre># Certificates<br>IDP_CERT_PATH=./certs/is-public.pem<br>SP_PRIVATE_KEY_PATH=./certs/sp-private.pem<br>SP_CERT_PATH=./certs/sp-public.pem</pre><pre># WSO2 APIM<br>APIM_TOKEN_ENDPOINT=https://localhost:9443/oauth2/token<br>APIM_CONSUMER_KEY=&lt;consumer key from Step 8&gt;<br>APIM_CONSUMER_SECRET=&lt;consumer secret from Step 8&gt;<br>APIM_API_ENDPOINT=https://localhost:8243/pizzashack/1.0.0/menu</pre><pre>PORT=3000</pre><h3>Step 10 — Run the App</h3><pre>npm install<br>node app.js</pre><p>Open http://localhost:3000 in your browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hJ8p9F_h6b444qMC72k9GA.png" /></figure><h3>Step 11 — Test the Full Flow</h3><p><strong>1.</strong> Click <strong>Login with SAML</strong> — you are redirected to the IS login page.</p><p><strong>2.</strong> Authenticate with your IS credentials.</p><p><strong>3.</strong> IS validates the login, signs the assertion, and POSTs the SAMLResponse to /callback. The app validates the response and brings you back to the home page.</p><p><strong>4.</strong> Click <strong>Get Token</strong> — the app extracts the raw &lt;saml:Assertion&gt; element, base64url-encodes it, and sends it to APIM&#39;s token endpoint with the SAML2 Bearer grant type.</p><blockquote>curl — location ‘https://localhost:9444/oauth2/token&#39; \<br> — header ‘Content-Type: application/x-www-form-urlencoded’ \<br> — header ‘Authorization: Basic base64encoded(clientID:clientSecret)’ \<br> — data-urlencode ‘assertion=&lt;base64 url encoded assertion&gt;’ \<br> — data-urlencode ‘grant_type=urn:ietf:params:oauth:grant-type:saml2-bearer’ \<br> — data-urlencode ‘scope=PRODUCTION’</blockquote><p><strong>5.</strong> APIM validates the assertion — checks the signature against IS’s cert, verifies the Audience and Recipient contain idpalias, and confirms the assertion hasn&#39;t expired.</p><p><strong>6.</strong> An OAuth2 access token appears on the page.</p><p><strong>7.</strong> Click <strong>Invoke API</strong> — the app calls PizzaShack with Authorization: Bearer &lt;token&gt;.</p><p><strong>8.</strong> The API response is displayed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iqEWohFy5h1ojl2VO_F-fg.png" /></figure><h3>Scenario 2 — IS as Resident Key Manager for APIM</h3><p>When IS is configured as the Resident Key Manager for APIM, the setup becomes significantly simpler from the client side since the DBs are shared between the 2 servers.</p><p>In Scenario 1, APIM managed its own OAuth2 token issuance and needed to be told about IS as an external Identity Provider. That’s why we had to create an Identity Provider in APIM, configure the idpalias, upload IS&#39;s cert, and match the Audience and Recipient values.</p><p>In this scenario, none of that is needed.</p><p>Because IS is the Key Manager, the client calls IS’s token endpoint directly — bypassing APIM’s token endpoint entirely. IS handles the SAML assertion validation and token issuance itself. There is no external IdP to configure on the APIM side because IS is already the one issuing the token.</p><p>The SAML SSO login works exactly the same way. The user authenticates against IS, the app receives the SAMLResponse, and the raw assertion is stored in the session. The token exchange also works exactly the same way — the app extracts the assertion and posts it with the SAML2 Bearer grant type.</p><p>There are two differences from Scenario 1.</p><p><strong>1. Token endpoint URL in </strong><strong>.env</strong> — points directly to IS instead of APIM:</p><pre>APIM_TOKEN_ENDPOINT=https://localhost:9444/oauth2/token</pre><p><strong>2. Audience and Recipient in the IS SP config</strong> — instead of idpalias, set both to the IS token endpoint URL directly:</p><p><strong>Audience:</strong> <a href="https://localhost:9444/oauth2/token">https://localhost:9444/oauth2/token</a></p><p><strong>Recipient:</strong> <a href="https://localhost:9444/oauth2/token">https://localhost:9444/oauth2/token</a></p><p>In Scenario 1, idpalias was an agreed-upon string between IS and APIM. APIM held that alias and checked that it appeared in the assertion. In Scenario 2, APIM is not involved in token issuance at all — the client calls IS directly. IS validates the assertion itself, and it checks that the assertion&#39;s Audience and Recipient match its own OAuth2 token endpoint URL. That URL is read automatically from IS&#39;s own server configuration (OAuth2TokenEPUrl in identity.xml), so on a default IS 6.1.0 with port offset 1 it resolves to https://localhost:9444/oauth2/token. No shared alias is needed — the token endpoint URL is the anchor.</p><p><strong>What you skip entirely in Scenario 2:</strong></p><ul><li>Creating an Identity Provider in APIM</li><li>Configuring a shared idpalias — the token endpoint URL is used directly instead</li><li>Setting the IdP Entity ID Alias (localhost2) — since APIM is no longer validating the assertion itself, the entity ID conflict doesn&#39;t apply.</li></ul><h3>Troubleshooting</h3><p><strong>Audience or Recipient mismatch</strong> APIM rejects the token exchange. The idpalias value in IS (Audience and Recipient) and APIM (Alias) don&#39;t match exactly. Check all three for typos or extra whitespace.</p><p><strong>Assertion issuer unknown to APIM</strong> APIM cannot identify the IdP. Verify that IS SP <strong>IdP Entity ID Alias</strong> is localhost2 and APIM IdP <strong>Identity Provider Entity ID</strong> is localhost2. Both must match.</p><p><strong>Signature validation failed on AuthnRequest</strong> IS rejected the AuthnRequest. The cert stored under alias samlapp-sp in IS keystore doesn&#39;t match the sp-public.pem the app is signing with. Re-run Step 4 to re-import the correct cert.</p><p><strong>Signature validation error on SAMLResponse</strong> The app can’t verify the SAMLResponse. certs/is-public.pem doesn&#39;t match the IS instance you&#39;re running against. Re-run Step 3 to re-export from the correct IS installation.</p><h3>A Note on the SAML Library</h3><p>The app uses @node-saml/passport-saml v5 — the actively maintained successor to the original passport-saml package.</p><p>The library choice matters here. When IS has Recipient Validation enabled, it includes two &lt;SubjectConfirmation&gt; elements in the assertion — one with the ACS URL as Recipient, and one with the configured Recipient (idpalias). Older versions of passport-saml (v2.x and v3.x) threw an error unconditionally whenever more than one SubjectConfirmation was present.</p><p>@node-saml/passport-saml v5 handles this correctly. It iterates over all SubjectConfirmation elements and accepts the assertion if any one of them is a valid bearer confirmation — which is exactly what RFC 7522 permits.</p><p>If you’re upgrading an existing app from passport-saml:</p><pre>npm uninstall passport-saml<br>npm install @node-saml/passport-saml</pre><p>Update the import:</p><pre>// before<br>const SamlStrategy = require(&#39;passport-saml&#39;).Strategy;<br><br>// after<br>const { Strategy: SamlStrategy } = require(&#39;@node-saml/passport-saml&#39;);</pre><p>And rename two options in the strategy configuration:</p><pre>cert        →  idpCert<br>signingCert →  publicCert</pre><h3>Wrapping Up</h3><p>The SAML2 Bearer Grant bridges the gap between SAML SSO and OAuth2-protected APIs cleanly and securely. With WSO2 IS and APIM, the pieces are all there — it’s mostly a matter of wiring them together correctly.</p><p>The two things that trip people up most often on same-machine setups are the <strong>IdP Entity ID Alias</strong> (localhost2) and the <strong>idpalias</strong> linking IS and APIM. Get those right and the rest follows.</p><p>The full source code for the client app is available at:</p><p><a href="https://github.com/AselaPathirage/saml2-bearer-token-exchange-client"><strong>https://github.com/AselaPathirage/saml2-bearer-token-exchange-client</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d57024163730" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beyond Vanilla Kubernetes: Deploying WSO2 API Manager 4.6.0 Distributed on OpenShift Local]]></title>
            <link>https://medium.com/@aselapathirage/beyond-vanilla-kubernetes-deploying-wso2-api-manager-4-6-0-distributed-on-openshift-local-9de9093e093c?source=rss-96a2b992c67b------2</link>
            <guid isPermaLink="false">https://medium.com/p/9de9093e093c</guid>
            <category><![CDATA[wso2]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[openshift]]></category>
            <category><![CDATA[helm]]></category>
            <category><![CDATA[wso2-api-manager]]></category>
            <dc:creator><![CDATA[Asela Pathirage]]></dc:creator>
            <pubDate>Mon, 04 May 2026 08:34:50 GMT</pubDate>
            <atom:updated>2026-05-04T08:34:50.709Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A practitioner’s guide to setting up OpenShift Local and running WSO2 API Manager 4.6.0 in a fully distributed pattern — covering OpenShift-specific configuration, custom Docker images, Helm chart tuning, and the real issues you’ll encounter along the way.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Lx61qeoyMN3cEDnggHL3BA.png" /></figure><p>OpenShift Local (CRC) makes it practical to run a full OpenShift cluster on a developer machine. In this post, I document how I deployed WSO2 API Manager 4.6.0 in a distributed pattern — Control Plane, Traffic Manager, and Gateway as separate pods — on CRC running on an Apple Silicon Mac. WSO2 provides official Helm charts and documentation for OpenShift deployments, and this is a hands-on companion to those resources, focusing on the step-by-step experience and the specific configuration decisions involved.</p><p>This guide covers the full journey — from installing and configuring OpenShift Local on your machine, to building OpenShift-compatible Docker images, configuring the Helm charts, and getting the WSO2 portals accessible in your browser. It also includes a troubleshooting section covering the key issues encountered during the process, with root causes and fixes for each.</p><p>If you are exploring WSO2 API Manager on OpenShift for local development or testing, this walkthrough should serve as a useful practical reference alongside the official resources.</p><h3>What Is OpenShift Local (CRC)?</h3><p><strong>OpenShift Local</strong> (formerly CodeReady Containers, or CRC) is Red Hat’s tool for running a single-node OpenShift cluster on your local machine. It’s the OpenShift equivalent of Minikube — a full OpenShift 4.x cluster that fits on a developer laptop, giving you the complete platform experience without needing a cloud environment.</p><p>Key characteristics:</p><ul><li>Runs inside a single VM using Apple Virtualization (on Mac), managed transparently by CRC</li><li>Provides the <strong>full OpenShift experience</strong>: HAProxy router, Route resources, SecurityContextConstraints (SCC), and the OpenShift web console</li><li>The VM disk is a sparse/qcow2 image — setting 60GB does not immediately consume 60GB on your Mac; it grows on demand</li><li>DNS for *.apps-crc.testing is handled automatically by CRC&#39;s built-in resolver</li></ul><p><strong>Why OpenShift instead of plain Kubernetes?</strong> OpenShift builds on Kubernetes and adds enterprise-focused capabilities — SecurityContextConstraints (SCCs) for pod security, a built-in HAProxy router with Route resources, an integrated image registry, and enhanced RBAC. If your production target is OpenShift, developing locally on CRC is a practical way to validate your deployment configuration — including SCC requirements, routing setup, and container permission settings — before deploying to a shared environment.</p><p>Understanding those differences is what makes an OpenShift deployment go smoothly — and that is what this guide focuses on.</p><h3>Deployment Architecture</h3><p>WSO2 API Manager supports several deployment patterns. For this guide, we use <a href="https://apim.docs.wso2.com/en/latest/install-and-setup/setup/distributed-deployment/deploying-wso2-api-m-in-a-distributed-setup/">a fully distributed setup</a> where each component runs as an independent pod. This is the pattern most relevant to production OpenShift deployments, where separation of concerns, independent scaling, and component-level resilience matter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vb6w0HEpvqUKdID9.png" /><figcaption>Ref: <a href="https://apim.docs.wso2.com/en/latest/install-and-setup/setup/distributed-deployment/deploying-wso2-api-m-in-a-distributed-setup/">https://apim.docs.wso2.com/en/latest/install-and-setup/setup/distributed-deployment/deploying-wso2-api-m-in-a-distributed-setup/</a></figcaption></figure><h3>Prerequisites</h3><p>Before starting, make sure you have the following installed on your Mac:</p><ul><li><strong>CRC (OpenShift Local)</strong> — download from <a href="https://developers.redhat.com/products/openshift-local/overview">Red Hat’s website</a> (free Red Hat account required)</li><li><strong>Helm 3</strong> — brew install helm</li><li><strong>oc CLI</strong> — comes bundled with CRC</li><li><strong>Docker</strong> (Rancher Desktop, Docker Desktop, or similar) — for building images</li><li><strong>MySQL</strong> running locally — for the APIM databases</li><li>A <strong>Docker Hub account</strong> (or any container registry) — for storing your custom images</li><li>The WSO2 APIM 4.6.0 distribution zip packs:</li><li>wso2am-acp-4.6.0.zip (Control Plane profile)</li><li>wso2am-tm-4.6.0.zip (Traffic Manager profile)</li><li>wso2am-universal-gw-4.6.0.zip (Universal Gateway profile)</li><li>The WSO2 APIM Helm charts — clone from <a href="https://github.com/wso2/helm-apim/tree/4.6.x">WSO2 Helm resources</a></li></ul><h3>Step 1 — Download and Install OpenShift Local</h3><h3>1.1 Download CRC</h3><ol><li>Go to the <a href="https://developers.redhat.com/products/openshift-local/overview">Red Hat OpenShift Local page</a></li><li>Click <strong>Download OpenShift Local</strong> — you’ll need a free Red Hat account to proceed</li><li>Sign in or create a Red Hat account if you don’t have one</li><li>On the download page, select your platform (<strong>macOS</strong> for Apple Silicon or Intel Mac) and download the installer</li><li>Also download the <strong>pull secret</strong> from the same page — this is a JSON file that authenticates your cluster to pull Red Hat images. Keep it handy, you’ll need it during crc setup</li></ol><p>Install the downloaded .pkg (macOS) by double-clicking it. This places the crc binary in /usr/local/bin.</p><p>Verify the installation:</p><pre>crc version</pre><h3>1.2 Install and Configure CRC</h3><p>Run the initial setup:</p><pre>crc setup</pre><p>During setup, CRC will prompt you to provide your <strong>pull secret</strong>. Paste the contents of the pull secret JSON file you downloaded from Red Hat. This step also downloads the CRC bundle (the OpenShift VM image) and configures the hypervisor — it takes several minutes on first run.</p><h3>1.3 Configure Resources</h3><p>CRC defaults are too small to run three WSO2 instances. Before starting, set appropriate resource limits:</p><pre># 12GB RAM — WSO2 is memory-heavy; each instance needs ~1.5GB<br>crc config set memory 12288<br><br># 60GB disk - three Docker images at ~2GB each plus OpenShift overhead<br>crc config set disk-size 60</pre><blockquote><strong><em>Note:</em></strong><em> The disk is a sparse qcow2 image. Setting 60GB doesn’t immediately use 60GB on your Mac — it only grows as data is written.</em></blockquote><p>If you’ve already started CRC with the defaults, you can change memory <strong>without deleting the cluster</strong>:</p><pre>crc config set memory 12288<br>crc stop &amp;&amp; crc start</pre><p>But to change disk size, you must delete and recreate the cluster:</p><pre>crc delete<br>crc config set disk-size 60<br>crc setup<br>crc start</pre><h3>1.4 Start CRC</h3><pre>crc start</pre><p>CRC will print a kubeadmin password at the end — <strong>save it immediately</strong>. The output looks like:</p><pre>Started the OpenShift cluster.<br><br>The server is accessible via web console at:<br>  https://console-openshift-console.apps-crc.testing<br>Log in as administrator:<br>  Username: kubeadmin<br>  Password: PqIS6-PAIi5-9RbG6-JGMeS</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-R8lc7sqRFNFTwlTQRREvQ.png" /><figcaption><em>Terminal output of </em><em>crc start completing successfully, showing the kubeadmin password and console URL</em></figcaption></figure><h3>1.5 Connect to the Cluster</h3><pre># Point oc to the CRC cluster&#39;s kubeconfig<br>export KUBECONFIG=~/.crc/machines/crc/kubeconfig<br><br># Log in as admin<br>oc login -u kubeadmin -p &lt;your-password&gt; https://api.crc.testing:6443</pre><blockquote><strong><em>Why </em></strong><strong><em>export KUBECONFIG?</em></strong><em> By default, </em><em>oc uses </em><em>~/.kube/config. CRC stores its kubeconfig separately. Exporting </em><em>KUBECONFIG ensures </em><em>oc commands hit the right cluster. You can also use </em><em>eval $(crc oc-env) which sets this and adds </em><em>oc to your PATH automatically.</em></blockquote><p>With the cluster running and the CLI connected, the next step is to set up the namespace with the right permissions for WSO2’s containers.</p><h3>Step 2 — Prepare the OpenShift Namespace and Security Context</h3><h3>2.1 Create the Namespace</h3><pre>oc new-project wso2-apim</pre><h3>2.2 Grant the Security Context Constraint</h3><p>OpenShift uses SecurityContextConstraints (SCCs) to control what a pod is allowed to do — including which user IDs it can run as. The default restricted SCC assigns a random UID from a namespace-allocated range at runtime.</p><p>The official WSO2 approach to OpenShift compatibility handles this through the Docker image itself — by granting group ownership to root (GID 0) and setting g=u permissions, the container filesystem becomes accessible to any UID that OpenShift assigns, since all OpenShift-assigned UIDs are members of GID 0. This is covered in detail in Step 3.</p><p>In this setup, anyuid was granted as a precautionary step:</p><pre>oc adm policy add-scc-to-user anyuid -z default -n wso2-apim</pre><blockquote><strong><em>Note:</em></strong><em> The official WSO2 OpenShift documentation does not require </em><em>anyuid — the GID 0 image fix is designed to work with OpenShift&#39;s default arbitrary UID assignment. Granting </em><em>anyuid is not harmful but may not be strictly necessary if your image is correctly prepared. Refer to the </em><a href="https://apim.docs.wso2.com/en/4.6.0/install-and-setup/setup/kubernetes-deployment/openshift/openshift-deployment-overview/#step-1-preparing-the-docker-images"><em>official WSO2 OpenShift deployment documentation</em></a><em> for the definitive guidance on SCC requirements.</em></blockquote><p>With the namespace and security policy in place, the next step is to prepare the container images. This is where most of the OpenShift-specific work lives.</p><h3>Step 3 — Build OpenShift-Compatible Docker Images</h3><h3>3.1 Why Custom Images?</h3><p>The official WSO2 documentation for OpenShift deployments requires building a custom Docker image for two reasons, both confirmed in the official guide:</p><ol><li><strong>OpenShift GID 0 compatibility</strong> — OpenShift injects a random UID at runtime rather than using the UID specified in the Dockerfile. The official approach is to grant group ownership of required directories to root (GID 0) and set g=u permissions — since all OpenShift-assigned UIDs are members of GID 0, this ensures the container filesystem remains writable regardless of which UID is assigned. Without this, the pod will encounter permission errors on startup.</li><li><strong>MySQL JDBC connector</strong> — WSO2 Docker images do not bundle database connectors. A custom image is required to include the connector for your specific database.</li></ol><blockquote><em>For reference, the official WSO2 sample Dockerfile for OpenShift compatibility looks like this:</em></blockquote><blockquote>FROM wso2/wso2am:4.6.0</blockquote><blockquote># Change directory permissions for OpenShift compatibility<br>USER root<br>RUN chgrp -R 0 ${USER_HOME} &amp;&amp; chmod -R g=u ${USER_HOME} \<br> &amp;&amp; chgrp -R 0 ${WSO2_SERVER_HOME} &amp;&amp; chmod -R g=u ${WSO2_SERVER_HOME} \<br> &amp;&amp; chgrp -R 0 ${USER_HOME}/solr &amp;&amp; chmod -R g=u ${USER_HOME}/solr<br>USER wso2carbon</blockquote><blockquote># Copy JDBC MySQL driver<br>COPY mysql-connector.jar ${WSO2_SERVER_HOME}/repository/components/lib</blockquote><blockquote><em>Our approach achieves the same result, built from the base Ubuntu image with the local zip packs.</em></blockquote><h3>3.2 Dockerfile Pattern</h3><p>All three Dockerfiles (ACP, TM, Universal GW) follow the same pattern. Here’s the key section that differs from the official images: Ref: <a href="https://github.com/wso2/docker-apim/tree/4.6.x/dockerfiles/ubuntu"><strong><em>docker-apim</em></strong></a></p><pre># Copy local WSO2 server zip into image (instead of downloading from GitHub)<br>COPY wso2am-acp-4.6.0.23.zip /tmp/wso2server.zip<br># Unzip and normalize directory name<br>RUN unzip -d ${USER_HOME} /tmp/wso2server.zip \<br>    &amp;&amp; EXTRACTED_DIR=$(ls -1 ${USER_HOME} | grep &quot;^${WSO2_SERVER_NAME}-&quot; | head -1) \<br>    &amp;&amp; if [ &quot;${EXTRACTED_DIR}&quot; != &quot;${WSO2_SERVER}&quot; ]; then \<br>         mv ${USER_HOME}/${EXTRACTED_DIR} ${USER_HOME}/${WSO2_SERVER}; \<br>       fi \<br>    &amp;&amp; chown wso2carbon:wso2 -R ${WSO2_SERVER_HOME} \<br>    &amp;&amp; mkdir ${USER_HOME}/wso2-tmp \<br>    &amp;&amp; bash -c &#39;mkdir -p ${USER_HOME}/solr/{indexed-data,database}&#39; \<br>    &amp;&amp; chown wso2carbon:wso2 -R ${USER_HOME}/solr \<br>    &amp;&amp; cp -r ${WSO2_SERVER_HOME}/repository/deployment/server/executionplans ${USER_HOME}/wso2-tmp \<br>    &amp;&amp; rm -f /tmp/wso2server.zip<br># Add MySQL connector<br>RUN curl -Lo ${WSO2_SERVER_HOME}/repository/components/lib/mysql-connector-j-8.4.0.jar \<br>    https://repo1.maven.org/maven2/com/mysql/mysql-connector-j/8.4.0/mysql-connector-j-8.4.0.jar<br># OpenShift compatibility - grant group ownership to root (GID 0)<br># OpenShift runs pods with arbitrary UIDs that are always members of GID 0<br>RUN chgrp -R 0 ${USER_HOME} \<br>    &amp;&amp; chmod -R g=u ${USER_HOME}<br># Run as the WSO2 UID (not root)<br>USER ${USER_ID}</pre><p>The critical OpenShift fix is the chgrp -R 0 and chmod -R g=u block. As confirmed in the official WSO2 OpenShift documentation, changing group ownership to GID 0 is mandatory — without it, the arbitrary UID that OpenShift assigns at runtime cannot write to the directories WSO2 needs (logs, tmp, solr, etc.), and the pod will fail to start.</p><h3>3.3 Why Push to a Container Registry?</h3><p>In this setup, images are built using Rancher Desktop — but Rancher Desktop and CRC cannot run at the same time due to a port 443 conflict. Pushing the images to a registry like Docker Hub means they remain accessible when CRC is running, without needing both tools up simultaneously.</p><p>We use <strong>Docker Hub</strong> with a private repository. To pull from a private registry, OpenShift needs credentials — configured as imagePullSecrets in the Helm values.</p><h3>3.4 Set Up Docker Hub</h3><ol><li>Create a free account at <a href="https://hub.docker.com/">hub.docker.com</a> if you don’t have one</li><li>Create a <strong>private repository</strong> — e.g., your-username/wso2-test</li><li>Generate a <strong>Personal Access Token (PAT)</strong> for authentication:</li></ol><ul><li>Go to <strong>Account Settings → Security → New Access Token</strong></li><li>Give it a name (e.g., crc-deploy) and select <strong>Read, Write, Delete</strong> permissions</li><li>Copy the token — you won’t see it again</li></ul><ol><li>Log in from your terminal:</li></ol><pre>docker login -u your-username<br># Enter your PAT as the password when prompted</pre><h3>3.5 Build and Push</h3><p>Place your zip pack in the same directory as the Dockerfile, then build and push directly with --push: Ref: <a href="https://github.com/wso2/docker-apim/tree/4.6.x/dockerfiles/ubuntu"><strong><em>docker-apim</em></strong></a></p><pre># Control Plane<br>cd dockerfiles/ubuntu/apim-acp/<br>docker buildx build --platform linux/arm64 -t your-username/wso2-test:acp-4.6.0-os --push .<br><br># Traffic Manager<br>cd ../apim-tm/<br>docker buildx build --platform linux/arm64 -t your-username/wso2-test:tm-4.6.0-os --push .<br><br># Universal Gateway<br>cd ../apim-universal-gw/<br>docker buildx build --platform linux/arm64 -t your-username/wso2-test:gw-4.6.0-os --push .</pre><blockquote><strong>--platform linux/arm64</strong> — CRC on Apple Silicon uses Apple Virtualization Framework and runs a native ARM64 VM. Build for arm64 to match the architecture of the CRC cluster.</blockquote><blockquote><strong><em>--push</em></strong><em> — Builds and pushes to the registry in one step using Docker Buildx. Equivalent to </em><em>docker build followed by </em><em>docker push.</em></blockquote><h3>3.6 Configure imagePullSecrets in Helm</h3><p>Since the repository is private, every Helm chart needs credentials to pull the image. The Helm chart handles this by creating a Kubernetes Secret of type kubernetes.io/dockerconfigjson from the credentials you provide in values.yaml:</p><pre>wso2:<br>  deployment:<br>    image:<br>      imagePullSecrets:<br>        enabled: true<br>        username: &quot;your-dockerhub-username&quot;<br>        password: &quot;your-dockerhub-pat-token&quot;   # Use PAT, not your account password<br>      registry: &quot;docker.io&quot;<br>      repository: &quot;your-username/wso2-test&quot;<br>      tag: &quot;acp-4.6.0-os&quot;        # acp-4.6.0-os / tm-4.6.0-os / gw-4.6.0-os<br>      imagePullPolicy: Always</pre><blockquote><strong><em>Use a PAT, not your Docker Hub password.</em></strong><em> PATs can be scoped (read-only is enough for pulling) and revoked independently. Never put your account password in a values file.</em></blockquote><p>With images built, pushed, and credentials configured, the next step is the database layer.</p><h3>Step 4 — Prepare the MySQL Databases</h3><p>WSO2 APIM requires two databases. Create them in your local MySQL instance:</p><pre>CREATE DATABASE apim_db_openshift CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;<br>CREATE DATABASE shared_db_openshift CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;</pre><p>Run the schema scripts from the WSO2 distribution:</p><pre>mysql -u root -p apim_db_openshift &lt; apimgt/mysql.sql<br>mysql -u root -p shared_db_openshift &lt; mysql.sql</pre><p>Allow remote connections from the OpenShift pods (which appear to MySQL as a non-localhost host):</p><pre>GRANT ALL PRIVILEGES ON *.* TO &#39;root&#39;@&#39;%&#39; IDENTIFIED BY &#39;root1234&#39;;<br>FLUSH PRIVILEGES;</pre><blockquote><em>Your host machine’s IP as seen from the CRC VM is typically </em><em>192.168.1.2 — verify with </em><em>ip route or </em><em>ifconfig from inside the VM. Use this IP in your JDBC URLs.</em></blockquote><p>With the databases ready, we need to give WSO2 the keystores it uses for TLS.</p><h3>Step 5 — Create the Keystore Secret in OpenShift</h3><p>WSO2 APIM uses JKS keystores for TLS. Create a Kubernetes secret containing the default keystore and truststore:</p><pre>#Create the secret from the default WSO2 JKS files<br>oc create secret generic apim-keystore-secret \<br>  --from-file=wso2carbon.jks=./wso2carbon.jks \<br>  --from-file=client-truststore.jks=./client-truststore.jks \<br>  -n wso2-apim</pre><p>The wso2carbon.jks and client-truststore.jks files come from any WSO2 APIM distribution under repository/resources/security/.</p><p>With the secret in place, we move to the most configuration-intensive part of this guide — tuning the Helm charts for OpenShift.</p><h3>Step 6 — Configure the Helm Charts for OpenShift</h3><p>Clone the WSO2 APIM Helm charts:</p><pre>git clone https://github.com/wso2/helm-apim.git<br>cd helm-apim/distributed<br>git checkout 4.6.x  </pre><p>You’ll work with three charts:</p><ul><li>control-plane/ — Publisher, DevPortal, Admin portals + Key Manager</li><li>traffic-manager/ — Throttling engine</li><li>gateway/ — API Gateway (Universal GW profile)</li></ul><h3>6.1 Common Changes Across All Three values.yaml Files</h3><p>These settings are required in <strong>all three</strong> charts.</p><p><strong>OpenShift security settings:</strong></p><pre>kubernetes:<br>  openshift:<br>    enabled: true        # Activates OpenShift-specific behaviors in templates<br>  securityContext:<br>    runAsUser: null      # Let OpenShift assign the UID (don&#39;t hardcode 10001)<br>  configMaps:<br>    scripts:<br>      defaultMode: &quot;0457&quot;  # Script files need execute permission for group (OpenShift GID 0)<br>  ingressClass: &quot;&quot;       # Don&#39;t use nginx — OpenShift has its own router (we&#39;ll create Routes manually)</pre><p><strong>Resource settings</strong> (tuned for a single-node CRC cluster):</p><pre>resources:<br>      requests:<br>        memory: &quot;256Mi&quot;<br>        cpu: &quot;250m&quot;<br>      limits:<br>        memory: &quot;1536Mi&quot;<br>        cpu: &quot;1000m&quot;<br>      jvm:<br>        memory:<br>          xms: &quot;256m&quot;<br>          xmx: &quot;1024m&quot;</pre><p><strong>Startup probe — increase failure threshold:</strong></p><pre>startupProbe:<br>      initialDelaySeconds: 60<br>      periodSeconds: 10<br>      failureThreshold: 30   # Gives WSO2 up to 360s to start (default 3 = only 90s)</pre><p><strong>Database configuration</strong> — note the &amp;amp; in JDBC URLs:</p><pre>databases:<br>        type: &quot;mysql&quot;<br>        jdbc:<br>          driver: &quot;com.mysql.cj.jdbc.Driver&quot;<br>        apim_db:<br>          url: &quot;jdbc:mysql://192.168.1.2:3306/apim_db_openshift?useSSL=false&amp;amp;allowPublicKeyRetrieval=true&quot;<br>          username: &quot;root&quot;<br>          password: &quot;root1234&quot;</pre><blockquote><strong><em>Why </em></strong><strong><em>&amp;amp; instead of </em></strong><strong><em>&amp;?</em></strong><em> The Helm chart writes the JDBC URL into an XML datasource configuration file. In XML, </em><em>&amp; is a special character that must be escaped as </em><em>&amp;amp;. If you use a bare </em><em>&amp;, the server throws an XML parsing error on startup and refuses to connect to the database.</em></blockquote><p><strong>TLS keystore and truststore — must set passwords:</strong></p><pre>security:<br>        jksSecretName: &quot;apim-keystore-secret&quot;<br>        keystores:<br>          tls:<br>            enabled: true<br>            name: &quot;wso2carbon.jks&quot;<br>            alias: &quot;wso2carbon&quot;<br>            password: &quot;wso2carbon&quot;<br>            keyPassword: &quot;wso2carbon&quot;<br>        truststore:<br>          name: &quot;client-truststore.jks&quot;<br>          password: &quot;wso2carbon&quot;</pre><blockquote><strong><em>Why explicitly set these passwords?</em></strong><em> The default Helm chart values have </em><strong><em>empty strings</em></strong><em> for both the TLS keystore passwords and the truststore password. WSO2 validates both on startup — an empty password on either causes a “keystore tampered or password incorrect” error and the server refuses to start. The default WSO2 password for both </em><em>wso2carbon.jks and </em><em>client-truststore.jks is </em><em>wso2carbon. Make sure you set </em><strong><em>all three</em></strong><em>: </em><em>keystores.tls.password, </em><em>keystores.tls.keyPassword, and </em><em>truststore.password.</em></blockquote><h3>6.2 Control Plane Specific Configuration (control-plane/values.yaml)</h3><pre>kubernetes:<br>  ingress:<br>    enabled: true   # Critical — controls the hostname used in deployment.toml<br>    controlPlane:<br>      enabled: true<br>      hostname: &quot;am.apps-crc.testing&quot;<br>      annotations:<br>        nginx.ingress.kubernetes.io/backend-protocol: &quot;HTTPS&quot;<br>        nginx.ingress.kubernetes.io/affinity: &quot;cookie&quot;<br>        nginx.ingress.kubernetes.io/session-cookie-name: &quot;route&quot;<br>        nginx.ingress.kubernetes.io/session-cookie-hash: &quot;sha1&quot;<br>wso2:<br>  apim:<br>    configurations:<br>      gateway:<br>        environments:<br>        - name: &quot;Default&quot;<br>          type: &quot;hybrid&quot;<br>          gatewayType: &quot;Regular&quot;<br>          provider: &quot;wso2&quot;<br>          displayInApiConsole: true<br>          description: &quot;This is a hybrid gateway that handles both production and sandbox token traffic.&quot;<br>          showAsTokenEndpointUrl: true<br>          serviceName: &quot;wso2am-gateway-service&quot;<br>          servicePort: 9443<br>          wsHostname: &quot;websocket.apps-crc.testing&quot;<br>          httpHostname: &quot;gw.apps-crc.testing&quot;<br>          websubHostname: &quot;websub.apps-crc.testing&quot;</pre><p>The resulting deployment.toml for the Control Plane:</p><pre>[server]<br>hostname = &quot;am.apps-crc.testing&quot;<br><br>[database.apim_db]<br>type = &quot;mysql&quot;<br>url = &quot;jdbc:mysql://192.168.1.2:3306/apim_db_openshift?useSSL=false&amp;amp;allowPublicKeyRetrieval=true&quot;<br>username = &quot;root&quot;<br>password = &quot;root1234&quot;<br>driver = &quot;com.mysql.cj.jdbc.Driver&quot;<br>[database.shared_db]<br>type = &quot;mysql&quot;<br>url = &quot;jdbc:mysql://192.168.1.2:3306/shared_db_openshift?useSSL=false&amp;amp;allowPublicKeyRetrieval=true&quot;<br>...<br><br>[[apim.gateway.environment]]<br>name = &quot;Default&quot;<br>http_endpoint = &quot;http://gw.apps-crc.testing&quot;<br>https_endpoint = &quot;https://gw.apps-crc.testing&quot;<br>ws_endpoint = &quot;ws://websocket.apps-crc.testing&quot;<br>...<br><br>[apim.devportal]<br>url = &quot;https://am.apps-crc.testing/devportal&quot;</pre><h3>6.3 Traffic Manager Specific Configuration (traffic-manager/values.yaml)</h3><pre>wso2:<br>  apim:<br>    configurations:<br>      km:<br>        serviceUrl: &quot;wso2am-cp-wso2am-acp-service&quot;  # Points to CP<br>        servicePort: 9443<br>      eventhub:<br>        serviceUrl: &quot;wso2am-cp-wso2am-acp-service&quot;<br>        urls:<br>          - &quot;wso2am-cp-wso2am-acp-1-service&quot;         # CP&#39;s per-instance service<br>  deployment:<br>    highAvailability: false   # Single TM instance for local dev</pre><p>The resulting deployment.toml for the Traffic Manager:</p><pre>[server]<br>hostname = &quot;tm.wso2.com&quot;<br>server_role = &quot;traffic-manager&quot;<br><br>[apim.key_manager]<br>service_url = &quot;https://wso2am-cp-wso2am-acp-service:9443/services/&quot;<br><br>[apim.event_hub]<br>service_url = &quot;https://wso2am-cp-wso2am-acp-service:9443/services/&quot;<br>event_listening_endpoints = [&quot;tcp://wso2am-cp-wso2am-acp-1-service:5672&quot;]</pre><h3>6.4 Gateway Specific Configuration (gateway/values.yaml)</h3><pre>wso2:<br>  apim:<br>    configurations:<br>      km:<br>        serviceUrl: &quot;wso2am-cp-wso2am-acp-service&quot;<br>        servicePort: 9443<br>      eventhub:<br>        enabled: true<br>        serviceUrl: &quot;wso2am-cp-wso2am-acp-service&quot;<br>        urls:<br>          - &quot;wso2am-cp-wso2am-acp-1-service&quot;<br>      throttling:<br>        serviceUrl: &quot;wso2am-tm-service&quot;         # Points to TM<br>        urls:<br>          - &quot;wso2am-tm-1-service&quot;<br>  deployment:<br>    replicas: 1<br>    minReplicas: 1</pre><p>The resulting deployment.toml for the Gateway:</p><pre>[server]<br>hostname = &quot;gw.wso2.com&quot;<br>server_role = &quot;gateway-worker&quot;<br><br>[apim.key_manager]<br>service_url = &quot;https://wso2am-cp-wso2am-acp-service:9443/services/&quot;<br><br>[apim.throttling]<br>service_url = &quot;https://wso2am-tm-service:9443/services/&quot;<br>throttle_decision_endpoints = [&quot;tcp://wso2am-tm-1-service:5672&quot;]<br><br>[[apim.throttling.url_group]]<br>traffic_manager_urls = [&quot;tcp://wso2am-tm-1-service:9611&quot;]<br>traffic_manager_auth_urls = [&quot;ssl://wso2am-tm-1-service:9711&quot;]<br><br>[apim.event_hub]<br>service_url = &quot;https://wso2am-cp-wso2am-acp-service:9443/services/&quot;<br>event_listening_endpoints = [&quot;tcp://wso2am-cp-wso2am-acp-1-service:5672&quot;]</pre><h3>Step 7 — Deploy with Helm</h3><p>With all three charts configured, we’re ready to deploy. Both the CP and TM must be up before GW connects to them.</p><pre># 1. Deploy Traffic Manager<br>helm install wso2am-tm ./traffic-manager -n wso2-apim<br><br># 2. Deploy Control Plane<br>helm install wso2am-cp ./control-plane -n wso2-apim<br><br># 3. Deploy Gateway<br>helm install wso2am-gw ./gateway -n wso2-apim</pre><p>Monitor pod status:</p><pre>oc get pods -n wso2-apim -w</pre><p>All three pods should eventually reach 1/1 Running:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/697/1*wp-xiomuq_wJj6Gz80nj9Q.png" /></figure><p>WSO2 APIM takes approximately <strong>2–3 minutes</strong> to start per instance. If a pod is still in 0/1 Running after 5 minutes, check the logs with oc logs &lt;pod-name&gt; -n wso2-apim — the troubleshooting section below covers the most common causes.</p><p>When all three pods show 1/1 Running, the deployment is healthy. The next step is making the services reachable from your browser — and this is where OpenShift diverges significantly from a standard Kubernetes setup.</p><h3>Step 8 — Expose Services with OpenShift Routes</h3><p>This is one of the most OpenShift-specific steps in the entire guide. OpenShift offers several options for exposing services externally: the native HAProxy router with <strong>Route</strong> resources, an nginx Ingress Controller installed as an add-on operator, or the Kubernetes Gateway API with an appropriate implementation. For this local CRC setup, we use Routes — they are built into OpenShift with no additional installation required, and CRC’s HAProxy router handles them out of the box. This avoids the overhead of installing and configuring an additional ingress controller in an already resource-constrained environment.</p><p>The Helm chart generates Ingress resources with ingressClassName: nginx, which OpenShift&#39;s built-in HAProxy router ignores entirely — so even with openshift.enabled: true in values, no routes appear automatically. We create them manually instead. We use <strong>passthrough</strong> TLS termination — the router forwards raw TLS traffic directly to the pod, and WSO2 handles TLS termination itself.</p><pre># Control Plane (Publisher, DevPortal, Admin portals)<br>oc create route passthrough wso2am-cp-route \<br>  --service=wso2am-cp-wso2am-acp-service \<br>  --hostname=am.apps-crc.testing \<br>  --port=9443 \<br>  -n wso2-apim<br># Gateway - HTTPS API traffic<br>oc create route passthrough wso2am-gw-route \<br>  --service=wso2am-gw-wso2am-universal-gw-service \<br>  --hostname=gw.apps-crc.testing \<br>  --port=8243 \<br>  -n wso2-apim<br># Gateway - WebSocket<br>oc create route passthrough wso2am-gw-websocket-route \<br>  --service=wso2am-gw-wso2am-universal-gw-service \<br>  --hostname=websocket.apps-crc.testing \<br>  --port=8099 \<br>  -n wso2-apim<br># Gateway - WebSub<br>oc create route passthrough wso2am-gw-websub-route \<br>  --service=wso2am-gw-wso2am-universal-gw-service \<br>  --hostname=websub.apps-crc.testing \<br>  --port=9021 \<br>  -n wso2-apim</pre><p>Verify:</p><pre>oc get routes -n wso2-apim</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vtiWBAJ7LOOAbD8kX9sn8g.png" /></figure><h3>8.2 Register Hostnames in /etc/hosts</h3><p>CRC automatically sets up DNS for the built-in OpenShift routes (console, OAuth, etc.), but your custom application hostnames — am.apps-crc.testing, gw.apps-crc.testing, etc. — are not registered automatically. Without an explicit entry, your Mac&#39;s DNS resolver won&#39;t know where to send requests for these hostnames, and the browser will fail to connect.</p><p>Add them to /etc/hosts:</p><pre>sudo nano /etc/hosts</pre><p>Add these two lines inside the existing CRC section (or at the end of the file if there is no CRC section):</p><pre>127.0.0.1   am.apps-crc.testing gw.apps-crc.testing<br>127.0.0.1   websocket.apps-crc.testing websub.apps-crc.testing</pre><p>Your CRC section in /etc/hosts will look like this:</p><pre># Added by CRC<br>127.0.0.1   api.crc.testing console-openshift-console.apps-crc.testing oauth-openshift.apps-crc.testing host.crc.testing<br>127.0.0.1   am.apps-crc.testing gw.apps-crc.testing<br>127.0.0.1   websocket.apps-crc.testing websub.apps-crc.testing<br># End of CRC section</pre><blockquote><strong><em>Why </em></strong><strong><em>127.0.0.1?</em></strong><em> CRC&#39;s HAProxy router listens on your Mac&#39;s loopback interface (</em><em>127.0.0.1) and forwards traffic to the correct pod based on the </em><em>Host header. All </em><em>*.apps-crc.testing routes resolve to </em><em>127.0.0.1 — the router does the internal dispatching.</em></blockquote><blockquote><strong><em>Heads up:</em></strong><em> If you run </em><em>crc stop and </em><em>crc start again, CRC may rewrite the </em><em>/etc/hosts file and remove your custom entries. Simply re-add them after each restart.</em></blockquote><h3>Step 9 — Access the WSO2 Portals</h3><p>With Routes created and hostnames registered, open your browser and navigate to the portals:</p><ul><li>PortalURLPublisherhttps://am.apps-crc.testing/publisher</li><li>Developer Portalhttps://am.apps-crc.testing/devportal</li><li>Admin Portalhttps://am.apps-crc.testing/admin</li><li>Carbon Consolehttps://am.apps-crc.testing/carbon</li></ul><p>Login with admin / admin.</p><blockquote><em>You will see a browser certificate warning — this is expected since WSO2 uses a self-signed certificate by default. Accept the warning and proceed.</em></blockquote><p>If the Publisher portal loads and you can log in successfully, the full distributed deployment is working — the Control Plane is serving the portal, authenticating via its Key Manager, and the Gateway has registered itself with the Control Plane.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Iq3UktR1MV5_zbgHKMAvuQ.png" /></figure><h3>Common Issues and How to Resolve Them</h3><p>The issues below are the ones most likely to surface during this deployment. Each is documented with the exact symptom, root cause, and fix — both to help you resolve it quickly and to explain <em>why</em> it happens so you can recognise similar patterns in the future.</p><h3>Issue 1 — oc login Fails Immediately After CRC Start (Port 443 Conflict with Rancher Desktop)</h3><p><strong>Symptom:</strong> oc login fails with &quot;connection reset&quot; error immediately after crc start.</p><p><strong>Root cause:</strong> Rancher Desktop’s Lima VM uses an SSH multiplexer that binds to port 443 on localhost. CRC’s router also requires port 443. When both are running, there is a port conflict.</p><p><strong>Fix:</strong></p><pre># Find the conflicting process<br>sudo lsof -iTCP:443 -sTCP:LISTEN</pre><pre># Kill the SSH mux process (not crc itself)<br>kill &lt;ssh-pid&gt;</pre><pre># Then retry<br>oc login -u kubeadmin -p &lt;password&gt; <a href="https://api.crc.testing:6443/">https://api.crc.testing:6443</a></pre><blockquote><em>Note: This conflict can recur after restarting your Mac or Rancher Desktop. If </em><em>oc login fails with a connection error, check port 443 first.</em></blockquote><h3>Issue 2 — Pod Reaches Running State but Cannot Connect to MySQL (Host-Based Access Denial)</h3><p><strong>Symptom:</strong> Pods reach Running state but logs show:</p><pre>Access denied for user &#39;root&#39;@&#39;aselap.bbrouter&#39; (using password: YES)</pre><p><strong>Root cause:</strong> MySQL’s root user was only permitted from localhost. The pod appears to MySQL with a non-localhost hostname.</p><p><strong>Fix:</strong></p><pre>GRANT ALL PRIVILEGES ON *.* TO &#39;root&#39;@&#39;%&#39; IDENTIFIED BY &#39;root1234&#39;;<br>FLUSH PRIVILEGES;</pre><h3>Issue 3— Server Refuses to Start with “Keystore Tampered” Error (Empty Passwords in Helm Values)</h3><p><strong>Symptom:</strong> Server fails to start with:</p><pre>Keystore was tampered with, or password was incorrect</pre><p><strong>Root cause:</strong> The Helm chart values had empty strings for TLS keystore passwords. WSO2 interprets an empty password as incorrect.</p><p><strong>Fix:</strong> Explicitly set the TLS keystore passwords in values.yaml:</p><pre>keystores:<br>  tls:<br>    enabled: true<br>    password: &quot;wso2carbon&quot;<br>    keyPassword: &quot;wso2carbon&quot;<br>truststore:<br>  password: &quot;wso2carbon&quot;</pre><h3>Issue 4— Pod Shows Evicted Status (CRC Disk Space Exhausted)</h3><p><strong>Symptom:</strong> CP pod shows Evicted status. oc describe node crc shows:</p><pre>Conditions: DiskPressure True</pre><p><strong>Root cause:</strong> The default CRC disk size is 32GB. Three WSO2 images at ~2GB each, plus OpenShift system images, filled the disk.</p><p><strong>Fix:</strong></p><pre>crc delete                    # Must delete — disk resize needs a fresh VM<br>crc config set disk-size 60<br>crc setup<br>crc start</pre><blockquote><em>Remember: after recreating CRC, you’ll need to re-create your namespace, SCC grants, secrets, and Helm releases.</em></blockquote><h3>Issue 5— Pod Restarts Repeatedly Despite WSO2 Starting Successfully (Startup Probe Timeout Too Short)</h3><p><strong>Symptom:</strong> Pod repeatedly restarts. Logs show WSO2 starting successfully but then Shutdown hook triggered around the same time.</p><p><strong>Root cause:</strong> The default startupProbe.failureThreshold: 3 with periodSeconds: 10 gives WSO2 only 90 seconds to start (after the initial delay). WSO2 APIM needs 2–3 minutes on constrained hardware.</p><p><strong>Fix:</strong></p><pre>startupProbe:<br>  initialDelaySeconds: 60<br>  periodSeconds: 10<br>  failureThreshold: 30    # 60 + 30×10 = 360 seconds maximum</pre><h3>Issue 6— Gateway Cannot Reach Control Plane (Helm Release Name Prefix in Service Names)</h3><p><strong>Symptom:</strong> Gateway logs show:</p><pre>Failed to connect to: wso2am-cp-service: Name or service not known</pre><p><strong>Root cause:</strong> Helm release names prefix all service names. The actual service name follows the pattern &lt;release-name&gt;-&lt;chart-service-name&gt;. The release name wso2am-cp and the chart&#39;s service name wso2am-acp-service combine to give wso2am-cp-wso2am-acp-service.</p><p><strong>Fix:</strong> Check actual service names and use them:</p><pre>oc get svc -n wso2-apim</pre><p>Then update all inter-service references accordingly:</p><pre>km:<br>  serviceUrl: &quot;wso2am-cp-wso2am-acp-service&quot;  # not &quot;wso2am-cp-service&quot;<br>eventhub:<br>  serviceUrl: &quot;wso2am-cp-wso2am-acp-service&quot;<br>  urls:<br>    - &quot;wso2am-cp-wso2am-acp-1-service&quot;</pre><h3>Issue 7— Publisher Portal Login Redirects to am.wso2.com Instead of Your Hostname</h3><p><strong>Symptom:</strong> Clicking the Publisher portal login button redirects to <a href="https://am.wso2.com/oauth2/authorize?...">https://am.wso2.com/oauth2/authorize?...</a></p><p><strong>Root cause:</strong> The deployment.toml hostname template:</p><pre>hostname = &quot;{{- if .Values.kubernetes.ingress.enabled -}}...{{- else -}}{{ gatewayAPI.hostname }}{{- end -}}&quot;</pre><p>checks kubernetes.ingress.enabled — a field that doesn&#39;t exist in the default values.yaml structure (only kubernetes.ingress.controlPlane.enabled exists). The falsy check falls to the else branch, which uses the Gateway API hostname defaulting to am.wso2.com.</p><p><strong>Fix:</strong> Add enabled: true directly under kubernetes.ingress:</p><pre>kubernetes:<br>  ingress:<br>    enabled: true    # This is the field the deployment.toml template checks<br>    controlPlane:<br>      enabled: true<br>      hostname: &quot;am.apps-crc.testing&quot;</pre><h3>Summary</h3><p><strong>Key OpenShift differences from vanilla Kubernetes:</strong></p><ol><li>Use SCC anyuid instead of PodSecurityPolicy</li><li>Add chgrp -R 0 &amp;&amp; chmod -R g=u to Dockerfiles for arbitrary UID support</li><li>Set runAsUser: null in values to let OpenShift assign the UID</li><li>Create Routes manually instead of relying on Ingress with nginx class</li><li>Set script defaultMode: &quot;0457&quot; for OpenShift&#39;s group execute requirement</li><li>kubernetes.openshift.enabled: true in Helm values activates OpenShift-specific template behaviors</li></ol><p>Repo: <a href="https://github.com/AselaPathirage/helm-apim/tree/4.6-openshift-local-demo">https://github.com/AselaPathirage/helm-apim/tree/4.6-openshift-local-demo</a></p><h3>Closing Thoughts</h3><p>Setting up WSO2 API Manager in a distributed pattern on OpenShift Local is a useful exercise for understanding how each component interacts and how OpenShift-specific requirements apply to a real WSO2 deployment. This guide has documented that process end-to-end. For production-grade deployments on OpenShift, the official WSO2 documentation provides the recommended configurations for high availability, storage, and security.</p><p>The setup described here is intended for local development and testing. For a production OpenShift deployment, additional considerations apply:</p><ul><li><strong>Persistent storage</strong> for Solr indexing (the Control Plane uses Solr for API search)</li><li><strong>High availability</strong> configuration for the Control Plane and Traffic Manager</li><li><strong>Proper certificate management</strong> — replacing the default self-signed wso2carbon.jks with a CA-signed certificate</li><li><strong>External secret management</strong> — using OpenShift Secrets or a vault integration rather than plaintext credentials in Helm values</li><li><strong>Resource tuning</strong> — the memory and CPU limits used here are conservative; production sizing depends on traffic volume</li></ul><p>The OpenShift-specific patterns documented here — GID 0 filesystem permissions, SCC grants, passthrough Routes, the runAsUser: null setting — apply broadly across WSO2 products and are a useful reference for any WSO2 workload targeting OpenShift.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9de9093e093c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[K8sGPT: Kubernetes Meets GenAI]]></title>
            <link>https://medium.com/@aselapathirage/k8sgpt-kubernetes-meets-genai-46f39918f525?source=rss-96a2b992c67b------2</link>
            <guid isPermaLink="false">https://medium.com/p/46f39918f525</guid>
            <category><![CDATA[k8sgpt]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Asela Pathirage]]></dc:creator>
            <pubDate>Tue, 27 Jan 2026 17:42:58 GMT</pubDate>
            <atom:updated>2026-01-27T17:42:58.269Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NRSYhNfSaTfjIyA0CjPWJw.png" /></figure><p>Kubernetes is the backbone of modern applications, but its complexity can be difficult to manage. As we move toward an <strong>“agentic future”</strong> in DevOps, tools like <strong>K8sGPT</strong> are becoming essential. K8sGPT is an open-source tool and a <strong>CNCF Sandbox project</strong> designed to scan clusters, diagnose issues, and provide explanations in simple English.</p><h3>Why AI-Assisted Management Matters</h3><p>Traditionally, troubleshooting Kubernetes requires manual log analysis and event tracing. Integrating AI shifts the role of an engineer from an executor to a <strong>strategic orchestrator</strong>.</p><p><strong>Key Features of K8sGPT:</strong></p><ul><li><strong>Cluster Scanning:</strong> Quickly identifies potential issues across various resource types.</li><li><strong>Issue Diagnosis:</strong> Uses AI-powered analyzers for pods, services, deployments, and more.</li><li><strong>Plain English Explanations:</strong> Explains complex issues in simple, human-readable language.</li><li><strong>Anonymization:</strong> Masks sensitive data before sending it to AI backends to ensure privacy.</li><li><strong>Actionable Advice:</strong> Provides suggestions and specific commands to fix problems.</li><li><strong>Multiple AI Providers</strong>: K8sGPT can be configured with many different AI providers to suit your team’s needs.</li></ul><h3>Supported AI Backends</h3><p>While this guide uses <strong>OpenAI</strong>, K8sGPT is designed to be flexible. You can configure many other AI agents and backends depending on your needs:</p><ul><li>OpenAI</li><li>Azure OpenAI</li><li>Google Vertex AI</li><li>Amazon Bedrock</li><li>IBM WatsonX</li><li>Local AI</li><li>Ollama</li><li>Cohere</li><li>Hugging Face</li></ul><h3>Basic Commands</h3><p>To use K8sGPT effectively, it is important to understand its core commands. These commands allow you to set up the tool and analyze your environment.</p><p>k8sgpt auth add<strong>- Authenticates</strong> the tool by adding your AI provider API key (e.g., OpenAI).</p><p>k8sgpt analyze<strong>- Scans</strong> the cluster for issues and lists detected errors.</p><p>k8sgpt analyze --explain<strong>- </strong>Requests the AI to provide a <strong>detailed explanation</strong> and a solution for the found errors.</p><p>k8sgpt analyze --filter=[Resource]<strong>- Targets</strong> the analysis to specific resource types like Pod, Service, or Deployment.</p><p>k8sgpt analyze --namespace=[Name]<strong>- Limits</strong> the scan to a specific namespace to reduce noise.</p><p>k8sgpt analyze --anonymize<strong>- Hides</strong> sensitive information (like IP addresses) before sending data to the AI.</p><h3>Getting Started: Setup and Core Commands</h3><p>In this setup, I’ll be using OpenAI in Rancher Desktop K8S Cluster.</p><h3>Prerequisites</h3><p>Before starting, ensure you have the following ready:</p><ul><li><strong>Kubernetes Cluster Access:</strong> You need a running cluster (like Minikube, Kind, or a cloud provider) with administrative access.</li><li><strong>kubectl Installed:</strong> The Kubernetes command-line tool must be installed and configured to communicate with your cluster.</li><li><strong>K8sGPT Installed:</strong> You can find the installation steps for your operating system at the <a href="https://k8sgpt.ai/docs/getting-started/installation">official K8sGPT documentation</a>.</li><li><strong>OpenAI API Key:</strong> You must have your own API key. If you don’t have one, you can obtain it from the <a href="https://platform.openai.com/api-keys">OpenAI Platform</a>.</li></ul><h4>Step 1: Authenticate K8sGPT</h4><p>Add your OpenAI API key to securely store your credentials:</p><pre>k8sgpt auth add</pre><p><em>When prompted, enter your API key.</em></p><h4>Step 2: The Verification Scan</h4><p>Perform a targeted analysis on the k8sgpt namespace to ensure the AI provider is responding correctly:</p><pre>k8sgpt analyze --explain --filter=Service --namespace k8sgpt</pre><blockquote><strong><em>Professional Tip:</em></strong><em> If the validation fails, double-check your OpenAI billing quota, as exceeded limits are a common cause for failure.</em></blockquote><h3>Practical Demonstration: Enhancing WSO2 APIM Deployments with AI</h3><p>To see K8sGPT in action, we will use a WSO2 API Manager (APIM) sample Helm chart as our demonstration case.</p><h4>Deployment Scenario</h4><p>Using the <a href="https://apim.docs.wso2.com/en/latest/install-and-setup/install/deploying-api-manager-with-kubernetes-resources/">official documentation</a>, let&#39;s initiate the Helm installation. To show the tool’s precision, we will assume the container image reference was accidentally left empty in the values.yaml file.</p><pre>helm install --dependency-update apim45 ./ --namespace apim45 --create-namespace</pre><h4>AI-Driven Diagnosis</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*luxEzH_Vr4Sw3M7BPvKcDg.png" /></figure><p>When the pods do not reach a Ready state, we can skip manual inspection and ask K8sGPT for a summary:</p><pre>k8sgpt analyze --explain --filter=Pod --namespace apim45</pre><p><strong>What K8sGPT reveals:</strong></p><p>Instead of a generic error, K8sGPT provides a clear explanation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xAlEV4jpyvRPpDXREL-Tew.png" /></figure><blockquote>“The pod is failing because the container image field is empty. This prevents Kubernetes from identifying the source registry. <strong>Solution:</strong> Update the image repository and tag in your values.yaml.”</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oBnSJuxfvA8wmgDH0x0C9g.png" /></figure><h4>Efficient Remediation</h4><p>Once identified, update the values.yaml and perform a seamless upgrade:</p><pre>helm upgrade apim45 ./ --namespace apim45 --dependency-update</pre><h3>Embracing the Future of Kubernetes Engineering</h3><p>The journey into AI-driven Kubernetes management is a marathon, not a sprint. As these tools become more sophisticated, the role of the engineer is fundamentally changing from performing manual implementation to designing resilient, self-healing architectures. By adopting AI agents today, you are not just simplifying your current troubleshooting; you are positioning yourself at the forefront of a new era in DevOps.</p><p>Preparing for this future requires continuous learning and a willingness to embrace a symbiotic relationship between human expertise and machine intelligence. As AI takes over routine diagnostics, it frees us to focus on high-level strategy and innovation, ultimately making our cloud-native ecosystems more stable and efficient than ever before.</p><p>If you found this guide helpful and want to dive deeper into the world of AI-enhanced operations, you can follow this excellent course: <a href="https://learn.kodekloud.com/user/courses/introduction-to-k8sgpt-and-ai-driven-kubernetes-engineering">Introduction to K8sGPT and AI-Driven Kubernetes Engineering</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=46f39918f525" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Debugging OSGi Issues in WSO2: A Simple, Practical Guide]]></title>
            <link>https://medium.com/@aselapathirage/debugging-osgi-issues-in-wso2-a-simple-practical-guide-6a1dd62328ee?source=rss-96a2b992c67b------2</link>
            <guid isPermaLink="false">https://medium.com/p/6a1dd62328ee</guid>
            <category><![CDATA[osgi]]></category>
            <category><![CDATA[troubleshooting]]></category>
            <dc:creator><![CDATA[Asela Pathirage]]></dc:creator>
            <pubDate>Fri, 19 Dec 2025 04:53:31 GMT</pubDate>
            <atom:updated>2025-12-19T05:02:52.736Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I_lRMwOBUW3ZIXGnkyfewg.png" /></figure><p>If you have ever deployed a custom JAR into a WSO2 product and thought:</p><blockquote><em>“The server started fine… but why isn’t my customization working?”</em></blockquote><p>You have probably run into an <strong>OSGi issue</strong>.</p><p>OSGi is powerful, but when something goes wrong, it can feel like a black box. This post breaks OSGi down into <strong>simple concepts</strong> and shows a <strong>step-by-step approach to troubleshooting OSGi issues in WSO2 products</strong>, without going deep into theory.</p><h3>What Is OSGi (In Simple Terms)?</h3><p>OSGi is a <strong>modular system for Java applications</strong>.</p><p>Instead of one large application, WSO2 products are made up of many <strong>small modules</strong>, called <strong>bundles</strong>.<br> Each bundle:</p><ul><li>Is a JAR file</li><li>Has its own lifecycle (start, stop, fail, restart)</li><li>Can depend on other bundles</li></ul><p>Think of OSGi like <strong>LEGO blocks</strong> 🧱<br>Each block must be present, compatible, and correctly connected for the system to work.</p><h3>How OSGi Issues Usually Appear</h3><p>OSGi problems rarely scream loudly. Instead, they show up as:</p><ul><li>“No errors, but my feature doesn’t work”</li><li>A custom JAR not being picked up</li><li>Class loading errors in logs</li><li>A bundle stuck in INSTALLED or RESOLVED state</li></ul><p>When this happens, <strong>OSGi is a place to look</strong>.</p><h3>Step 1: Confirm the Bundle Is Deployed</h3><p>Custom JARs are typically placed in:</p><pre>&lt;PRODUCT_HOME&gt;/repository/components/dropins/</pre><p>Third-party libraries that are not OSGi bundles usually go into:</p><pre>&lt;PRODUCT_HOME&gt;/repository/components/lib/</pre><p>After copying the files, <strong>restart the server</strong>.</p><h3>Step 2: Enable the OSGi Console</h3><p>Start the server with the OSGi console enabled:</p><pre>./api-manager.sh -DosgiConsole</pre><p>This gives you direct access to the OSGi runtime — your most powerful debugging tool.</p><h3>Step 3: Check Bundle Status</h3><p>Once the server starts, list all bundles:</p><pre>osgi&gt; ss</pre><p>Search for your bundle:</p><pre>osgi&gt; ss | grep &lt;your-bundle-name&gt;</pre><p>Example:</p><pre>135  ACTIVE  example-custom-bundle_1.0.0</pre><h3>What These States Mean</h3><ul><li><strong>ACTIVE</strong> → Bundle is running and fully operational</li><li><strong>RESOLVED</strong> → Dependencies are OK, but the bundle is not yet active. It is ready to be started</li><li><strong>INSTALLED</strong> → The bundle has been installed but not resolved. This state occurs if there are missing dependencies or unresolved constraints</li><li><strong>FAILED</strong> → Activation error</li></ul><p>If it’s not ACTIVE, you’ve found your problem.</p><h3>Step 4: Diagnose the Bundle</h3><p>To understand <em>why</em> a bundle isn’t active:</p><pre>osgi&gt; diag &lt;bundle_id&gt;</pre><p>This command often reveals:</p><ul><li>Missing packages</li><li>Version mismatches</li><li>Unresolved imports</li></ul><p>If OSGi cannot resolve dependencies, the bundle will never start.</p><h3>Step 5: Inspect Bundle Details</h3><p>For deeper inspection:</p><pre>osgi&gt; bundle &lt;bundle_id&gt;</pre><p>This shows:</p><ul><li>Imported packages</li><li>Exported packages</li><li>Current state</li></ul><p>This is especially useful when debugging <strong>dependency conflicts</strong>.</p><h3>Step 6: Don’t Forget Product Configuration</h3><p>Some OSGi components require <strong>explicit configuration</strong> in:</p><pre>&lt;PRODUCT_HOME&gt;/repository/conf/deployment.toml</pre><p>A bundle can be:</p><ul><li>ACTIVE</li><li>Error-free</li><li>Still completely ignored by the product</li></ul><p>If required configuration is missing, the feature won’t appear or work.</p><h3>Step 7: Watch the Logs Carefully</h3><p>Always check:</p><pre>&lt;PRODUCT_HOME&gt;/repository/logs/wso2carbon.log</pre><p>Look for:</p><ul><li>Bundle activation errors</li><li>Class loading issues</li><li>Service registration failures</li></ul><p>Adding clear log lines inside your bundle’s activation logic helps a lot:</p><pre>log.info(&quot;Activating custom bundle...&quot;);<br>log.info(&quot;Custom bundle activated successfully.&quot;);</pre><p>If you don’t see these logs, the bundle was never activated.</p><h3>Step 8: Dependency Compatibility Matters (A Lot)</h3><p>One of the biggest causes of OSGi issues is <strong>dependency mismatch</strong>.</p><p>Best practices:</p><ul><li>Align dependency versions with the product’s official POM</li><li>Avoid packaging libraries already provided by WSO2</li><li>Prefer provided scope where possible</li><li>Ensure third-party libraries are OSGi-compatible</li></ul><p>OSGi is strict — even a <strong>minor version mismatch</strong> can break resolution.</p><h3>A Simple OSGi Troubleshooting Checklist ✅</h3><p>When something doesn’t work:</p><ol><li>Any hidden errors in wso2carbon.log?</li><li>Is the JAR in the correct location?</li><li>Are required configs added?</li><li>Is the bundle ACTIVE?</li><li>Do activation logs appear?</li><li>Does diag show missing dependencies?</li><li>Are dependencies compatible?</li></ol><p>Answer these in order — most issues resolve before step 5.</p><p>OSGi issues are usually not complex, just hidden. A quick check of the bundle state, dependencies, configuration, and logs is often enough to uncover the problem. With a structured approach and regular use of the OSGi console, debugging OSGi becomes a routine task instead of a time-consuming guess.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6a1dd62328ee" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Role-Based Access Control (RBAC): Enhancing API Management with WSO2 APIM 4.3]]></title>
            <link>https://medium.com/@aselapathirage/role-based-access-control-rbac-enhancing-api-management-with-wso2-apim-4-3-wso2-is-7-db329f030ec5?source=rss-96a2b992c67b------2</link>
            <guid isPermaLink="false">https://medium.com/p/db329f030ec5</guid>
            <category><![CDATA[api-management]]></category>
            <category><![CDATA[role-based-access-control]]></category>
            <category><![CDATA[wso2]]></category>
            <dc:creator><![CDATA[Asela Pathirage]]></dc:creator>
            <pubDate>Sun, 15 Dec 2024 04:09:31 GMT</pubDate>
            <atom:updated>2025-03-25T12:26:24.134Z</atom:updated>
            <content:encoded><![CDATA[<h3>Role-Based Access Control (RBAC): Enhancing API Management with WSO2 APIM 4.3 &amp; WSO2 IS 7 (Updated APIM 4.4)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EvsHx14GP6DCWY-inVIX7Q.jpeg" /></figure><h3>What is RBAC?</h3><p>Role-Based Access Control (RBAC) is a foundational security model that restricts system access based on user roles, streamlining permission management and ensuring adherence to organizational policies. Instead of assigning permissions to individual users, RBAC associates them with roles, enabling users to inherit permissions through their assigned roles. This approach simplifies access control, enhances security, and aligns resource access with business responsibilities. In the realm of API management, RBAC is indispensable for enforcing the principle of least privilege, ensuring only authorized users or systems interact with APIs, and mitigating risks associated with unauthorized access.</p><p>Now let’s try this out with WSO2 products. :)</p><h4>1. Configuring IS 7 as 3rd party key manager with APIM 4.3.0</h4><p>First things first! Let’s configure IS 7.0.0 as 3rd party key manager with APIM 4.3.0. You can follow the steps aligned in the official documentation for the steps. [<a href="https://apim.docs.wso2.com/en/4.3.0/administer/key-managers/configure-wso2is7-connector/">1</a>]</p><p>Note: Ensure that the products are updated to the versions listed below or later and that IS7 is correctly configured as a third-party manager following the documentation [1].</p><ul><li>WSO2 API Manager 4.3.0.23.</li><li>WSO2 Identity Server 7.0.0.65.</li></ul><h4>2. Implementation</h4><p>2.1. As the first step, log in to the APIM Management Console and create a new role.</p><p>Identity -&gt; Users and Roles -&gt; Add New Role</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ocbc6QPD2Z8WNO65" /></figure><p>2.2. Next, log in to the Publisher Portal, create or select an API, and start creating a local scope. Provide a name for the scope, assign the created role, and save the changes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5P5qiEAQBaPFstu-" /></figure><p>2.3. Now, let’s add the scope to an API resource and save the API.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3yOxOjCF44YbZb8_" /></figure><p>On the IS7 side, now we will see a new scope and role being created with the binding.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YfPKsZmG-MBFnwb-" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1021/0*lT-Nl0CkSIFxanoe" /></figure><p><strong>Note: If it’s a shared scope, a corresponding IS7 scope will be created once the shared scope is created in APIM.</strong></p><p>Furthermore, we can view the created scopes under the <em>User-defined OAuth2 Resource</em> in the API Resources section of the IS7 Console. Kindly note that only a single API Resource in IS7 has been created, which contains all the scopes created when used with APIM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ym7jL88N1-YL4OEX" /></figure><p>And we won’t need to manually authorize the API resource for the created applications, as the configuration is in the deployment.toml file of IS7 will eliminate the need for manual authorization.</p><blockquote>[oauth]<br>authorize_all_scopes = true</blockquote><p>Once you deploy the API, subscribe to the API using an Application whose keys are generated in the IS7.</p><p>To test the scenario, you can create users with and without the roles with required scopes and generate access tokens requesting the relevant scopes. You can then use these access tokens to invoke the APIs in the Dev Portal.</p><p>Here is a sample curl command to generate an access token using the password grant type:</p><pre>curl --location &#39;https://localhost:9444/oauth2/token&#39; \<br>--header &#39;Content-Type: application/x-www-form-urlencoded&#39; \<br>--data-urlencode &#39;client_id=&lt;client_id&gt;&#39; \<br>--data-urlencode &#39;client_secret=&lt;client_secret&gt;&#39; \<br>--data-urlencode &#39;password=&lt;password&gt;&#39; \<br>--data-urlencode &#39;scope=&lt;scope&gt;&#39; \<br>--data-urlencode &#39;username=&lt;username&gt;&#39; \<br>--data-urlencode &#39;grant_type=password&#39;</pre><p>If you found this guide helpful, feel free to share it with your network or leave your thoughts in the comments below.</p><h3>APIM 4.4.0 UPDATE</h3><p>To create roles in WSO2 Identity Server 7, you need to enable the relevant setting in the Connector Configurations while configuring the Key Manager [2].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*C4gmSukvcAdHV9oP" /></figure><p>Note: Enabling role creation according to this convention is supported from WSO2 API Manager 4.4.0.5 and WSO2 Identity Server 7.0.0.81 update levels onwards.</p><p>Additionally, the following configuration must be enabled in the deployment.toml file of WSO2 IS 7 to allow the creation of system_primary_&lt;roleName&gt; roles:</p><blockquote>[role_mgt]<br>allow_system_prefix_for_role = true</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*36gN66374gjKIS2Y" /></figure><p><strong>Note: </strong>From the Identity Server console, check whether the roles audience is set to Organization for the relevant application for which you are generating the token.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P6lJoqIbVLAKlI4vsX640A.png" /></figure><p>Thank you for reading! :)</p><p><strong>References</strong></p><p>[1] <a href="https://apim.docs.wso2.com/en/4.3.0/administer/key-managers/configure-wso2is7-connector/">https://apim.docs.wso2.com/en/4.3.0/administer/key-managers/configure-wso2is7-connector/</a></p><p>[2] <a href="https://apim.docs.wso2.com/en/4.4.0/administer/key-managers/configure-wso2is7-connector/">https://apim.docs.wso2.com/en/4.4.0/administer/key-managers/configure-wso2is7-connector/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=db329f030ec5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Use of HMS Kits in a Food Delivery App]]></title>
            <link>https://medium.com/@aselapathirage/use-of-hms-kits-in-a-food-delivery-app-4dfd084a9e41?source=rss-96a2b992c67b------2</link>
            <guid isPermaLink="false">https://medium.com/p/4dfd084a9e41</guid>
            <category><![CDATA[huawei]]></category>
            <category><![CDATA[hms]]></category>
            <category><![CDATA[huawei-developers]]></category>
            <category><![CDATA[mobile-app-development]]></category>
            <category><![CDATA[hms-core]]></category>
            <dc:creator><![CDATA[Asela Pathirage]]></dc:creator>
            <pubDate>Thu, 06 Jan 2022 12:26:14 GMT</pubDate>
            <atom:updated>2022-01-06T12:26:14.125Z</atom:updated>
            <content:encoded><![CDATA[<p>Dining out used to be a huge part of our life, but these days, ordering in is the next best thing. You may use a delivery app to order food and drinks online. Food delivery apps are extremely popular and sought after mobile applications, as many hotels and restaurants began offering food delivery services during the pandemic. Food delivery services use a variety of mobile and online programs or apps. Some applications allow you to choose from a number of restaurants, cafes, and shops in your area, while others allow you to select meals and dishes from a specific restaurant or franchise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eAnTh5VkW2wcj7xX1aclqA.jpeg" /></figure><p>This article describes how a food delivery app is going to be developed using some of the Huawei Core kits. HMS Core is a mobile service framework based on Huawei devices and the Android platform that gives app developers access to a variety of service capabilities. It provides Huawei end consumers with fundamental services such as HUAWEI ID, payments, and notifications.</p><p>When considering a food delivery app it should consist of the following features and more.</p><ul><li>Push notifications</li><li>Providing contact information for the delivery person</li><li>Discount programs, cash back programs, and loyalty programs</li><li>Food delivery GPS tracking in real-time</li><li>Simple payment methods</li><li>Search filters</li><li>Profile customization and registration</li><li>Messaging within the app</li><li>Order history</li><li>Ratings and reviews</li><li>Assist with virtual work</li><li>The easy learning curve for the users</li><li>Personal favourites</li></ul><p>Using the Huawei HMS kits to integrate the above features in our app will make the work fast, easy and efficient and have a quality output. Here we look at the kits and their features which helps the application.</p><h4><a href="https://developer.huawei.com/consumer/en/hms/huawei-pushkit"><strong>Push kit</strong></a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/752/1*qf4sc557oE949I71LtrzCQ.jpeg" /></figure><p>Push notifications are a mandatory feature in a food delivery app. Huawei push kit offers responsive, reliable, and precise push services with functions like automated notifications, message receipts, analysis reports, deep linking etc. We can be confident that the app reaches the right users at the right time.</p><h4><a href="https://developer.huawei.com/consumer/en/hms/huawei-accountkit"><strong>Account kit</strong></a></h4><p>To make the consumers retain in the app, the logins, registrations should be fast and secure. The faster a consumer signs in to your app, the more likely they are to use it. There’s no faster or safer way than using their HUAWEI IDs, which use two-factor authentication. It also gives more features like one-click sign-in and easy SMS verifications.</p><h4><a href="https://developer.huawei.com/consumer/en/hms/huawei-MapKit"><strong>Map kit</strong></a><strong> and </strong><a href="https://developer.huawei.com/consumer/en/hms/huawei-locationkit"><strong>Location kit</strong></a></h4><p>Customers may track the location of their food using real-time GPS, which is one of the essential features of food delivery apps. To make this feature more reliable we can use the Map kit and the Location kit. Map kit provides standard maps with layers to customize and allows route planning which supports the drivers to deliver the food on time. Location kit locates the app users’ locations faster and more accurately. It can also be used in navigating the users by providing accurate location information. It also presents a large amount of information about food and drink restaurants.</p><h4><a href="https://developer.huawei.com/consumer/en/hms/huawei-sitekit/"><strong>Site kit</strong></a></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/752/1*YhoxIQg80XugpdUueWJkPA.jpeg" /></figure><p>When we integrate the text search in our app, the Site kit provides core capabilities to developers to quickly build apps more efficiently. Its features like keyword search, nearby place search, place detail search, suggestions, auto-complete will increase the user-friendliness.</p><h4><a href="https://developer.huawei.com/consumer/en/hms/huawei-iap/"><strong>In-App purchases</strong></a></h4><p>Giving proper secure payment methods to the users will make users confident about the app. So In-App purchase kit by Huawei can be used to manage payment methods. It supports multiple payment methods and tracks the order status to ensure that all purchased products are delivered to the users. Its development guide and API reference make the developers work easier.</p><h4><a href="https://developer.huawei.com/consumer/en/hms/huawei-analyticskit"><strong>Analytics kit</strong></a></h4><p>This service offers free data analysis for a wide range of devices and platforms, allowing you to make informed product optimization and marketing decisions based on the behaviour of your consumers. This feature can be used to make recommendations which is a cool feature nowadays.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AWOwhC-qIDFglGN_noCK_Q.png" /></figure><p>Like these kits, we can use more Huawei core kits to make our app run smooth. So, Huawei core kits make it developers easy to integrate the features as required on their domain. You can find more details on the following links.</p><p><a href="https://developer.huawei.com/consumer/en/hms/"><em>https://developer.huawei.com/consumer/en/hms/</em></a></p><p><a href="https://developer.huawei.com/consumer/en/doc/development/hmscore-common-Guides/get-started-hmscore-0000001212585589?fbclid=IwAR1ctgb4tfS03ESaFAaCMYEbXz6nsXg283OuPUyHS-j585q1svKNQS2RzTw"><em>https://developer.huawei.com/consumer/en/doc/development/hmscore-common-Guides/get-started-hmscore-0000001212585589?fbclid=IwAR1ctgb4tfS03ESaFAaCMYEbXz6nsXg283OuPUyHS-j585q1svKNQS2RzTw</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4dfd084a9e41" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>