<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Dinuka Caldera on Medium]]></title>
        <description><![CDATA[Stories by Dinuka Caldera on Medium]]></description>
        <link>https://medium.com/@dinuka.caldera?source=rss-856a3ebca6c9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*78gGa9OZC8L0Wa6xNGimTg.jpeg</url>
            <title>Stories by Dinuka Caldera on Medium</title>
            <link>https://medium.com/@dinuka.caldera?source=rss-856a3ebca6c9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 19:36:27 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dinuka.caldera/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Changing Backend Endpoints at Runtime for API Resources in WSO2 API Manager 4.3.0]]></title>
            <link>https://medium.com/@dinuka.caldera/changing-backend-endpoints-at-runtime-for-api-resources-in-wso2-api-manager-4-3-0-27fa8ed6af9f?source=rss-856a3ebca6c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/27fa8ed6af9f</guid>
            <category><![CDATA[wso2]]></category>
            <category><![CDATA[endpoints]]></category>
            <category><![CDATA[microservices-testing]]></category>
            <category><![CDATA[api]]></category>
            <category><![CDATA[api-management-tool]]></category>
            <dc:creator><![CDATA[Dinuka Caldera]]></dc:creator>
            <pubDate>Wed, 11 Feb 2026 04:06:18 GMT</pubDate>
            <atom:updated>2026-02-11T04:06:18.549Z</atom:updated>
            <content:encoded><![CDATA[<p>Usually, in <strong>WSO2 API Manager</strong>, when you create an API, the backend endpoint is configured at the API level. As a result, any request sent to any resource of that API is forwarded to the same backend endpoint.</p><p>However, there can be scenarios where a single API contains multiple resources, and each resource needs to forward requests to a different backend service. In some cases, only specific resources may need to route requests to a different backend endpoint.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DoKaxzhxrQZeyutnqq0n6Q.png" /></figure><h3>So, how can we achieve this in <strong>WSO2 API Manager 4.3.0</strong>?</h3><p>This can be achieved by using the <strong>dynamic endpoint [1] </strong>, which allows you to set the backend endpoint <strong>dynamically at runtime</strong> through mediation policies.</p><p>Usually, we configure the backend endpoint when creating an API in the <strong>Publisher Portal</strong> of <strong>WSO2 API Manager</strong> by providing the backend endpoint URL, as shown below. When the endpoint is configured in this way, it is <strong>static</strong> and applies to the entire API.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aggpMTAL4G3ZkmiBCyDcNg.png" /></figure><p>This means that when a static endpoint is set at the API level, any request sent to any resource of that API is forwarded to the same configured backend endpoint. In other words, all API resources share the same backend endpoint when a static endpoint configuration is used.</p><p>Dynamic endpoints allow you to overcome this limitation by enabling the backend endpoint to be changed <strong>at runtime</strong> using mediation logic.</p><p><strong>Now, let’s see how we can achieve the above requirement using dynamic endpoints in WSO2 API Manager 4.3.0.</strong></p><p>First, we need to create a <strong>mediation policy</strong>. This mediation policy can then be selected <strong>as the API endpoint when creating the API and configuring the endpoint in the Publisher Portal.</strong></p><p>To achieve this requirement, there are <strong>two possible</strong> approaches in WSO2 API Manager 4.3.0.</p><p>The mediation sequence can be <strong>configured</strong>:</p><p><strong>1.) </strong>At the resource level, or</p><p><strong>2.) </strong>At the API level</p><p>Let’s understand this with an example.</p><p><strong>Assume you have an API with the following resources:</strong></p><ol><li>/resource_A</li><li>/resource_B</li><li>/resource_C</li></ol><p><strong>The routing requirement is:</strong></p><p>Requests to /resource_A → <a href="https://endpoint_a.com/">https://endpoint_A.com</a></p><p>Requests to /resource_B and /resource_C → <a href="https://endpoint_x.com/">https://endpoint_X.com</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J3W8_AYSnuD4-mKUwoKzXw.png" /></figure><p>— — — — — — — — — — — — — — — — — — — -</p><p>Please refer to documentation [2] for detailed steps on creating and adding mediation policies.</p><h4><strong>Approach 1: Resource-Level Mediation Policies</strong></h4><p>In this approach, separate mediation policies are created and attached to the required API resources.</p><p><strong>Mediation Policy for /resource_A</strong></p><blockquote>&lt;sequence xmlns=”<a href="http://ws.apache.org/ns/synapse">http://ws.apache.org/ns/synapse</a>&quot;<br> name=”dynamic_endpoint_policy_for_resource_A”&gt;</blockquote><blockquote>&lt;header name=”To”<br> value=”<a href="https://endpoint_a.com/">https://endpoint_A.com</a>&quot;/&gt;</blockquote><blockquote>&lt;/sequence&gt;</blockquote><p>This mediation policy is attached only to <strong>/resource_A.</strong></p><p><strong>Mediation Policy for /resource_B and /resource_C</strong></p><blockquote>&lt;sequence xmlns=”<a href="http://ws.apache.org/ns/synapse">http://ws.apache.org/ns/synapse</a>&quot;<br> name=”dynamic_endpoint_policy_for_other_resources”&gt;</blockquote><blockquote>&lt;header name=”To”<br> value=”<a href="https://endpoint_x.com/">https://endpoint_X.com</a>&quot;/&gt;</blockquote><blockquote>&lt;/sequence&gt;</blockquote><p>This mediation policy is attached to <strong>/resource_B</strong> and <strong>/resource_C.</strong></p><p><strong>How This Works</strong></p><p>In each mediation policy, a specific backend endpoint is configured by setting the To header.</p><p>As shown above, separate mediation policies are attached to the required resources. Each policy contains a configured endpoint value.</p><p>When a request is received, the mediation policy associated with the invoked resource is executed. Based on the configured To header in that policy, the request is forwarded to the corresponding backend endpoint.</p><p>In other words, the backend endpoint is selected according to the API resource that is being accessed.</p><p>This approach works well. However, it requires multiple mediation policies, and as the number of resources increases, managing these policies can become more complex and harder to maintain.</p><h4><strong>Approach 2: API-Level Mediation Policy (Recommended)</strong></h4><p>Another approach also uses dynamic endpoints. However, instead of defining separate mediation policies at the resource level, a single API-level mediation policy is used.</p><p>This policy identifies the invoked resource at runtime and dynamically sets the appropriate backend endpoint by updating the To header.</p><p>With this approach, routing logic is centralized in one mediation sequence, making it easier to manage and maintain.</p><p><strong>API-Level Mediation Policy Example</strong></p><blockquote>&lt;sequence xmlns=”<a href="http://ws.apache.org/ns/synapse">http://ws.apache.org/ns/synapse</a>&quot;<br> name=”dynamic_endpoint_policy”&gt;</blockquote><blockquote>&lt;filter source=”get-property(‘api.ut.resource’)” regex=”/resource_A”&gt;<br> &lt;then&gt;<br> &lt;header name=”To”<br> value=”<a href="https://endpoint_a.com/">https://endpoint_A.com</a>”/&gt;<br> &lt;/then&gt;<br> &lt;else&gt;<br> &lt;header name=”To”<br> value=”<a href="https://endpoint_x.com/">https://endpoint_X.com</a>”/&gt;<br> &lt;/else&gt;<br> &lt;/filter&gt;</blockquote><blockquote>&lt;/sequence&gt;</blockquote><p><strong>How This Works</strong></p><p>If the request is received for /resource_A, it is routed to <a href="https://endpoint_a.com/">https://endpoint_A.com</a>.</p><p>Requests for all other resources are routed to <a href="https://endpoint_x.com/">https://endpoint_X.com</a>, as defined in the mediation policy.</p><p>Since the routing logic is handled within a single API-level mediation policy, only one policy is required.</p><p>This approach is easier to maintain and scales better, especially when the API contains multiple resources.</p><p>[1] <a href="https://apim.docs.wso2.com/en/4.3.0/design/api-policies/regular-gateway-policies/adding-dynamic-endpoints/">https://apim.docs.wso2.com/en/4.3.0/design/api-policies/regular-gateway-policies/adding-dynamic-endpoints/</a></p><p>[2] <a href="https://apim.docs.wso2.com/en/4.3.0/design/api-policies/create-policy/">https://apim.docs.wso2.com/en/4.3.0/design/api-policies/create-policy/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=27fa8ed6af9f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Configuring HAProxy as a Reverse Proxy for WSO2 API Manager 4.3.0 (Single-Node Deployment)]]></title>
            <link>https://medium.com/@dinuka.caldera/configuring-haproxy-as-a-reverse-proxy-for-wso2-api-manager-4-3-0-single-node-deployment-73f34c22a7b7?source=rss-856a3ebca6c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/73f34c22a7b7</guid>
            <category><![CDATA[haproxy]]></category>
            <category><![CDATA[reverse-proxy]]></category>
            <category><![CDATA[ssl]]></category>
            <category><![CDATA[apim]]></category>
            <category><![CDATA[wso2]]></category>
            <dc:creator><![CDATA[Dinuka Caldera]]></dc:creator>
            <pubDate>Sun, 11 May 2025 16:30:08 GMT</pubDate>
            <atom:updated>2025-05-11T16:30:08.311Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/511/1*ZzIxIxNr7_iwW3vvXMzF3Q.png" /></figure><p>Hi all ,</p><p>This is a reference article that includes the necessary sample HAProxy configuration and WSO2 API Manager configuration. It is intended for readers who have a basic understanding of WSO2 API Manager and HAProxy, and are looking to configure HAProxy as a reverse proxy for WSO2 API Manager. Since I couldn’t find any official documentation or articles detailing this process, I hope this will be a helpful resource for those who need guidance on setting up HAProxy as a reverse proxy for WSO2 API Manager.</p><p><strong>Please note:</strong> I have used <strong>Ubuntu 24.04.2 LTS</strong> for this setup. Therefore, all the commands and configuration examples provided from this point onward are based on my working environment configured on Ubuntu 24.04.2 LTS.So</p><p>Once you have successfully completed the setup, you will be able to access the WSO2 API Manager portals using the following URLs:</p><p><a href="https://knnect.lk/apim/devportal/">https://knnect.lk/apim/devportal/</a><br><a href="https://knnect.lk/apim/publisher/">https://knnect.lk/apim/publisher/</a><br><a href="https://knnect.lk/apim/admin/">https://knnect.lk/apim/admin/</a><br><a href="https://knnect.lk/apim/carbon/">https://knnect.lk/apim/carbon/</a></p><h3>Installing HAProxy on Ubuntu</h3><p>Please use below commands to Install HAProxy</p><pre>sudo apt update<br>sudo apt install haproxy -y</pre><p>To verify that HAProxy was installed successfully, run:</p><pre>haproxy -v</pre><p>If the installation was successful, you should see a response similar to:</p><pre>HAProxy version 2.8.xxxxx</pre><h3>Creating an SSL Certificate for HAProxy</h3><p>Use the following steps — each containing a command you need to run in your terminal — to generate a self-signed SSL certificate for HAProxy. This enables HAProxy to securely serve HTTPS traffic using the generated certificate.</p><p><strong>Step 1: Create an OpenSSL configuration file</strong></p><pre>cat &lt;&lt;EOF &gt; openssl.cnf<br>[req]<br>default_bits = 2048<br>prompt = no<br>default_md = sha256<br>distinguished_name = dn<br>req_extensions = req_ext<br><br>[dn]<br>CN = localhost<br>C = LK<br>ST = Western<br>L = Piliyandala<br>O = DSC<br><br>[req_ext]<br>subjectAltName = @alt_names<br><br>[alt_names]<br>DNS.1 = api.am.wso2.com<br>DNS.2 = gw.am.wso2.com<br>DNS.3 = knnect.lk<br>DNS.4 = office.knnect.com<br>EOF</pre><p>This block creates a file named openssl.cnf.</p><p>It defines the SSL certificate’s subject details (like Common Name and DNS names).</p><p>The subjectAltName section lists the domain names that the certificate will be valid for.</p><p><strong>Step 2: Generate a private key</strong></p><pre>sudo openssl genrsa -out haproxy.key 2048</pre><p>This generates a 2048-bit private key named haproxy.key which is required for SSL encryption.</p><p><strong>Step 3: Create a Certificate Signing Request (CSR)</strong></p><pre>sudo openssl req -new -key haproxy.key -out server.csr -config openssl.cnf</pre><p>Uses the private key and config file to create a CSR (server.csr), which includes the information needed for the SSL certificate.</p><p><strong>Step 4: Generate a self-signed certificate</strong></p><pre>sudo openssl x509 -req -days 3650 -in server.csr -signkey haproxy.key -out haproxy.crt -extfile openssl.cnf -extensions req_ext</pre><p>Creates a self-signed certificate (haproxy.crt) that is valid for 10 years (3650 days).</p><p><strong>Step 5: Verify Subject Alternative Names (SAN)</strong></p><pre>sudo openssl x509 -in haproxy.crt -text -noout | grep -A1 &quot;Subject Alternative Name&quot;</pre><p>Displays the SAN entries in the certificate to confirm that all your DNS names are correctly included</p><p><strong>Step 6: Combine certificate and key into one file</strong></p><pre>sudo cat haproxy.crt haproxy.key &gt; server.pem</pre><p>Combines the certificate and key into a single file (server.pem), which is the format HAProxy expects for SSL termination.</p><p><strong>Step 7: Copy the SSL Certificate to HAProxy’s Configuration Directory</strong></p><pre>sudo cp server.pem /etc/haproxy/certs/</pre><p>This command copies the server.pem file (which contains your SSL certificate and private key) into the /etc/haproxy/certs/ directory. This is the standard location where HAProxy looks for SSL certificate files to use for secure (HTTPS) connections</p><h3>Configure HAProxy as a Reverse Proxy for WSO2 API Manager</h3><p>Open the /etc/haproxy/haproxy.cfg file and <strong>replace its existing contents</strong> with the following configuration. This setup allows HAProxy to act as a reverse proxy for the WSO2 API Manager:</p><pre>global<br>    log /dev/log local0<br>    maxconn 4096<br>    tune.ssl.default-dh-param 2048<br>    ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets<br>    ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5<br><br>defaults<br>    log     global<br>    mode    http<br>    option  httplog<br>    option  dontlognull<br>    timeout connect 5000<br>    timeout client  300000<br>    timeout server  300000<br><br>frontend http-in<br>    bind *:80<br>    # Redirect all HTTP traffic to HTTPS with 301 (permanent)<br>    redirect scheme https code 301 if !{ ssl_fc }<br><br>frontend https-in<br>    bind *:443 ssl crt /etc/haproxy/certs/server.pem alpn h2,http/1.1<br>    mode http<br>    option forwardfor  # Automatically add the X-Forwarded-For header<br><br>    # Host-based routing<br>    acl host_api hdr(host) -i api.am.wso2.com<br>    acl host_gw hdr(host) -i gw.am.wso2.com<br>    acl host_knnect hdr(host) -i knnect.lk office.knnect.com<br><br>    use_backend backend_sslapi if host_api<br>    use_backend backend_sslgw if host_gw<br>    use_backend backend_knnect if host_knnect<br><br>backend backend_sslapi<br>    http-request set-header X-Forwarded-Port 443<br>    http-request set-header X-Forwarded-Proto https<br>    server wso2_sslapi 127.0.0.1:9443 ssl verify none check inter 10s<br><br>backend backend_sslgw<br>    http-request set-header X-Forwarded-Port 443<br>    http-request set-header X-Forwarded-Proto https<br>    server wso2_sslgw 127.0.0.1:8243 ssl verify none check inter 10s<br><br>backend backend_knnect<br>    # Path-based redirects<br>    http-request redirect location /admin/ code 301 if { path,regsub(^/admin$,/admin/,i) }<br>    http-request redirect location /devportal/ code 301 if { path,regsub(^/devportal$,/devportal/,i) }<br>    http-request redirect location /publisher/ code 301 if { path,regsub(^/publisher$,/publisher/,i) }<br><br>    # Header modifications<br>    http-request set-header X-Forwarded-Host %[req.hdr(Host)]<br>    http-request set-header X-Forwarded-Server %[req.hdr(Host)]<br>    http-request set-header X-Forwarded-Port 443<br>    http-request set-header X-Forwarded-Proto https<br><br>    # Path rewriting for /apim/ prefix<br>    http-request replace-path /apim/(.*) /\1<br><br>    # Backend server with health check<br>    server knnect_local localhost:9443 ssl verify none check inter 10s</pre><p>Once the file is saved, run the following commands to restart and enable HAProxy</p><pre>sudo systemctl restart haproxy<br>sudo systemctl enable haproxy</pre><h3><strong>Update the /etc/hosts File for Local Domain Resolution</strong></h3><p>Open the /etc/hosts file with a text editor:</p><pre>sudo nano /etc/hosts</pre><p>Then add the following lines:</p><pre>127.0.0.1 api.am.wso2.com<br>127.0.0.1 gw.am.wso2.com<br>127.0.0.1 knnect.lk<br>127.0.0.1 office.knnect.com</pre><p>Adding entries to the /etc/hosts file ensures that <strong>any request to those domain names is routed to </strong><strong>127.0.0.1 (your local machine)</strong>, where HAProxy is running.</p><h3>Update WSO2 API Manager Configurations for HAProxy Compatibility</h3><p>1.) Update the API Manager configuration — deployment.toml</p><p>Open repository/conf/deployment.toml and add or update the following configurations.</p><pre>[server]<br>hostname = &quot;knnect.lk&quot;<br>node_ip = &quot;127.0.0.1&quot;<br>mode = &quot;single&quot; #single or ha<br>base_path = &quot;${carbon.protocol}://${carbon.host}:${carbon.management.port}/apim&quot;<br>server_role = &quot;default&quot;<br>proxy_context_path = &quot;/apim&quot;<br><br>[apim.devportal]<br>url = &quot;https://knnect.lk/apim/devportal&quot;<br><br>[transport.https.properties]<br>proxyPort = 443</pre><p>2.) Add the following configuration to each web application.</p><p><em>devportal/site/public/theme/settings.json</em></p><pre>&quot;context&quot;: &quot;/apim/devportal&quot;, <br>&quot;proxy_context_path&quot;: &quot;/apim&quot;,</pre><p><em>publisher/site/public/conf/settings.json</em></p><pre>&quot;context&quot;: &quot;/apim/publisher&quot;, <br>&quot;proxy_context_path&quot;: &quot;/apim&quot;,</pre><p><em>admin/site/public/conf/settings.json</em></p><pre>&quot;context&quot;: &quot;/apim/admin&quot;, <br>&quot;proxy_context_path&quot;: &quot;/apim&quot;,</pre><p>For more information regarding the above configurations, please refer to the [1] documentation.</p><h3>Start WSO2 API Manager and Access Portals</h3><p>Now that the configurations are complete:</p><p>1.) Navigate to the WSO2 API Manager bin directory:</p><pre>cd wso2am-4.3.0/bin</pre><p>2.) Start the API Manager server using:</p><pre>./api-manager.sh</pre><p>Once the server has started, you can access the APIM portals using the URLs mentioned at the beginning of this article:</p><p><strong><em>Publisher</em></strong><em>: </em><a href="https://knnect.lk/apim/publisher/"><em>https://knnect.lk/apim/publisher/</em></a><em><br></em><strong><em>Developer Portal</em></strong><em>: </em><a href="https://knnect.lk/apim/devportal"><em>https://knnect.lk/apim/devportal</em></a><em>/</em><em><br></em><strong><em>Admin Portal</em></strong><em>: </em><a href="https://knnect.lk/apim/admin/"><em>https://knnect.lk/apim/admin/</em></a></p><p>[1] <a href="https://apim.docs.wso2.com/en/4.4.0/install-and-setup/setup/setting-up-proxy-server-and-the-load-balancer/adding-a-custom-proxy-path/#access-wso2-api-manager-through-a-custom-proxy-path">https://apim.docs.wso2.com/en/4.3.0/install-and-setup/setup/setting-up-proxy-server-and-the-load-balancer/adding-a-custom-proxy-path/#access-wso2-api-manager-through-a-custom-proxy-path</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=73f34c22a7b7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using DataService Call Mediator to Insert Data into a Database via AP]]></title>
            <link>https://medium.com/@dinuka.caldera/using-dataservice-call-mediator-to-insert-data-into-a-database-via-ap-fe4a89c2813e?source=rss-856a3ebca6c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/fe4a89c2813e</guid>
            <dc:creator><![CDATA[Dinuka Caldera]]></dc:creator>
            <pubDate>Mon, 16 Dec 2024 04:12:32 GMT</pubDate>
            <atom:updated>2024-12-16T04:12:32.946Z</atom:updated>
            <content:encoded><![CDATA[<p>Hey everyone,</p><p>Today, I am going to explain how you can create an API, then a DataService, and through the API, use the DataService Call Mediator to call the DataService. This will allow you to connect to a database and insert data into a table.</p><p>In this article, I mainly want to explain the concept of DataServices and how to create them, as well as how to interact with a database through a DataService.</p><p>While this article mainly focuses on helping you understand what DataServices and DataService Call Mediators are, it’s always better to have a high-level idea of what you’re going to do (I assume you already know what an API and a mediator are, and I’ll be explaining the new concepts here: DataServices and DataService Call Mediators).</p><p>First, let’s understand what a DataService is [1].</p><p>A DataService is simply a way to connect your application to a database. For example, through the API we are going to create, we will use the DataService to interact with the database and perform operations like inserting, deleting, updating, and fetching data. This is the simplest way to explain a DataService.</p><p>Now, let’s understand what a Data Service Call Mediator [2] is.</p><p>As the name indicates, it’s a tool that calls the DataService. To elaborate, the Data Service Call Mediator allows you to call a DataService from within an integration flow. It acts like a messenger that connects your integration logic to the DataService.</p><p>I will explain the entire flow with a flowchart, which will make it easier for everyone to understand how each component connects with each other.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aKyl3hzfLcPW4X8ESZFgtQ.png" /></figure><p>Let’s say a client (an app or a person) sends an API request to the backend. Instead of the request going directly to the backend, there is a middleman (WSO2 MI 4.2.0) where the request is modified before it reaches the backend. This modification is done through mediators [3], and one of those mediators is the Data Service Call Mediator.</p><p>In simple terms, the client sends an API request, and it goes to the MI server. The server has mediation logic that makes changes to the request, including interacting with the database using a DataService before sending the request to the backend. The high-level idea is as simple as that. You can better understand this with the diagram I shared earlier.</p><p>Let’s go through an example to explain further. In this example, I have set up an API with a DataService Call Mediator. To make it more technical, I am sending the required parameters through request headers. In the sequence, the API takes the “name” and “job” values from the request headers using $trp:name and $trp:job [4].</p><p>The DataService Call Mediator then calls the RESTDataService and passes the extracted values (name and job) to the insertEmployeeDetails operation.</p><p>You will find the relevant DataService below. In the DataService, there is an operation called insertEmployeeDetails. This operation uses the parameters and runs a query to insert data into the database:<br>INSERT INTO employee_details (name, job) VALUES (:name, :job);.</p><p>Once the operation is successful, you will receive a response.</p><p>I’ve also provided a sample cURL command that you can use to insert data into the database. As you can see, it’s that simple — you just need to call the API with the correct headers (I included these headers to show how you can get content from them). You can also pass the parameters in the request body as well.</p><p><strong>API with Data Service Call Mediator</strong></p><pre>&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;<br>&lt;api context=&quot;/testAPI&quot; name=&quot;testAPI&quot; xmlns=&quot;http://ws.apache.org/ns/synapse&quot;&gt;<br>    &lt;resource methods=&quot;POST GET&quot;&gt;<br>        &lt;inSequence&gt;<br>            &lt;dataServiceCall serviceName=&quot;RESTDataService&quot;&gt;<br>                &lt;operations type=&quot;single&quot;&gt;<br>                    &lt;operation name=&quot;insertEmployeeDetails&quot;&gt;<br>                        &lt;param evaluator=&quot;xml&quot; expression=&quot;$trp:name&quot; name=&quot;name&quot;/&gt;<br>                        &lt;param evaluator=&quot;xml&quot; expression=&quot;$trp:job&quot; name=&quot;job&quot;/&gt;<br>                    &lt;/operation&gt;<br>                &lt;/operations&gt;<br>                &lt;source type=&quot;inline&quot;/&gt;<br>                &lt;target type=&quot;body&quot;/&gt;<br>            &lt;/dataServiceCall&gt;<br>            &lt;respond/&gt;<br>        &lt;/inSequence&gt;<br>        &lt;outSequence/&gt;<br>        &lt;faultSequence/&gt;<br>    &lt;/resource&gt;<br>&lt;/api&gt;</pre><p><strong>Data service</strong></p><pre>&lt;data name=&quot;RESTDataService&quot; transports=&quot;http https&quot;&gt;<br>  &lt;description&gt;Exposing the data service as a REST service.&lt;/description&gt;<br>  &lt;config id=&quot;testDataSource&quot;&gt;<br>    &lt;property name=&quot;driverClassName&quot;&gt;com.mysql.jdbc.Driver&lt;/property&gt;<br>    &lt;property name=&quot;url&quot;&gt;jdbc:mysql://localhost:3306/employees&lt;/property&gt;<br>    &lt;property name=&quot;username&quot;&gt;*****&lt;/property&gt;<br>    &lt;property name=&quot;password&quot;&gt;*****&lt;/property&gt;<br>  &lt;/config&gt;<br>  &lt;query id=&quot;employeeDetails&quot; useConfig=&quot;testDataSource&quot;&gt;<br>    &lt;sql&gt;INSERT INTO employee_details (name, job) VALUES (:name , :job);<br>&lt;/sql&gt;<br>    &lt;param name=&quot;name&quot; paramType=&quot;SCALAR&quot; sqlType=&quot;STRING&quot; type=&quot;IN&quot; optional=&quot;false&quot; /&gt;<br>  &lt;/query&gt;<br>  &lt;operation name=&quot;insertEmployeeDetails&quot;&gt;<br>    &lt;call-query href=&quot;employeeDetails&quot;&gt;<br>      &lt;with-param name=&quot;name&quot; query-param=&quot;name&quot; /&gt;<br>      &lt;with-param name=&quot;job&quot; query-param=&quot;job&quot; /&gt;<br>    &lt;/call-query&gt;<br>  &lt;/operation&gt;<br>  &lt;resource method=&quot;POST&quot; path=&quot;employee&quot;&gt;<br>    &lt;description /&gt;<br>    &lt;call-query href=&quot;employeeDetails&quot;&gt;<br>      &lt;with-param name=&quot;name&quot; query-param=&quot;name&quot; /&gt;<br>    &lt;/call-query&gt;<br>  &lt;/resource&gt;<br>&lt;/data&gt;</pre><pre>curl --location &#39;https://localhost:8253/testAPI&#39; \<br>--header &#39;name: Dinuka&#39; \<br>--header &#39;job: testJob&#39; \<br>--header &#39;Content-Type: application/json&#39; \<br>--data &#39;{ &quot;sample&quot; : &quot;sample&quot;}&#39;</pre><p>[1] <a href="https://apim.docs.wso2.com/en/4.2.0/reference/synapse-properties/data-services/">https://apim.docs.wso2.com/en/4.2.0/reference/synapse-properties/data-services/</a></p><p>[2] <a href="https://apim.docs.wso2.com/en/4.2.0/reference/mediators/dss-mediator/">https://apim.docs.wso2.com/en/4.2.0/reference/mediators/dss-mediator/</a></p><p>[3] <a href="https://apim.docs.wso2.com/en/4.2.0/reference/mediators/about-mediators/">https://apim.docs.wso2.com/en/4.2.0/reference/mediators/about-mediators/</a></p><p>[4] <a href="https://ei.docs.wso2.com/en/7.0.0/micro-integrator/references/mediators/property-reference/accessing-properties-with-xpath/#trp">https://ei.docs.wso2.com/en/7.0.0/micro-integrator/references/mediators/property-reference/accessing-properties-with-xpath/#trp</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fe4a89c2813e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python Script for Searching Any Record in MySQL Databases]]></title>
            <link>https://medium.com/@dinuka.caldera/python-script-for-searching-any-record-in-mysql-databases-5298563b069e?source=rss-856a3ebca6c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/5298563b069e</guid>
            <dc:creator><![CDATA[Dinuka Caldera]]></dc:creator>
            <pubDate>Sun, 12 May 2024 04:39:38 GMT</pubDate>
            <atom:updated>2024-05-12T04:54:46.008Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*81gVBQyD8Dg02FsgBlAa5A.jpeg" /></figure><p>Today, I’d like to tell you about a problem I faced and how I fixed it. When I was testing something, I needed to find specific information in a database. This included checking if a particular piece of data existed and where it was stored. Doing this manually was tough ( Obviously), especially when I wasn’t sure about the data type.</p><p>For example, I was looking for something like ‘<strong>63bf0748–0d89–41c4–8021–0a651f9bd2d3</strong>’.</p><p>To overcome this issue, Python came as the savior.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/202/1*hVvkFP_N_S6Sl7-_DScBDA.gif" /></figure><p>To simplify this process , I wrote a Python script. This script helps to quickly pinpoint information within a database.I hope it will be helpful to people who may face the same kind of difficulty. You can modify it according to your requirement.</p><p>Using this script is pretty simple. Simply enter the database host, port, name, username, password, and the value you want to search for. Then, the script will do the work.</p><p><strong>For an Example :</strong></p><p>Let’s say I want to search for the record “63bf0748–0d89–41c4–8021–0a651f9bd2d3” in the database “MY_DB”. Here are the values I have entered in the script:</p><blockquote>host = “localhost”<br>port = 3306<br>database_name = “MY_DB”<br>username = “root”<br>password = “******”<br>search_value = “63bf0748–0d89–41c4–8021–0a651f9bd2d3”<br>search_value_in_database(host, port, database_name, username, password, search_value)</blockquote><p>And when I run the script and if the record exists, it will provide the output as follows:</p><p><strong>To run the script :</strong></p><p>~/Downloads$<strong> python3 search_database.py</strong></p><p><strong>Output :</strong></p><p>Found in table ‘ABC’ column ‘EFG’ value ‘63bf0748–0d89–41c4–8021–0a651f9bd2d3’:<br>(319, None, 400, None, ‘system.user’, datetime.datetime(2024, 4, 3, 17, 42, 59), ‘system.user’, datetime.datetime(2024, 4, 3, 17, 42, 59), None, None, -1234, ‘63bf0748–0d89–41c4–8021–0a651f9bd2d3’)</p><p>It’s as simple as that!</p><p><strong>Before you run the script , make sure you have Python and the MySQL Connector for Python installed.</strong></p><p>Python Script :</p><blockquote>import mysql.connector</blockquote><blockquote>def search_value_in_database(host, port, database_name, username, password, search_value):<br> # Connect to the MySQL server<br> conn = mysql.connector.connect(<br> host=host,<br> port=port,<br> user=username,<br> password=password,<br> database=database_name<br> )<br> <br> cursor = conn.cursor()<br> <br> # Get a list of tables in the database<br> cursor.execute(“SHOW TABLES”)<br> tables = cursor.fetchall()<br> <br> for table_name in tables:<br> table_name = table_name[0]<br> cursor.execute(f”DESCRIBE {table_name}”)<br> columns = [column[0] for column in cursor.fetchall()]<br> <br> for column in columns:<br> query = f”SELECT * FROM {table_name} WHERE {column} LIKE %s”<br> cursor.execute(query, (‘%’ + search_value + ‘%’,))<br> <br> # Fetch all records where the value is present<br> records = cursor.fetchall()<br> <br> # If records are found, print them along with table, column name and value<br> if records:<br> print(f”Found in table ‘{table_name}’ column ‘{column}’ value ‘{search_value}’:”)<br> for record in records:<br> print(record)<br> <br> cursor.close()<br> conn.close()</blockquote><blockquote># Usage<br>host = “DATABASE_HOST:DATABASE_PORT”<br>database_name = “YOUR_DATABASE_NAME”<br>username = “YOUR_USERNAME”<br>password = “YOUR_PASSWORD”<br>search_value = “YOUR_SEARCH_VALUE”<br>search_value_in_database(host, database_name, username, password, search_value)</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5298563b069e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Implementing Resource Level Throttling for Each User on Each Resource in WSO2 API Manager 4.0.0]]></title>
            <link>https://medium.com/@dinuka.caldera/implementing-resource-level-throttling-for-each-user-on-each-resource-in-wso2-api-manager-4-0-0-302b1b1c11bf?source=rss-856a3ebca6c9------2</link>
            <guid isPermaLink="false">https://medium.com/p/302b1b1c11bf</guid>
            <dc:creator><![CDATA[Dinuka Caldera]]></dc:creator>
            <pubDate>Mon, 06 Mar 2023 08:52:48 GMT</pubDate>
            <atom:updated>2023-03-06T08:52:48.278Z</atom:updated>
            <content:encoded><![CDATA[<p>First I would like to explain the exact requirement. What you can achieve using this blog article. Using that you will be able to decide whether you should read the article further or not.</p><p>I hope everyone who read this article have some knowledge about WSO2 API Manager . If you want to know more about API Manager , go to the <a href="https://apim.docs.wso2.com/en/4.0.0/get-started/overview/">https://apim.docs.wso2.com/en/4.0.0/get-started/overview/</a> .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/636/0*wFe4STP-Y4TKSsJ4.jpeg" /></figure><p>Lets get back the with the requirement, lets say there are 3 users (User A, User B, User C) who has access to the same API ( example API) and there are 3 resources in the API ( Resource A , Resource B, Resource C)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/607/0*M4nYBbtLipKkpb4q" /></figure><h4>User A :</h4><ul><li>should only be able to send 2 requests (Max) per minute to “Resource A”</li><li>should only be able to send 4 requests (Max) per minute to “Resource B”</li><li>should only be able to send 6 requests (Max) per minute to “Resource C”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/616/0*9lEfyQ6eWUvUrj4n" /></figure><h4>User B :</h4><ul><li>should only be able to send 1 requests (Max) per minutes to “Resource A”</li><li>should only be able to send 5 requests (Max) per minute to “Resource B”</li><li>should only be able to send 7 requests (Max) per minute to “Resource C”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/0*GOgkTXQ5X_-f21mz" /></figure><h4>User C :</h4><ul><li>should only be able to send 3 requests (Max) per minutes to “Resource A”</li><li>should only be able to send 6 requests (Max) per minute to “Resource B”</li><li>should only be able to send 9 requests (Max) per minute to “Resource C”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/593/0*uOHhJYuIZ-RVGShd" /></figure><p>This requirement can be achieved using “<strong>Advanced Rate Limiting Policies</strong>”.</p><p>To learn more about Advanced Rate Limiting Policies, please refer to [1] documentation. In brief, using these advanced rate limiting policies, we can control access to the API or API resources based on the following properties and their combinations:</p><ul><li><strong>IP address and address range</strong>: You can control/restrict access to your API or its selected resources for a given IP address or address range.</li><li><strong>HTTP REQUEST HEADERS</strong>: Apply limits to APIs by filtering requests based on HTTP headers.</li><li><strong>JWT CLAIMS</strong>: Advanced throttling policies based on JWT claims allow you to filter requests by JWT claim values and apply limits for requests.</li><li><strong>QUERY PARAMETERS</strong>: Filtering based on query parameters almost always applies to HTTP GET requests when doing search-type operations. For example, if you have a search API with category as a query parameter, you can have different limits for searching different categories.</li></ul><p>As I mentioned earlier, to get an in-depth understanding of these advanced throttling policies, please refer to [1] documentation. In this scenario, we need a <strong>property </strong>to identify each user who sends an API request, so that we can <strong>filter out</strong> the user based on that property. As you all know, in general, when sending an API request, you have to do it with an Access Token. WSO2 API Manager uses JSON Web Tokens (JWTs) as access tokens to provide secure access to APIs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/0*zJntbdVBfXgH-sZp" /></figure><p><strong><em>These JWT Access Tokens contain JWT claims. JWT claims are pieces of information encoded into a JSON Web Token (JWT) to represent information about the authenticated user or client.</em></strong></p><p>Here’s an example of some common JWT claims that might be included in a JWT access token:</p><ul><li><strong>“iss”: The issuer of the token, typically the authorization server.</strong></li><li><strong>“sub”: The subject of the token, typically the user ID or client ID.</strong></li><li><strong>“aud”: The audience of the token, typically the API or resource server.</strong></li><li><strong>“exp”: The expiration time of the token.</strong></li><li><strong>“nbf”: The time before which the token must not be accepted.</strong></li><li><strong>“iat”: The time at which the token was issued.</strong></li><li><strong>“jti”: The unique identifier for the token.</strong></li></ul><p>I will explain this using a JWT token that I have used for API invocation. In the below scenario, I have made an API invocation. As you can see, I have included the JWT Access token for the curl request under the Authorization header. When you decode this JWT token using [2], you will be able to find the JWT claims under the payload. I have explained the meanings of these claims above.</p><blockquote><strong><em>curl -X GET “https://localhost:8243/pizzashack/1.0.0/menu&quot; -H “accept: application/json” -H “Authorization: Bearer eyJ4NXQiOiJNell4TW1Ga09HWXdNV0kwWldObU5EY3hOR1l3WW1NNFpUQTNNV0kyTkRBelpHUXpOR00wWkdSbE5qSmtPREZrWkRSaU9URmtNV0ZoTXpVMlpHVmxOZyIsImtpZCI6Ik16WXhNbUZrT0dZd01XSTBaV05tTkRjeE5HWXdZbU00WlRBM01XSTJOREF6WkdRek5HTTBaR1JsTmpKa09ERmtaRFJpT1RGa01XRmhNelUyWkdWbE5nX1JTMjU2IiwiYWxnIjoiUlMyNTYifQ.eyJzdWIiOiJkaW51a2EiLCJhdXQiOiJBUFBMSUNBVElPTiIsImF1ZCI6Ingwd3lTMjl0UHlUS0lUNWRaZVB6VThCX1lUMGEiLCJuYmYiOjE2NzgwODU3NzMsImF6cCI6Ingwd3lTMjl0UHlUS0lUNWRaZVB6VThCX1lUMGEiLCJzY29wZSI6ImRlZmF1bHQiLCJpc3MiOiJodHRwczpcL1wvbG9jYWxob3N0Ojk0NDNcL29hdXRoMlwvdG9rZW4iLCJleHAiOjE2NzgwODkzNzMsImlhdCI6MTY3ODA4NTc3MywianRpIjoiMGVmZDMyYjAtODA5ZC00NjUwLTg1NDItOWVjM2ExMjU3YTdlIn0.aVdXN0csjlRdeMkNZkvUANEekdmVNv-_1L31t37qkcr8vw86XF6wCW4kUMhWKURH69ng9XFXHzgqSFSlo7f7KnaiybMG15frXsUe9H2tntB-E8-bhJ4Y1H7rvOj8NalhzE3sqCNp9w9R2J3GsI-z4_efS5hLdpFONLN_jU9YL_ZdXJfWd0diUEVL1o4UHnLWNZUof4E0GLa4TE8nyYiSyw9sUAwhm8DdL60Y31C3F9O94ODSMlwc-s384jJrrEme9VC2LbofKIZb3JjEwfNMyrSNKFO8zNL5h8hmBuz9KYj-IREL97MKcEvchDIHselm1QdQ94pKQySjJr3x6BuaJQ</em>”</strong></blockquote><p>Decoded jwt payload :</p><h4>“sub”: “dinuka”,<br> “aut”: “APPLICATION”,<br> “aud”: “x0wyS29tPyTKIT5dZePzU8B_YT0a”,<br> “nbf”: 1678085773,<br> “azp”: “x0wyS29tPyTKIT5dZePzU8B_YT0a”,<br> “scope”: “default”,<br> “iss”: “https://localhost:9443/oauth2/token&quot;,<br> “exp”: 1678089373,<br> “iat”: 1678085773,<br> “jti”: “0efd32b0–809d-4650–8542–9ec3a1257a7e”</h4><p>Through which claim do you think we can achieve our requirement? Yes, you are correct! 😃 It is the sub-claim that we can use.</p><p>A “sub” claim represents the subject of the token, typically the user ID or client ID. In the above scenario, it’s Dinuka (the username I used to log in to the developer portal to get the access token).</p><p><strong>So now we have a way to identify each user. </strong>😃😃</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*ZA9DG-9tEftprX5tC0cmCg.jpeg" /></figure><p>Now let’s go back to creating an advanced level throttling policy to achieve the requirement. Before starting the API Manager, you need to make some configuration changes in the <strong>&lt;Product_Home&gt;/repository/conf/deployment.toml</strong> file. Please open that file and add the following configurations. You can find more details about these configurations in [1] documentation.</p><h4>[apim.throttling]<br>enable_jwt_claim_based_throttling=true</h4><h4>[apim.jwt]<br>enable = true<br>#encoding = “base64” # base64,base64url<br>#generator_impl = “org.wso2.carbon.apimgt.keymgt.token.JWTGenerator”<br>claim_dialect = “http://wso2.org/claims&quot;<br>#convert_dialect = false<br>#header = “X-JWT-Assertion”<br>#signing_algorithm = “SHA256withRSA”<br>#enable_user_claims = true<br>#claims_extractor_impl = “org.wso2.carbon.apimgt.impl.token.ExtendedDefaultClaimsRetriever”</h4><p>Now I will show the next steps using screenshots , it will be easier for you to follow the steps.</p><p><strong>1.)</strong> We can “Add New Policy” through the admin portal of the API Manager. After starting the API Manager, please log in to the admin portal and click on “Advanced Policies” as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*iD803OE2saW3nAko" /></figure><p><strong>2.)</strong> Click on “Add New Policy” and fill in the required details (Please refer to the screenshot below). You will see the “Default Limit” option, which is the general limit for every request for that resource (for example, 20 requests per 1 minute). I have added a policy named “ThrottleForResourceA”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FF0JOP8-5yyPyJuO" /></figure><p><strong>3.) </strong>Click on “Add Conditional Group”. In there, you can see the JWT Condition Policy. This policy filters the user using the “sub” attribute in the access token.</p><p>(Sub is the username of the user who logs in to the developer portal, as I explained before.)<br>Please refer to the image below. <br>In the screenshot, you can see that I have defined that user1 can send 3 requests per 1 minute. (In these example screenshots, I have used 3 users: user 1, user 2, and user 3. For each user, you have to add a conditional group )</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yQBljD3BswFAkmAa" /></figure><p><strong>4.)</strong> Add another condition group for user2 and define 6 requests per 1 minute</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_WIomxhY4ojFkYo2" /></figure><p><strong>5.)</strong> Add another condition group for user3 and define 10 requests per 1 minute</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8JFkzyicFwrAeRh1" /></figure><p><strong>6.)</strong> Then save the policy, Go to the publisher portal, Go to the relevant resource and select the added policy from Rate Limiting policy ( Refer 6.png image)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8i64oNRsFNnPtbHX" /></figure><p>I have only provided the steps to add an Advanced Policy for one resource (Resource A). For the other two resources (Resource B and Resource C), you have to add new policies by following the same steps depending on the requirements.</p><p>[1] <a href="https://apim.docs.wso2.com/en/4.0.0/design/rate-limiting/adding-new-throttling-policies/#adding-a-new-advanced-rate-limiting-policy">https://apim.docs.wso2.com/en/4.0.0/design/rate-limiting/adding-new-throttling-policies/#adding-a-new-advanced-rate-limiting-policy</a></p><p>[2] <a href="https://jwt.io/">https://jwt.io/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=302b1b1c11bf" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>