<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Dany Drouin on Medium]]></title>
        <description><![CDATA[Stories by Dany Drouin on Medium]]></description>
        <link>https://medium.com/@dany.drouin?source=rss-13f5331e37ff------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Stories by Dany Drouin on Medium</title>
            <link>https://medium.com/@dany.drouin?source=rss-13f5331e37ff------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 07:37:21 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dany.drouin/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Connecting IBM watsonx.data intelligence as a Service on AWS to AWS Data Sources]]></title>
            <link>https://medium.com/@dany.drouin/connecting-ibm-watsonx-data-intelligence-as-a-service-on-aws-to-aws-data-sources-7e752b277ac7?source=rss-13f5331e37ff------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e752b277ac7</guid>
            <category><![CDATA[watsonx]]></category>
            <category><![CDATA[data-intelligence]]></category>
            <category><![CDATA[ibm]]></category>
            <category><![CDATA[watson-knowledge-catalog]]></category>
            <dc:creator><![CDATA[Dany Drouin]]></dc:creator>
            <pubDate>Tue, 16 Jun 2026 20:27:04 GMT</pubDate>
            <atom:updated>2026-06-16T20:27:04.107Z</atom:updated>
            <content:encoded><![CDATA[<h3>Connecting IBM watsonx.data intelligence as a Service on AWS to AWS Data Sources: Three Proven Approaches</h3><p><em>A comprehensive guide to securely connecting IBM watsonx.data intelligence (on AWS SaaS) to AWS data sources including Databricks, Aurora PostgreSQL, RDS, Redshift, and more</em></p><h3>Introduction</h3><p><a href="https://aws.amazon.com/marketplace/pp/prodview-l3fj6yu5ubsnk">IBM watsonx.data intelligence as a Service on AWS</a> is a comprehensive data governance and catalog solution that helps organizations discover, curate, and govern their data assets across hybrid cloud environments. As organizations increasingly adopt multi-cloud data architectures, connecting watsonx.data intelligence running as a SaaS offering on AWS to various AWS data sources has become a critical requirement for unified data governance.</p><p>This comprehensive guide walks through <strong>three proven connectivity approaches</strong> that work with <strong>multiple AWS data sources</strong> including:</p><ul><li><strong>Databricks</strong> (primary example throughout this guide)</li><li><strong>Amazon Aurora PostgreSQL</strong> (featured in Option 3)</li><li><strong>Amazon RDS</strong> (PostgreSQL, MySQL, SQL Server, Oracle)</li><li><strong>Amazon Redshift</strong></li><li><strong>Other TCP-based data sources</strong></li></ul><p>Each approach is designed for different security requirements, operational complexity, and cost considerations.</p><h3>Why Connect watsonx.data intelligence to AWS Data Sources?</h3><p>watsonx.data intelligence needs to connect to AWS data sources for several key governance and catalog functions:</p><ol><li><strong>Metadata Discovery</strong>: Automatically scanning tables, schemas, and catalogs (Databricks Unity Catalog, RDS databases, etc.)</li><li><strong>Data Profiling</strong>: Analyzing data quality, statistics, and patterns across all data sources</li><li><strong>Data Lineage</strong>: Tracking data flow and transformations across your data ecosystem</li><li><strong>Policy Enforcement</strong>: Applying governance rules and access controls consistently</li><li><strong>Quality Rules</strong>: Running automated data quality checks and validations</li><li><strong>Business Glossary</strong>: Linking technical assets to business terms and definitions</li></ol><h3>What You’ll Learn</h3><ul><li>Three connectivity architectures applicable to multiple AWS data sources</li><li>Detailed implementation steps with Databricks and PostgreSQL examples</li></ul><h3>Applicability to Different AWS Data Sources</h3><p><strong>Important</strong>: While this guide uses <strong>Databricks as the primary example</strong> for Options 1 and 2, and <strong>Aurora PostgreSQL for Option 3</strong>, all three connectivity approaches work with any TCP-based AWS data sources</p><h3>Current Connectivity Landscape</h3><p><strong>Important Note</strong>: As of June 2026, IBM watsonx.data intelligence AWS SaaS does not support VPC PrivateLink or Virtual Private Endpoints (VPE) for outbound connections to external data sources. Until then, the three approaches outlined in this guide provide secure, production-ready alternatives for connecting to:</p><ul><li>Databricks (HTTPS/TLS on port 443)</li><li>Aurora PostgreSQL and RDS databases (TCP on ports 5432, 3306, etc.)</li><li>Redshift (TCP on port 5439)</li><li>Other AWS data sources accessible via TCP protocols</li></ul><h3>Architecture Overview: Three Options Compared</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/857/1*W_rH0IUiAuiq1No7DEZI_A.png" /></figure><h3>Quick Comparison</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/1*5mhhg_hmh7YJIc0LBznyMA.png" /></figure><h3>Option 1: Direct Connection with IP Allowlisting</h3><h3>Overview</h3><p>This is the simplest approach, using Databricks’ public endpoints with IP allowlisting to restrict access to only watsonx.data intelligence platform IPs. It’s ideal for development environments, proof-of-concepts, or organizations with less stringent security requirements.</p><h3>How It Works</h3><p>watsonx.data intelligence connects directly to Databricks’ public endpoint over the internet. Databricks’ IP Access List feature ensures that only requests originating from IBM’s known egress IPs are allowed, blocking all other traffic.</p><h3>Step-by-Step Configuration</h3><h4>1. Obtain watsonx.data intelligence Egress IPs</h4><p>IBM watsonx.data intelligence publishes the egress IP addresses used by the SaaS platform:</p><p><strong>Documentation URL</strong>: <a href="https://us-east-1.aws.data.ibm.com/docs/content/wsj/admin/firewall_cfg.html?context=wx&amp;locale=en">https://us-east-1.aws.data.ibm.com/docs/content/wsj/admin/firewall_cfg.html?context=wx&amp;locale=en</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5rrd7pOEYsuUK-EXjj_L7w.jpeg" /><figcaption><em>Example of firewall configuration for watsonx.data intelligence egress IPs</em></figcaption></figure><p>For <strong>us-east</strong> SaaS deployments, you’ll typically see IP ranges like:</p><ul><li>52.163.106.49/32</li><li>54.163.212.8/32</li><li>98.95.187.177/32</li><li>Additional CIDR blocks specific to the IBM watsonx data platform</li></ul><p><strong>Important</strong>: These IPs may change over time. Subscribe to IBM’s change notifications or regularly check the documentation for updates.</p><h4>2. Configure Databricks IP Access Lists</h4><p>In your Databricks workspace:</p><ol><li>Navigate to <strong>Admin Console</strong> → <strong>Settings</strong> → <strong>Network</strong></li><li>Click on <strong>IP Access Lists</strong></li><li>Create a new access list:<br>-<strong>Name</strong>: watsonx-intelligence-access<br>-<strong>IP Addresses</strong>: Add each watsonx.data intelligence egress IP/CIDR<br>-<strong>Type</strong>: Allow</li><li>Enable the access list</li></ol><pre>{<br>  &quot;label&quot;: &quot;watsonx-intelligence-access&quot;,<br>  &quot;list_type&quot;: &quot;ALLOW&quot;,<br>  &quot;ip_addresses&quot;: [<br>    &quot;52.163.106.49/32&quot;,<br>    &quot;54.163.212.8/32&quot;,<br>    &quot;98.95.187.177/32&quot;<br>  ],<br>  &quot;enabled&quot;: true<br>}</pre><h4>3. Create Databricks Personal Access Token</h4><p>Generate a Personal Access Token (PAT) for watsonx.data intelligence:</p><ol><li>In Databricks, go to <strong>User Settings</strong> → <strong>Access Tokens</strong></li><li>Click <strong>Generate New Token</strong></li><li>Set a descriptive comment: watsonx-intelligence-metadata-access</li><li>Set expiration (recommend 90 days for security)</li><li>Copy and securely store the token</li></ol><p><strong>Best Practice</strong>: Use a service account rather than a personal user account for production deployments.</p><h4>4. Configure Connection in watsonx.data intelligence</h4><p>In watsonx.data intelligence:</p><ol><li>Navigate to <strong>Data Manager</strong> → <strong>Platform Connections</strong> → <strong>Add Connection</strong></li><li>Select <strong>Databricks</strong> as the connection type</li><li>Enter connection details:<br>-<strong>Connection Name</strong>: Databricks Production<br>-<strong>Host</strong>: &lt;workspace-id&gt;.cloud.databricks.com<br>-<strong>Port</strong>: 443<br><strong>-HTTP Path</strong>: /sql/1.0/warehouses/&lt;warehouse-id&gt;<br><strong>-Authentication Type</strong>: Personal Access Token<br><strong>-Token</strong>: Paste the token from step 3</li><li>Test the connection</li><li>Save the connection</li></ol><h3>Traffic Flow Diagram</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/790/1*ReuuwSS2CJ1Z-Lf6p1rJDQ.png" /></figure><h3>Pros and Cons</h3><p><strong>Advantages</strong>:</p><ul><li>✅ <strong>Simple to implement</strong> — Can be set up in under 30 minutes</li><li>✅ <strong>No additional infrastructure required</strong> — No AWS resources to manage</li><li>✅ <strong>Lower latency</strong> — Direct connection without intermediaries</li><li>✅ <strong>No additional costs</strong> — Zero infrastructure costs</li><li>✅ <strong>Easy to troubleshoot</strong> — Fewer components means simpler debugging</li><li>✅ <strong>Quick to deploy</strong> — Ideal for POCs and development environments</li></ul><p><strong>Disadvantages</strong>:</p><ul><li>❌ <strong>Traffic traverses public internet</strong> — Not suitable for highly sensitive data but is still encrypted</li><li>❌ <strong>Relies on IP allowlisting</strong> — Less secure than private connectivity</li><li>❌ <strong>Requires monitoring for IP changes</strong> — IBM may update egress IPs</li><li>❌ <strong>May not meet strict compliance requirements</strong> — Some regulations require private connectivity</li><li>❌ <strong>No additional security layers</strong> — Single point of security control</li><li>❌ <strong>Public endpoint exposure</strong> — Databricks endpoint is internet-accessible</li></ul><p><strong>Best For</strong>: Development environments, POCs, small teams, non-regulated data, cost-conscious deployments</p><h3>Option 2: Application Load Balancer with Reverse Proxy</h3><h3>Overview</h3><p>For organizations requiring additional security layers or needing to connect to private Databricks deployments, a reverse proxy architecture in your AWS account provides a robust solution. This approach uses an Application Load Balancer (ALB) for IP filtering and an nginx reverse proxy for application-layer routing and SSL termination.</p><h3>Architecture Components</h3><p>This solution consists of:</p><ul><li><strong>Application Load Balancer (ALB)</strong>: Internet-facing, handles IP allowlisting</li><li><strong>nginx Reverse Proxy</strong>: Runs on EC2 in private subnets, handles SSL and routing</li><li><strong>NAT Gateway</strong>: Allows proxy to reach external endpoints</li><li><strong>Auto Scaling Group</strong>: Ensures high availability of proxy instances</li></ul><h3>Key Configuration Steps</h3><h4>1. VPC Setup</h4><pre>VPC: 10.0.0.0/16<br>Public Subnets: 10.0.1.0/24, 10.0.2.0/24 (for ALB)<br>Private Subnets: 10.0.10.0/24, 10.0.11.0/24 (for proxy)<br>NAT Gateway: In public subnet with Elastic IP</pre><h4>2. ALB Configuration</h4><ul><li><strong>Scheme</strong>: Internet-facing</li><li><strong>Listeners</strong>: HTTPS (443) with ACM certificate</li><li><strong>Security Group</strong>: Allow only watsonx.data intelligence IPs on port 443</li><li><strong>Target Group</strong>: Points to proxy instances on port 8443</li><li><strong>Health Check</strong>: /health endpoint on proxy</li></ul><h4>3. nginx Reverse Proxy</h4><pre>server {<br>    listen 8443 ssl http2;<br>    <br>    # Health check<br>    location /health {<br>        return 200 &quot;healthy\n&quot;;<br>    }<br>    <br>    # Databricks proxy<br>    location /databricks/ {<br>        proxy_pass https://&lt;workspace-id&gt;.cloud.databricks.com/;<br>        proxy_set_header Host &lt;workspace-id&gt;.cloud.databricks.com;<br>        proxy_ssl_server_name on;<br>        <br>        # Timeouts for long-running queries<br>        proxy_connect_timeout 300s;<br>        proxy_read_timeout 300s;<br>    }<br>}</pre><h3>Traffic Flow</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*WCTZ4vRGlocojHqvpbykmQ.png" /></figure><h3>Pros and Cons</h3><p><strong>Advantages</strong>:</p><ul><li>✅ <strong>Private endpoint connectivity</strong> — Full private network access</li><li>✅ <strong>SSL inspection capability</strong> — Can inspect encrypted traffic</li><li>✅ <strong>Advanced routing</strong> — Path-based and host-based routing</li><li>✅ <strong>Detailed logging</strong> — Application-layer logs</li><li>✅ <strong>Multi-backend support</strong> — Can proxy to RDS, S3, Redshift, etc.</li><li>✅ <strong>Request manipulation</strong> — Can modify headers and add authentication</li></ul><p><strong>Disadvantages</strong>:</p><ul><li>❌ <strong>High complexity</strong> — Multiple components to manage</li><li>❌ <strong>Higher costs</strong> — ~$200–400/month (ALB, EC2, NAT Gateway)</li><li>❌ <strong>Maintenance overhead</strong> — Patching, monitoring, scaling</li><li>❌ <strong>Higher latency</strong> — Additional hop adds ~15–20ms</li><li>❌ <strong>Certificate management</strong> — Need to manage SSL certificates</li></ul><p><strong>Best For</strong>: Large enterprises, highly regulated industries, organizations requiring SSL inspection, multi-source connectivity, strict security requirements</p><h3>Option 3: Network Load Balancer with TCP Passthrough</h3><h3>Overview</h3><p>This option provides the <strong>sweet spot</strong> between direct connection and the full reverse proxy architecture. It uses AWS Network Load Balancer as a simple TCP/TLS passthrough to private endpoints, without requiring application-layer proxying. This approach offers the security benefits of private connectivity with significantly lower complexity and cost than Option 2.</p><p><strong>Note</strong>: This section uses <strong>Aurora PostgreSQL and RDS</strong> as primary examples (alongside Databricks) to demonstrate the multi-protocol capabilities of NLB. The same approach works for any TCP-based data source by simply adjusting the port number and target configuration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7hLHjkZIft36u7M-GZlWlg.jpeg" /><figcaption><em>Network Load Balancer configuration with target groups</em></figcaption></figure><h3>Why NLB Instead of ALB?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/648/1*s74OIAOoUAZo3WdrS0YA4g.png" /></figure><h3>Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*DRTgLyztizcAU7bWcs20eg.png" /></figure><h3>Key Configuration Steps</h3><h4>1. Simplified VPC Setup</h4><pre>VPC: 10.0.0.0/16<br>Public Subnets: 10.0.1.0/24, 10.0.2.0/24 (for NLB)<br># No private subnets or NAT Gateway needed!</pre><h4>2. NLB Configuration</h4><pre>LoadBalancer:<br>  Type: network<br>  Scheme: internet-facing<br>  <br>  SecurityGroup:<br>    Ingress:<br>      - Port: 443, Protocol: TCP<br>        Source: watsonx Intelligence IPs<br>      - Port: 5432, Protocol: TCP  # For RDS<br>        Source: watsonx Intelligence IPs<br>  <br>  TargetGroups:<br>    - Name: databricks-tg<br>      Port: 443<br>      Protocol: TLS<br>      TargetType: ip<br>      Target: &lt;databricks-private-endpoint-ip&gt;<br>    <br>    - Name: rds-tg<br>      Port: 5432<br>      Protocol: TCP<br>      TargetType: ip<br>      Target: &lt;rds-private-ip&gt;<br>  <br>  Listeners:<br>    - Port: 443, Protocol: TLS<br>      Certificate: ACM-Certificate-ARN<br>      Action: Forward to databricks-tg<br>      # TLS passthrough - no termination</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P6cnto8mKvoeHnsahA7CeQ.jpeg" /><figcaption><em>Security Group configuration</em></figcaption></figure><h4>3. Traffic Flow</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/766/1*8k5zLHqweMzgrDnoSthysQ.png" /></figure><h4>4. Configure Connection in watsonx.data intelligence</h4><p>In watsonx.data intelligence:</p><ol><li>Navigate to <strong>Data Manager</strong> → <strong>Platform Connections</strong> → <strong>Add Connection</strong></li><li>Select <strong>Postgres</strong> as the connection type</li><li>Enter connection details:<br><strong>-Connection Name</strong>: Postgres Production<br><strong>-Host</strong>: watsonx-nlb-xxx.elb.us-east-1.amazonaws.com<br><strong>-Port</strong>: 5432<br><strong>-Database</strong>: postgres</li><li>Test the connection</li><li>Save the connection</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yWIAYcFgGCAnMH70sq7x0A.jpeg" /><figcaption><em>Testing a Aurora Postgres connection in watsonx.data intelligence</em></figcaption></figure><h3>Pros and Cons</h3><p><strong>Advantages</strong>:</p><ul><li>✅ <strong>Simpler than ALB + Proxy</strong> — No EC2 instances to manage</li><li>✅ <strong>Lower cost</strong> — ~$100–280/month savings vs Option 2</li><li>✅ <strong>Better performance</strong> — Layer 4 is faster than Layer 7</li><li>✅ <strong>End-to-end encryption</strong> — TLS passthrough maintains encryption</li><li>✅ <strong>Multi-protocol support</strong> — Works with any TCP protocol</li><li>✅ <strong>Built-in HA</strong> — No Auto Scaling Groups needed</li><li>✅ <strong>No NAT Gateway required</strong> — Additional cost savings</li></ul><p><strong>Disadvantages</strong>:</p><ul><li>❌ <strong>No SSL inspection</strong> — Cannot inspect encrypted traffic</li><li>❌ <strong>Basic routing</strong> — No path-based or host-based routing</li><li>❌ <strong>Limited logging</strong> — Only VPC Flow Logs available</li><li>❌ <strong>No caching</strong> — Every request goes to backend</li><li>❌ <strong>No request manipulation</strong> — Cannot modify headers</li></ul><p><strong>Best For</strong>: Production environments, cost-conscious enterprises, organizations needing private connectivity without SSL inspection, multi-protocol support</p><h3>Conclusion</h3><p>Connecting IBM watsonx.data intelligence to Databricks or other AWS data sources requires careful consideration of security, cost, and operational requirements. This guide presented three proven approaches:</p><h3>Summary</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/707/1*No3PVhexsqoRiXif3v37qw.png" /></figure><h3>Recommendation</h3><p>For most organizations:</p><ol><li><strong>Start with Option 1</strong> for development and testing</li><li><strong>Move to Option 3</strong> for production workloads</li><li><strong>Only use Option 2</strong> if you have specific requirements for SSL inspection or advanced routing</li></ol><p><strong>Option 3 (NLB) provides the best balance</strong> of security, cost, and operational simplicity for production workloads, saving $100–280/month compared to Option 2 while still providing private connectivity.</p><h3>Additional Resources</h3><h3>Official Documentation</h3><ul><li><a href="https://us-east-1.aws.data.ibm.com/docs/content/wsj/admin/firewall_cfg.html?context=wx&amp;locale=en">IBM watsonx.data intelligence Firewall Configuration</a></li><li><a href="https://docs.databricks.com/security/network/ip-access-list.html">Databricks IP Access Lists</a></li><li><a href="https://docs.aws.amazon.com/elasticloadbalancing/latest/application/">AWS Application Load Balancer</a></li><li><a href="https://docs.aws.amazon.com/elasticloadbalancing/latest/network/">AWS Network Load Balancer</a></li><li><a href="https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/">nginx Reverse Proxy Configuration</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e752b277ac7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Troubleshooting Watson Knowledge Catalog Installs / Upgrades]]></title>
            <link>https://medium.com/@dany.drouin/troubleshooting-watson-knowledge-catalog-installs-upgrades-b30c58bff4ce?source=rss-13f5331e37ff------2</link>
            <guid isPermaLink="false">https://medium.com/p/b30c58bff4ce</guid>
            <category><![CDATA[troubleshooting]]></category>
            <category><![CDATA[wkc]]></category>
            <category><![CDATA[install]]></category>
            <category><![CDATA[operators]]></category>
            <category><![CDATA[upgrade]]></category>
            <dc:creator><![CDATA[Dany Drouin]]></dc:creator>
            <pubDate>Tue, 22 Aug 2023 13:15:44 GMT</pubDate>
            <atom:updated>2024-01-08T20:21:39.533Z</atom:updated>
            <content:encoded><![CDATA[<h3>Troubleshooting IBM Knowledge Catalog (WKC) Installs / Upgrades</h3><h3>Overview</h3><p>IBM Knowledge Catalog is a powerful application with different components working in harmony to enable metadata sharing across Cloud Pak for Data… CCS, Data Refinery, DataStage, Analytics Engine, Foundation DB, DB2u and DB2 are some of it’s core components.</p><p>In this blog, I will share some tips on how to troubleshoot installs/upgrades and dive into how to interpret the operator logs to help resolve any obstacles encountered during the install/upgrade process.</p><h3>WKC CR Status</h3><p>You can monitor the current state of WKC by looking at the CR (Custom Resource) status field.<br>Available states are:<br>· <strong>Completed</strong> — Install / Upgrade / Reconcile was successful.<br>· <strong>InProgress</strong> — Reconcile in progress.<br>· <strong>InMaintenance</strong> — ignoreForMaintenance mode is enabled and reconciles will be ignored.<br>· <strong>Failed</strong> — Reconcile failed, consult the WKC CR status field for more details.</p><p>To get the latest status you can run either commands bellow:</p><pre>oc get wkc wkc-cr -o yaml</pre><p>or</p><pre>cpd-cli manage get-cr-status --cpd_instance_ns=wkc --components=wkc</pre><p>The first option above will provide you a more in-depth status including the last known failure in the case the CR failed to reconcile. In case of failure, the operator logs can be consulted for analysis following the instructions bellow.</p><h3>Operator Logs</h3><p>The operator container logs are often not useful and contain lots of noise making diagnosis of WKC install issues very painful. Instead, I recommend using the cleaner per reconcile ansible runner logs available in the operator’s container.</p><p>Here’s how to get better logs from the operators.</p><p><strong><em>Locate the WKC operator</em></strong></p><pre>oc get po --all-namespaces -l app.kubernetes.io/name=ibm-cpd-wkc-operator</pre><pre>NAMESPACE           NAME                                  READY STATUS  RESTARTS AGE<br>ibm-common-services ibm-cpd-wkc-operator-55fcbfd6c7-tdhdq 1/1   Running 0        7d5h</pre><p><strong><em>Viewing ansible logs from the operator</em></strong></p><p>For this task, you will want to rsh into the operator container.</p><pre>oc rsh -n &lt;namespace&gt; ibm-cpd-wkc-operator-55fcbfd6c7-tdhdq</pre><p>*Replace namespace and pod names accordingly.</p><p>Each operator reconcile of the CR will create an ansible run log. These logs can be found within the operator pod under the following location: <strong><em>/tmp/ansible-operator/runner</em></strong></p><p>For WKC, the latest log can be found under: <strong><em>/tmp/ansible-operator/runner/wkc.cpd.ibm.com/v1beta1/WKC/&lt;namespace&gt;/wkc-cr/artifacts/latest</em></strong></p><p>*Note to replace the namespace accordingly.</p><p>The log file is named <strong><em>stdout</em></strong></p><p>To view the latest logs for WKC CR</p><pre>cat /tmp/ansible-operator/runner/wkc.cpd.ibm.com/v1beta1/WKC/&lt;namespace&gt;/wkc-cr/artifacts/latest/stdout</pre><p>If you are looking for logs for a previous reconcile loop, you can go to the <strong><em>/artifacts</em></strong> parent folder instead of the latest. Each reconcile loop creates a unique folder with the timestamp.</p><p><strong><em>Getting all logs from the operator</em></strong></p><p>Run the following command to tar package all the runner logs from the WKC operator. <br>Replace and the actual operator name accordingly.</p><pre>oc exec -n &lt;namespace&gt; ibm-cpd-wkc-operator-&lt;xyv&gt; -- tar -czvf /tmp/logs.tar /tmp/ansible-operator/runner/</pre><p>Copy the logs out of the operator pod using:</p><pre>oc cp -n &lt;namespace&gt; ibm-cpd-wkc-operator-&lt;xyz&gt;:/tmp/logs.tar ./logs.tar</pre><p><strong><em>Debug level logs</em></strong></p><p>You can enable verbose logging for the ansible runner by adding the following annotation to the CR. Once annotated, you can force a reconcile loop and then collect the logs as mentioned above.</p><p><em>Annotate WKC CR:</em></p><pre>oc annotate wkc wkc-cr &quot;ansible.sdk.operatorframework.io/verbosity&quot;=&quot;5&quot;</pre><p><strong><em>Forcing a reconcile loop</em></strong></p><p>Simply deleting the operator pod will force a reconcile on next startup.</p><p>The other option is to modify the CR with a bogus value, but if the operator is already in the process of reconciling, it will not reconcile the change right away, not until the previous loop completes or fails.</p><p>The fastest way is to force the reconcile by deleting the operator pod.</p><h3>Must-gather</h3><p>The must-gather output is a very useful method to help diagnose issues with the install. Support and internal WKC development will require this package as a starting point.</p><p>The must-gather tool can be ran using the following:</p><pre>oc adm must-gather --image=icr.io/cpopen/cpfs/must-gather:latest --gather -n ibm-common-services,wkc -m overview,cloudpak</pre><p>*Replace -n with the namespaces you want to collect. In the example above it will collect data for the operators from <em>ibm-common-services</em> and the cpd-instance from <em>wkc</em> namespaces where WKC is installed.</p><p>It’s important to include both the operators namespace as well as the cpd-instance (where WKC is installed).</p><p>More info can be found here: <a href="https://www.ibm.com/docs/en/cpfs?topic=support-collecting-information-mustgather">https://www.ibm.com/docs/en/cpfs?topic=support-collecting-information-mustgather</a></p><h3>Interpreting the Ansible Runner Logs</h3><p>It’s good practice to consult the operator logs from the last reconcile loop and starting from the end of the logs, slowly making your way up the log until you encounter a <em>Failed</em> task.</p><p>Failures can come from common tasks, here are a few examples:</p><p>· <strong>Waiting for a service to become ready:<br></strong>Failure message: Deploy resource and wait for it to complete<br>Diagnosing: Consult the service’s pod logs and describe the pod for hints as to why the service is not starting and ready to serve requests.</p><p>· <strong>Waiting for a job to complete:<br></strong>Failure message: Wait for {{ job_name }} to complete<br>Diagnosing: Consult the job’s pod logs</p><p>· <strong>Creating a resource:<br></strong>Failure message: Deploying resource<br>Diagnosing: Consult the operator ansible logs to hints as to why the resource couldn’t be created/updated.</p><p>· <strong>Waiting until a service is ready:<br></strong>Failure message: Wait until the service is ready<br>Diagnosing: Consult the service’s pod logs and describe the pod for hints as to why the service is not starting and ready to serve requests.</p><p>Depending on the error you observed in the WKC CR status or operator logs, you may need to consult another operator’s logs (CCS for example).</p><blockquote>Message: Unable to install CCS prerequisite</blockquote><p>The same process as above can be followed to get the CCS operator logs, simply replace <em>wkc</em> with <em>ccs</em> everywhere.</p><p><strong><em>Failure scenario:</em></strong></p><p>After consulting the CCS CR status and logs we found the following:</p><p>CCS CR status:</p><blockquote>Message: One or more items failed<br>The playbook has failed at task — ‘Deploy app_name resource and wait for it to complete’<br>The error was: ‘One or more items failed’</blockquote><p>CCS operator logs:</p><blockquote>failed: [localhost] (item=wkc-base-roles-init) =&gt; {\”ansible_loop_var\”: \”item\”, \”changed\”: true, \”duration\”: 300, \”item\”: \”wkc-base-roles-init\”, \”method\”: \”create\”, \”msg\”: \”Resource creation timed out\</blockquote><p>The above logs indicate a failure in the <em>wkc-base-roles-init</em> job.</p><p>Let’s consult its logs.</p><p>Get the pods corresponding to the job:</p><pre>oc get po -l job-name=wkc-base-roles-init</pre><pre>NAME READY STATUS RESTARTS AGE<br>wkc-base-roles-init-mxz4n 0/1 Failed 0 4m56s<br>wkc-base-roles-init-nxzad 0/1 Failed 0 5m12s</pre><p>Consult the oldest pod first and then the most recent one.</p><pre>oc logs wkc-base-roles-init-nxzad</pre><pre>&gt;&gt; Requesting for a refresh of roles from usermgmt-svc…<br>&gt;&gt; Running with uninstall set to ( False )<br>&gt;&gt; Found role that needs to be refreshed — zen_administrator_role<br>&gt;&gt; Error refreshing role with extension_name — zen_administrator_role — status_code — 400<br>{<br>“exception”: “Request body contains permissions that are not known to the platform&quot;,<br>“_messageCode_”: “updateRole_fail”,<br>“message”: “Updating role failed”,<br>“_statusCode_”: 400<br>}<br>&gt;&gt; Found role that needs to be refreshed — zen_user_role<br>&gt;&gt; Successfully refreshed role with extension_name — zen_user_role — status_code — 200<br>&gt;&gt; Found role that needs to be refreshed — wkc_data_scientist_role<br>&gt;&gt; Successfully refreshed role with extension_name — wkc_data_scientist_role — status_code — 200<br>&gt;&gt; Found role that needs to be refreshed — zen_developer_role<br>&gt;&gt; Successfully refreshed role with extension_name — zen_developer_role — status_code — 200<br>&gt;&gt; Found role that needs to be refreshed — wkc_business_analyst_role<br>&gt;&gt; Error refreshing role with extension_name — wkc_business_analyst_role — status_code — 400<br>{<br>“exception”: “Request body contains permissions that are not known to the platform”,<br>“_messageCode_”: “updateRole_fail”,<br>“message”: “Updating role failed”,<br>“_statusCode_”: 400<br>}<br>&gt;&gt; Found role that needs to be refreshed — zen_data_engineer_role<br>&gt;&gt; Error refreshing role with extension_name — zen_data_engineer_role — status_code — 400<br>{<br>“exception”: “Request body contains permissions that are not known to the platform”,<br>“_messageCode_”: “updateRole_fail”,<br>“message”: “Updating role failed”,<br>“_statusCode_”: 400<br>}<br>&gt;&gt; Found role that needs to be refreshed — wkc_data_quality_analyst_role<br>&gt;&gt; Error refreshing role with extension_name — wkc_data_quality_analyst_role — status_code — 400<br>{<br>“exception”: “Request body contains permissions that are not known to the platform”,<br>“_messageCode_”: “updateRole_fail”,<br>“message”: “Updating role failed”,<br>“_statusCode_”: 400<br>}<br>&gt;&gt; Found role that needs to be refreshed — wkc_data_steward_role<br>&gt;&gt; Error refreshing role with extension_name — wkc_data_steward_role — status_code — 400<br>{<br>“exception”: “Request body contains permissions that are not known to the platform”,<br>“_messageCode_”: “updateRole_fail”,<br>“message”: “Updating role failed”,<br>“_statusCode_”: 400<br>}<br>some roles were not refreshed — exit and retry</pre><p>The above errors indicate a problem while refreshing the user permission and roles.</p><p>This can happen if permissions were suddenly removed from the platform while they are still referenced by a role. An improper uninstall of WKC for example can cause this.</p><p>It’s also good practice to consult the Cloud Pak for Data known issues for WKC and CCS for potential workarounds.</p><p><a href="https://www.ibm.com/docs/en/cloud-paks/cp-data/4.6.x?topic=issues-watson-knowledge-catalog">https://www.ibm.com/docs/en/cloud-paks/cp-data/4.6.x?topic=issues-watson-knowledge-catalog</a></p><p><a href="https://www.ibm.com/docs/en/cloud-paks/cp-data/4.6.x?topic=issues-common-core-services">https://www.ibm.com/docs/en/cloud-paks/cp-data/4.6.x?topic=issues-common-core-services</a></p><p>In the case of the <em>wkc-base-roles-init</em> job failure example above, there’s a known issue already documenting the steps to take to get resolution.</p><p>If no known issues have been reported, you may need to further debug the job to get better insight on the root cause. See Debugging a Failed Job section for instructions.</p><p><strong><em>Second install failure scenario</em></strong></p><p><strong>Db2u install failed:</strong></p><p>After consulting the WKC CR status and logs we found the following:</p><p>WKC CR status:</p><blockquote>message: “WKC DB2U post install job failed (‘wkc-db2u-init’ job). Please consult the ‘wkc-db2u-init’ pod logs for more info. Job has reached the specified backoff limit\nFailed to deploy WKC prereqs\nFailed at task: \nThe error was: WKC DB2U post install job failed (‘wkc-db2u-init’ job). Please consult the ‘wkc-db2u-init’ pod logs for more info. Job has reached the specified backoff limit”</blockquote><p>WKC operator logs:</p><blockquote>TASK [0071-wkc-prereqs/wkc-prereqs : fail] *************************************<br>task path: /opt/ansible/4.7.0/roles/wkc-core/roles/0071-wkc-prereqs/wkc-prereqs/tasks/wait-wkc-db2u.yml:16<br>Tuesday 07 February 2023 01:01:41 +0000 (0:08:10.191) 0:15:22.639 ******<br>Tuesday 07 February 2023 01:01:41 +0000 (0:08:10.191) 0:15:22.639 ******<br>fatal: [localhost]: FAILED! =&gt; {“changed”: false, “msg”: “WKC DB2U post install job failed (‘wkc-db2u-init’ job). Please consult the ‘wkc-db2u-init’ pod logs for more info. Job has reached the specified backoff limit”}</blockquote><pre>oc get po -l job-name=wkc-db2u-init</pre><pre>NAME READY STATUS RESTARTS AGE<br>wkc-db2u-init-8xcww 0/1 Error 0 16m<br>wkc-db2u-init-bqkp8 0/1 Error 0 14m<br>wkc-db2u-init-6r6xv 0/1 Error 0 13m<br>wkc-db2u-init-k4fc7 0/1 Error 0 11m</pre><p>It’s also good to consult the db2u <em>statefulset</em> for signs of issues:</p><pre>oc get sts c-db2oltp-wkc-db2u</pre><pre>c-db2oltp-wkc-db2u-0      0/1           SysctlForbidden       0         0s</pre><p>The statefulset in this case is complaining about sysctls being forbidden. This is a sign that the necessary pre-req to allow unsafe sysctls was not applied on the cluster. <a href="https://www.ibm.com/docs/en/cloud-paks/cp-data/4.7.x?topic=manage-apply-db2-kubelet">https://www.ibm.com/docs/en/cloud-paks/cp-data/4.7.x?topic=manage-apply-db2-kubelet</a></p><h3>Debugging a Failed Job</h3><p>Debugging a job allows us to launch a pod and container configured the same way as if the job was triggered automatically but with the advantage that it will not actually run the startup script. This allows us to manually invoke the script and make adjustments if necessary for further debugging.</p><pre>oc debug job/wkc-base-roles-init --as-user 123456</pre><p>We must specify the — <em>as-user</em> option so make sure the permissions are set accordingly within the container, else it will try to run as root which we don’t tolerate. You can use any UID here… or use the actual projects UID.</p><pre>oc get project -o yaml | grep uid</pre><pre>Starting pod/wkc-base-roles-init-debug, command was: /bin/sh /wkc/init.sh<br>Pod IP: 11.254.7.11<br>If you don’t see a command prompt, try pressing enter.<br>sh-4.4$</pre><p>The startup script command will be displayed when the container launches, but it will not actually run.</p><p>To manually run the script simply type the command as indicated in the shell.</p><pre>sh-4.4$ /wkc/init.sh<br><br>Updating the base user with all the base roles and permissions<br>...<br>...</pre><p>If you must alter the startup shell script in any way, you can enable our container debugging tools which will install <em>vi</em> amongst other useful utilities.</p><pre>sh-4.4$ . /debug/debug_tools.sh<br>The following utilities will be installed and made available in the container:<br>Archive: /debug/tools.zip<br>inflating: /tmp/debug/free<br>inflating: /tmp/debug/pgrep<br>inflating: /tmp/debug/pidof<br>inflating: /tmp/debug/pkill<br>inflating: /tmp/debug/pmap<br>inflating: /tmp/debug/ps<br>inflating: /tmp/debug/pwdx<br>inflating: /tmp/debug/skill<br>inflating: /tmp/debug/slabtop<br>inflating: /tmp/debug/snice<br>inflating: /tmp/debug/tload<br>inflating: /tmp/debug/top<br>inflating: /tmp/debug/uptime<br>inflating: /tmp/debug/vmstat<br>inflating: /tmp/debug/w<br>inflating: /tmp/debug/watch<br>inflating: /tmp/debug/libprocps.so.7<br>inflating: /tmp/debug/libprocps.so.7.1.0<br>inflating: /tmp/debug/dnsdomainname<br>inflating: /tmp/debug/domainname<br>inflating: /tmp/debug/hostname<br>inflating: /tmp/debug/nisdomainname<br>inflating: /tmp/debug/ypdomainname<br>inflating: /tmp/debug/libtirpc.so.3<br>inflating: /tmp/debug/libtirpc.so.3.0.0<br>inflating: /tmp/debug/find<br>inflating: /tmp/debug/tar<br>inflating: /tmp/debug/vi<br>inflating: /tmp/debug/less<br>inflating: /tmp/debug/gzip<br>inflating: /tmp/debug/which<br>inflating: /tmp/debug/lsof</pre><h3>Conclusion</h3><p>It can be challenging to diagnose install/upgrade issues with WKC and its dependencies. I hope this blog was useful in providing guidance and basics on how to approach a failure and get to a potential resolution.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b30c58bff4ce" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Scaling IBM Knowledge Catalog (WKC) on Cloud Pak for Data]]></title>
            <link>https://medium.com/@dany.drouin/scaling-watson-knowledge-catalog-on-cloud-pak-for-data-11623f41f7df?source=rss-13f5331e37ff------2</link>
            <guid isPermaLink="false">https://medium.com/p/11623f41f7df</guid>
            <category><![CDATA[scaling]]></category>
            <category><![CDATA[wkc]]></category>
            <category><![CDATA[cp4d]]></category>
            <category><![CDATA[watson-knowledge-catalog]]></category>
            <category><![CDATA[cloud-pak-for-data]]></category>
            <dc:creator><![CDATA[Dany Drouin]]></dc:creator>
            <pubDate>Thu, 29 Jun 2023 17:12:40 GMT</pubDate>
            <atom:updated>2024-01-08T20:23:08.366Z</atom:updated>
            <content:encoded><![CDATA[<h3>Overview</h3><p>Scaling an application essentially ensures it can handle more users and their load at peak times by splitting up its workload and distributing it.</p><p>Scaling IKC / WKC on RedHat OpenShift can be achieved using the scale config feature of Cloud Pak for Data where the services deployed can scale and shrink when required as per certain workload specifications.</p><p>There are two types of application scaling available:</p><ul><li>Vertical Scaling</li><li>Horizontal Scaling</li></ul><p>Vertical scaling is the process of adding more resources to an existing application service. This is done by increasing the CPU and/or memory. This type of scaling is useful when you need to increase the capacity of a single service instance.</p><p>Horizontal scaling is the process of adding more pods (replicas) for an existing application service. This type of scaling is useful when you need to increase the capacity of an application but also make it more resilient.</p><p>In this article, we’ll share a few different approaches on how to scale IKC / WKC and its services for optimal performance and resiliency.</p><h3>Scale Config T-shirt sizes</h3><p>WKC supports 3 default t-shirt sizes for scaling services out of the box.</p><ul><li>small (default configuration)</li><li>medium</li><li>large</li></ul><p>To scale WKC and CCS (Common Core Services) using scaleConfig, the following patch commands can be used: (replace the SIZE variable with one of the sizes noted above)</p><pre>export SIZE=&lt;size&gt;<br><br>cpd-cli manage update-cr \<br>--component=ccs \<br>--cpd_instance_ns=${PROJECT_CPD_INSTANCE} \<br>--patch=&quot;{\&quot;scaleConfig\&quot;:\&quot;${SIZE}\&quot;}&quot;<br><br>cpd-cli manage update-cr \<br>--component=wkc \<br>--cpd_instance_ns=${PROJECT_CPD_INSTANCE} \<br>--patch=&quot;{\&quot;scaleConfig\&quot;:\&quot;${SIZE}\&quot;}&quot;</pre><p>For more details, consult the following documentation topic:</p><p><a href="https://www.ibm.com/docs/en/cloud-paks/cp-data/4.7.x?topic=services-manually-scaling">https://www.ibm.com/docs/en/cloud-paks/cp-data/4.7.x?topic=services-manually-scaling</a></p><p>It is recommended to always scale CCS along with WKC. Specially scale CCS to medium or large if you have multiple components installed on the environment, such as Watson Studio, Watson Machine Learning, DataStage. CCS contains shared services used by many CloudPak for Data components and requires specific tuning based on not only WKCs workload but other components as well.</p><h3>Comparing t-shirt sizes:</h3><p><strong>Small</strong> — Basic workloads (under 10k assets), single replica of most services. Ideal for dev and POC deployments.</p><p><strong>Medium</strong> — For small to medium size workloads (under 100k assets)</p><p><strong>Large</strong> — For larger workloads (1M assets) and production environments. Increases service resiliency and enables HA deployment.</p><p>When the OOTB t-shirt sizes are not sufficient, we can opt to scale beyond the t-shirt sizes individual services. We will cover this in detail in the custom scaling topic bellow.</p><h3>Custom Scaling</h3><p>Often the default scale configs offered by the OOTB t-shirt sizes are not sufficient. For this we can use custom scale configs in combination with the t-shirt size to scale beyond the defaults to meet workload requirements. Example, the business requires running extensive MDE jobs or very large discovery jobs, scaling to large size and beyond might be necessary.</p><p><strong>How to determine what to scale and when?</strong></p><p>Using monitoring solutions such as Grafana which is available as part of RedHat OpenShift, we can analyze the individual services resource consumption (CPU and Memory) to help determine if a given service should be scaled up (or down). There are great blogs covering this topic in detail written by fellow IBMers:</p><ul><li><a href="https://medium.com/ibm-data-ai/cloud-pak-for-data-36ba41d6dd80">https://medium.com/ibm-data-ai/cloud-pak-for-data-36ba41d6dd80</a></li><li><a href="https://community.ibm.com/community/user/cloudpakfordata/blogs/hong-wei-jia1/2021/09/12/cpd-performance-troubleshooting-best-practice-p2">https://community.ibm.com/community/user/cloudpakfordata/blogs/hong-wei-jia1/2021/09/12/cpd-performance-troubleshooting-best-practice-p2</a></li></ul><p>Note these articles were written a while back for version 3.5.x but are still relevant today with newer levels of Cloud Pak for Data.</p><p><strong>Key things to look for:</strong></p><ul><li>Extensive CPU throttling</li><li>Memory usage at or near the limit.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/803/1*Bef3HZaXTnO4FsMLRp7jLQ.png" /><figcaption>Sample Grafana pod view showing very high CPU throttling</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/803/1*tnIBRX-ADMNGeXSr6HfC6Q.png" /><figcaption>Sample Grafana pod view of the memory usage/limit. Usage is at 5.71GB of 6GB Limit</figcaption></figure><p>For a list of services which support scaling and their different sizes, consult the scale-config deployment yamls from within the CCS and WKC operators.</p><p>To view the t-shirt sizes for WKC:</p><pre>oc exec -it -n ibm-common-services $(oc get po -n ibm-common-services -l app.kubernetes.io/name=ibm-cpd-wkc-operator -o name) -- bash -c &quot;cat /opt/ansible/4.7.0/roles/wkc-core/scale-config/small.yaml&quot;</pre><p>To view the t-shirt sizes for CCS:</p><pre>oc exec -it -n ibm-common-services $(oc get po -n ibm-common-services -l app.kubernetes.io/name=ibm-cpd-ccs-operator -o name) -- bash -c &quot;cat /opt/ansible/7.0.0/scale-config/small.yaml&quot;</pre><p><em>**Replace the namespace and WKC/CCS version accordingly</em></p><p>Sample output of the first few services:</p><pre>oc exec -it -n ibm-common-services $(oc get po -n ibm-common-services -l app.kubernetes.io/name=ibm-cpd-wkc-operator -o name) -- bash -c &quot;cat /opt/ansible/4.7.0/roles/wkc-core/scale-config/small.yaml&quot;<br># finley-public<br>finley_public_min_replicas: 1<br>finley_public_max_replicas: 1<br>finley_public_resources:<br>  requests:<br>    cpu: 30m<br>    memory: 300Mi<br>    ephemeral-storage: 50Mi<br>  limits:<br>    cpu: 1000m<br>    memory: 4096Mi<br>    ephemeral-storage: 1Gi<br><br># wkc-data-rules<br>wkc_data_rules_min_replicas: 1<br>wkc_data_rules_max_replicas: 1<br>wkc_data_rules_resources:<br>  requests:<br>    cpu: 100m<br>    memory: 800Mi<br>    ephemeral-storage: 50Mi<br>  limits:<br>    cpu: 1<br>    memory: 2048Mi<br>    ephemeral-storage: 1Gi<br><br># wkc-gov-ui<br>wkc_gov_ui_min_replicas: 1<br>wkc_gov_ui_max_replicas: 1<br>wkc_gov_ui_resources:<br>  requests:<br>    cpu: 50m<br>    memory: 300Mi<br>    ephemeral-storage: 50Mi<br>  limits:<br>    cpu: 1000m<br>    memory: 600Mi<br>    ephemeral-storage: 1Gi<br>…</pre><p>Let’s compare small to the large t-shirt size:</p><pre>oc exec -it -n ibm-common-services $(oc get po -n ibm-common-services -l app.kubernetes.io/name=ibm-cpd-wkc-operator -o name) -- bash -c &quot;cat /opt/ansible/4.7.0/roles/wkc-core/scale-config/large.yaml&quot;<br># finley-public<br>finley_public_min_replicas: 3<br>finley_public_max_replicas: 3<br>finley_public_resources:<br>  requests:<br>    cpu: 30m<br>    memory: 300Mi<br>    ephemeral-storage: 50Mi<br>  limits:<br>    cpu: 1000m<br>    memory: 4096Mi<br>    ephemeral-storage: 1Gi<br><br># wkc-data-rules<br>wkc_data_rules_min_replicas: 3<br>wkc_data_rules_max_replicas: 3<br>wkc_data_rules_resources:<br>  requests:<br>    cpu: 100m<br>    memory: 800Mi<br>    ephemeral-storage: 50Mi<br>  limits:<br>    cpu: 1<br>    memory: 2048Mi<br>    ephemeral-storage: 1Gi<br><br># wkc-gov-ui<br>wkc_gov_ui_min_replicas: 3<br>wkc_gov_ui_max_replicas: 3<br>wkc_gov_ui_resources:<br>  requests:<br>    cpu: 50m<br>    memory: 300Mi<br>    ephemeral-storage: 50Mi<br>  limits:<br>    cpu: 1000m<br>    memory: 600Mi<br>    ephemeral-storage: 1Gi<br>…</pre><p>You can see that many of the services scale up horizontally by increasing the number of replicas. This increases service resiliency and enables an HA deployment.</p><p><strong><em>Scaling Individual Services</em></strong></p><p>Scaling services using a custom set of resource definitions consist of patching the WKC or CCS CR (Custom Resources). These CRs are the source of truth for any customizations done to the services. Users should not patch/edit the Deployments or StatefulSets directly since an operator reconcile will revert any changes made. Instead, the user should add any customization to the CR, and wait for the operator to finish reconciling, monitor the WKC and/or CCS CR status until they are <em>Completed</em>. We’ll show some examples of some of the most popular services that require special tuning, but others can be scaled using the same approach. Consult the scale-config yamls for a complete list of services and their resource definitions.</p><p><strong><em>Scaling CouchDB</em></strong></p><p>CouchDB is used for storing projects, catalogs, and asset metadata on the platform. The CouchDB cluster is fully HA and consist of 3 replicas, this, regardless of the scale config selected. Currently, we do not support scaling CouchDB horizontally beyond the 3 default replicas due to complex rebalancing needed, but we do support scaling vertically by allocating more CPU and memory to the containers.</p><p>Here’s an example patch command scaling up the couchdb and search engine containers.</p><pre>oc patch ccs ccs-cr --type merge --patch &#39;{&quot;spec&quot;: {<br>&quot;couchdb_resources&quot;:{<br>    &quot;requests&quot;:{&quot;cpu&quot;: &quot;3&quot;, &quot;memory&quot;: &quot;256Mi&quot;}, <br>    &quot;limits&quot;:{&quot;cpu&quot;: &quot;16&quot;, &quot;memory&quot;: &quot;16Gi&quot;}<br>},<br>&quot;couchdb_search_resources&quot;:{<br>    &quot;requests&quot;:{&quot;cpu&quot;: &quot;250m&quot;, &quot;memory&quot;: &quot;256Mi&quot;},<br>    &quot;limits&quot;:{&quot;cpu&quot;: &quot;4&quot;, &quot;memory&quot;: &quot;6Gi&quot;}<br>}<br>}}&#39;</pre><p>**Sample patch command increasing cpu to 16 and memory to 16GB for CouchDB core process. The search container is also increased to 4 and 6GB. This will patch the CCS CR.</p><p>This will trigger a reconciliation of CCS. You can monitor its progress by watching the ccsStatus field of the CCS CR. You can confirm the changes were applied to the CCS CR using:</p><pre>oc get ccs ccs-cr -o yaml</pre><p><strong><em>Scaling WKC Services</em></strong></p><p>Here’s a look at key services that may require custom scaling to better improve the performance of WKC workloads.</p><p>Governance UI service. An increased number of concurrent users managing glossaries, business terms, workflows may require scaling the Gov. UI Service.</p><pre>oc patch wkc wkc-cr --type merge --patch &#39;{&quot;spec&quot;: {<br>&quot;wkc_gov_ui_min_replicas&quot;: 4, &quot;wkc_gov_ui_max_replicas&quot;: 4,<br>&quot;wkc_gov_ui_resources&quot;:{<br>    &quot;requests&quot;:{&quot;cpu&quot;: &quot;1&quot;, &quot;memory&quot;: &quot;300Mi&quot;}, <br>    &quot;limits&quot;:{&quot;cpu&quot;: &quot;4&quot;, &quot;memory&quot;: &quot;4Gi&quot;}<br>}<br>}}&#39;</pre><p>**Sample patch command increasing cpu to 4 and memory to 4GB for Gov UI. It also scales replicas to 4. This will patch the WKC CR.</p><p>Lineage Activities Service. An increased in the number of asset creation (due to an import for example) could lead to activities service not being able to keep up with requests. Scaling up the cpu and replicas might be required.</p><pre>oc patch wkc wkc-cr --type merge --patch &#39;{&quot;spec&quot;: {<br>&quot;wdp_lineage_min_replicas&quot;: 4, &quot;wdp_lineage_max_replicas&quot;: 4,<br>&quot;wdp_lineage_resources&quot;:{<br>    &quot;requests&quot;:{&quot;cpu&quot;: &quot;80m&quot;, &quot;memory&quot;: &quot;450Mi&quot;},<br>    &quot;limits&quot;:{&quot;cpu&quot;: &quot;2&quot;, &quot;memory&quot;: &quot;2Gi&quot;}<br>}<br>}}&#39;</pre><p>**Sample patch command increasing cpu to 2 and replicas to 4 for the Activities service. This will patch the WKC CR.</p><p><strong><em>Scaling WKC Db2u instance</em></strong></p><p>Consult the following documentation on how to scale the WKC db2u instance:</p><p><a href="https://www.ibm.com/docs/en/cloud-paks/cp-data/4.7.x?topic=administering-scaling-up-db2-watson-knowledge-catalog">https://www.ibm.com/docs/en/cloud-paks/cp-data/4.7.x?topic=administering-scaling-up-db2-watson-knowledge-catalog</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=11623f41f7df" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>