Authors: Kuassi Mensah — Paramdeep Saini— Sanjay.Singh

The journey to making enterprise databases truly Kubernetes-native just took a significant leap forward. Oracle has released version 2.0.0 of the Oracle Database Operator for Kubernetes (a.k.a. OraOperator), bringing enterprise-grade database management into the cloud-native era with enhanced automation, expanded platform support, and powerful new features.

Lifecycle Automation at Scale

OraOperator v2.0.0 automates complex database operations that traditionally required manual intervention:

• Provisioning and cloning across multiple database types
• In-place and out-of-place patching for seamless updates
• Backup and restore operations with long-term backup support
• Scaling operations for both compute and storage resources
• Data Guard configuration and switchover operations

What Makes This Release Special?

Version 2.0.0 represents a major milestone in Oracle’s commitment to Kubernetes-native database operations.

Red Hat OpenShift Partner Validation: Enterprise Ready

One of the most significant additions is full validation of Red Hat OpenShift. OraOperator and its controllers have been validated and included in the Red Hat Operators Catalog, making it easier than ever for enterprises running OpenShift to deploy and manage Oracle databases with confidence.

The Oracle Container Images (OCR) are available for deployment on Red Hat OpenShift, including: Oracle AI Database 26ai Free, SIDB 19.28, GDD(GSM) 26ai Free, and Oracle Restart Database 19.28.

Oracle Restart Controller: Enhanced Infrastructure Management

The new Restart Controller enables dynamic infrastructure management, such as provisioning, adding, and deleting ASM disks, bringing advanced storage automation to Kubernetes-managed databases and supporting deployments with GI and ASM.
When storage is added or removed using Kubernetes Persistent Volume Claims (PVCs), Oracle Restart and ASM automatically handle these changes.

Broader Integration with Private Cloud Appliance (PCA)

The Database Operator for Kubernetes add-on on the Oracle Kubernetes Engine (OKE) now supports Oracle Private Cloud Appliance (PCA) with platform release 3.12 and is available through the OKE console with Database Operator V1.2. Additionally, Database Operator V2.0 offers on-demand updates, with planned integration in the next platform release 3.13, enhancing deployment flexibility and supporting a wider range of Oracle environments.

ORDS Services Enhancements

The enhanced Oracle REST Data Services (ORDS) support now includes:

• Service Account integration with OpenShift support
• Compatibility with ORDS image 25.1 and above
• Flexible APEX installation: provide files via download URL or persistent volume, addressing the need for missing installation files

True Cache Support

Preview support for True Cache with Free SIDB databases brings Oracle’s high-performance caching to Kubernetes deployments.

Expanded Database and Platform Support

Version 2.0.0 supports an impressive range of Oracle database configurations:

• Autonomous Database (ADB-S and ADB-D): Includes manual failover and switchover
• Oracle Database 26ai: Full support, including Free and Free-lite editions
• Globally Distributed Databases 26ai: Manage sharded topologies and Raft replication
• Multitenant: Improved PDB management via a new LRES-based controller, enhanced config map support, and deletion policy control

Broad Kubernetes Platform Support

This production release has been thoroughly tested across the Kubernetes ecosystem:

• Oracle Container Engine for Kubernetes (OKE)
• RedHat OpenShift (v4.16+)
• Oracle Linux Cloud Native Environment (OLCNE)
• Google Kubernetes Engine (GKE)
• Azure Kubernetes Service (AKS)
• Amazon Elastic Kubernetes Service (EKS)
• Minikube (for development)

Also:

• Operator Lifecycle Manager (OLM) support
• Upgraded Kubernetes API (v4) for enhanced compatibility

High Availability, Observability, and Integrations

Data Guard Enhancements

Includes an Observer for FastStartFailover and Snapshot Standby, raising high availability for databases in containers.

Database Observability

Native support for database logs and metrics through the latest exporter container images enables seamless integration with Kubernetes monitoring stacks such as Prometheus and Grafana.

Enterprise Cloud Integrations

Added deployment support for Database Operator on OKE console for Private Cloud Appliance (PCA) and Compute Cloud@Customer (C3) for true hybrid cloud operations.

Getting Started

Ready to make your Oracle databases Kubernetes-native?

The OraOperator v2.0.0 is available via:

• GitHub
• Oracle Container Registry
• Operatorhub.io
• Red Hat Operators Catalog (upon final approval)

The Bottom Line

Oracle Database Operator for Kubernetes v2.0.0 represents a mature, production-ready solution for running Oracle databases in Kubernetes. Whether you’re managing a single containerized database or orchestrating a globally distributed database topology, OraOperator brings the power of declarative configuration and Kubernetes automation to Oracle Database operations.

For organizations committed to Kubernetes as their application platform, v2.0.0 removes the traditional barriers between stateful database workloads and cloud-native infrastructure. The result is faster deployments, simplified operations, and the ability to manage databases using the same tools and patterns as your applications.

How We Automated Database Provisioning from Days to Minutes (And the Problems We Solved Along the Way)

Kuassi Mensah

“Can you provision a database for my feature branch?”

This simple request used to trigger a 2–3 day process: create a ticket, wait for the DBA team to pick it up, manual database creation, configuration, connection details sent back, inevitably something is misconfigured, back to the DBA team…

By the time the database was ready, the developer had moved on to something else, context was lost, and momentum was dead.

We decided to fix this. What started as a simple automation project became a deep dive into Kubernetes internals, authentication patterns, and the challenges of managing stateful applications in cloud-native environments.

This is the story of that implementation — the problems we faced, the solutions we found, and the lessons we learned.
I presented this with a live workflow at the recent Oracle AI World in Las Vegas. See my slides.

The Problem: Development Database Bottleneck

Our development workflow was broken. We had three teams working on different features of our application, all needing their own database environments for testing. But our infrastructure couldn’t keep up:

For Developers:

• Shared development database meant constant conflicts — your schema change broke someone else’s work
• Manual provisioning took days, killing momentum
• No two development environments were configured the same way
• “Works on my machine” problems were constant

For DBAs:

• Managing 15+ manual database requests per week
• Tracking which database belongs to which feature in spreadsheets
• Forgotten test databases are consuming cloud resources and budget
• No standardization — every database was a snowflake

For the Business:

• Features are taking 20–30% longer due to infrastructure delays
• Monthly cloud bills included $5,000 in orphaned test databases
• Production issues from schema changes that weren’t properly tested
• Developer frustration leading to retention concerns

We needed a better way.

The Vision: Database-as-Code

The initial implementation was performed by Norman Aberin’s work; we envisioned a simple workflow:

1. The developer creates a feature branch: feature/user-authentication
2. Developer pushes code
3. Magic happens
4. 15 minutes later, they have a fully configured Oracle database with schemas deployed
5. A developer works on their feature with an isolated database
6. The developer deletes the branch
7. The database automatically cleans up

No tickets. No waiting. No manual intervention.

Infrastructure-as-Code, but for databases.

The Architecture

Here’s what you can achieve (I’ve only implemented a test-db feature branch)

Key Components:

• Oracle Kubernetes Engine (OKE): Our container orchestration platform
• Oracle Database Operator: Manages the database lifecycle as a Kubernetes resource
• GitHub Actions: Orchestrates the entire workflow
• Kustomize: Handles configuration management
• Liquibase: Manages schema versioning and deployment

Each feature branch gets its own namespace in the Kubernetes cluster, complete with an isolated Oracle database instance.

The flow looks like this:

GitHub Repository (feature branches)
           ↓
    GitHub Actions (triggered on push)
           ↓
    OKE Cluster
      ├─ Namespace: feature-user-auth
      │    ├─ Oracle Database Pod
      │    ├─ LoadBalancer Service
      │    └─ Persistent Volume
      └─ Namespace: feature-payment-system
           ├─ Oracle Database Pod
           ├─ LoadBalancer Service
           └─ Persistent Volume

The Journey: Four Major Challenges

What seemed straightforward in theory hit reality hard. Here are the four major technical challenges we faced and how we solved them.

Challenge 1: Authentication Hell

The Problem

We needed GitHub Actions to authenticate with our OKE cluster. Oracle provides a GitHub Action (oracle-actions/configure-kubectl-oke) that's supposed to handle this. We followed the documentation, configured our secrets, ran the workflow, and...

Error: Authentication failed
The required information to complete authentication 
was not provided or was incorrect.

We spent three days trying different combinations of credentials, regions, and configuration options. The error messages were generic and unhelpful.

The Investigation

OCI authentication uses a private key and a fingerprint of that key’s public component. The Oracle action expected us to provide the fingerprint as a GitHub secret. But we kept getting mismatches — the fingerprint we stored didn’t match what the actual key generated.

Then the breakthrough: What if we don’t store the fingerprint at all? What if we generate it from the private key at runtime?

The Solution

- name: Setup OCI CLI and Configure kubectl
  run: |
    # Install OCI CLI
    bash -c "$(curl -L https://raw.githubusercontent.com/oracle/oci-cli/master/scripts/install/install.sh)" -- --accept-all-defaults
    export PATH="$HOME/bin:$PATH"
    
    # Create private key from GitHub secret
    mkdir -p ~/.oci
    echo "${{ secrets.OCI_KEY_FILE }}" > ~/.oci/key.pem
    chmod 600 ~/.oci/key.pem
    
    # Generate fingerprint from actual key (not from secret!)
    FINGERPRINT=$(openssl rsa -pubout -outform DER \
      -in ~/.oci/key.pem 2>/dev/null | \
      openssl md5 -c | awk -F'= ' '{print $2}')
    
    # Create OCI config with generated fingerprint
    cat > ~/.oci/config << EOF
    [DEFAULT]
    user=${{ secrets.OCI_USER_OCID }}
    fingerprint=${FINGERPRINT}
    tenancy=${{ secrets.OCI_TENANCY_OCID }}
    region=${{ secrets.OCI_REGION }}
    key_file=$HOME/.oci/key.pem
    EOF
    
    # Configure kubectl
    $HOME/bin/oci ce cluster create-kubeconfig \
      --cluster-id ${{ secrets.OKE_CLUSTER_OCID }} \
      --file ~/.kube/config \
      --region '${{ secrets.OCI_REGION }}' \
      --token-version 2.0.0

Key Insight

Store only what you must. Generate what you can.

The fingerprint is derived from the key, so derive it at runtime. This guarantees they match because they come from the same source.

Impact: Authentication went from 0% success rate to 100%.

Challenge 2: The Kustomize Naming Nightmare

The Problem

We had a base database configuration named freedb-lite-sample. We wanted to add the branch name as a suffix, so for the branchtest-db, we'd get db-test-db. Simple, right?

We used Kustomize’s namesuffix feature:

kustomize edit set namesuffix test-db

The result? freedb-lite-sampletest-db

Kustomize appended the suffix directly to the existing name. Our monitoring scripts were looking for db-test-db, but the database was actually named freedb-lite-sampletest-db.

We spent hours debugging “database not found” errors when the database was right there — just with the wrong name.

The Solution

We switched namesuffix to JSON patches, which give complete control:

cat > kustomization.yaml << EOF
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: feature-test-db

resources:
  - base/sidb-free-lite

patches:
  - patch: |-
      - op: replace
        path: /metadata/name
        value: db-test-db
    target:
      kind: SingleInstanceDatabase
EOF

The patch says: “Find the SingleInstanceDatabase resource and replace its metadata name with exactly this value." No ambiguity, full control.

Lesson Learned

Kustomize’s convenience features are great for simple cases, but for precise control, JSON patches are worth the extra verbosity.

Challenge 3: Namespaces That Won’t Die

The Problem

This was perhaps the most frustrating issue. We’d run our delete workflow to clean up an old database. The workflow completed successfully. But when we tried to create a new database with the same name:

Error from server (AlreadyExists): object is being deleted: 
namespaces "feature-test-db" already exists

The namespace was stuck in the “Terminating” state. For hours. In one case, over 6 hours.

Kubernetes was trying to delete the namespace, but couldn’t because resources inside it had “finalizers” — cleanup handlers that must complete before deletion. The SingleInstanceDatabase custom resource had a finalizer for proper database shutdown. Something was preventing that finalizer from completing, so the namespace was stuck in limbo forever.

Developers couldn’t create new databases because the namespace name was technically “taken” even though nothing was actually running.

The Investigation

When I examined the namespace status, I found:

"message": "Some content in the namespace has finalizers remaining: 
database.oracle.com/singleinstancedatabasefinalizer in 1 resource instances"

The database resource was blocking namespace deletion. But the database pod was long gone. The finalizer was waiting for something that would never happen.

The Solution

We had to force the issue by removing finalizers:

- name: Force delete stuck namespace and resources
  run: |
    NAMESPACE="feature-${{ steps.getname.outputs.idname }}"
    
    if kubectl get namespace $NAMESPACE 2>/dev/null | grep -q Terminating; then
      echo "Namespace stuck, removing resources..."
      
      # Remove finalizers from database resources first
      for db in $(kubectl get singleinstancedatabase -n $NAMESPACE -o name); do
        kubectl patch $db -n $NAMESPACE \
          -p '{"metadata":{"finalizers":[]}}' --type=merge
      done
      
      # Remove finalizers from namespace
      kubectl patch namespace $NAMESPACE \
        -p '{"metadata":{"finalizers":[]}}' --type=merge
      
      # Force delete if still exists
      sleep 15
      kubectl delete namespace $NAMESPACE --force --grace-period=0
    fi

We run this check before creating a new namespace. If an old one is stuck, we forcibly remove the finalizers and delete it.

Lessons Learned

Kubernetes finalizers are powerful for ensuring proper cleanup, but they can create deadlocks.

Always have a force-cleanup mechanism for production workflows. Custom Resource Definitions are especially prone to this issue.

Impact: Workflow success rate went from ~60% (blocked by stuck namespaces) to 95%+.

Challenge 4: The Connectivity Conundrum

The Problem

Database created successfully. Pod running. Everything looked perfect in the Kubernetes dashboard. But when Liquibase tried to connect from GitHub Actions:

ORA-12541: Cannot connect. No listener at host 10.0.10.10 port 30328
nc: connect to 10.0.10.10 port 30328 (tcp) failed: Connection timed out

We verified credentials. Checked firewall rules. Tested from different networks. Nothing worked.

Then I realized: 10.0.10.10 is an internal Kubernetes IP (ClusterIP). It's only accessible from inside the cluster. GitHub Actions runs on GitHub's infrastructure, completely outside our cluster. They're in different networks with no connectivity.

The Root Cause

The Oracle Database Operator created a ClusterIP service by default:

apiVersion: v1
kind: Service
metadata:
  name: db-service
spec:
  type: ClusterIP  # Internal only!
  clusterIP: 10.0.10.10
  ports:
  - port: 1521

ClusterIP is perfect for pod-to-pod communication within the cluster. But it’s invisible to the outside world.

The Solution

We needed to expose the database externally via a LoadBalancer:

- name: Expose database externally
  run: |
    kubectl apply -f - <<EOF
    apiVersion: v1
    kind: Service
    metadata:
      name: db-external
      namespace: feature-test-db
    spec:
      type: LoadBalancer
      selector:
        app: db-test-db
      ports:
      - port: 1521
        targetPort: 1521
    EOF
    
    # Wait for external IP
    kubectl wait --for=jsonpath='{.status.loadBalancer.ingress[0].ip}' \
      service/db-external --timeout=300s

In Oracle Cloud, the LoadBalancer service automatically provisions an OCI Load Balancer with a public IP address. The flow becomes:

Liquibase (GitHub Actions) 
  → External IP (OCI Load Balancer) 
  → ClusterIP (Kubernetes Service) 
  → Database Pod

We wait for the external IP assignment (takes 2–3 minutes) before proceeding with Liquibase.

Lessons Learned

Kubernetes networking has layers. ClusterIP is internal-only, NodePort exposes on node IPs, and LoadBalancer gets a cloud provider’s load balancer. Choose based on your access patterns.

Security Note: For production, you’d restrict access to this LoadBalancer (IP allowlists, VPN, etc.). For temporary development databases with non-sensitive data, external access is acceptable.

Alternative Approach: We’re considering running Liquibase as a Kubernetes Job inside the cluster for future iterations. This would keep the database internal-only and eliminate the need for external exposure.

Real World Performance

Here’s what a typical execution looks like:

What you get:

• 1 Oracle Database instance (19c or 21c)
• 1 Kubernetes namespace (isolated)
• 3+ application schemas (fully deployed)
• 1 external IP for connectivity
• Automatic cleanup on branch deletion

Compare this to our previous process: a minimum of 2–3 days.

The Impact

Quantitative Results

Before:

• Average database provisioning time: 2–3 days
• Monthly orphaned database costs: ~$5,000
• Developer satisfaction score (infrastructure): 4.2/10
• Features delayed by database availability: 20–30%

After:

• Average database provisioning time: 15–20 minutes (99.5% reduction)
• Monthly orphaned database costs: ~$200 (96% reduction)
• Developer satisfaction score (infrastructure): 8.7/10
• Features delayed by database availability: <2%

Qualitative Benefits

Developers:

“I can experiment freely without worrying about breaking someone else’s work”

“Testing schema changes is so much easier now”

“I love that I can just push code and come back to a ready database”

DBAs:

“I’ve gone from managing 15+ manual requests per week to zero”

“Everything is standardized now — I know exactly how every database is configured”

“The automatic cleanup is amazing for cost control”

Project Managers:

“Features are shipping faster”

“Our cloud bill dropped significantly”

“Developers are happier, which helps with retention”

Lessons Learned

1. Authentication: Store Less, Generate More

Don’t store credentials you can derive. We went from storing five pieces of authentication data to storing two (private key and region), generating everything else at runtime. This reduced secret management complexity and eliminated mismatch errors.

2. Kubernetes: Plan for Finalizers

Custom Resource Definitions often have finalizers. In a perfect world, they clean up gracefully. In reality, they can deadlock. Always have force-cleanup mechanisms for production workflows.

3. Networking: Internal vs External Matters

ClusterIP vs LoadBalancer isn’t just about exposure — it fundamentally changes who can access your services. Understand your access patterns before choosing service types.

4. Debugging: Add Observability Early

Every step in our workflow now echoes what it’s doing and the results. When something fails at 2 AM, these logs are invaluable. Don’t wait for production failures to add debugging.

5. Automation: Perfect is the Enemy of Done

Our first version worked for 60% of cases. We shipped it, learned from failures, and iterated. Waiting for perfection would have meant developers still waiting days for databases.

Future Improvements

We’re not done. Here’s what we’re planning:

Security:

• Run Liquibase as a Kubernetes Job (keep databases internal)
• Add network policies (restrict database access to specific pods)
• Migrate from GitHub Secrets to OCI Vault (better secret management)
• Implement TLS/SSL for database connections

Operations:

• Automated backups before schema changes
• Prometheus metrics for database health
• Cost tracking by feature branch
• Slack notifications for deployment status

Developer Experience:

• Database seeding with realistic test data
• Schema diff tool (compare feature branch to main)
• Direct kubectl access for troubleshooting
• Rollback capability for schema changes

Tools and Resources

This implementation wouldn’t have been possible without standing on the shoulders of others:

Core Technologies:

• Oracle Database Operator for Kubernetes
• Oracle Kubernetes Engine (OKE)
• GitHub Actions
• Liquibase
• Kustomize

Inspiration:

• Norman Aberin’s blog post provided the initial architecture and approach

Documentation & Community:

• Oracle Cloud Infrastructure documentation
• Kubernetes documentation (especially on finalizers and service types)
• Stack Overflow for specific troubleshooting
• Claude AI for debugging strategies and troubleshooting guidance

Try It Yourself

Want to implement this in your environment? Here’s where to start:

GitHub - Kuassim/AiWorld: Oracle AI WOrld demo

1. Set up an OKE cluster with the Oracle Database Operator installed
2. Configure GitHub secrets for OCI authentication
3. Create base database configuration using Oracle DB Operator manifests
4. Implement the create workflow, starting with authentication
5. Add Kustomize for configuration management
6. Integrate Liquibase for schema deployment
7. Build the delete workflow for cleanup

Start simple. Get authentication working first. Then add one piece at a time. Each challenge we faced taught us something valuable about the technology stack.

Conclusion

From 2–3 days to 15–20 minutes. From manual spreadsheet tracking to automated GitOps. From $5,000 in wasted resources to $200. From frustrated developers to delighted ones.

But more than the metrics, what we built is a culture shift. Databases are no longer special snowflakes that require manual care. They’re infrastructure-as-code, just like everything else in our cloud-native stack.

Every developer on our team can now provision a database with a git push. They can experiment freely, test thoroughly, and ship confidently. The database is no longer a bottleneck — it's an enabler.

That’s the real value of automation: not just time saved, but possibilities created.

Have you implemented similar database automation in your environment? What challenges did you face? I’d love to hear your experiences in the comments.

Tags: #Kubernetes #DevOps #Oracle #GitHubActions #DatabaseAutomation #CloudNative #InfrastructureAsCode #OCI #Liquibase #Kustomize

As part of Oracle’s Multi-Cloud partnership with third-party Clouds, including Azure, GCP, and AWS, this blog post describes the steps for installing the Oracle Database Operator for Kubernetes (a.k.a. OraOperator) on the Google Kubernetes Engine (GKE) and provisioning a Single Instance Database (a.k.a. SIDB).

Prerequisites — Create a GKE Cluster

You first need to create a Google Kubernetes Engine (GKE) cluster. The Google Cloud Platform (GCP) offers two modes for provisioning GKE:

1. The Autopilot mode, allows you to focus on your services and applications, while GKE manages your nodes and infrastructure. We did not perform our tests using this mode.
2. The standard mode lets you control the configuration of the individual nodes.

For our validation of OraOperator on GKE, we created a single node cluster in standard mode, using the Google Cloud Console.

Alternatively, you may use this script by the Google folks to create the same cluster.

./oopgke.bash step1_create_infra

Then use the following links to configure your client machine for accessing the GKE cluster using kubectl

1. Install gcloud CLI
2. Install gcloud plugin to access GKE using kubectl

At this stage, you have a 1-node GKE cluster installed and your client has been configured to access it using kubectl.

kubectl get nodes -o wide

The next steps, which consist of installing OraOperatore and provisioning a Single Instance Database, are business as usual, i.e., similar for all CNCF-compliant Kubernetes engines.

Install OraOperator on GKE

1. Install the certificate manager using the following command

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.2/cert-manager.yaml

2. Namespace deployment

To reduce the scope of OraOperator deployment and monitoring to the default namespace, as opposed to a cluster-wide deployment, you need to grant service account with resource access to the default namespace, using the following command

kubectl apply -f rbac/default-ns-role-binding.yaml

3. Edit the oracle-database-operator.yaml to add the default namespace under WATCH_NAMESPACE then save it.

- name: WATCH_NAMESPACE
  value: "default"

4. Finally, apply the just saved yaml file using the following command; this will install/deploy OraOperator.

kubectl apply -f oracle-database-operator.yaml

5. Check that the operator is up and running using the following command

For a wide view use the following command:

$ kubectl get all -n oracle-database-operator-system -o wide

For a narrow view, use the following command:

$ kubectl get pods -n oracle-database-operator-system

Provisioning a Single Instance Database (SIDB)

A single-instance database has a one-to-one relationship between the database server instance (memory and processes) and the database itself (i.e., the database files). It is the most basic deployment option of the Oracle Database.

Ensure Sufficient Disk Space in the Kubernetes Worker Node

The SIDB container images are stored on the Kubernetes worker node. These images can be as large as 10 GB therefore, you need to ensure the K8s nodes have enough free space

Resource Privileges

The SIDB controller needs the following Kubernetes privileges.

The controller automatically acquires these privileges from the cluster, you need not take any action. Additional optional Kubernetes resource privileges may also be specified (see the OraOperator SIDB doc for more details).

Prepare the Oracle Container Image

The supported Oracle Database releases include Enterprise and Standard Editions for Oracle Database 19c and later, Express Edition for Oracle Database 21.3.0 only, Oracle Database Free 23.6.0 and later, and Oracle Database Free lite 23.6 and later.

You can build a SIDB image from the source by following these instructions or, better, use the pre-built images available here (for the Enterprise edition) and here (for the Free edition).
You need to sign in and accept the required license agreement if you have not already done so.

To pull the image from the Oracle Container Registry, you need to create a Kubernetes secret (named oracle-container-registry-secret for example) as follows:

$ kubectl create secret docker-registry oracle-container-registry-secret --docker-server=container-registry.oracle.com --docker-username='<oracle-sso-email-address>' --docker-password='<container-registry-auth-token>' --docker-email='<oracle-sso-email-address>'

Alternatively, use the docker config.json or podman auth.json (after successfully logging in).

$ docker login container-registry.oracle.com
$ kubectl create secret generic oracle-container-registry-secret  --from-file=.dockerconfigjson=.docker/config.json --type=kubernetes.io/dockerconfigjson

or

$ podman login container-registry.oracle.com
$ kubectl create secret generic oracle-container-registry-secret  --from-file=.dockerconfigjson=${XDG_RUNTIME_DIR}/containers/auth.json --type=kubernetes.io/dockerconfigjson

Edit the ## Database image details section of the singleinstancedatabase_create.yaml file as follows:

## Database image details
image:
  pullFrom: container-registry.oracle.com/database/enterprise_ru:21.17.0.0
  pullSecrets: oracle-container-registry-secret

Set Up Volumes for Database Persistence

Volumes are required for the persistent storage of the database files. There are two options: dynamic persistence or static persistence.

• Dynamic Persistence: in this mode, a persistent volume is provisioned by mentioning a Cloud infrastructure-specific storage class in the singleinstancedatabase_create.yaml file. The supported access mode for this class is ReadWriteOnce.The Reclaim Policy of such dynamically provisioned volumes is Deletei.e., meaning that the volumes are deleted when their corresponding database deployment is deleted.
• Static Persistence: in this mode, you must create a volume manually, and then use the name of this volume with the <.spec.persistence.datafilesVolumeName> field, which corresponds to the datafilesVolumeName field of the persistence section in the singleinstancedatabase_create.yaml file.

For this validation test, we will use the dynamic persistence option. Let’s get details of the available storage classes in GCP using the following command:

$ kubectl get sc

We will use “standard-rwo”, the default storage class.

Kubernetes Secret

The following command creates a Kubernetes secret named “db-admin-secret” (for SYS, SYSTEM, and SYSADMIN users), with the oracle_pwd key mapped to the actual password specified in the command.

$ kubectl create secret generic db-admin-secret --from-literal=oracle_pwd=<specify password here>

Note: theadminPassword field in the singleinstancedatabase.yaml file refers to that secret.

Check the successful creation of the secret using the following command:

$ kubectl get secret -n default

Provision the SIDB

Edit the singleinstancedatabase_create.yaml file and replace:
storageClass: “oci-bv” by “standard-rwo”

Save then apply the just edited file, using

$ kubectl apply -f singleinstancedatabase_create.yaml

Wait for the creation notification singleinstancedatabase.database.oracle.com/sidb-sample created

After the creation, the SIDB needs to be deployed, which takes a few minutes. You can monitor the progress of the provisioning in the log file using the following command:

$ kubectl logs -f  pod/sidb-sample-8hmw7

The verbose output is not displayed here however, the following message will indicate the completion of the deployment.

Get the status of the deployment using the following command (the status should be “Running”):

$ kubectl get all -o wide

Get a summary of the deployed SIDB as follows:

$ kubectl get singleinstancedatabase sidb-sample -n default

Get more details as follows

$ kubectl describe singleinstancedatabase sidb-sample -n default

Output not shown, too verbose and lengthy!

Confirm that the database is healthy

$ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}"

Healthy

Get the CDB connect string

$ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectString}"

34.70.98.61:31625/ORCL1

Get the PDB connect string

$ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.pdbConnectString}"

34.70.98.61:31625/ORCLPDB1

Configuring a FrontEnd Load Balancer

By default, external connections from outside the K8s cluster to the database are performed through the NodePort service (30000-32767 range) however, in Cloud environments, the best practice is to front-end the SIDB with a load balancer. Configuring it is as simple as setting the loadBalancer attribute to true in the singleinstancedatabase_create.yaml file as follows.

loadBalancer: true

Alternatively, use the kubectl patch command as follows:

$ kubectl --type=merge -p '{"spec":{"loadBalancer": true}}' patch singleinstancedatabase sidb-sample

After a few minutes, the load balancer will be provisioned, and an external IP will be assigned.

$ kubectl get all

Secure TCPS Connection

To increase security, you can configure the tcpsListenerPort service thereby restricting external communications to the SIDB through the TCPs listener (or TCPs endpoint).

$ kubectl patch --type=merge singleinstancedatabases.database.oracle.com sidb-sample -p '{"spec": {"enableTCPS": true}}'

After a few minutes, the service will be enabled and confirmed using the following commands (checking the status)

$ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.isTcpsEnabled}"

When TCPS is enabled, a self-signed certificate is generated and stored in a wallet under/opt/oracle/oradata/clientWallet/$ORACLE_SID in the pod.

Regular Connections

At this stage, you can connect to the database using the connect strings returned by the commands above. You may use any supported client or the good old Sqlplus command line interface:

$ sqlplus sys/<.spec.adminPassword>@34.70.98.61:316/ORCL1 as sysdba

TCPS Connections

Copy the generated wallet directory onto a client machine, under “wallet_dir” (for example), using the following commands (from the client machine)

$ mkdir wallet_dir
$ kubectl cp $(kubectl get pods -l app=sidb-sample -o=jsonpath='{.items[0].metadata.name}'):/opt/oracle/oradata/clientWallet/ORCL1 ./wallet_dir

The wallet dir includes several files such astnsnames.ora and sqlnet.ora files.

$ ls wallet_dir
cwallet.sso  cwallet.sso.lck  ewallet.p12  ewallet.p12.lck  sqlnet.ora  tnsnames.ora

The tnsnames.ora file has short TNS aliases for connecting to the database (instead of using the lengthy full descriptor). The TNS_ADMIN environment variable refers to the location of the tnsnames.ora file.

$ cd wallet_dir
$ export TNS_ADMIN=$(pwd)
$ sqlplus sys@ORCL1 as sysdba

This is it!

Clean Up

Housekeeping!

Run the following commands to delete all the CRD instances, APIService, RBAC, and deployments in the default namespace.

$ kubectl delete singleinstancedatabase.database.oracle.com --all -n default

$ cat rbac/* | kubectl delete -f -

$ kubectl delete -f oracle-database-operator.yaml --ignore-not-found=true

Further Readings

• The SIDB documentation.
• The Oracle Database Operator for Kubernetes.
• Google’s “Deploying the Oracle Operator on GKE” was a source, published prior to this blog post.

Thank you!

Oracle Database Operator for Kubernetes on Google Cloud was originally published in Oracle Developers on Medium, where people are continuing the conversation by highlighting and responding to this story.

Learn how to configure the Universal Connection Pool (UCP) to leverage the Oracle Databases’ High Availability (HA) capabilities for continuous service during planned and unplanned downtime.

All the recommendations have been tested using the latest Long Term Support (LTS) Oracle Database release, i.e., 19c (19.21).

About UCP

Oracle Database furnishes the Universal Connection Pool (UCP), a feature-rich Java connection pool—one of the Spring Boot choices—that supports all Oracle database configurations and mission-critical capabilities (availability, scalability, and load balancing), out-of-the-box, including Oracle Real Application Cluster (RAC), Data Guard, Oracle Sharding, Asynchronous API, Runtime Load Balancing, XA, and front-end to Database Resident Connection Pool (DRCP).

UCP is released quarterly and fully supported by Oracle.

See more details @ https://www.oracle.com/jdbc/.

The Basics

With Spring Boot v2.4.0 and later, configuring UCP is easier and does not require any extra code. Spring identifies UCP as a data source like any other from the application.properties file. Spring retrieves the configuration from the application.properties file and autowires (injects) the values to the datasource.

The basic steps for using the UCP pool with the Oracle Database are (1) configuring the Oracle JDBC driver and (2) configuring UCP.

Configuring the Oracle JDBC Driver

Add Oracle JDBC driver as a dependency in Spring Initializer or manually add it to your project pom.xml file:

<properties>
    <oracle.jdbc.version>19.21.0.0</<oracle.jdbc.version>
</properties>

<dependencies>
...
<dependency>
    <groupId>com.oracle.database.jdbc</groupId>
    <artifactId>ojdbc8</artifactId>
    <version>${oracle.jdbc.version}</version>
</dependency>
...
</dependencies>

Check out this page for more information on Oracle RDBMS and JDK Version Compatibility for Oracle JDBC Drivers.

Configuring UCP

Pull the latest UCP release by adding the following dependency to your project pom.xml file:

<dependencies>
...
<dependency>
<groupid>com.oracle.database.jdbc</groupId>
    <artifactId>ucp</artifactId>
</dependency>
...
</dependencies>

Then, add the following properties to the application.properties file. Here is an example of the application.properties file with the minimum set of properties required for using the Oracle database:

#Oracle Database DataSource Configuration
spring.datasource.url=${JDBC_URL}
spring.datasource.username=${DB_USER}
spring.datasource.password=${DB_PASSWORD}

#UCP Settings
spring.datasource.type=oracle.ucp.jdbc.PoolDataSource
spring.datasource.oracleucp.connection-factory-class-name=oracle.jdbc.pool.OracleDataSource
spring.datasource.oracleucp.initial-pool-size=4
spring.datasource.oracleucp.min-pool-size=4
spring.datasource.oracleucp.max-pool-size=4

The next sections cover the steps for configuring UCP to leverage Oracle Databases’ high availability capabilities.

Oracle Database High Availability

With the Oracle Real Application Cluster system (RAC), any Oracle database service is accessible equally from several server nodes. If a node or a subset of the RAC cluster fails or is taken offline for maintenance, the database is still accessible through the remaining active nodes.

The building blocks for implementing Application High Availability with the Oracle database are: (1) using a database service, (2) configuring a URL or a connection string for High Availability, (3) enabling Fast Application Notification (FAN), (4) implementing draining and enabling continuous database availability for Java applications.
You may choose the HA protection level that suits your requirements. A detailed explanation of HA protection levels can be found in Oracle’s “High Availability Overview and Best Practices” under the Application High Availability Levels chapter.

The following steps allow you to configure the Oracle RAC system, the Oracle JDBC driver, and Oracle UCP to sustain planned and unplanned outages. Oracle’s “High Availability Overview and Best Practices” documentation covers the detailed steps and explanations for maintaining your applications' continuous availability.

Configuring for Basic High Availability — Level 1

This HA level furnishes fast dead connection detection thereby allowing applications to immediately react to instance, node, or database failures and quickly establish new connections to surviving database instances. With this level, downtime is minimized for both unplanned and planned outages.

The steps for implementing High Availability Level 1 are:

1. Configure High Availability Database Services
2. Setup the Connect String for High Availability
3. Enable Fast Application Notification

Configuring High Availability Database Services

Create a dedicated service to support HA features as follows.

$ srvctl add service -db mydb -service MYSERVICE -pdb mypdb
-notification TRUE

Setting Up the Connection String for High Availability

Use the connect string shown here to connect to the dedicated database service, created above.

jdbc:oracle:thin:@(DESCRIPTION =
  (CONNECT_TIMEOUT= 3)(RETRY_COUNT=4)(RETRY_DELAY=2)
  (TRANSPORT_CONNECT_TIMEOUT=3) (ADDRESS_LIST = (LOAD_BALANCE=on)
  (ADDRESS = (PROTOCOL = TCP)(HOST=clu_site_scan)(PORT=1521)))
  (CONNECT_DATA=(SERVICE_NAME = my_service)))

Enabling Fast Application Notification (FAN)

FAN is a high-availability notification mechanism that allows an active transaction to be immediately terminated when an instance or server fails. FAN is auto-configured and is enabled out of the box; no application code changes. FAN uses Oracle Clusterware’s Oracle Notification Service (ONS) to receive events from the cluster. ONS requires some ports to be available between the client and the servers.

The database uses the URL specified above to auto-configure FAN on the client side. To enable Fast Application Notification (FAN), add ons jar file to your project’s pom.xml file.

<dependencies>
...
<dependency>
<groupid>com.oracle.database.ha</groupId>
    <artifactId>ons</artifactId>
    <version>${oracle.jdbc.version}</version>
</dependency>
...
</dependencies>

Configuring HA Level 1 for Spring

Configure the Oracle database and UCP in your project’s application.properties file as follows.

#Oracle Database DataSource Configuration
spring.datasource.url=${JDBC_URL}
spring.datasource.username=${DB_USER}
spring.datasource.password=${DB_PASSWORD}

#UCP Settings
spring.datasource.type=oracle.ucp.jdbc.PoolDataSource
spring.datasource.oracleucp.connection-factory-class-name=oracle.jdbc.pool.OracleDataSource
spring.datasource.oracleucp.fast-connection-failover-enabled=true
spring.datasource.oracleucp.initial-pool-size=4
spring.datasource.oracleucp.min-pool-size=4
spring.datasource.oracleucp.max-pool-size=4
spring.datasource.oracleucp.validate-connection-on-borrow=false

You may notice that we’ve added spring.datasource.oracleucp.validate-connection-on-borrow=false to disable connection validation. For more details on connection validation, see the section titled Session Validation of this blog.

At this stage, the basic High Availability has been configured for Level 1. If your application has been designed to handle planned and unplanned outages via retry logic, you don’t need anything else.

Next, we will walk you through HA Level 2 and Level 3 which furnish the following benefits:

1. FAN event notification of unplanned connection failures. This is especially useful when the failure may have resulted in an application hang, such as when a database node fails, and the TCP/IP sockets are not cleaned up.
2. A connection string that results in connection requests being load-balanced across multiple RAC nodes using Single Client Access Name (SCAN) with retry on failure.

Configuring for High-Availability — Level 2

Building on application HA Level 1, Level 2 adds session draining for minimal application impact during planned maintenance. You can use planned operations to relocate, stop services, or switch over, allowing for graceful completion of the users’ work.

UCP responds to FAN events and takes immediate action, either balancing connections or establishing new connections and routing work to the available database instances or where the service was started. When a DOWN event (Reason = User) occurs, connections to the databases scheduled for maintenance will be cleaned up gracefully. When a UP event occurs (i.e., new instance up), FAN informs the connection pool that the new instance is available, allowing sessions to be created on this instance.

Performing Planned Maintenance

Draining database sessions is the safest way to migrate work without interrupting applications. The following command initiates the draining of active sessions.
The drain_timeout 60 parameter allows active sessions to finish their requests within a predefined timeout (60 seconds).

srvctl stop instance -db mydb -node node1 -stopoption immediate
drain_timeout 60 -force -failover

Planned Outage Success Test

For the planned outage test, we will be using a two-node Oracle RAC environment. During normal operation, both RAC nodes are up and running and serving the application. Depending on your load-balancing strategy, the number of sessions might be different or equally distributed across both nodes. Under load (i.e., database activities), observe that the connections drain away from that instance and that the instance/node can be shut down.

We will monitor the draining process using the following SQL statement :

select systimestamp time, machine, inst_id, count(*) from v$session where username='XXXX' group by systimestamp, machine, inst_id;

In our test run, we perform the following steps:

• Start the applications — app1 and app2 against nodes db1 and db 2.
• Run the workload.
• Drain the instance on node db1.
• Monitor the draining.
• Shut down the applications.

During normal operation, both RAC nodes db1 and db2 (inst_id 1 and inst_id2) are up and running. The following SQL output demonstrates how sessions for app1 and app2 are being distributed (count * column). Each application is using a connection pool that is configured to hold 4 sessions.

SQL>
TIME                                MACHINE     INST_ID   COUNT(*)
----------------------------------- --------- ---------- ----------
17-MAY-24 08.47.55.092067 PM +00:00 app1         1    1
17-MAY-24 08.47.55.092067 PM +00:00 app1         2    3
17-MAY-24 08.47.55.092067 PM +00:00 app2         2    1
17-MAY-24 08.47.55.092067 PM +00:00 app2         1    3

To begin the planned maintenance test, we first stop the database services on node 1 using the following command:

srvctl stop instance -db $(srvctl config database) -node db1 -stopoption immediate -drain_timeout 10 -force -failover

Fast Application Notification (FAN) sends a STOP event. The connection pool reacts to FAN and starts closing connections on node 1. At the same time, new connections are being opened on node 2. The active session on node 1 is still executing its work. If it finishes within the drain timeout (10 sec) and returns the connection to the pool, then the session is closed and a new one is established on node 2 when requested.

SQL>
TIME                                MACHINE    INST_ID   COUNT(*)
----------------------------------- --------- ---------- ----------
17-MAY-24 08.47.58.090673 PM +00:00 app2        2        3
17-MAY-24 08.47.58.090673 PM +00:00 app2        1        1
17-MAY-24 08.47.58.090673 PM +00:00 app1        2        4

If the drain timeout kicks in but some sessions are still connected to node 1, these will be disconnected. With Application Continuity in place, the sessions reconnect to node 2, and the interrupted work will be replayed transparently to the end user. See the next section for replay configuration details.

SQL>
TIME                                MACHINE     INST_ID   COUNT(*)
----------------------------------- --------- ---------- ----------
17-MAY-24 08.48.08.111763 PM +00:00 app1         2    4
17-MAY-24 08.48.08.111763 PM +00:00 app2         2    4

Configuring for High Availability — Level 3

For planned maintenance (Level 2), draining allows active sessions to finish their work (drain) before the database instance is shut down. However, in the case of unplanned events such as an instance, node, or database service failure, all sessions belonging to the faulty node/instance will be terminated immediately, with no time for draining. The Fast Application Notification (FAN) mechanism will detect those unplanned events and notify the UCP that has been instrumented to receive and process the FAN messages.

The primary difference between unplanned (Level 3) and planned shutdown (Level 2) scenarios is how borrowed connections are handled. In the unplanned shutdown scenario, stale connections idle in the pool (i.e., not borrowed) are removed.

Starting with Oracle Database19c, Transparent Application Continuity (TAC) automatically tracks and records the session and transactional states to recover and rebuild the database session and its states following recoverable errors (i.e., node, server, outages; SQL exceptions requiring application or database schema changes are classified as unrecoverable errors). TAC protects your work without application knowledge or code changes.

To configure TAC, perform the following steps: (1) add extra failover parameters to the database service definition and (2) add a replay data source to the application.properties file.

1. Add the following parameters to the database service definition

• -failovertype AUTO — enables Transparent Application Continuity (TAC).
• -failover_restore AUTO — restores client states automatically
• -replay_init_time 600 — specifies the time in seconds when the replay will not be performed.

srvctl stop instance -db $(srvctl config database) -node db1 -stopoption immediate -drain_timeout 10 -force -failover AUTO -failover_restore AUTO -replay_init_time 600

2. Add the replay data source spring.datasource.oracleucp.connection-factory-class-name=oracle.jdbc.replay.OracleDataSourceImpl to enable continuous service during unplanned outages to your application.properties file.

#Oracle Database DataSource Configuration
spring.datasource.url=${JDBC_URL}
spring.datasource.username=${DB_USER}
spring.datasource.password=${DB_PASSWORD}

#UCP Settings
spring.datasource.type=oracle.ucp.jdbc.PoolDataSource
spring.datasource.oracleucp.connection-factory-class-name=oracle.jdbc.replay.OracleDataSourceImpl
spring.datasource.oracleucp.fast-connection-failover-enabled=true
spring.datasource.oracleucp.initial-pool-size=4
spring.datasource.oracleucp.min-pool-size=4
spring.datasource.oracleucp.max-pool-size=4
spring.datasource.oracleucp.validate-connection-on-borrow=false

Session Validation

Connection validation on borrow in 19c UCP is configured by enabling or disabling spring.datasource.oracleucp.validate-connection-on-borrow property in the application.properties file.

• If spring.datasource.oracleucp.validate-connection-on-borrow is set to false, no connection validation is performed.
• If spring.datasource.oracleucp.validate-connection-on-borrow set to true, the level of validation is controlled by the oracle.jdbc.defaultConnectionValidation connection property (see more details in the next paragraph). The connection validation will be performed on every borrow unless the setSecondsToTrustIdleConnection() method specifies a positive value (in seconds), in which case, validation will only be performed if a connection has not been used for that period.

The connection property oracle.jdbc.defaultConnectionValidation controls what connection.isValid() method call does under the covers. This connection property specifies the level of connection validation. The possible values for this property are::

• “SERVER” (or “COMPLETE”) makes the driver execute a basic SQL query “SELECT ‘x’ FROM DUAL”,
• “NETWORK” causes the driver to issue an OPING TTC function. It’s the default.
• “SOCKET” causes the driver to write a zero-length NS data packet on the socket, which the server ignores.
• “LOCAL” same as above
• “NONE” checks the connection’s lifecycle (variable check)

Here is an example for setting validation properties in the application.properties file:

#Oracle Database DataSource Configuration
spring.datasource.url=${JDBC_URL}
spring.datasource.username=${DB_USER}
spring.datasource.password=${DB_PASSWORD}

#UCP Settings
spring.datasource.type=oracle.ucp.jdbc.PoolDataSource
spring.datasource.oracleucp.connection-factory-class-name=oracle.jdbc.pool.OracleDataSource
spring.datasource.oracleucp.initial-pool-size=4
spring.datasource.oracleucp.min-pool-size=4
spring.datasource.oracleucp.max-pool-size=4

#Session Validation
spring.datasource.oracleucp.validate-connection-on-borrow=true
spring.datasource.oracleucp.connection-properties=oracle.jdbc.defaultConnectionValidation=SOCKET
spring.datasource.oracleucp.seconds-to-trust-idle-connection=Wrap

Wrap-up

The steps described in this blog post are based on a RAC system and Oracle JDBC Drivers v19.21.0.0. A follow-up blog will describe the steps for planned and unplanned outages using the Autonomous Databases and a more recent Oracle JDBC driver release.

Further Readings

• The Oracle JDBC Developer Guide v19c
• High Availability Overview and Best Practices
• Application Checklist for Continuous Service for Maximum Availability Architecture solutions

UCP Best Practices for Oracle Database 19c and Spring Boot was originally published in Oracle Developers on Medium, where people are continuing the conversation by highlighting and responding to this story.

Irina Granat — Richard Exley— Kuassi Mensah

This previously published blog now covers the new Oracle Database 23ai release.
HikariCP is a popular Java connection pool, commonly used with Spring Boot. This blog post furnishes the best practices for configuring HikariCP with Spring Boot for the Oracle Database.

You will learn how to configure HikariCP to leverage the Oracle Databases’ High Availability (HA) capabilities for continuous service during planned and unplanned downtime.
All the recommendations in this blog post have been tested using the latest Long Term Support (LTS) Oracle Database release i.e., 19c (19.21) and 23ai.

About UCP

The Oracle Database furnishes the Universal Connection Pool (UCP), a feature-rich Java connection pool — one of the Spring Boot choices — that supports out-of-the-box, all Oracle database configurations, and mission-critical capabilities such as Oracle Real Application Cluster (RAC), Data Guard, Oracle Sharding, Asynchronous API, Runtime Load Balancing, XA, and front-end to Database Resident Connection Pool (DRCP).

UCP is released quarterly and fully supported by Oracle.
See more details at: https://www.oracle.com/jdbc/

You may have standardized on HikariCP and do not necessarily need the advanced and mission-critical UCP features. However, if you’d like to migrate from HikariCP to UCP, this blog post is for you.

The Basics

The basic steps for using HikariCP with the Oracle Database are (i) configuring the Oracle JDBC driver and (ii) configuring HikariCP.

Configure the Oracle JDBC Driver

Add Oracle Driver as a dependency in Spring Initializr or manually add it to your project’s pom.xml.

<properties>
    <oracle.jdbc.version>19.21.0.0</<oracle.jdbc.version>  
</properties>
 
<dependencies>

<dependency>
      <groupId>com.oracle.database.jdbc</groupId>
      <artifactId>ojdbc8</artifactId>
      <version>${oracle.jdbc.version}</version>
</dependency>

</dependencies>

For Oracle database 23ai, the version of JDBC is 23.4.0.24.05:

<properties>
    <oracle.jdbc.version>23.4.0.24.05</<oracle.jdbc.version>
</properties>

<dependencies>
...
<dependency>
    <groupId>com.oracle.database.jdbc</groupId>
    <artifactId>ojdbc8</artifactId>
    <version>${oracle.jdbc.version}</version>
</dependency>
...
</dependencies>

Use this page for more information on Oracle RDBMS and JDK Version Compatibility for Oracle JDBC Drivers.

Configure HikariCP

1. Starting with version 2, Spring Boot uses HikariCP as the default connection pool and it is transitively imported with the following Spring Boot starters: spring-boot-starter-jdbc or spring-boot-starter-data-jpa.

<parent>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-parent</artifactId>
  <version>3.0.4</version>
  <relativePath/> <!-- lookup parent from repository -->
 </parent>

<dependencies>
...
<dependency>
<! -- Assume Spring Data JDBC -->
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
...
</dependencies>

2. Then configure the application properties using one of the following methods: (i) using aapplication.properties file (Spring Boot automatically reads these files and applies the configuration), (ii) creating a configuration class annotated with @Configuration and using @Bean to define the DataSource bean with HikariCP settings — we are using this method hereafter, (iii) using aapplication.yaml file, and (iv) using a Kubernetes secret, for a more secure method. See this UCP-related blog post for more details about the approaches. Pick the approach that best fits your project and preferences. The first method might be simpler and is often sufficient for basic setups, while the second method provides more flexibility and control over the configuration in a Java class.

HikariCP has default settings that perform well in most deployments without additional tweaking. Spring Boot exposes Hikari-specific properties using thespring.datasource.hikarinamespace. Every property is optional except for the few essential ones that must be set. HikariCP’s official GitHub page explains various configuration options. In addition, you can review the Common Application Properties page listing all Spring Boot application.properties options.

Here are examples of an application.properties file and DataSourceConfig.java class with the minimum set of properties required to use the Oracle Database.

i. An application.properties file with the minimum set of properties required for using the Oracle Database.

# Oracle DataSource Configuration
spring.datasource.url=${JDBC_URL}
spring.datasource.username=${DB_USER}
spring.datasource.password=${DB_PASSWORD}
spring.datasource.driver-class-name=oracle.jdbc.OracleDriver

# HikariCP settings
spring.datasource.hikari.maximum-pool-size=4
spring.datasource.hikari.data-source-properties.oracle.jdbc.defaultConnectionValidation=LOCAL

Setting the oracle.jdbc.defaultConnectionValidation property set to LOCAL or SOCKET enables lightweight connection validation by the JDBC driver when you call the isValid(timeout) method. We strongly recommend setting up such a property.

You may add other HikariCP properties based on your application requirements. However, when setting multiple data source properties, beware of the syntax and side effects.

We also recommend enabling additional logging to verify that your properties were correctly registered:

logging.level.com.zaxxer.hikari.HikariConfig=DEBUG 
logging.level.com.zaxxer.hikari=TRACE

Please refer to the HikariCP documentation for more details.

ii. An example of a DataSourceConfig class defining the minimum set of properties that the DataSource requires to work with the Oracle Database.

package maa.spring;
import org.springframework.context.annotation.Bean;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;
import com.zaxxer.hikari.HikariDataSource;
import com.zaxxer.hikari.HikariConfig;

import java.util.Properties;
import javax.sql.DataSource;

@Configuration
public class DataSourceConfig {
    @Bean
    public HikariDataSource getDataSource () {
        HikariConfig config = new HikariConfig();
        config.setJdbcUrl(System.getenv("JDBC_URL"));
        config.setUsername(System.getenv("DB_USER"));
        config.setPassword(System.getenv("DB_PASSWORD"));
        config.setMaximumPoolSize(4);
        config.addDataSourceProperty("oracle.jdbc.defaultConnectionValidation", "LOCAL");
        return new HikariDataSource(config);
    }
}

At this stage, you have successfully configured your Spring Boot application to work with the Oracle Database using HikariCP.

The next sections cover the steps for configuring HikariCP to leverage Oracle Databases’ high availability capabilities.

Oracle Database’s High Availability

With the Oracle Real Application Cluster system (RAC), any Oracle database service is accessible equally from several server nodes. If a node or a subset of the RAC cluster fails or is taken offline for maintenance, the database is still accessible through the remaining active nodes.

The building blocks for implementing Application High Availability with the Oracle database are: using a database service, configuring a URL or a connection string for High Availability, enabling Fast Application Notification (FAN), implementing draining, and enabling continuous database availability for Java applications.
You may choose the level of HA that suits your requirements. A detailed explanation of HA protection levels can be found in Oracle’s “High Availability Overview and Best Practices” under the Application High Availability Levels chapter.

The following steps allow you to configure the Oracle RAC system, the Oracle JDBC driver, and HikariCP for sustaining planned and unplanned outages. The detailed steps and explanations for the continuous availability of your applications are covered in Oracle’s “High Availability Overview and Best Practices” documentation.

Configuring for Basic High Availability — Level 1

Implement a level of high availability that allows applications to immediately react to instance, node, or database failures, and quickly establish new connections to surviving database instances.

With application HA Level 1, downtime is minimized for unplanned and planned outages.

Configure High Availability Database Services

With Oracle RAC, the Oracle database service may be deployed over several nodes in the cluster. Planned outage allows performing maintenance operations on a subset of the service nodes/hosts. The database admin issues a command that generates a planned down event. As HikariCP is not instrumented to handle such event type, it will be handled by the Oracle JDBC. The driver will drain, i.e., transparently and smoothly close all active connections, thereby allowing the nodes scheduled for maintenance, to be shut down without impacting the applications (the remaining active nodes will absorb the workload).

Create a dedicated service to support HA features as follows.

$ srvctl add service -db mydb -service MYSERVICE -pdb mypdb
-notification TRUE

The -notification TRUE enables FAN for this service. Your application will connect to that service.

Configure the JDBC Connection String for High Availability

Configure your Java application to use the following connect string.

jdbc:oracle:thin:@(DESCRIPTION = 
  (CONNECT_TIMEOUT= 3)(RETRY_COUNT=4)(RETRY_DELAY=2)
  (TRANSPORT_CONNECT_TIMEOUT=3) (ADDRESS_LIST = (LOAD_BALANCE=on) 
  (ADDRESS = (PROTOCOL = TCP)(HOST=clu_site_scan)(PORT=1521))) 
  (CONNECT_DATA=(SERVICE_NAME = my_service)))

Enable Fast Application Notification (FAN) at the Driver Level

As HikariCP has not been instrumented to handle FAN events, the planned maintenance will be managed by the Oracle JDBC driver. To configure the driver to handle FAN events, add simplefan and ons jar files to your project’s pom.xml.

<dependencies>

...
    <dependency>
      <groupId>com.oracle.database.ha</groupId>
      <artifactId>ons</artifactId>
      <version>${oracle.jdbc.version}</version>
    </dependency>
    <dependency>
      <groupId>com.oracle.database.ha</groupId>
      <artifactId>simplefan</artifactId>
      <version>${oracle.jdbc.version}</version>
    </dependency>

...
  </dependencies>

At this stage, the basic High Availability has been configured and furnishes the following benefits:

1. FAN event notification of unplanned connection failures. This is especially useful when the failure may have resulted in an application hang, such as when a database node is failing and the TCP/IP sockets are not cleaned up.
2. A connection string that results in connection requests being load-balanced across multiple RAC nodes using Single Client Access Name (SCAN) with retry on failure.

If your application has been designed to handle planned and unplanned outages via retry logic, then you don’t need anything else.

Configuring for Planned Maintenance — Level 2

To prepare applications for planned maintenance, add the following connection validation settings to your application code.

Enabling Predictable Connection Validation

By design, HikariCP validates connections that have been idle for more than 500ms (in the latest version). If a connection is constantly being checked out and checked in, it won’t be validated and, hence, won’t be drained by the Oracle JDBC driver.
Set the following property either as a JVM system property or programmatically in your DataSourceConfig.java class (as shown above) to force connection validation at checkout.

• -Dcom.zaxxer.hikari.aliveBypassWindowMs=-1
• System.property("com.zaxxer.hikari.aliveBypassWindowsMs","-1");

Please be aware that a system property has JVM scope, so a setting like this will affect all HikariCP pools in that JVM instance.

A systematic connection validation on borrow may have a slight performance impact; Oracle recommends setting a lightweight connection validation property oracle.jdbc.defaultConnectionValidation to SOCKET or LOCAL. SOCKET has a small performance penalty while LOCAL might limit the driver's ability to detect the outage) to mitigate the impact.

props.setProperty("oracle.jdbc.defaultConnectionValidation", "LOCAL");

Setting the data source class name using the application.properties or the applications.yaml files throw the following error message: “Caused by: java.lang.IllegalStateException: both driverClassName and dataSourceClassName.” We need to set it programmatically as follows (at this point, we recommend using only the DataSourceConfig class file for any additional settings and not combining it with the application.properties file).

package maa.spring;
import org.springframework.context.annotation.Bean;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;
import com.zaxxer.hikari.HikariDataSource;
import com.zaxxer.hikari.HikariConfig;

import java.util.Properties;
import javax.sql.DataSource;

@Configuration
public class DataSourceConfig {
    @Bean
    public HikariDataSource getDataSource () {
        System.setProperty("com.zaxxer.hikari.aliveBypassWindowMs", "-1");
        HikariConfig config = new HikariConfig();
        config.setJdbcUrl(System.getenv("JDBC_URL"));
        config.addDataSourceProperty("url",System.getenv("JDBC_URL"));
        config.setUsername(System.getenv("DB_USER"));
        config.setPassword(System.getenv("DB_PASSWORD"));
        config.setMaximumPoolSize(4);
        Properties props = new Properties();
        props.setProperty("oracle.jdbc.defaultConnectionValidation", "LOCAL");
        config.addDataSourceProperty("connectionProperties", props);
        return new HikariDataSource(config);
    }
}

For Oracle database 23ai, change config.setDataSourceClassName value for data source creation to oracle.jdbc.datasource.impl.OracleDataSource:

config.setDataSourceClassName("oracle.jdbc.datasource.impl.OracleDataSource");

Performing Planned Maintenance

The following command initiates the draining of active sessions.
The -drain_timeout 60parameter, allows active sessions to finish their requests within a predefined timeout (60 sec).

srvctl stop instance -db mydb -node node1 -stopoption immediate 
–drain_timeout 60 -force -failover

Planned Outage Success Test

For the planned outage test, we will be using a two-node Oracle RAC environment. During normal operation, both RAC nodes are up and running and serving the application. Depending on your load-balancing strategy, the number of sessions might be different or equally distributed across both nodes. Under load (i.e., database activities), observe that the connections drain away from that instance and that the instance/node can be shut down.

We will monitor the draining process using the following SQL statement:

select systimestamp time, machine, inst_id, count(*) from v$session where username='XXXX' group by systimestamp, machine, inst_id;

During normal operation, both RAC nodes db1 and db2 (inst_id 1 and inst_id2) are up and running. The following SQL output demonstrates how sessions for app1 and app2 are being distributed (count * column). Each application is using a connection pool that is configured to hold 4 sessions.

SQL>
TIME                        MACHINE      INST_ID   COUNT(*)
----------------------------------- --------- ---------- ----------
24-JUN-24 04.44.50.330975 PM +00:00 app2         2    2
24-JUN-24 04.44.50.330975 PM +00:00 app1         2    1
24-JUN-24 04.44.50.330975 PM +00:00 app1         1    3
24-JUN-24 04.44.50.330975 PM +00:00 app2         1    2

To begin the planned maintenance test, we first stop the database services on node 1 using the following command:

srvctl stop instance -db $(srvctl config database) -node db1 -stopoption immediate -drain_timeout 10 -force -failover

Fast Application Notification (FAN) sends a stop event. The connection pool reacts to FAN and starts closing connections on node 1. At the same time, new connections are being opened on node 2. The active session on node 1 is still executing its work. If it finishes within the drain timeout (10 sec) and returns the connection to the pool, then the session is closed and a new one is established on node 2 when requested.

SQL>
TIME                        MACHINE      INST_ID   COUNT(*)
----------------------------------- --------- ---------- ----------
24-JUN-24 04.44.58.313179 PM +00:00 app1         2    4
24-JUN-24 04.44.58.313179 PM +00:00 app2         2    2
24-JUN-24 04.44.58.313179 PM +00:00 app2         1    2

If the drain timeout completes but the remaining session is still connected to node 1, it will be disconnected. With Application Continuity in place, the session reconnects to node 2, and the interrupted work will be replayed transparently to the end user. See the next section for replay configuration details.

SQL>
TIME                        MACHINE   INST_ID   COUNT(*)
----------------------------------- --------- ---------- ----------
24-JUN-24 04.45.04.310911 PM +00:00 app1         2    4
24-JUN-24 04.45.04.310911 PM +00:00 app2         2    4

Configuring for Unplanned Outages — Level 3

For planned maintenance, draining is used to allow active sessions to finish their work (drain) before the database instance is shut down. However, in unplanned events such as an instance, node, or database service failures, all sessions connected to the faulty node/instance will be terminated immediately — there is no time for draining. The Fast Application Notification (FAN) mechanism will detect those unplanned events and notify the JDBC driver or any connection pool such as UCP that has been instrumented to receive and process the FAN messages.

As HikarCPi is not capable of handling FAN events, the Oracle JDBC driver transparently cleans up the orphan connections and replays in-flight requests on a surviving database instance (using new connections); the failures will, therefore, be transparent to the applications that were using those connections.

1. Add the following parameters to the database service definition:

-failovertype AUTO — enables Transparent Application Continuity (TAC).

-failover_restore AUTO — restores client states automatically

-replay_init_time 600 — specifies the time in seconds upon which the replay will not be performed.

$ srvctl add service -db mydb -service MYSERVICE -pdb mypdb
-notification TRUE -failover_restore AUTO -failovertype AUTO -replay_init_time 600

2. To enable continuous service during unplanned outages, include the replay data source oracle.jdbc.replay.OracleDataSourceImplin your DataSourceConfig class.

package maa.spring;

import org.springframework.context.annotation.Bean;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;
import com.zaxxer.hikari.HikariDataSource;
import com.zaxxer.hikari.HikariConfig;

import java.util.Properties;
import javax.sql.DataSource;

public class DataSourceConfig {
    @Bean
    public HikariDataSource getDataSource () {
        System.setProperty("com.zaxxer.hikari.aliveBypassWindowMs", "-1");
        HikariConfig config = new HikariConfig();
        config.setJdbcUrl(System.getenv("JDBC_URL"));
        config.addDataSourceProperty("url",System.getenv("JDBC_URL"));
        config.setUsername(System.getenv("DB_USER"));
        config.setPassword(System.getenv("DB_PASSWORD"));
        config.setDataSourceClassName("oracle.jdbc.replay.OracleDataSourceImpl");
        config.setMaximumPoolSize(4);
        Properties props = new Properties();
        props.setProperty("oracle.jdbc.defaultConnectionValidation", "LOCAL");
        config.addDataSourceProperty("connectionProperties", props);
        return new HikariDataSource(config);
    }
}

You may notice that we added config.addDataSourceProperty(“url”,System.getenv(“JDBC_URL”)) setting to HikariDataSource in addition to config.setJdbcUrl(System.getenv(“JDBC_URL”)) (this setting will be ignored in this case) — it is required to resolve java.sql.SQLException: Invalid Oracle URL specified: OracleDataSource.makeURL exception.

For Oracle database 23ai, change the value of config.setDataSourceClassName for the data source creation to
oracle.jdbc.datasource.impl.OracleDataSource.

JDBC Diagnosability

New in Oracle Database 23ai: A single ojdbc jar (e.g., ojdbc8.jar, ojdbc11.jar) for all use cases (production, debug, metrics). In other words, no more ojdbc8_g.jar or ojdbc11_g.jar for debugging.

Check out this technical brief—What’s in ‘Oracle DB 23c Free Developer Release’ for Java Developers—for more information about the new JDBC Self-Driven Diagnosability in Oracle database 23ai.

Wrap-up

The steps described in this blog post are based on an Oracle RAC system and Oracle JDBC Drivers v19.21.0.0 and 23.4.0.24.05.

Thank you for reading!

Further Readings

• The Oracle JDBC Developer Guide v19c
• High Availability Overview and Best Practices
• A Guide to Migrating from HikariCP to Oracle UCP (Universal Connection Pool)
• Application Checklist for Continuous Service for Maximum Availability Architecture solutions

HikariCP Best Practices for Oracle Database and Spring Boot was originally published in Oracle Developers on Medium, where people are continuing the conversation by highlighting and responding to this story.

What are the pain points in deploying Java Apps? Infrastructure as Code (IaC) allows automating most tasks involved with provisioning Cloud infrastructure resources thereby eliminating those pains.
This post identifies the main steps in deploying your Java app and then introduces the App Stack for Java, a stack in the Oracle Cloud
Infrastructure marketplace for automating these steps.

Pain Points for Deploying Java Apps

Here are the common pains for deploying your Java applications over a Cloud infrastructure.

1. Configuring the virtual network (VCN, subnets, security lists, route tables, gateways).
2. Provisioning of the deployment platform.
3. Provisioning a new database or reusing an existing one.
4. Configuring the JDBC data source.
5. Configuring the load balancer.
6. Configuring DNS.
7. Generating the build and deployment of CI/CD pipelines.
8. Activating the monitoring service.

An IaC solution could take care of all these steps; all you’ll have to do is fill up some parameters then click a button and, after a few minutes, your application is up and listening for requests.

Introducing the App Stack for Java

The App Stack for Java is a Terraform-based IaC product (or “solution”) that provisions serverless resources in the Oracle Cloud Infrastructure and deploys your Java App.
Note: this is not a managed Cloud service.

Target Apps & Use Cases

• Web applications (server-side HTML applications, aka servlets or Java Server Pages).
• Back-end servers (REST APIs).
• Standalone Java app that embeds a runtime (Netty for example).
• Servlet or JSP packaged as a Web Application Archive (WAR) that the stack deploys on Tomcat.
• Spring or SpringBoot application (with or without Hibernate) that configures Tomcat as a dependency.
• Generic use case: container image that includes a web app (doesn’t have to be Java).

The Key Features

• Simple to configure and launch.
• Provisions the serverless Cloud resources required/configured by the Java App.
• Mirror the Application source code from GitHub to OCI DevOps.
• High Availability with a minimum of two container instances.

How Does the App Stack for Java Work?

For deploying your Java app with the App Stack for Java, you need to

1. Fulfill the OCI prerequisites.
2. Fill out the parameters; this is the only hard part.
3. Apply the stack -> one-click deployment.

That’s it, the Application is ready to handle requests!
A complete step-by-step tutorial and videos are furnished hereafter.

Under the Covers

The App Stack performs the following actions

• Create a new repository “xxxx_config”.
• Leverage OCI services such as VCN, Object stores.
• Download the database wallet (wallet.zip): the zip file includes Tnsnames.ora, cwallet.sso, and ojdbc.properties. If the database wallet is rotated this zip file needs to be updated.
• Generate the Dockerfile for building the container image.
• Generate the spec for the build pipeline.
• Generate the self-signed keystore for the internal HTTPS connection between the load balancer and the container instances.
• Pull orapki.jar for use by the build pipeline to add the DB password to cwallet.sso.
• Generate a container image of the user’s application, including an HTTP/HTTPS server and pre-configured ADB access.
• Add the generated image to the Container Registry.
• Deploy the application in Container Instances.
• Provisions the Autonomous Database.
• Generates the dependencies to pull JDBC, UCP, R2DBC, and so on from Maven Central.
• It provisions Oracle Application Performance Monitoring (Oracle APM) and connects your app.
• And much more …

These actions constitute pain relief and the main purpose of the App Stack.

Prerequisites and Cloud Resources

What are the prerequisites and Cloud resources used for deploying your Java App?

Prerequisites

This step is required whether you are using the App Stack for Java or not.

1. Select an application in your GitHub repo and create a personal token if you don’t already have one.
2. Create an OCI compartment
3. Create an OCI Vault in that compartment then add an encryption key
4. Add the GitHub token as a secret
5. Create a dynamic group and related policies
6. For CI/CD managing, create an empty project (with its topic) in OCI DevOps and enable logging
7. Allow OCI DevOps to connect to your GitHub repo by creating a connection
8. Mirror your GitHub project in OCI DevOps.

The Cloud Resources Used by Your Java App

Cloud resources are provisioned when you execute the stack based on its configuration. Most of these are free, even always free!

Free-Tier and Always-Free Resources

The Oracle Cloud Free Tier service allows you to build, test, and deploy your applications on Oracle Cloud for free. Upon signing up, the service comes with a $300 credit with 30 days expiration; following the expiration or the exhaustion of the credit, most of the provisioned services remain available as Always Free. You may add additional credit for services that do not fall under Always-Free.

Container Instances

OCI Container Instances are serverless compute resources (Open Container Initiative-compliant), optimized for containerized workloads that do not require a container orchestration platform such as Kubernetes.
You pay only for the CPU and memory resources allocated to your container instance, which are the same price as the regular Compute Instances for the chosen shape.

Resources Map and List

Here is the map and the list of OCI resources provisioned for your Java app by the App Stack.

• Container Instances for building and running the app. Not an Always-Free resource.
• DevOps project (optional): used for the build pipeline and CI/CD of the Java app; an OCI DevOps project is a mirror of your GitHub project. This service isn’t required if the application is already packaged as a container image.
• Database: an existing Autonomous Database — Shared Infrastructure (ADB-S) can be used with the stack. The stack may create a new one if specified during the Stack configuration. This is an Always-Free
• Vault (optional): necessary to avoid the limit on the number of tokens the current user has; we do not recommend using the current user, however, the Vault isn’t required if the application is provided as a container image.
• A load balancer; part of Always-Free.
• DNS (optional): A DNS zone for creating the application URL (for example https://myapp.domain.com). If not provided during the stack configuration, the application will be available through the load balancer’s public IP (Always-Free). You can then configure your third-party DNS provider to point to this IP address.
• HTTPS certificate (optional): if no certificate is provided, HTTP will be used against the IP address; this resource is part of Always-Free
• Security policies
• Monitoring service (Oracle APM); part of Always-Free.

With an average burn rate of $3 and the initial $300 Free-Tier credit, you have sufficient free and cheap resources (i.e., when Free-Tier ends) to build, test, and deploy your Java apps with the App Stack.

Get Started — Tutorial & Videos

Before using the App Stack for your own apps, we urge you to go through this complete step-by-step tutorial for deploying a SpringBoot app on OCI using the App Stack for Java.

Here is the playlist of the walk-through videos including

• Prerequisites for source code deployment.
• Configuring and provisioning for source code deployment.
• CI/CD workflow, and Observability.

We will soon release more videos related to apps packaged as Java artifacts (WAR, JAR) and Containerized images.

Get Help

• Submit a GitHub issue
• Email appstack_ww at oracle.com.

Follow @kmensah and #AppStack for notifications, questions, and discussions.

I am very pleased to announce that the long overdue production release (v 1.0.0) of the Oracle Database Operator for Kubernetes (a.k.a. OraOperator) has been released.

What’s In It?

In this release, OraOperator supports containerized databases, database services deployed on the Oracle Cloud Infrastructure (OCI), and on-premises databases.

Containerized Databases

The Oracle database images (custom-built or from the container registry) are deployed within the Kubernetes cluster alongside OraOperator. These databases can be deployed in any Kubernetes cluster where the operator runs including OKE, 3rd party Kubernetes clusters, on-premises Kubernetes (Oracle Linux), and so on.

Let’s sum up the lifecycle operations supported for each kind.

Single Instance Databases (a.k.a. SIDB)

The Single Instance DB controller supports the following Oracle database editions: Standard Edition (SE), Enterprise Edition (EE), eXpress Edition (XE), and DB23c Free.

The supported lifecycle operations include: provisioning a new database instance, provisioning a new pre-built database instance, cloning a database, patching a database (in-place/out-of-place), deleting a database, and updating the database init parameters.
The SIDB controller furnishes additional features such as REST enabling a database, running multiple replicas of a database, setting up a database with LoadBalancer, installing APEX, enabling TCPS connections, and getting the connect string to the CDB.
For more details, please refer to the SIDB Readme (under the …/doc/sidb folder).

Sharded Databases

An Oracle Sharding database is made of Shards. Containerized Shards are deployed over several Kubernetes pods.

The Sharding Database controller supports the following lifecycle operations: deploy the Shard topology (i.e., the Sharding infrastructure) using various options (check out the …/doc/sharding/Readme.md for more details), Add a Shard, Delete a Shard, Clone a Shard.

Data Guard (Preview)

In this release, a new Data Guard controller is introduced as a preview.

This controller allows creating a Standby Database for a Single Instance database (see the prerequisites in …/doc/sidb), adding databases in the Data Guard configuration, and deleting a database configured for Data Guard.
Additional operations include connecting to the Primary, performing a Switchover, and patching the Primary and the Standby databases.

OCI-Only Databases

The operator manages also the lifecycle of Cloud database services deployed in the Oracle Cloud Infrastructure (OCI), outside the OKE Kubernetes cluster. Unlike containerized databases, these Cloud database services are only available/deployable in Oracle Cloud.

Autonomous Database (Shared, Dedicated)

Autonomous Databases are the flagship Oracle Cloud database services and come in two flavors: shared infrastructure (a.k.a. ADB-S) and dedicated infrastructure (a.k.a. ADB-D). ADB refers to both flavors. The infrastructure is known as the Autonomous Container Database (ACD).

The Autonomous Database controller supports the following lifecycle operations: provisioning an Autonomous Database (ADB), binding the operator to an existing ADB, scaling Up/Down the OCPU core count or the storage, managing the database ADMIN password, downloading ADB instance credentials (i.e., wallets), Starting/Stopping/Terminating an ADB, and deleting anADB from the resources (CRDs) known by the operator.

Base DB (formerly DBCS)

The Oracle Base Database Cloud Service allows deploying single-node and two-node database systems on virtual machines. See more details about the supported database editions and versions, here.

The BaseDB controller supports the following lifecycle operations: provision a BaseDB, binding to an existing BaseDB, scaling UP/DOWN the shape of an existing BaseDB, scaling UP the storage of an existing Base DB, deploying a two-nodes RAC, updating the license type, and terminating an existing BaseDB.

On-Premises Databases

Multitenant Databases (Exadata, PDBs)

A multitenant controller enables managing the lifecycle of pluggable databases (a.k.a. PDBs) associated with a Container database (a.k.a. CDB). The Oracle REST Data Service is required for reaching the PDBs.

The controller supports the creation of a new PDB, cloning a PDB, modifying the state of a PDB, unplugging a PDB from a CDB, plugging an unplugged PDB into a CDB, and deleting a PDB.

Pre-installed on OKE

On OCI, OraOperator is pre-installed and ready to use as an Add-on to OKE Console deployment. In future releases, a button will automatically provision Cloud database services (i.e., the Autonomous Databases and BaseDB).

Resources

Livelab: Oracle Database Operator for Kubernetes + DevOps.

Get help either by submitting a GitHub issue and/or filing an Oracle Support service request against the Oracle Database Operator for Kubernetes (product ID: 14423).

Visit our website for the latest news about Oracle Databases, Containers, and Kubernetes.

Follow @kmensah and #OraOperator for notifications, questions, and discussions.

Thanks for reading.

Free as in Free Beer

The “Oracle Database 23c Free — Developer Release” is a full-featured, no-click-through, developer focussed Oracle database. It has been released under the Free Use Terms & Conditions license; see more details here.

It brings new features for Java developers. This blog post highlights my favorite features; others will be covered in depth in subsequent blog posts.

My Favorites Cool Java Features

I am totally biased here! I’ll cover Java support for pipelined database operations, Relational-JSON Duality Views, and Self-driven diagnosability.

1 — Java and Pipelined Database Operations

A database pipeline consists of a sequence of multiple database requests submitted without waiting for a response between operations. The database sends a response when the results of each query are ready.
Pipelined database operations foster an asynchronous programming model, in which a user thread returns immediately upon submitting a SQL statement for execution without waiting for its execution and the ResultSets.

Java developers would leverage database pipelining through the Oracle JDBC Reactive Extensions, the Reactive Streams libraries, Virtual Threads, and the standard JDBC Batching.

JDBC Reactive Extensions and Database Pipelining

Introduced in Oracle JDBC release 21.1, the JDBC Reactive Extensions implement the Java Flow API and extend it with the asynchronous forms of connection creation, SQL execution, row data fetching LOB I/O, and transaction control (i.e., commit/rollback). The API is fully compliant with the Reactive Streams 1.0.3 specification and interoperates with the ecosystem of reactive libraries including Spring, jOOQ, Helidon, Hibernate, Project Reactor, RxJava, Akka Streams, and more.
The following code introduces the concept of calling AsyncOracle methods — part of Oracle JDBCs’ Reactive Extensions — to compose a pipeline.
In the later examples, the intricacies of closing statements and consuming results for reactive and synchronous-using-Virtual-Threads programming models will be shown.

In the example above, a call to unwrap(OraclePreparedStatement.class) is used to access the methods of the Reactive Extensions API i.e., executeUpdateAsyncOracle and executeQueryAsyncOracle. These methods return a Flow.Publisher. As the result of each SQL statement is received, the Publisher emits a ResultSet to a Subscriber. How exactly? JDBC “listens” for the database response to each SQL statement by polling the network using a single Selector across the driver. Upon the reception of the database response, the Selector invokes a callback that reads the message.

Reactive Streams Libraries and Database Pipelining
Reactive libraries such as Project Reactor, RxJava, Akka Streams, Spring, jOOQ, Helidon MP Reactive Streams, Hibernate Reactive, and so on — expose an API that is similar to java.util.stream.Stream. The code example from the previous section can be expanded to use the API of Project Reactor.

In the example above, instances of Mono and Flux are created by calling their “from” methods. The argument to the method is an org.reactivestreams.Publisher, which is adapted from a Flow.Publisher by calling org.reactivestreams.FlowAdapters.toPublisher. A “subscribe” method is called to asynchronously consume the results with a callback function. To consume a result of multiple rows, a call to Mono.flatMapMany converts a stream of a single ResultSet into a stream of multiple row values.

Virtual Threads and Database Pipelining
Looking ahead, Virtual Threads (Project Loom) is an alternative to Reactive Streams and is touted as the future of Java scalability.
Oracle JDBC has been instrumented with Virtual Threads since the 21.1 release. Specifically, the driver will not cause thread pinning by holding an intrinsic lock during I/O calls. For more details, see: http://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.html#pinning
With Virtual Threads, asynchronous APIs like Reactive Streams can easily be translated into synchronous code. A Virtual Thread only needs to block its execution until an asynchronous result has been received. To demonstrate this, the code example from the previous section can be re-written as follows:

Just like the code sample of the previous section, each Flow.Publisher is converted into a Mono or Flux from the Project Reactor API. However, with Virtual Threads, there is no longer a need to consume the results asynchronously. Instead of calling the subscribe method with a callback function, a Virtual Thread can simply call the block() method. The block method blocks the Virtual Thread until the result of the asynchronous call is received. The result may then be processed in a traditional style using imperative programming.

Standard JDBC Batching

The Oracle JDBC driver will start a database pipeline as soon as you start using the JDBC batching APIs i.e., stmt.addBatch() or prepstmt.addBatch(); the driver generates asynchronous calls under the covers, using the JDBC Reactive Extensions APIs.

2 — Java and the JSON-Relational Duality Views

Many applications require simplicity (self-describing, self-contained, and schema-less), ease of development (i.e., access JSON data programmatically), hierarchical data, common interchange format (i.e., easy to transmit across the network), and ease of conversion to Java types of JSON — and by the same token — the efficiency of space management, the queryability, consistency, and the powerful analytics and reporting capabilities of SQL . The JSON-relational duality views consist in presenting relational data as JSON documents. This feature gives developers the best of the Relational and JSON worlds. More details can be found here.

How to Setup and Use JSON-Relational Duality views?

1. Create and populate the regular relational tables (or reuse existing ones). E.g., Students, Teachers, and Classes tables.
2. Create one or several JSON-Relational Duality Views based on use cases, using either a SQL visual tool, a SQL CLI, or programmatically using JDBC Statement.execute(“ …”).
   Each view consists in leveraging several capabilities. The following SQL code fragment maps base table columns to JSON name/values pairs.

The following SQL code extends the previous one using a subquery to assemble nested data into a JSON array.

The following SQL statement completes the previous ones by specifying update rules.

That’s it! You can now work directly with the JSON documents using the following query which returns John’s schedule as a JSON document.

Using the oracle.sql.json package in Oracle JDBC, you can retrieve the value of specific properties from John’s schedule.

JSON document transfer to the Java app can be performed either in binary JSON (OSON, suitable for large documents) or String (for small documents).

3 — JDBC Self-Driven Diagnosability

Every Java developer using the Oracle database will appreciate the following statement: “No more switching between the production jar and the debug jar to investigate an issue”!
A single ojdbc jar (e.g., ojdbc8.jar, ojdbc11.jar) for all use cases (production, debug, metrics). In other words, no more ojdbc8_g.jar or ojdbc11_g.jar for debugging, no more ojdbc8dms.jar or ojdbc11dms.jar for the Oracle Dynamic Monitoring Service (DMS) metrics, and no more ojdbc8dms_g.jar or ojdbc11dms_g.jar for DMS debugging.

Although the debug and metrics jars have been eliminated, the core JDBC jar furnishes enhanced logging, debugging (diagnose-on-first-failure), and execution tracing.

Logging: the core JDBC Jars (i.e., ojdbc8.jar or ojdbc11.jar) include the logging capabilities which need to be turned on using the following properties:
-Doracle.jdbc.diagnostic.enableLogging=true
in addition, the location of the logging configuration file
-Djava.util.logging.config.file=./logging.config
and the content of the logging configuration file
handlers = java.util.logging.ConsoleHandler
oracle.jdbc.level = FINEST
java.util.logging.ConsoleHandler.level = FINEST
java.util.logging.ConsoleHandler.formatter =
oracle.jdbc.diagnostics.OracleSimpleFormatter

Diagnose-on-first-failure: this feature diagnoses the first occurrence of a failure. It records the critical execution state in memory, then dumps the recording on error. It’s ON by default however, you may disable it either by setting the CONNECTION_PROPERTY_ENABLE_DIAGNOSE_FIRST_FAILURE property to FALSE (the default value) or via the DiagnosticMBeans interface.

For security, each diagnosability feature has two modes, public and sensitive. In the public mode, these features do not record or persist sensitive information. This severely limits the amount of information captured and reduces the effectiveness of the diagnosability features. In the sensitive mode, these features record and persist sensitive information. Java developers can share traces with the redaction of sensitive data. The sensitive mode can only be enabled by a privileged user and is controlled by two switches; one for enabling and another one for permitting. Only the permit property is required. If the permit property is included in the java command line then sensitive mode can be enabled and disabled programmatically at runtime, or via MBean. It can also be enabled via the enable property as shown.
-Doracle.jdbc.diagnostic.enableSensitiveDiagnostics=true
-Doracle.jdbc.diagnostic.permitSensitiveDiagnostics=true

You must configure java.util.logging (see above) to get diagnostic output on diagnose-on-first-failure.

Execution Trace: this feature records the execution sequence details.
The JDBC driver has also been instrumented with a hook for tracing database round trips; this hook enables the support of OpenTelemetry.
In a nutshell, OpenTelemetry is enabled using 3rd party libraries such as OpenTelemetry-JDBC without code change. All top-level calls can be monitored; the Java agent creates a span for JDBC calls.
To be covered in detail in an upcoming blog post.

DMS Metrics: The DMS metrics are recorded if the dms.jar is in the classpath.

Caution: you must balance the volume of state information recording against the cost of recording. As execution trace has a significant cost involved, you are advised to enable it only in limited contexts.

Other Java Goodies

• Native Boolean datatype in oracle.jdbc.OracleType.

• JDBC-Thin support for passwords up to 1024 bytes in length.
• JDBC-Thin generic support for token-based authentication including Oracle Cloud Infrastructure (OCI) Identity Access Management (IAM), and Azure Active Directory (Azure AD). See more here.
• JDBC-Thin support for RADIUS and RADIUS two-factor authentication where the RADIUS server asks for a valid response to a displayed challenge. See more here.
• JDBC Support for Directory-based Sharding and Split Partition Set.
  No public Java API however, under the covers, during the partition-set split, the client-side connection pools receive ONS events about data in a chunk being split and moved across partition sets and update their sharding topology appropriately.
• UCP Support for XA Transactions with Oracle Database Sharding. Java applications that use the UCP native data source in WebLogic Server to connect to Sharded Oracle databases can participate in JTA/XA transactions managed by the WebLogic Transaction Manager (TM).
• Oracle JVM Support for HTTP and TCP access while disabling other OS calls. See more details in Database Security in a Multitenant Environment.

Wrap Up

I trust that this post has inspired you to design new projects or re-architect existing ones, based on the new Java features in Oracle Database 23c Free — Developer Release.
Thank you for reading.
@kmensah #javaOracleDB http://oracle.com/jdbc

The introductory release (v0.1.0) of Oracle Database Operator for Kubernetes (#OraOperator) lets you manage the lifecycle of Single Instance Databases, Sharded Databases, and the Autonomous Database on shared infrastructure (ADB-S).

In this feature-packed v0.2.0 release, #OraOperator extends its reach over even more database configurations, and enhances existing features. Hopefully, you’ll appreciate why it took us a while, after v0.1.0, to get it out. But don’t just take my word for it — check it out yourself.

What’s in #OraOperator v0.2.0?

While the operator itself is deployed on a Kubernetes cluster (see “Where does OraOperator Run?”), its operational reach straddles databases deployed within the Kubernetes cluster and those deployed outside of it.

Here’ s what you’ll find in the v0.2.0 release.

In-k8s Cluster Databases

These databases run in containers are (such as Docker) deployed within the Kubernetes cluster.

Single Instance databases (SIDB)

This release brings new enhancements to the SIDB controller such as support for pre-built XE for faster provisioning. The list of features include provision, clone, patch (in-place/out-of-place), update database initialization parameters, update database configuration (Flashback, archiving), Oracle Enterprise Manager (EM) Express (a basic observability console), Oracle REST Data Service (ORDS) to support REST-based SQL, PDB management, SQL Developer Web, and Application Express (Apex). Get more details in the SIDB controller doc.

Sharded Databases

The operator can provision the shard topology, add a new shard, and delete an existing shard. Get more details in the Sharded Database controller doc.

Off-k8s Cluster Databases

These databases run either in the Oracle Cloud or on-premises.

Autonomous Databases & Container

The autonomous database runs in the Oracle Cloud Infrastructure (OCI), off the Kubernetes cluster.

• ADB-S: stands for Autonomous Database on a shared infrastructure. In other words, multiple tenants share the same infrastructure but each deploy their own database. In this release, the operator can provision, bind, start, stop, terminate (soft/hard), and scale (up/down) an ADB; it can also perform manual or on-demand backup as well as manual restore.
  Note: the Autonomous Database already performs automatic backup and restore.
• ADB-D: stands for Autonomous Database on a dedicated infrastructure; the tenant has exclusive use of the infrastructure. The support for ADB-D is a new in this release. The operator can provision, bind, start, stop, terminate (soft/hard), and scale (up/down) an ADB. In addition — new to this release — it can also perform manual backup and manual restore.
• Autonomous Container Database (ACD): this is the database infrastructure where Autonomous Databases are plugged in. The support for ACD is new, in this release. The operator can provision, bind, restart, and terminate (soft/hard) an ACD.

Get more details in the ADB doc as well as the ACD doc.

Database Cloud Service (DBCS)

DBCS is made of single-node DB systems and two-nodes Oracle RAC systems on virtual machines; this is part of co-managed databases in OCI (i.e., managed by Oracle and customers). The support for the Database Cloud Service is new in this release. The operator can provision, bind, scale Up/Down, liveness probe a DBCS, and it can also perform manual backup. Get more details in the DBCS controller doc.

On-Premises Databases

Requested by many customers, the automation of the lifecycle of on-premises multi-tenant databases is an indispensable part of IT modernization. With #OraOperator, multi-tenant databases (CDB/PDBs) can be integrated in DevOps CI/CD. In this release the operator can bind to an existing CDB, create a PDB, plug a PDB, unplug a PDB, delete a PDB, clone a PDB, and open/close a PDB. Get more details in the On-Premises controller doc.

Where Does OraOperator Run?

#OraOperator runs on almost every Kubernetes platform, in the Cloud or on-premises. It has been flawlessly installed and tested on the following Kubernetes platforms.

• Oracle Container Engine for Kubernetes (OKE) with Kubernetes 1.17 or later
• Oracle Linux Cloud Native Environment(OLCNE) 1.3 or later
• Minikube with version v1.21.0 or later
• Azure Kubernetes Service
• Amazon Elastic Kubernetes Service
• Red Hat OKD
• Red Hat OpenShift (we’ve heard from partners)
• Custom K8s platforms (we’ve heard of)

The key difference between those platforms is the configuration of persistent volumes for in-cluster databases. Please share your experience installing (#oraoperator) if you have tested any of the k8s platforms listed above or others.

Looking Forward …

In the upcoming releases, #OraOperator will move to production status and add support for more database deployments, as well as furnishing additional services such as OLM install, OpenTelemetry-based unified observability (traces, logs metrics) with a Grafana dashboard, and so on.

@kmensah and #OraOperator for notifications, questions, and discussions.

Thanks for reading.

Note: If you use k8s and swoon at DevOps at large, we know you’d love Oracle Cloud Infrastructure’s Free Tier account. Sign up for a Free Tier account today!

Join the conversation!

If you’re curious about the goings-on of Oracle Developers in their natural habitat, come join us on our public Slack channel! We don’t mind being your fish bowl 🐠

Oracle Database Operator for Kubernetes — Take 2 was originally published in Oracle Developers on Medium, where people are continuing the conversation by highlighting and responding to this story.

Kubernetes has become the most popular and ubiquitous open-source framework for automating Continuous Integration and Continuous Delivery (CI/CD) pipelines. Its core functionality coupled with Helm Charts (i.e., a package manager for template YAML files and other capabilities) manages the lifecycle of simple and stateless applications (i.e., define, install, deploy, and delete). Operators, on the other hand, extend Kubernetes’s core functionality with custom resources definitions (i.e., complex and stateful applications such as databases) and controllers that encapsulate the knowledge or business logic involved in managing the lifecycle of those custom resources.

Oracle is releasing the Oracle Database Operator for Kubernetes (a.k.a. OraOperator) which extends the Kubernetes API with custom resources definitions and controllers for automating Oracle Databases operations.

What’s in OraOperator for Developers, DBAs, DevOps, and GitOps Teams?

By making Oracle Databases operatable by Kubernetes similarly to its native objects, OraOperator helps developers, DBAs, DevOps and GitOps teams reduce the time and complexity of deploying and managing Oracle Databases. It eliminates the dependency on a human operator or administrator for the majority of database operations.

This first release supports Oracle Database deployed in and off Kubernetes clusters, more specifically: the Autonomous Databases in OCI (Shared infrastructure), these do not run in Kubernetes; containerized Single Instance Databases in the OCI Kubernetes Engine (OKE), and the Oracle Linux Cloud Native Environment (OLCNE); finally, containerized Oracle Sharded Databases in OKE and OLCNE.

Here is a brief description of the lifecycle operations supported by each controller.

Managing Autonomous Databases with OraOperator

Oracle Autonomous Databases (ADB) are currently deployed in the Oracle Cloud Infrastructure (OCI), not in a Kubernetes cluster.
In this release, the Autonomous Database controller supports all current workloads on shared OCI infrastructure (as opposed to dedicated infrastructure). The workloads include the Autonomous Transaction Processing (ATP), the Autonomous Data Warehouse (ADW), the Autonomous JSON Database (AJD), and Oracle APEX Application Development (APEX Service).
The ADB controller allows provisioning a new instance or binding the operator to an existing one, then the following operation can be performed: start, stop, terminate, delete, rename, scale-up/down (CPU, storage), enable/disable auto-scaling, download the wallet associated with the database instance (a regional wallet is an alternative choice), and manage the password of the database ADMIN user.

Managing Containerized Single Instance Databases with OraOperator

Oracle Database Single Instance (SIDB) are instances of either Standard Edition (SE), or Enterprise Edition (EE) databases. These can be containerized and deployed over a Kubernetes cluster (OKE and OLCNE as of this release) and managed by the operator.
The SIDB controller furnishes the following lifecycle operations: provision a new database instance, clone an existing instance, in-place or out-of-place patching to a higher release update, or rolling back to a lower release update.

Managing Containerized Oracle Database Sharding with OraOperator

Oracle Database Sharding (SDB) is made of a Catalog database, Global Data Services, and Shards (i.e., databases). All these (a.k.a. the Sharding Database topology) can be containerized and deployed as a Statefulset in a Kubernetes cluster (OKE and OLCNS as of this release) and managed by the Operator.
The SDB controller furnishes the following lifecycle operations: provision a Sharding Database topology, Shard scale up/down, Sharding Database topology cleanup.

Security Considerations

Kubernetes Secrets are the usual means for storing credentials or passwords. The operator reads the Secrets programmatically, which limits the exposure of sensitive data. However, Oracle strongly recommends that you set and get sensitive data from Oracle Cloud Infrastructure Vault, or from third-party Vaults.

The following YAML file fragment specifies Oracle Cloud Infrastructure Vault as the repository for the admin password.
adminPassword:
ociSecretOCID: ocid1.vaultsecret.oc1...

Community Contribution

Members of the Oracle database community who were briefed about this product are very excited about it. We are eager to incorporate suggestions and contributions.
See Contributing to this Repository

Looking Forward …

In upcoming releases, OraOperator will extend the capabilities of existing controllers, add support for more deployments in-and-off Kubernetes clusters, support for more Kubernetes clusters or equivalent platforms, and additional services.

@kmensah and #OraOperator for notifications, questions, and discussions.

Thanks for reading.

Making Oracle Database Kubernetes-native was originally published in Oracle Developers on Medium, where people are continuing the conversation by highlighting and responding to this story.