Deploy a Weaviate vector database on GKE

Set up your environment

To set up your environment with Cloud Shell, follow these steps:

1. Set environment variables for your project, region, and a Kubernetes cluster resource prefix:

   export PROJECT_ID=PROJECT_ID
   export KUBERNETES_CLUSTER_PREFIX=weaviate
   export REGION=us-central1
   

   Replace PROJECT_ID with your Google Cloud project ID.

   This tutorial uses us-central1 region to create your deployment resources.

2. Check the version of Helm:

   helm version
   

   Update the version if it's older than 3.13:

   curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
   

3. Clone the sample code repository from GitHub:

   git clone https://github.com/GoogleCloudPlatform/kubernetes-engine-samples
   

4. Navigate to the weaviate directory:

   cd kubernetes-engine-samples/databases/weaviate
   

Create your cluster infrastructure

In this section, you run a Terraform script to create a private, highly-available, regional GKE cluster to deploy your Weaviate database.

You can choose to deploy Weaviate using a Standard or Autopilot cluster. Each has its own advantages and different pricing models.

export GOOGLE_OAUTH_ACCESS_TOKEN=$(gcloud auth print-access-token)
terraform -chdir=terraform/gke-autopilot init
terraform -chdir=terraform/gke-autopilot apply \
-var project_id=${PROJECT_ID} \
-var region=${REGION} \
-var cluster_prefix=${KUBERNETES_CLUSTER_PREFIX}

...
Apply complete! Resources: 9 added, 0 changed, 0 destroyed.

Outputs:

kubectl_connection_command = "gcloud container clusters get-credentials weaviate-cluster --region us-central1"

export GOOGLE_OAUTH_ACCESS_TOKEN=$(gcloud auth print-access-token)
terraform -chdir=terraform/gke-standard init
terraform -chdir=terraform/gke-standard apply \
-var project_id=${PROJECT_ID} \
-var region=${REGION} \
-var cluster_prefix=${KUBERNETES_CLUSTER_PREFIX}

...
Apply complete! Resources: 10 added, 0 changed, 0 destroyed.

Outputs:

kubectl_connection_command = "gcloud container clusters get-credentials weaviate-cluster --region us-central1"

Connect to the cluster

Configure kubectl to fetch credentials and communicate with your new GKE cluster:

gcloud container clusters get-credentials \
    ${KUBERNETES_CLUSTER_PREFIX}-cluster --location ${REGION}

Deploy the Weaviate database to your cluster

To use Helm chart to deploy the Weaviate database to your GKE cluster, follow these steps:

1. Add the Weaviate database Helm Chart repository before you can deploy it on your GKE cluster:

   helm repo add weaviate https://weaviate.github.io/weaviate-helm
   

2. Create the namespace weaviate for the database:

   kubectl create ns weaviate
   

3. Create a secret to store the API key:

   kubectl create secret generic apikeys --from-literal=AUTHENTICATION_APIKEY_ALLOWED_KEYS=$(openssl rand -base64 32) -n weaviate
   

4. Deploy an internal load balancer to access Weaviate from within the virtual network:

   kubectl apply -n weaviate -f manifests/05-ilb/ilb.yaml
   

   The ilb.yaml manifest describes the load balancer service:

   apiVersion: v1
   kind: Service
   metadata:
     annotations:
       #cloud.google.com/neg: '{"ingress": true}'
       networking.gke.io/load-balancer-type: "Internal"
     labels:
       app.kubernetes.io/name: weaviate
     name: weaviate-ilb
   spec:
     ports:
     - name: http
       port: 8080
       protocol: TCP
       targetPort: 8080
     - name: grpc
       port: 50051
       protocol: TCP
       targetPort: 50051
     selector:
       app: weaviate
     type: LoadBalancer

5. Apply the manifest to deploy Weaviate cluster:

   helm upgrade --install "weaviate" weaviate/weaviate \
   --namespace "weaviate" \
   --values ./manifests/01-basic-cluster/weaviate_cluster.yaml
   

   The weaviate_cluster.yaml manifest describes the Deployment. A Deployment is a Kubernetes API object that lets you run multiple replicas of Pods that are distributed among the nodes in a cluster.:

   initContainers:
     sysctlInitContainer:
       enabled: false
     extraInitContainers: {}
   resources: 
      requests:
        cpu: '1'
        memory: '4Gi'
      limits:
        cpu: '2'
        memory: '4Gi'
   replicas: 3
   storage:
     size: 10Gi
     storageClassName: "premium-rwo"
   service:
     name: weaviate
     ports:
       - name: http
         protocol: TCP
         port: 80
     type: ClusterIP
   grpcService:
     enabled: true
     name: weaviate-grpc
     ports:
       - name: grpc
         protocol: TCP
         port: 50051
     type: ClusterIP
   authentication:
     anonymous_access:
       enabled: false
   authorization:
     admin_list:
       enabled: true
       users:
         - [email protected]
   modules:
     text2vec-palm:
       enabled: true
   env:
     AUTHENTICATION_APIKEY_ENABLED: 'true'
     AUTHENTICATION_APIKEY_USERS: '[email protected]'
     PROMETHEUS_MONITORING_ENABLED: true
   envSecrets:
     AUTHENTICATION_APIKEY_ALLOWED_KEYS: apikeys
   tolerations:
     - key: "app.stateful/component"
       operator: "Equal"
       value: "weaviate"
       effect: NoSchedule

   Wait for a few minutes for the Weaviate cluster to fully start.

6. Check the Deployment status:

   kubectl get weaviate -n weaviate --watch
   

   The output is similar to following, if the weaviate database is successfully deployed:

   NAME: weaviate
   LAST DEPLOYED: Tue Jun 18 13:15:53 2024
   NAMESPACE: weaviate
   STATUS: deployed
   REVISION: 1
   TEST SUITE: None
   

7. Wait for Kubernetes to start the resources:

   kubectl wait pods -l app.kubernetes.io/name=weaviate --for condition=Ready --timeout=300s -n weaviate
   

Run queries with Vertex AI Colab Enterprise notebook

This section explains how to connect to your Weaviate database using Colab Enterprise. You can use a dedicated runtime template to deploy to the weaviate-vpc, so the notebook can communicate with resources in the GKE cluster.

For more information about Vertex AI Colab Enterprise, see Colab Enterprise documentation.

Create a runtime template

To create a Colab Enterprise runtime template:

1. In the Google Cloud console, go to the Colab Enterprise Runtime Templates page and make sure your project is selected:

   Go to Runtime Templates

2. Click add_box New Template. The Create new runtime template page appears.

3. In the Runtime basics section:

   • In the Display name field, enter weaviate-connect.
   • In the Region drop-down list, select us-central1. It's the same region as your GKE cluster.

4. In the Configure compute section:

   • In the Machine type drop-down list, select e2-standard-2.
   • In the Disk size field, enter 30.

5. In the Networking and security section:

   • In the Network drop-down list, select the network where your GKE cluster resides.
   • In the Subnetwork drop-down list, select a corresponding subnetwork.
   • Clear the Enable public internet access checkbox.

6. To finish creating the runtime template, click Create. Your runtime template appears in the list on the Runtime templates tab.

Create a runtime

To create a Colab Enterprise runtime:

1. In the runtime templates list for the template you just created, in the Actions column, click more_vert and then click Create runtime. The Create Vertex AI Runtime pane appears.

2. To create a runtime based on your template, click Create.

3. On the Runtimes tab that opens, wait for the status to transition to Healthy.

Import the notebook

To import the notebook in Colab Enterprise:

1. Go to the My Notebooks tab and click Import. The Import notebooks pane appears.

2. In Import source, select URL.

3. Under Notebook URLs, enter the following link:

   https://raw.githubusercontent.com/GoogleCloudPlatform/kubernetes-engine-samples/main/databases/weaviate/manifests/02-notebook/vector-database.ipynb
   

4. Click Import.

Connect to the runtime and run queries

To connect to the runtime and run queries:

1. In the notebook, next to the Connect button, click arrow_drop_down Additional connection options. The Connect to Vertex AI Runtime pane appears.

2. Select Connect to a runtime and then select Connect to an existing Runtime.

3. Select the runtime that you launched and click Connect.

4. To run the notebook cells, click the Run cell button next to each code cell.

The notebook contains both code cells and text that describes each code block. Running a code cell executes its commands and displays an output. You can run the cells in order, or run individual cells as needed.

View Prometheus metrics for your cluster

The GKE cluster is configured with Google Cloud Managed Service for Prometheus, which enables collection of metrics in the Prometheus format. This service provides a fully managed solution for monitoring and alerting, allowing for collection, storage, and analysis of metrics from the cluster and its applications.

The following diagram shows how Prometheus collects metrics for your cluster:

The GKE private cluster in the diagram contains the following components:

• Weaviate Pods that expose metrics on the path /metrics and port 2112.
• Prometheus-based collectors that process the metrics from the Weaviate Pods.
• A PodMonitoring resource that sends the metrics to Cloud Monitoring.

To export and view the metrics, follow these steps:

1. Create the PodMonitoring resource to scrape metrics by labelSelector:

   kubectl apply -n weaviate -f manifests/03-prometheus-metrics/pod-monitoring.yaml
   

   The pod-monitoring.yaml manifest describes the PodMonitoring resource:

   apiVersion: monitoring.googleapis.com/v1
   kind: PodMonitoring
   metadata:
     name: weaviate
   spec:
     selector:
       matchLabels:
         app: weaviate
     endpoints:
     - port: 2112
       interval: 30s
       path: /metrics

2. To import a custom Cloud Monitoring dashboard with the configurations defined in dashboard.json:

   gcloud --project "${PROJECT_ID}" monitoring dashboards create --config-from-file monitoring/dashboard.json
   

3. After the command runs successfully, go to the Cloud Monitoring Dashboards:

   Go to Dashboards overview

4. From the list of dashboards, open the Weaviate Overview dashboard. It might take some time to collect and display metrics. The dashboard shows amount of Shards, Vectors and operations latency