Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion pages/data-warehouse/how-to/connect-applications.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ To connect your deployment with BI tools, refer to the [dedicated documentation]

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.
- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

1. Click **ClickHouse®** under **Data & Analytics** on the side menu. The Data Warehouse deployment page displays.
Expand Down
1 change: 0 additions & 1 deletion pages/data-warehouse/how-to/connect-bi-tools.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ This page explains how to integrate your Data Warehouse for ClickHouse® deploym

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.
- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

## Tableau
Expand Down
1 change: 0 additions & 1 deletion pages/data-warehouse/how-to/create-deployment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ A Data Warehouse is a centralized repository designed to store, manage, and anal

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.

## Creating a Data Warehouse Deployment

Expand Down
2 changes: 1 addition & 1 deletion pages/data-warehouse/how-to/edit-autoscaling.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ This page explains how to edit the autoscaling configuration of your Data Wareho

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.

- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

<Message type="important">
Expand Down
2 changes: 1 addition & 1 deletion pages/data-warehouse/how-to/import-data.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Scaleway Data Warehouse for ClickHouse® allows you to quickly import any type o

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.

- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

## How to import data using the ClickHouse® CLI
Expand Down
50 changes: 39 additions & 11 deletions pages/data-warehouse/how-to/manage-databases.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,29 +9,57 @@ dates:
import Requirements from '@macros/iam/requirements.mdx'


This page explains how to manage databases in your Data Warehouse for ClickHouse® deployment using the [Scaleway console](https://console.scaleway.com/).

<Message type="note">
During the private beta phase, database management can only be done with the **scwadmin** & via the ClickHouse® CLI, MySQL or HTTPS protocols.
</Message>
This page explains how to manage databases in your Data Warehouse for ClickHouse® deployment.

<Requirements />

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.
- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

## How to manage databases using the Scaleway console

During the private beta phase, the management of databases in your Data Warehouse for ClickHouse® deployment is limited to viewing databases in your deployment.
The Scaleway console allows you to create and delete databases for your Data Warehouse for ClickHouse® deployment.

### How to create a database using the Scaleway console

1. Click **ClickHouse®** under **Data & Analytics** on the side menu. The Data Warehouse deployment page displays.

2. Click the name of the desired Data Warehouse deployment. The **Overview** tab of the deployment displays.

3. Click the **Databases** tab. A list of your current databases displays.

4. Click **+ Create database**. A pop-up displays.

5. Enter a name for your new database. It can can only contain alphanumeric characters, underscores, and dashes.

6. Click **Create database** to confirm.

Your new database appears in the list.

### How to delete a database using the Scaleway console

<Message type="note">
The `default` database cannot be deleted.
</Message>

1. Click **ClickHouse®** under **Data & Analytics** on the side menu. The Data Warehouse deployment page displays.

2. Click the name of the desired Data Warehouse deployment. The **Overview** tab of the deployment displays.

3. Click <Icon name="delete" /> next to the name of the database you want to delete. A confirmation pop-up displays.

4. Enter `DELETE`, then click **Delete database** to confirm.

The deleted database no longer appears in the list.

## How to manage databases using frameworks

[Connect to your deployment](/data-warehouse/how-to/connect-applications/) using your preferred framework.
<Message type="tip">
You can also connect using the ClickHouse HTTP console from the **Overview** tab of your deployment. Make sure to enter valid credentials in the top-right fields.
</Message>
[Connect to your deployment](/data-warehouse/how-to/connect-applications/) using your preferred framework.

<Message type="tip">
You can also connect using the ClickHouse HTTP console from the **Overview** tab of your deployment. Make sure to enter valid credentials in the top-right fields.
</Message>

Once connected, you can run SQL queries to `CREATE/ALTER/DROP` your `DATABASE/TABLE`.

Expand Down
1 change: 0 additions & 1 deletion pages/data-warehouse/how-to/manage-delete-deployment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ This page explains how to manage and delete your Data Warehouse deployment.

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.
- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

## How to manage a Data Warehouse deployment
Expand Down
141 changes: 141 additions & 0 deletions pages/data-warehouse/how-to/manage-storage.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
---
title: How to manage data storage in a Data Warehouse for ClickHouse® deployment
description: This page explains how to manage data storage in a Data Warehouse for ClickHouse® deployment using storage policies
tags: data storage policy strategy options object block
dates:
validation: 2025-10-16
posted: 2025-10-16
---

import Requirements from '@macros/iam/requirements.mdx'

<Requirements />

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization

## Overview

Scaleway Data Warehouse for ClickHouse® uses ClickHouse®'s native storage policies to decide where table data lives: on Scaleway Block storage, and/or on Object Storage (Amazon S3). Policies are applied at table-level during table creation, using the settings presented below. You can choose between three policies for your deployment:

- `tiered` (default in Scaleway Data Warehouse for ClickHouse®)
- `s3_cache` (cache mode)
- `default` (local-only mode)

The table below provides an overview of each storage policy's behavior and use case.

| Storage&nbsp;policy | Behavior | When to use |
|------|---------|------------|
| `tiered` | Writes on Block Storage first, and parts are automatically moved to Object Storage when disk is 90% full. | General purpose setting for fast local writes and transparent spillover to Object Storage as the dataset grows. |
| `s3_cache` | Data is stored on Object Storage. A local Block Storage cache layer keeps frequently-read parts to accelerate repeated reads. | Large datasets that mostly live in Object Storage, with repeated reads on smaller subsets that benefit from local caching. |
| `default` | Data is stored on Block storage only. | Small datasets where lowest latency for reads/writes on local disk is desired and capacity fits the Block Storage volume. |

## Tiered storage policy

### Applying the tiered policy

**Tiered** is the deployment's default policy. Data is stored on the `hot` volume (Block Storage) until it reaches 90% of its capacity, then data is moved to the `cold` volume. This mechanism is controlled by `move_factor:0.1`.

[Connect to you deployment](/data-warehouse/how-to/connect-applications/), then run the SQL query below to apply the `tiered` storage policy.

<Tabs id="tiered-explicit-implicit">
<TabsTab label="Explicit">

Storage policy is explicitly defined as `tiered`.

```sql
CREATE TABLE tiered_table
(
id UInt64,
ts DateTime,
value Float32
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(ts)
ORDER BY id
SETTINGS storage_policy = 'tiered';
```

</TabsTab>
<TabsTab label="Implicit">

Storage policy is not defined, ClickHouse® applies the `tiered` storage policy by default.

```sql
CREATE TABLE dw.tiered_table_implicit
(
id UInt64,
ts DateTime,
value Float32
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(ts)
ORDER BY id;
```
</TabsTab>
</Tabs>

<Message type="note">
If you omit the `storage_policy` setting, ClickHouse® will automatically apply the `tiered` policy.
</Message>

### Moving data with the tiered policy

When using the `tiered` storage policy, you can manually move partitions between `hot` (Block Storage) and `cold` (Object Storage) volumes using the SQL queries below.

- Moving specific partitions to the `cold` volume (Object Storage):

```sql
ALTER TABLE tiered_table MOVE PARTITION '2024-08' TO VOLUME 'cold';
```

- Moving a non-partitioned table (`tuple()`) to the `cold` volume (Object Storage):

```sql
ALTER TABLE tiered_table_without_partitions MOVE PARTITION tuple() TO VOLUME 'cold';
```

- Moving a partition to the `hot` volume (Block Storage):

```sql
ALTER TABLE tiered_table MOVE PARTITION '2024-08' TO VOLUME 'hot';
```
- Moving a non-partitioned table (`tuple()`) to the `hot` volume (Block Storage):

```sql
ALTER TABLE tiered_table_without_partitions MOVE PARTITION tuple() TO VOLUME 'hot';
```

## S3 cache policy

[Connect to your deployment](/data-warehouse/how-to/connect-applications/), then run the SQL query below to create a new table with the `s3_cache` policy enabled.
```sql
CREATE TABLE cache_table
(
id UInt64,
ts DateTime,
value Float32
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(ts)
ORDER BY id
SETTINGS storage_policy = 's3_cache';
```

## Local-only policy

[Connect to your deployment](/data-warehouse/how-to/connect-applications/), then run the SQL query below to create a new table with the `default` (Block Storage only) policy enabled.

```sql
CREATE TABLE local_table
(
id UInt64,
ts DateTime,
value Float32
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(ts)
ORDER BY id
SETTINGS storage_policy = 'default';
```

107 changes: 97 additions & 10 deletions pages/data-warehouse/how-to/manage-users.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,24 +9,63 @@ dates:
import Requirements from '@macros/iam/requirements.mdx'


This page explains how to manage users in your Data Warehouse for ClickHouse® deployment using the [Scaleway console](https://console.scaleway.com/).

<Message type="note">
During the private beta phase, user management can only be done with the **scwadmin** user via the ClickHouse® CLI, MySQL or HTTPS protocols.
</Message>
This page explains how to manage users in your Data Warehouse for ClickHouse® deployment.

<Requirements />

- A Scaleway account logged into the [console](https://console.scaleway.com)
- [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization
- [Signed up to the private beta](https://www.scaleway.com/fr/betas/) and received a confirmation email.
- Created a [Data Warehouse deployment](/data-warehouse/how-to/create-deployment/)

## How to manage users using the Scaleway console

During the private beta phase, the management of users for your Data Warehouse for ClickHouse® deployment is limited to:
- Viewing users in your deployment
- Changing the password of an existing user
The Scaleway console allows you to create users for your Data Warehouse for ClickHouse® deployment, to grant them admin rights, to update their password, and to delete them.

### How to create a user using the Scaleway console

1. Click **ClickHouse®** under **Data & Analytics** on the side menu. The Data Warehouse deployment page displays.

2. Click the name of the desired Data Warehouse deployment. The **Overview** tab of the deployment displays.

3. Click the **Users** tab. A list of your current users displays.

4. Click **+ Create user**. A pop-up displays.

5. Enter a username. It must be unique, contain up to 63 characters, and must start with a letter. Only alphanumeric characters, underscores, and dashes are accepted.

6. Enter a password, or generate one automatically.

7. Enable the toggle if you want to grant admin rights to the user.

8. Click **Create user** to proceed.

The newly created user appears in the list of your deployment's users.

### How to update a user's password and admin rights

1. Click **ClickHouse®** under **Data & Analytics** on the side menu. The Data Warehouse deployment page displays.

2. Click the name of the desired Data Warehouse deployment. The **Overview** tab of the deployment displays.

3. Click the **Users** tab. A list of your current users displays.

4. Click <Icon name="more" /> next to the name of the user you want to update, then click **Update user**. A pop-up displays.

5. Update the user's password and/or admin rights, then click **Update user** to confirm.

### How to delete a user using the Scaleway console

1. Click **ClickHouse®** under **Data & Analytics** on the side menu. The Data Warehouse deployment page displays.

2. Click the name of the desired Data Warehouse deployment. The **Overview** tab of the deployment displays.

3. Click the **Users** tab. A list of your current users displays.

4. Click <Icon name="delete" /> next to the name of the user you want to delete. A confirmation pop-up displays.

5. Enter `DELETE`, then click **Delete user** to confirm.``

The user is deleted and no longer appears in the list.

## How to manage users using frameworks

Expand All @@ -37,4 +76,52 @@ During the private beta phase, the management of users for your Data Warehouse f

Once connected, you can run SQL queries using `USER`, `ROLE`, and `ROW POLICY` to create new users and grant them the desired permissions via roles.

Refer to the official ClickHouse® documentation on [users and roles](https://clickhouse.com/docs/sql-reference/statements/create/user) for more information.
Refer to the official ClickHouse® documentation on [users and roles](https://clickhouse.com/docs/sql-reference/statements/create/user) for more information.

## Manage user permissions

[Connect to your deployment](/data-warehouse/how-to/connect-applications/) using your preferred framework.

Once connected, you can run SQL as shown below to grant the desired permissions to users.

**Grant permission to create databases**

```sql
GRANT CREATE DATABASE ON . TO user;
```

**Grant permission to create tables in any database**

```sql
GRANT CREATE TABLE ON . TO user;
```

**Grant permission to create tables in a specific database**

```sql
GRANT CREATE TABLE ON database_name.* TO user;
```

**Grant data manipulation permissions for any table in any database**

```sql
GRANT INSERT, UPDATE, DELETE ON . TO user;
```

**Grant data manipulation permissions for any table in a specific database**

```sql
GRANT INSERT, UPDATE, DELETE ON database_name.* TO user;
```

**Grant data manipulation permissions for a specific table in a specific database**

```sql
GRANT INSERT, UPDATE, DELETE ON database_name.table_name TO user;
```

**Grant every permissions on every database**

```sql
GRANT CREATE DATABASE, CREATE TABLE, INSERT, UPDATE, DELETE ON . TO user;
```
Loading
Loading