• help build development and testing environments involving full-size database,
• provide temporary full-size database clones for SQL query optimization (see also: Joe bot, which works on top of Database Lab),
• help verify database migrations (DB schema changes) and massive data operations.

As an example, the cloning of 10 TiB PostgreSQL database takes less than 2 seconds.

In rapidly developing businesses, excellent production health requires powerful non-production environments. With Database Lab, provisioning of multi-terabyte database clones doesn't imply much waiting time or extra budgets spent anymore. Therefore, Database Lab gives necessary power to developers, DBAs, and QA engineers, boosting development and testing processes.

The project is in its early stage. However, it is already being extensively used in some teams in their daily work. Since production is not involved, it is quite easy to try and start using it.

Please support the project giving a GitLab star (it's on the main page, at the upper right corner):

To discuss Database Lab, try a demo, or ask any questions, join our community Slack.

• Perform SQL optimization in a convenient way (see postgres-ai/joe)
• Check database schema changes (database migrations) on full-sized database clones using Database Lab in CI (see postgres-ai/ci-example)

TL;DR: you need:

• any machine with a separate disk that is big enough to store a single copy of your database,
• Linux with Docker,
• ZFS to enable thin cloning (the default option; also, LVM is supported as an alternative),
• initial copy of your Postgres database.

Details:

• for each Database Lab instance, a separate machine (either physical or virtual) is needed,
• both on-premise and cloud setups are possible,
• for each Postgres cluster (a single Postgres server with databases), a separate Database Lab instance is required,
• the machine needs to have a separate disk partition with size enough to store the target Postgres directory (PGDATA),
• any modern Linux is supported,
• Docker needs to be installed on the machine,
• currently, you need to take care yourself of the initial copying of the database to this disk ("thick cloning" stage), use pg_basebackup, restoration from an archive (such as WAL-G, Barman, pgBackRest or any), or dump/restore (the only way supported for RDS, until AWS guys decide to allow replication connections),
• upon request, Database Lab will do "thin cloning" of PGDATA, providing fully independent writable Postgres clones to users;
• currently, two technologies are supported for thin cloning:
  • ZFS: ZFS on Linux needs to be installed on the machine,
  • LVM: with LVM, all thin clones will be based on the latest version of the database (with ZFS, you can have multiple snapshots prepared and choose which one to use when requesting a new thin clone).
• it is easy to extend and add, Ceph - please contact us if you need it;
• comments, bug reports, and contributions are highly welcome.

See detailed tutorial in our documentation.

For stable Docker images see postgresai/dblab-server repository on DockerHub.

Install Database Lab client CLI on a Linux architecture (e.g., Ubuntu):

curl https://gitlab.com/postgres-ai/database-lab/-/raw/master/scripts/cli_install.sh | bash

Also, binaries available for download: Alpine, other.

See the full client CLI reference here.

Once you have Database Lab server(s) intalled, and client CLI installed on your machine, initialize CLI and start communicating with the Database Lab server(s).

dblab init \
  --environment-id=tutorial \
  --url=http://$IP_OR_HOSTNAME:3000 \
  --token=secret_token

Access your Database Lab instance and check its status.

dblab instance status

When your Database Lab instance is up and running you can use it to create thin clones, work with them, delete the existing clones, and see the list of existing clones. To create a thin clone, you need to execute a dblab clone create and fill all the required options, as illustrated below:

dblab clone create \
  --username dblab_user_1 \
  --password secret_password \
  --id my_first_clone

After a few seconds, if everything is configured correctly, you will see that the clone is ready to be used:

{
    "id": "botcmi54uvgmo17htcl0",
    "snapshot": {
        "id": "dblab_pool@initdb",
        ...
    },
    "status": {
        "code": "OK",
        "message": "Clone is ready to accept Postgres connections."
    },
    "db": {
        "connStr": "host=localhost port=6000 user=dblab_user_1",
        "host": "localhost",
        "port": "6000",
        "username": "dblab_user_1",
        "password": ""
    },
    ...
}

Now you can work with this clone using any PostgreSQL client, for example psql:

PGPASSWORD=secret_password \
  psql "host=${IP_OR_HOSTNAME} port=6000 user=dblab_user_1 dbname=test" \
  -c '\l+'

• API reference
• CLI reference

• Check our Q&A
• or join our community (links below)

• Community Slack (English)
• Telegram (Russian)

See our GitLab Container Registry for develop Docker images.

1. Install golangci-lint: https://github.com/golangci/golangci-lint#install