How it Works#

The Database Lab Engine is an open source experimentation platform for PostgreSQL databases. Instantly create full-size clones of your production database and use them to test your database migrations, optimize SQL, or deploy full-size staging apps.

Architecture#

Database Lab Architecture

The Database Lab Engine (DLE) runs on an independent server within your own infrastructure. The engine stores a single, full-size copy of production data on the specialized ZFS filesystem.1

Using the copy-on-write capabilities of ZFS, the DLE is able to generate full-size replicas of the production database in seconds. These writable "thin clones" will behave identically to production: they will have the same data and generate the same query plans.

Learn more about the DLE

Security#

Security Model

Instances of the Database Lab Engine (DLE) and all thin clones reside completely within your own infrastructure at all times.

The DLE is managed via an API which can be exposed either over HTTPS or an SSH tunnel. The team at Postgres.ai and all platform components:

  • cannot reach your databases
  • cannot read or modify the data in your databases
  • cannot SSH to your machines

Production data is sensitive - even within your own organization. You can configure the DLE to generate fully masked clones so that engineers without production authorization can also gain the benefits of experimenting with thin clones.

Learn more about our Security Model

Setup#

You can install the Database Lab Engine (DLE) on a linux2 server where you can provision a ZFS volume. If you run your infrastructure on AWS, you can follow our Getting Started Guide for RDS.

Here's what you can expect when setting up the DLE:

  • Experienced engineers can setup the DLE in less than 1 hour
  • The DLE host instance should have a disk 30% larger than the production DB
  • Sensitive data masking can be achieved with a set of declarative rules
  • 1 If desired, alternate copy-on-write filesystems may be used. See the  documentation
  • 2 The DLE has been tested on Ubuntu and RHEL/CentOS.