Action required to migrate from a previous version. See Migration notes.
The Database Lab Engine (DLE) is an open-source technology that enables thin cloning for PostgreSQL. Thin clones are exceptionally useful when you need to scale the development process. DLE can manage dozens of independent clones of your database on a single machine, so each engineer or automation process works with their own database provisioned in seconds without extra costs.
DLE 2.5 significantly expands the capabilities of automatic preparation of snapshots directly from managed database services, as well as from logical dumps, namely:
- restoring of multiple databases
- various pg_dump output formats and file compression formats
Since version 2.5, it becomes possible to reset the clone's database state to a specific snapshot if multiple snapshots are available. See DLE CLI reference. There is also a new option for the
--latest, that allows resetting to the latest available state not knowing the snapshot name. This can be very useful in situations when a clone lives long, occupying a specific port, and some applications (e.g., analytical tools) are configured to work with it – users can periodically switch to the freshest database state without a need to reconfigure their applications.
All new restore features are also already available in the Terraform module (currently works with AWS only).
Additionally, this release has a lot of improvements and fixes. Read the full changelog.
The Database Lab ecosystem is growing with new products and services. Most of them have configuration files and metadata, but their structure and naming are not uniform.
To simplify work with the configuration, we decided to redesign the configuration structure of all Database Lab services and standardize the naming of configuration files and directories.
- The main directory of Database Lab configuration is
~/.dblab. There are directories for each service inside this main directory:
cli- configs of Database Lab command-line interface
engine- configs and metadata of Database Lab Engine
joe- configs and metadata of Joe Bot assistant
ci_checker- configs of DB Migration Checker
- Rename the configuration file of Database Lab Engine to
server.ymland store it to
- In the running command
docker run ...replace the mounting flagwith
--volume ~/.dblab/engine/configs:/home/dblab/configs:ro--volume ~/.dblab/engine/meta:/home/dblab/meta
Check the example of configuration structure on a host machine:
~/.dblab/ cli/ - cli.yml engine/ configs/ - server.yml meta/ - state.json joe/ configs/ - joe.yml meta/ - sessions.json ci_checker/ configs/ - ci_checker.yml
Database Lab Engine starts a number of Docker containers with a PostgreSQL instance inside. A powerful configuration system allows controlling various properties of running containers (PostgreSQL version, Docker image, container parameters, extensions loaded, PostgreSQL configuration parameters).
However, such flexibility may be inconvenient in the case of a number of the configuration section. On the other hand, YAML anchors and aliases can be used, help you conveniently manage your configuration sections
YAML allows defining a "global" binding with
& where you can set properties for all PostgreSQL containers and then refer to it using an alias denoted by
⚠ Note, the "local" configuration of each section would still be supported, overriding particular parts.
There is an example using an anchor:
# Configure database containersdatabaseContainer: &db_container dockerImage: 'postgresai/extended-postgres:13' containerConfig: 'shm-size': 1gb retrieval: jobs: - logicalDump - logicalRestore - logicalSnapshot spec: logicalDump: options: <<: *db_container dumpLocation: '/var/lib/dblab/dblab_pool/dump'
See more examples of configuration files in the Database Lab repository
If you have problems or questions, please contact our communities for help: https://postgres.ai/docs/questions-and-answers#where-to-get-help
Feedback and contributions would be greatly appreciated:
- Database Lab Community Slack: https://slack.postgres.ai/
- DLE & DB Migration Checker issue tracker: https://gitlab.com/postgres-ai/database-lab/-/issues
- Issue tracker of the Terraform module for Database Lab: https://gitlab.com/postgres-ai/database-lab-infrastructure/-/issues
Share this blog post:
Database Lab by Postgres.ai
An open-source experimentation platform for PostgreSQL databases. Instantly create full-size clones of your production database and use them to test your database migrations, optimize SQL, or deploy full-size staging apps.