Create an EC2 instance with Ubuntu 18.04 or 20.04, and add an EBS volume to store data. You can find detailed instructions on how to create an AWS EC2 instance here (if you want to use Google Cloud, see the GCP documentation).
You will need to allow working with the following ports (outbound rules in your Security Group):
22: to connect to the instance using SSH
2345: to work with Database Lab Engine API (can be changed in the Database Lab Engine configuration file)
6000-6100: to connect to PostgreSQL clones (this is default port range used in the Database Lab Engine configuration file, can be chanfed if needed)
For real-life use, it is not a good idea to open ports to the public. Instead, it is recommended to use VPN or SSH port forwarding to access both Database Lab API and PostgreSQL clones, or to enforce encryption for all connections using NGINX with SSL and configuring SSL in PostgreSQL configuration.
Additionally, to be able to install software, allow accessing external resources using HTTP/HTTPS (edit inbound rule in your Security Group):
Here is how the inbound and outbound rules in your Security Group may look like:
If needed, you can find the detailed installation guides for Docker here.
Further, we will need
$DBLAB_DISK environment variable. It must contain the device name corresponding the disk where all the Database Lab Engine data will be stored.
To understand what needs to be specified in
$DBLAB_DISK in your case, check the output of
AWS local ephemeral NVMe disks; EBS volumes for instances built on the Nitro system:$ sudo lsblkNAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT...xvda 202:0 0 8G 0 disk└─xvda1 202:1 0 8G 0 part /nvme0n1 259:0 0 777G 0 disk$ export DBLAB_DISK="/dev/nvme0n1"
AWS EBS volumes for older (pre-Nitro) EC2 instances:$ sudo lsblkNAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT...xvda 202:0 0 8G 0 disk└─xvda1 202:1 0 8G 0 part /xvdb 202:16 0 777G 0 disk$ export DBLAB_DISK="/dev/xvdb"
ZFS is a recommended way to enable thin cloning in Database Lab. LVM is also available, but has certain limitations:
- much less flexible disk space consumption and risks for a clone to be destroyed during massive operations in it
- inability to work with multiple snapshots ("time travel"), cloning always happens based on the most recent version of data
Create a new ZFS storage pool (make sure
$DBLAB_DISK has the correct value, see the previous step!):
And check the result using
zfs list and
lsblk, it has to be like this:
Create an LVM volume (make sure that
$DBLAB_DISK has a correct value, see the previous step):
Logical volume size needs to be defined at volume creation time. By default, we allocate 10% of the available memory. If the volume size exceeds the allocated memory volume will be destroyed, potentially leading to data losses. To prevent volumes from being destroyed, consider enabling the LVM auto-extend feature.
To enable the auto-extend feature, the following LVM configuration options need to be updated:
snapshot_autoextend_threshold: auto-extend a "snapshot" volume when its usage exceeds the specified percentage
snapshot_autoextend_percent: auto-extend a "snapshot" volume by the specified percentage of the available space once the usage exceeds the threshold
Update LVM configuration (located in
/etc/lvm/lvm.conf by default):