Increase postgres storage using RAID
Step1: Create a Linux VM
I am using azure cloud services to deploy a linux VM with Ubuntu 16.04
Step2: Add data disks
Once VM is ready, add data disk to it. I am using azure online portal to add data disks. You can also use azure cli or powershell for same.
Get Deepak Shakya’s stories in your inbox
Join Medium for free to get updates from this writer.
After adding disks, verify:
$ lsscsi
[0:0:0:0] disk Msft Virtual Disk 1.0 /dev/sda
[1:0:1:0] disk Msft Virtual Disk 1.0 /dev/sdb
[3:0:0:0] disk Msft Virtual Disk 1.0 /dev/sdc
[3:0:0:1] disk Msft Virtual Disk 1.0 /dev/sdd
[5:0:0:0] cd/dvd Msft Virtual CD/ROM 1.0 /dev/sr0/dev/sdc and /dev/sdd are newly added data disks.
Step3: Partition disks & format them for raid
$ sudo fdisk /dev/sdcWelcome to fdisk (util-linux 2.27.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xb1a55336.Command (m for help): n
Partition type
p primary (0 primary, 0 extended, 4 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (1–4, default 1):
First sector (2048–2145386495, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048–2145386495, default 2145386495):Created a new partition 1 of type ‘Linux’ and of size 1023 GiB.Command (m for help): t
Selected partition 1
Partition type (type L to list all types): fd
Changed type of partition ‘Linux’ to ‘Linux raid autodetect’.Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.$ sudo fdisk /dev/sddWelcome to fdisk (util-linux 2.27.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x5ae06a92.Command (m for help): n
Partition type
p primary (0 primary, 0 extended, 4 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (1–4, default 1):
First sector (2048–2145386495, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048–2145386495, default 2145386495):Created a new partition 1 of type ‘Linux’ and of size 1023 GiB.Command (m for help): t
Selected partition 1
Partition type (type L to list all types): fd
Changed type of partition ‘Linux’ to ‘Linux raid autodetect’.Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
Step4: Create RAID using 2 disks
$ sudo mdadm — create /dev/md0 — level 0 — raid-devices 2 /dev/sdc1 /dev/sdd1
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.Create the file system on the new RAID device
$ sudo mkfs -t ext4 /dev/md0
mke2fs 1.42.13 (17-May-2015)
Discarding device blocks: done
Creating filesystem with 536280576 4k blocks and 134070272 inodes
Filesystem UUID: 3d614d7a-e43a-4d56-b84e-de3f0fa84f10
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848, 512000000Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
Create the desired mount point for your new file system, for example:
sudo mkdir /dataWhen editing /etc/fstab, the UUID should be used to reference the file system rather than the device name. Use the blkid utility to determine the UUID for the new file system:
$ sudo /sbin/blkid
...........
/dev/md127: UUID="aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee" TYPE="ext4"Open /etc/fstab in a text editor and add an entry for the new file system, for example:
UUID=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee /data ext4 defaults 0 2Test that the /etc/fstab entry is correct:
sudo mount -aIf no error in above step(s), then you should see new mounted drive
$ df -h
Filesystem Size Used Avail Use% Mounted on
………
/dev/md0 2.0T 71M 1.9T 1% /dataStep5: Install postgresql
sudo apt-get install postgresqlStep6: Configure postgres to point to new data location
$ sudo -u postgres psql
psql (9.5.9)
Type “help” for help.postgres=# show data_directory;
data_directory
— — — — — — — — — — — — — — —
/var/lib/postgresql/9.5/main
(1 row)
Step6: Stop postgres service before moving any data
$ sudo systemctl stop postgresql$ sudo systemctl status postgresql
● postgresql.service — PostgreSQL RDBMS
Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
Active: inactive (dead) since Thu 2017–10–26 10:43:36 UTC; 6s ago
Main PID: 4734 (code=exited, status=0/SUCCESS)Oct 26 10:24:24 testing-raid systemd[1]: Starting PostgreSQL RDBMS…
Oct 26 10:24:24 testing-raid systemd[1]: Started PostgreSQL RDBMS.
Oct 26 10:24:31 testing-raid systemd[1]: Started PostgreSQL RDBMS.
Oct 26 10:43:36 testing-raid systemd[1]: Stopped PostgreSQL RDBMS.
Step7: Copy postgres data to new location
We’re going to start the rsync from the postgresql directory in order to mimic the original directory structure in our new location. By creating that postgresql directory within the mount-point directory and retaining ownership by the PostgreSQL user, we can avoid permissions problems for future upgrades. The version directory, 9.5 isn’t strictly necessary since we’ve defined the location explicitly in the postgresql.conf file, but following the project convention certainly won’t hurt, especially if there’s a need in the future to run multiple versions of PostgreSQL.
$ sudo rsync -av /var/lib/postgresql /data
sending incremental file list
postgresql/
postgresql/.psql_history
postgresql/9.5/……sent 38,545,949 bytes received 16,458 bytes 25,708,271.33 bytes/sec
total size is 38,485,955 speedup is 1.00$
Step7.1: Make backup of existing data in same location
Once the copy is complete, we’ll rename the current folder with a .bak extension and keep it until we’ve confirmed the move was successful. By re-naming it, we’ll avoid confusion that could arise from files in both the new and the old location:
$ sudo mv /var/lib/postgresql/9.5/main /var/lib/postgresql/9.5/main.bakStep8: Configure postgres to point to new data location
Edit file at this location: /etc/postgresql/9.5/main/postgresql.conf
. . .
data_directory = '/data/postgresql/9.5/main'
. . .Step9: Restart postgres service and verify new data location
$ sudo systemctl start postgresql$ sudo systemctl status postgresql
● postgresql.service — PostgreSQL RDBMS
Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
Active: active (exited) since Thu 2017–10–26 10:52:19 UTC; 6s ago
Process: 8311 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
Main PID: 8311 (code=exited, status=0/SUCCESS)Oct 26 10:52:19 testing-raid systemd[1]: Starting PostgreSQL RDBMS…
Oct 26 10:52:19 testing-raid systemd[1]: Started PostgreSQL RDBMS.$ sudo -u postgres psql
psql (9.5.9)
Type “help” for help.postgres=# show data_directory;
data_directory
— — — — — — — — — — — — — -
/data/postgresql/9.5/main
(1 row)postgres=#









