Disk configuration for 'charm'

home | airgap | charisma | verve | trail | commitment | discernment | lore

For more info see 'charm'.

nvme0n1 (1TB)
PNY21242106180100095
nvme0n1p1 (1.049G)
[unused]
nvme0n1p2 (2G)
md0
nvme0n1p3 (8G)
md1
nvme0n1p4 (128G)
md2
nvme0n1p5 (800G)
fast
nvme1n1 (1TB)
PNY21242106180100093
nvme1n1p1 (1.049G)
[unused]
nvme1n1p2 (2G)
md0
nvme1n1p3 (8G)
md1
nvme1n1p4 (128G)
md2
nvme1n1p5 (800G)
fast
nvme2n1 (1TB)
PNY21242106180100092
nvme2n1p1 (1.049G)
/boot/efi
nvme2n1p2 (2G)
md0
nvme2n1p3 (8G)
md1
nvme2n1p4 (128G)
md2
nvme2n1p5 (800G)
fast
nvme3n1 (1TB)
PNY21242106180100094
nvme3n1p1 (1.049G)
[unused]
nvme3n1p2 (2G)
md0
nvme3n1p3 (8G)
md1
nvme3n1p4 (128G)
md2
nvme3n1p5 (800G)
fast
Drive Description Model Commissioned WRL
nvme[0123]n1 PNY CS3140 1TB SSD PNY CS3140 1TB NVMe Gen4 SSD M.2 2022-03-23 700TBW

MD RAID

Note: all MD RAID devices are used with a single partition.

Device RAID Components Capacity
/dev/md0 RAID1 4x 2GB 2GB
/dev/md1 RAID0 4x 8GB 32GB
/dev/md2 RAID1 4x 128GB 128GB

Mounts

Partition Capacity File-system Mount point Mount options
/dev/nvme2n1p1 1GB vfat /boot/efi defaults
/dev/md0p1 2GB ext4 /boot noatime
/dev/md1p1 32GB swap swap
/dev/md2p1 128GB btrfs / noatime

ZFS zpools

Pool RAID Components Capacity
fast RAID10 4x 800G 1.6TB

ZFS datasets

These are the datasets we create on our zpools.

Dataset Mount Compression Dedup
fast /fast lz4 on
fast/data /data gzip-9 on
fast/data/mysql /data/mysql off off
fast/download /temp/download lz4 on
fast/home /home lz4 on
fast/home/jj5 /fast lz4 on
fast/mysql /var/lib/mysql lz4 on
fast/vbox /vbox lz4 on

Commands

DISK1=/dev/disk/by-id/nvme-eui.6479a74fb0c00509
DISK2=/dev/disk/by-id/nvme-eui.6479a74fb0c00507
DISK3=/dev/disk/by-id/nvme-eui.6479a74fb0c004b7
DISK4=/dev/disk/by-id/nvme-eui.6479a74fb0c00508

zpool create \
    -o ashift=12 -o autotrim=on \
    -O acltype=posixacl -O compression=lz4 \
    -O dnodesize=auto -O normalization=formD -O atime=off -O dedup=on \
    -O xattr=sa \
    fast mirror ${DISK1}-part5 ${DISK2}-part5 mirror ${DISK3}-part5 ${DISK4}-part5

Photos

Here are some happy snaps from the install.

Configuration status

-------------------
Thu Mar 24 08:55:46 [bash:5.0.17 jobs:0 error:0 time:0]
jj5@charm:/home/jj5
$ cat /proc/mdstat
Personalities : [raid0] [raid1] [linear] [multipath] [raid6] [raid5] [raid4] [raid10]
md2 : active raid1 nvme0n1p4[3] nvme2n1p4[0] nvme3n1p4[2] nvme1n1p4[1]
      126245888 blocks super 1.2 [4/4] [UUUU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md0 : active raid1 nvme0n1p2[3] nvme2n1p2[0] nvme1n1p2[1] nvme3n1p2[2]
      2094080 blocks super 1.2 [4/4] [UUUU]

md1 : active raid0 nvme0n1p3[1] nvme2n1p3[2] nvme1n1p3[3] nvme3n1p3[0]
      33517568 blocks super 1.2 512k chunks

unused devices: 
-------------------
Thu Mar 24 08:55:48 [bash:5.0.17 jobs:0 error:0 time:2]
jj5@charm:/home/jj5
$ zpool list
NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
fast  1.55T   589G  1003G        -         -    16%    37%  1.30x    ONLINE  -
-------------------
Thu Mar 24 08:55:55 [bash:5.0.17 jobs:0 error:0 time:9]
jj5@charm:/home/jj5
$ zpool status
  pool: fast
 state: ONLINE
  scan: none requested
config:

        NAME                                 STATE     READ WRITE CKSUM
        fast                                 ONLINE       0     0     0
          mirror-0                           ONLINE       0     0     0
            nvme-eui.6479a74fb0c00509-part5  ONLINE       0     0     0
            nvme-eui.6479a74fb0c00507-part5  ONLINE       0     0     0
          mirror-1                           ONLINE       0     0     0
            nvme-eui.6479a74fb0c004b7-part5  ONLINE       0     0     0
            nvme-eui.6479a74fb0c00508-part5  ONLINE       0     0     0

errors: No known data errors
-------------------
Wed Mar 30 05:26:04 [bash:5.0.17 jobs:0 error:0 time:194]
root@charm:/var/backups
# zfs list
NAME              USED  AVAIL     REFER  MOUNTPOINT
fast             1.57T   393G       96K  /fast
fast/data         827G   393G      827G  /data
fast/data/mysql    96K   393G       96K  /data/mysql
fast/home         119G   393G      224K  /home
fast/home/jj5     119G   393G      105G  /home/jj5
fast/mysql       23.7G   393G     23.7G  /var/lib/mysql
fast/vbox         363G   393G      363G  /vbox
-------------------
Wed Mar 30 05:30:26 [bash:5.0.17 jobs:0 error:0 time:456]
root@charm:/var/backups
# zfs get dedup
NAME                             PROPERTY  VALUE          SOURCE
fast                             dedup     on             local
fast/data                        dedup     on             inherited from fast
fast/data/mysql                  dedup     off            local
fast/home                        dedup     on             inherited from fast
fast/home/jj5                    dedup     on             inherited from fast
fast/mysql                       dedup     on             inherited from fast
fast/vbox                        dedup     on             inherited from fast
-------------------
Wed Mar 30 05:30:30 [bash:5.0.17 jobs:0 error:0 time:460]
root@charm:/var/backups
# zfs get compression
NAME                             PROPERTY     VALUE     SOURCE
fast                             compression  lz4       local
fast/data                        compression  gzip-9    local
fast/data/mysql                  compression  off       local
fast/home                        compression  lz4       inherited from fast
fast/home/jj5                    compression  lz4       inherited from fast
fast/mysql                       compression  lz4       inherited from fast
fast/vbox                        compression  lz4       inherited from fast
-------------------

2023-01-06 jj5 - adding fast/session and fast/scratch

# zpool upgrade fast
# zfs create fast/session
# zfs set compression=zstd fast/session
# zfs set dedup=off fast/session
# zfs set mountpoint=/var/log/session fast/session
# chmod 1777 /var/log/session
#
# zfs create fast/scratch
# zfs set dedup=off fast/scratch
# zfs set compression=off fast/scratch