HW Setup
GMKtec NucBox G9
CPU: Intel N150,
RAM: 12GB LPDDR5,
DISK: 4x4TB Samsung 990 Pro with heatsink (Gen4x4 inserted in Gen3x2 slots),
OS: TrueNAS Scale 25.10 RC1,
Power Limits: 15W/15W (BIOS: Performance mode)
Disk pool record size: 128K
TrueNAS 4x4TB NVMe RAID disk speed comparison
By RAID Mode RAIDZ1 (RAID5), Stripe (RAID0), Mirror (RAID10) and compression (OFF, LZ4, GZIP-1)
4xSamsung 990 Pro GMKtec NucBox G9 TrueNAS | Sequental | Random | |||||||
---|---|---|---|---|---|---|---|---|---|
Read (MiB/s) | Write (MiB/s) | Read (MiB/s) | Write (MiB/s) | ||||||
RAID | compression | bs = 1M | bs = 4K | bs = 1M | bs = 4K | bs = 1M | bs = 4K | bs = 1M | bs = 4K |
RAIDZ1 | GZIP-1 | 3050 | 511 | 98.7 | 478 | 3146 | 76.3 | 96.3 | 21.9 |
RAIDZ1 | LZ4 | 3996 | 894 | 1334 | 568 | 3173 | 99.2 | 1167 | 82.5 |
RAIDZ1 | OFF | 4117 | 998 | 2513 | 589 | 2059 | 167 | 1577 | 62.4 |
Stripe | GZIP-1 | 4055 | 517 | 89.0 | 333 | 4039 | 26.6 | 89.3 | 17.9 |
Stripe | LZ4 | 4294 | 1046 | 1527 | 617 | 4180 | 97.7 | 1551 | 65.4 |
Stripe | OFF | 4357 | 986 | 3405 | 508 | 4258 | 168 | 3007 | 80.3 |
Mirror | GZIP-1 | 4496 | 529 | 93.2 | 348 | 4390 | 28.0 | 91.1 | 19.8 |
Mirror | LZ4 | 4201 | 1047 | 1517 | 617 | 4096 | 66.7 | 1446 | 49.4 |
Mirror | OFF | 4491 | 1076 | 2015 | 595 | 4443 | 173 | 2012 | 60.2 |
Mirror DS220+ | OFF 2xHDD | 179 | 62.7 | 168 | 20.9 | 182 | 61.5 | 169 | 9.6 |
In RAID10 mode by dataset write size (4K, 16K, 128K) and compression (OFF, LZ4, GZIP-1)
With additional test for Crypto dataset OFF/128K/AES-256-GCM compared to AES-256-CBC of Synology DS220+
4xSamsung 990 Pro GMKtec NucBox G9 TrueNAS and Synology | Sequental | Random | |||||||
---|---|---|---|---|---|---|---|---|---|
Read (MiB/s) | Write (MiB/s) | Read (MiB/s) | Write (MiB/s) | ||||||
RAID | compression | bs = 1M | bs = 4K | bs = 1M | bs = 4K | bs = 1M | bs = 4K | bs = 1M | bs = 4K |
Mirror | GZIP-1/4K | 1261 | 635 | 102 | 107 | 1269 | 489 | 102 | 96.9 |
Mirror | GZIP-1/16K | 2876 | 360 | 147 | 224 | 2866 | 115 | 148 | 70.4 |
Mirror | GZIP-1/128K | 2870 | 529 | 93.2 | 348 | 4390 | 28.0 | 91.1 | 19.8 |
Mirror | GZIP-1/1M | 5082 | 560 | 93.0 | 375 | 4929 | 29.7 | 92.5 | 0.4 |
Mirror | LZ4/4K | 1197 | 641 | 628 | 340 | 1272 | 484 | 567 | 241 |
Mirror | LZ4/16K | 2790 | 882 | 1168 | 489 | 2821 | 377 | 1147 | 186 |
Mirror | LZ4/128K | 4201 | 1047 | 1517 | 617 | 4096 | 66.7 | 1446 | 49.4 |
Mirror | LZ4/1M | 4506 | 1064 | 1428 | 652 | 4768 | 14.8 | 1585 | 6.7 |
Mirror | OFF/4K | 1219 | 665 | 646 | 335 | 1274 | 598 | 526 | 246 |
Mirror | OFF/16K | 2872 | 890 | 1550 | 502 | 2796 | 444 | 1454 | 223 |
Mirror | OFF/128K | 4491 | 1076 | 2015 | 595 | 4443 | 173 | 2012 | 60.2 |
Mirror | OFF/1M | 5057 | 1100 | 2054 | 636 | 5051 | 29.9 | 2052 | 7.5 |
Mirror | OFF/4M | 4227 | 1028 | 1925 | 477 | 1622 | 7.4 | 541 | 1.9 |
Mirror | AES-256-GCM | 4462 | 1046 | 1931 | 461 | 1896 | 170 | 1876 | 34.5 |
Mirror DS220+ | OFF 2xHDD | 179 | 62.7 | 168 | 20.9 | 182 | 61.5 | 169 | 9.6 |
Mirror DS220+ | AES-256-CBC (not direct) | 139 | 178 | 129 | 138 | 32.4 | 3.5 | 142 | 5.6 |
RAID0 on Ubuntu 24.04 using LVM, BTRFS and MDADM
4xSamsung 990 Pro GMKtec NucBox G9 Ubuntu 24.04 RAID0 | Sequental | Random | |||||||
---|---|---|---|---|---|---|---|---|---|
Read (MiB/s) | Write (MiB/s) | Read (MiB/s) | Write (MiB/s) | ||||||
RAID | compression | bs = 1M | bs = 4K | bs = 1M | bs = 4K | bs = 1M | bs = 4K | bs = 1M | bs = 4K |
ZFS Mirror | OFF/4K | 1219 | 665 | 646 | 335 | 1274 | 598 | 526 | 246 |
ZFS Stripe | OFF/128K | 4357 | 986 | 3405 | 508 | 4258 | 168 | 3007 | 80.3 |
LVM Stripe (ext4) | OFF/4K | 2596 | 213 | 2434 | 165 | 2302 | 65.3 | 2437 | 162 |
LVM Stripe (ext4) | OFF/16K | 2574 | 220 | 2357 | 167 | 2163 | 64.9 | 2362 | 157 |
LVM Stripe (ext4) | OFF/128K | 2619 | 51 | 2369 | 148 | 2322 | 65.6 | 2372 | 143 |
btrfs raid 0 | OFF/4K | 1947 | 126 | 2017 | 134 | 1943 | 42.5 | 1988 | 118 |
btrfs raid 0 | ON/4K | 1960 | 44.7 | 2036 | 129 | 1930 | 40.0 | 1993 | 117 |
mdraid (ext4) | OFF/4K | 2292 | 207 | 2132 | 157 | 2051 | 65.4 | 2037 | 148 |
mdraid (ext4) | OFF/16K | 2271 | 211 | 2080 | 153 | 2048 | 64.5 | 2067 | 155 |
mkfs.ext4 -C 16384 -O bigalloc -L EXT4-16K /dev/storage/fs mkfs.btrfs --data raid0 --sectorsize 4k /dev/nvme0n1p1 /dev/nvme1n1p1 /dev/nvme2n1p1 /dev/nvme3n1p1
Table for comparison with non-NAS devices
Sequental | Random | ||||||||
---|---|---|---|---|---|---|---|---|---|
Read (MiB/s) | Write (MiB/s) | Read (MiB/s) | Write (MiB/s) | ||||||
device | disk type | bs = 1M | bs = 4K | bs = 1M | bs = 4K | bs = 1M | bs = 4K | bs = 1M | bs = 4K |
R100N | 1xNVMe | 1302 | 123 | 1047 | 104 | 1211 | 49.1 | 1056 | 103 |
R100S | 1xSSD | 405 | 57.4 | 347 | 49.9 | 321 | 20.0 | 341 | 48.5 |
N200N | 1xNVMe | 1293 | 89.9 | 1212 | 202 | 1255 | 74.0 | 1152 | 193 |
I1071N | 1xNVMe | 2720 | 278 | 2691 | 280 | 2462 | 75.0 | 2414 | 216 |
I1355N | 1xNVMe | 1756 | 289 | 16.0 | 2.5 | 1348 | 57.4 | 15.9 | 2.6 |
DS220+ | 2xHDD | 179 | 62.7 | 168 | 20.9 | 182 | 61.5 | 169 | 9.6 |
For disk performance comparison next system were use,
code | Hardware | Disk | space used,% | Power On, hours | Life Time Remain, % |
---|---|---|---|---|---|
N150R | Intel N150, 12GB LPDDR5, Debian, TrueNAS 25.04 RC1, ZFS | Samsung SSD 990 PRO with Heatsink 4TB (4 in RAID) | 1 | 56 | 100 |
R100N | Intel N100, 16GB DDR4, Ubuntu 24.04, CasaOS, ext4 | Movespeed YSSDHB-4TN7000 4TB NVMe | 8 | 4822 | 100 |
R100S | Intel N100, 16GB DDR4, Ubuntu 24.04, CasaOS, ext4 | Transcend SSD230S 4 TB SATA SSD | 49 | 1522 | 100 |
N200N | Intel N200, 32GB DDR5, Ubuntu 24.04, ext4 | Fanxiang S500PRO 2TB | 97 | 8343 | 100 |
I1071N | Intel Core i7-10710U, 64GB DDR4, Ubuntu 24.0 | Lexar SSD ARES 4TB | 74 | 12238 | 99 |
I1355N | Intel Core i7-1355U (15W), 96GB DDR5, Ubuntu 24.04 | XPG GAMMIX S70 BLADE 8TB | 85 | 6823 | 100 |
DS220+ | Synology DS220+, J4025, 2GB, RAID1 | WD Gold Enterprise Class 18 TB 7200 RPM(2 in RAID1) | 77 | 37877 | N/A |
1 Year is 8766 hours (37877 hours = 4.3 Years)
Commands used to test disk speed
cd /mnt/pool/shared/user # each command executed 3 times # 1M fio --ioengine=libaio --direct=1 --name=test --filename=test0 --bs=1M --size=4G --readwrite=write --ramp_time=4 # Col 3 fio --ioengine=libaio --direct=1 --name=test --filename=test0 --bs=1M --size=4G --readwrite=read --ramp_time=4 # Col 1 fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test1 --filename=test2 --bs=1M --size=4G --readwrite=randwrite --ramp_time=4 # Col 7 fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test1 --filename=test2 --bs=1M --size=4G --readwrite=randread --ramp_time=4 # Col 5 # 4K fio --ioengine=libaio --direct=1 --name=test --filename=test0 --bs=4K --size=4G --readwrite=write --ramp_time=4 # Col 4 fio --ioengine=libaio --direct=1 --name=test --filename=test0 --bs=4K --size=4G --readwrite=read --ramp_time=4 # Col 2 fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test1 --filename=test2 --bs=4k --size=4G --readwrite=randwrite --ramp_time=4 # Col 8 fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test1 --filename=test2 --bs=4k --size=4G --readwrite=randread --ramp_time=4 # Col 6