HW Setup

GMKtec NucBox G9

CPU: Intel N150,

RAM: 12GB LPDDR5,

DISK: 4x4TB Samsung 990 Pro with heatsink (Gen4x4 inserted in Gen3x2 slots),

OS: TrueNAS Scale 25.10 RC1,

Power Limits: 15W/15W (BIOS: Performance mode)

Disk pool record size: 128K

TrueNAS 4x4TB NVMe RAID disk speed comparison

By RAID Mode RAIDZ1 (RAID5), Stripe (RAID0), Mirror (RAID10) and compression (OFF, LZ4, GZIP-1)

4xSamsung 990 Pro GMKtec NucBox G9 TrueNAS		Sequental				Random
4xSamsung 990 Pro GMKtec NucBox G9 TrueNAS		Read (MiB/s)		Write (MiB/s)		Read (MiB/s)		Write (MiB/s)
RAID	compression	bs = 1M	bs = 4K	bs = 1M	bs = 4K	bs = 1M	bs = 4K	bs = 1M	bs = 4K
RAIDZ1	GZIP-1	3050	511	98.7	478	3146	76.3	96.3	21.9
RAIDZ1	LZ4	3996	894	1334	568	3173	99.2	1167	82.5
RAIDZ1	OFF	4117	998	2513	589	2059	167	1577	62.4
Stripe	GZIP-1	4055	517	89.0	333	4039	26.6	89.3	17.9
Stripe	LZ4	4294	1046	1527	617	4180	97.7	1551	65.4
Stripe	OFF	4357	986	3405	508	4258	168	3007	80.3
Mirror	GZIP-1	4496	529	93.2	348	4390	28.0	91.1	19.8
Mirror	LZ4	4201	1047	1517	617	4096	66.7	1446	49.4
Mirror	OFF	4491	1076	2015	595	4443	173	2012	60.2
Mirror DS220+	OFF 2xHDD	179	62.7	168	20.9	182	61.5	169	9.6

In RAID10 mode by dataset write size (4K, 16K, 128K) and compression (OFF, LZ4, GZIP-1)

With additional test for Crypto dataset OFF/128K/AES-256-GCM compared to AES-256-CBC of Synology DS220+

4xSamsung 990 Pro GMKtec NucBox G9 TrueNAS and Synology		Sequental				Random
4xSamsung 990 Pro GMKtec NucBox G9 TrueNAS and Synology		Read (MiB/s)		Write (MiB/s)		Read (MiB/s)		Write (MiB/s)
RAID	compression	bs = 1M	bs = 4K	bs = 1M	bs = 4K	bs = 1M	bs = 4K	bs = 1M	bs = 4K
Mirror	GZIP-1/4K	1261	635	102	107	1269	489	102	96.9
Mirror	GZIP-1/16K	2876	360	147	224	2866	115	148	70.4
Mirror	GZIP-1/128K	2870	529	93.2	348	4390	28.0	91.1	19.8
Mirror	GZIP-1/1M	5082	560	93.0	375	4929	29.7	92.5	0.4
Mirror	LZ4/4K	1197	641	628	340	1272	484	567	241
Mirror	LZ4/16K	2790	882	1168	489	2821	377	1147	186
Mirror	LZ4/128K	4201	1047	1517	617	4096	66.7	1446	49.4
Mirror	LZ4/1M	4506	1064	1428	652	4768	14.8	1585	6.7
Mirror	OFF/4K	1219	665	646	335	1274	598	526	246
Mirror	OFF/16K	2872	890	1550	502	2796	444	1454	223
Mirror	OFF/128K	4491	1076	2015	595	4443	173	2012	60.2
Mirror	OFF/1M	5057	1100	2054	636	5051	29.9	2052	7.5
Mirror	OFF/4M	4227	1028	1925	477	1622	7.4	541	1.9
Mirror	AES-256-GCM	4462	1046	1931	461	1896	170	1876	34.5
Mirror DS220+	OFF 2xHDD	179	62.7	168	20.9	182	61.5	169	9.6
Mirror DS220+	AES-256-CBC (not direct)	139	178	129	138	32.4	3.5	142	5.6

RAID0 on Ubuntu 24.04 using LVM on ext4 fs

4xSamsung 990 Pro GMKtec NucBox G9 Ubuntu 24.04 LVM RAID0		Sequental				Random
4xSamsung 990 Pro GMKtec NucBox G9 Ubuntu 24.04 LVM RAID0		Read (MiB/s)		Write (MiB/s)		Read (MiB/s)		Write (MiB/s)
RAID	compression	bs = 1M	bs = 4K	bs = 1M	bs = 4K	bs = 1M	bs = 4K	bs = 1M	bs = 4K
ZFS Stripe	OFF/128K	4357	986	3405	508	4258	168	3007	80.3
LVM Stripe (ext4)	OFF/4K	2596	213	2434	165	2302	65.3	2437	162
LVM Stripe (ext4)	OFF/16K	2574	220	2357	167	2163	64.9	2362	157
LVM Stripe (ext4)	OFF/128K	2619	51	2369	148	2322	65.6	2372	143
btrfs raid 0	4K	1947	126	2017	134	1943	42.5	1988	118

VG PE Size 4M, PV PE Size 4M ext fs default cluster size 4k

mkfs.ext4 -C 16384 -O  bigalloc -L EXT4-16K /dev/storage/fs
mkfs.btrfs --data raid0 --sectorsize 4k /dev/nvme0n1p1 /dev/nvme1n1p1 /dev/nvme2n1p1 /dev/nvme3n1p1

Table for comparison with non-NAS devices

		Sequental				Random
		Read (MiB/s)		Write (MiB/s)		Read (MiB/s)		Write (MiB/s)
device	disk type	bs = 1M	bs = 4K	bs = 1M	bs = 4K	bs = 1M	bs = 4K	bs = 1M	bs = 4K
R100N	1xNVMe	1302	123	1047	104	1211	49.1	1056	103
R100S	1xSSD	405	57.4	347	49.9	321	20.0	341	48.5
N200N	1xNVMe	1293	89.9	1212	202	1255	74.0	1152	193
I1071N	1xNVMe	2720	278	2691	280	2462	75.0	2414	216
I1355N	1xNVMe	1756	289	16.0	2.5	1348	57.4	15.9	2.6
DS220+	2xHDD	179	62.7	168	20.9	182	61.5	169	9.6

For disk performance comparison next system were use,

code	Hardware	Disk	space used,%	Power On, hours	Life Time Remain, %
N150R	Intel N150, 12GB LPDDR5, Debian, TrueNAS 25.04 RC1, ZFS	Samsung SSD 990 PRO with Heatsink 4TB (4 in RAID)	1	56	100
R100N	Intel N100, 16GB DDR4, Ubuntu 24.04, CasaOS, ext4	Movespeed YSSDHB-4TN7000 4TB NVMe	8	4822	100
R100S	Intel N100, 16GB DDR4, Ubuntu 24.04, CasaOS, ext4	Transcend SSD230S 4 TB SATA SSD	49	1522	100
N200N	Intel N200, 32GB DDR5, Ubuntu 24.04, ext4	Fanxiang S500PRO 2TB	97	8343	100
I1071N	Intel Core i7-10710U, 64GB DDR4, Ubuntu 24.0	Lexar SSD ARES 4TB	74	12238	99
I1355N	Intel Core i7-1355U (15W), 96GB DDR5, Ubuntu 24.04	XPG GAMMIX S70 BLADE 8TB	85	6823	100
DS220+	Synology DS220+, J4025, 2GB, RAID1	WD Gold Enterprise Class 18 TB 7200 RPM(2 in RAID1)	77	37877	N/A

1 Year is 8766 hours (37877 hours = 4.3 Years)

Commands used to test disk speed

cd /mnt/pool/shared/user
# each command executed 3 times
# 1M
fio --ioengine=libaio --direct=1 --name=test --filename=test0 --bs=1M --size=4G --readwrite=write --ramp_time=4 # Col 3
fio --ioengine=libaio --direct=1 --name=test --filename=test0 --bs=1M --size=4G --readwrite=read --ramp_time=4 # Col 1
fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test1 --filename=test2 --bs=1M --size=4G --readwrite=randwrite --ramp_time=4  # Col  7
fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test1 --filename=test2 --bs=1M --size=4G --readwrite=randread --ramp_time=4 # Col 5
# 4K
fio --ioengine=libaio --direct=1 --name=test --filename=test0 --bs=4K --size=4G --readwrite=write --ramp_time=4 # Col 4
fio --ioengine=libaio --direct=1 --name=test --filename=test0 --bs=4K --size=4G --readwrite=read --ramp_time=4 # Col 2
fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test1 --filename=test2 --bs=4k --size=4G --readwrite=randwrite --ramp_time=4 # Col 8
fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test1 --filename=test2 --bs=4k --size=4G --readwrite=randread --ramp_time=4 # Col 6

Check Power Limits

root@truenas:/mnt/pool/shared/user# turbostat 2>&1|grep Watt
RAPL: 43691 sec. Joule Counter Range, at 6 Watts
cpu0: MSR_RAPL_POWER_UNIT: 0x000a0e03 (0.125000 Watts, 0.000061 Joules, 0.000977 sec.)
cpu0: PKG Limit #1: ENabled (15.000 Watts, 8.000000 sec, clamp ENabled)
cpu0: PKG Limit #2: ENabled (15.000 Watts, 0.002441* sec, clamp DISabled)
cpu0: PKG Limit #4: 78.000000 Watts (UNlocked)
cpu0: DRAM Limit: DISabled (0.000 Watts, 0.000977 sec, clamp DISabled)
cpu0: Cores Limit: DISabled (0.000 Watts, 0.000977 sec, clamp DISabled)
cpu0: GFX Limit: DISabled (0.000 Watts, 0.000977 sec, clamp DISabled)

RAIDZ1 (LZ4) commands output

root@truenas:/mnt/pool/shared/user# zfs get all pool/shared |grep comp
pool/shared  compressratio            1.41x                    -
pool/shared  compression              lz4                      inherited from pool
pool/shared  refcompressratio         1.41x                    -

root@truenas:/mnt/pool/shared/user# sync;fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test --filename=test --bs=4k --size=4G --readwrite=randwrite --ramp_time=4
test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.33
Starting 1 process
test: Laying out IO file (1 file / 4096MiB)
Jobs: 1 (f=0): [f(1)][100.0%][w=147MiB/s][w=37.6k IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=41307: Thu Apr 10 12:49:43 2025
  write: IOPS=21.1k, BW=82.5MiB/s (86.5MB/s)(3672MiB/44503msec); 0 zone resets
    slat (usec): min=4, max=80062, avg=45.37, stdev=167.42
    clat (nsec): min=614, max=4072.6k, avg=1133.86, stdev=6985.89
     lat (usec): min=5, max=80072, avg=46.50, stdev=167.70
    clat percentiles (nsec):
     |  1.00th=[  740],  5.00th=[  836], 10.00th=[  884], 20.00th=[  924],
     | 30.00th=[  956], 40.00th=[  972], 50.00th=[  996], 60.00th=[ 1020],
     | 70.00th=[ 1048], 80.00th=[ 1080], 90.00th=[ 1192], 95.00th=[ 1384],
     | 99.00th=[ 2008], 99.50th=[ 3312], 99.90th=[24704], 99.95th=[41728],
     | 99.99th=[72192]
   bw (  KiB/s): min=48766, max=137119, per=98.81%, avg=83487.14, stdev=11581.15, samples=87
   iops        : min=12191, max=34279, avg=20872.08, stdev=2895.09, samples=87
  lat (nsec)   : 750=1.20%, 1000=50.42%
  lat (usec)   : 2=47.36%, 4=0.59%, 10=0.14%, 20=0.16%, 50=0.10%
  lat (usec)   : 100=0.02%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%
  cpu          : usr=3.69%, sys=85.07%, ctx=107206, majf=0, minf=37
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,940003,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=82.5MiB/s (86.5MB/s), 82.5MiB/s-82.5MiB/s (86.5MB/s-86.5MB/s), io=3672MiB (3850MB), run=44503-44503msec


root@truenas:/mnt/pool/shared/user# sync;fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test --filename=test --bs=4k --size=4G --readwrite=randread --ramp_time=4
test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.33
Starting 1 process
Jobs: 1 (f=1): [r(1)][91.3%][r=160MiB/s][r=41.0k IOPS][eta 00m:04s]
test: (groupid=0, jobs=1): err= 0: pid=41962: Thu Apr 10 12:50:58 2025
  read: IOPS=25.4k, BW=99.2MiB/s (104MB/s)(3708MiB/37391msec)
    slat (usec): min=2, max=467, avg=38.12, stdev=11.50
    clat (nsec): min=587, max=54938, avg=738.69, stdev=251.78
     lat (usec): min=3, max=470, avg=38.86, stdev=11.53
    clat percentiles (nsec):
     |  1.00th=[  636],  5.00th=[  668], 10.00th=[  684], 20.00th=[  692],
     | 30.00th=[  700], 40.00th=[  708], 50.00th=[  716], 60.00th=[  724],
     | 70.00th=[  740], 80.00th=[  756], 90.00th=[  804], 95.00th=[  876],
     | 99.00th=[ 1012], 99.50th=[ 1096], 99.90th=[ 2416], 99.95th=[ 3056],
     | 99.99th=[14144]
   bw (  KiB/s): min=91048, max=157648, per=98.65%, avg=100191.19, stdev=8691.21, samples=74
   iops        : min=22762, max=39412, avg=25047.76, stdev=2172.79, samples=74
  lat (nsec)   : 750=76.93%, 1000=21.93%
  lat (usec)   : 2=1.03%, 4=0.08%, 10=0.01%, 20=0.01%, 50=0.01%
  lat (usec)   : 100=0.01%
  cpu          : usr=2.95%, sys=96.97%, ctx=420, majf=0, minf=37
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=949364,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=99.2MiB/s (104MB/s), 99.2MiB/s-99.2MiB/s (104MB/s-104MB/s), io=3708MiB (3889MB), run=37391-37391msec

root@truenas:/mnt/pool/shared/user# sync;fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test --filename=test --bs=4k --size=4G --readwrite=write --ramp_time=4
test: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.33
Starting 1 process
Jobs: 1 (f=1): [W(1)][53.8%][w=555MiB/s][w=142k IOPS][eta 00m:06s]
test: (groupid=0, jobs=1): err= 0: pid=43581: Thu Apr 10 12:53:42 2025
  write: IOPS=145k, BW=568MiB/s (595MB/s)(1808MiB/3184msec); 0 zone resets
    slat (usec): min=3, max=7144, avg= 5.85, stdev=13.87
    clat (nsec): min=581, max=149242, avg=722.02, stdev=731.07
     lat (usec): min=4, max=7154, avg= 6.57, stdev=13.94
    clat percentiles (nsec):
     |  1.00th=[  636],  5.00th=[  644], 10.00th=[  652], 20.00th=[  660],
     | 30.00th=[  668], 40.00th=[  676], 50.00th=[  684], 60.00th=[  692],
     | 70.00th=[  700], 80.00th=[  732], 90.00th=[  844], 95.00th=[  892],
     | 99.00th=[  972], 99.50th=[ 1144], 99.90th=[ 1576], 99.95th=[ 4384],
     | 99.99th=[47360]
   bw (  KiB/s): min=565370, max=595054, per=99.72%, avg=579799.00, stdev=12657.07, samples=6
   iops        : min=141342, max=148763, avg=144949.50, stdev=3164.21, samples=6
  lat (nsec)   : 750=82.16%, 1000=17.03%
  lat (usec)   : 2=0.73%, 4=0.02%, 10=0.02%, 20=0.01%, 50=0.01%
  lat (usec)   : 100=0.01%, 250=0.01%
  cpu          : usr=13.51%, sys=84.32%, ctx=1523, majf=0, minf=37
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,462792,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=568MiB/s (595MB/s), 568MiB/s-568MiB/s (595MB/s-595MB/s), io=1808MiB (1896MB), run=3184-3184msec

RAIDZ1 (OFF) commands output

root@truenas:/mnt/pool/shared/user# zfs get all pool/shared |grep comp
pool/shared  compressratio            1.41x                    -
pool/shared  compression              off                      local
pool/shared  refcompressratio         1.41x                    -

root@truenas:/mnt/pool/shared/user# cd /mnt/pool/shared/user
sync;fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test --filename=test --bs=4k --size=4G --readwrite=write --ramp_time=4
sync;fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test --filename=test --bs=4k --size=4G --readwrite=read --ramp_time=4
sync;fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test --filename=test --bs=4k --size=4G --readwrite=randwrite --ramp_time=4
sync;fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test --filename=test --bs=4k --size=4G --readwrite=randread --ramp_time=4
test: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.33
Starting 1 process
Jobs: 1 (f=1): [W(1)][53.8%][w=606MiB/s][w=155k IOPS][eta 00m:06s]
test: (groupid=0, jobs=1): err= 0: pid=48964: Thu Apr 10 13:03:16 2025
  write: IOPS=151k, BW=589MiB/s (617MB/s)(1754MiB/2980msec); 0 zone resets
    slat (usec): min=3, max=8392, avg= 5.53, stdev=23.09
    clat (nsec): min=593, max=3842.7k, avg=762.98, stdev=6482.39
     lat (usec): min=4, max=8402, avg= 6.30, stdev=24.11
    clat percentiles (nsec):
     |  1.00th=[  644],  5.00th=[  660], 10.00th=[  660], 20.00th=[  668],
     | 30.00th=[  676], 40.00th=[  684], 50.00th=[  692], 60.00th=[  700],
     | 70.00th=[  724], 80.00th=[  812], 90.00th=[  892], 95.00th=[  940],
     | 99.00th=[ 1048], 99.50th=[ 1176], 99.90th=[ 1928], 99.95th=[ 4960],
     | 99.99th=[53504]
   bw (  KiB/s): min=569664, max=630376, per=99.81%, avg=601526.00, stdev=21788.41, samples=5
   iops        : min=142416, max=157594, avg=150381.40, stdev=5447.09, samples=5
  lat (nsec)   : 750=75.24%, 1000=22.93%
  lat (usec)   : 2=1.74%, 4=0.04%, 10=0.03%, 20=0.01%, 50=0.01%
  lat (usec)   : 100=0.01%, 250=0.01%, 500=0.01%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%
  cpu          : usr=14.54%, sys=81.71%, ctx=1094, majf=0, minf=37
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,449002,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=589MiB/s (617MB/s), 589MiB/s-589MiB/s (617MB/s-617MB/s), io=1754MiB (1839MB), run=2980-2980msec
test: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.33
Starting 1 process
Jobs: 1 (f=0): [/(1)][-.-%][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=49098: Thu Apr 10 13:03:24 2025
  read: IOPS=264k, BW=1033MiB/s (1083MB/s)(4096MiB/3966msec)
  cpu          : usr=18.54%, sys=81.36%, ctx=11, majf=0, minf=10
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=1048576,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=1033MiB/s (1083MB/s), 1033MiB/s-1033MiB/s (1083MB/s-1083MB/s), io=4096MiB (4295MB), run=3966-3966msec
test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.33
Starting 1 process
Jobs: 1 (f=1): [w(1)][91.8%][w=101MiB/s][w=25.9k IOPS][eta 00m:05s]
test: (groupid=0, jobs=1): err= 0: pid=49133: Thu Apr 10 13:04:21 2025
  write: IOPS=18.8k, BW=73.4MiB/s (77.0MB/s)(3839MiB/52277msec); 0 zone resets
    slat (usec): min=4, max=46730, avg=50.43, stdev=165.10
    clat (nsec): min=615, max=21921k, avg=1583.11, stdev=39330.27
     lat (usec): min=5, max=46739, avg=52.01, stdev=172.39
    clat percentiles (nsec):
     |  1.00th=[   764],  5.00th=[   852], 10.00th=[   892], 20.00th=[   940],
     | 30.00th=[   988], 40.00th=[  1020], 50.00th=[  1064], 60.00th=[  1096],
     | 70.00th=[  1144], 80.00th=[  1208], 90.00th=[  1496], 95.00th=[  1944],
     | 99.00th=[  3856], 99.50th=[ 10560], 99.90th=[ 85504], 99.95th=[100864],
     | 99.99th=[329728]
   bw (  KiB/s): min=53085, max=136798, per=98.79%, avg=74293.55, stdev=11284.08, samples=103
   iops        : min=13271, max=34199, avg=18573.24, stdev=2820.98, samples=103
  lat (nsec)   : 750=0.75%, 1000=32.80%
  lat (usec)   : 2=61.85%, 4=3.62%, 10=0.46%, 20=0.11%, 50=0.17%
  lat (usec)   : 100=0.18%, 250=0.04%, 500=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=3.45%, sys=77.24%, ctx=153481, majf=0, minf=38
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,982878,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=73.4MiB/s (77.0MB/s), 73.4MiB/s-73.4MiB/s (77.0MB/s-77.0MB/s), io=3839MiB (4026MB), run=52277-52277msec
test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
fio-3.33
Starting 1 process
Jobs: 1 (f=1): [r(1)][86.2%][r=198MiB/s][r=50.7k IOPS][eta 00m:04s]
test: (groupid=0, jobs=1): err= 0: pid=49874: Thu Apr 10 13:04:51 2025
  read: IOPS=42.7k, BW=167MiB/s (175MB/s)(3451MiB/20689msec)
    slat (usec): min=2, max=489, avg=22.17, stdev= 6.69
    clat (nsec): min=546, max=57914, avg=731.80, stdev=296.20
     lat (usec): min=2, max=493, avg=22.91, stdev= 6.74
    clat percentiles (nsec):
     |  1.00th=[  596],  5.00th=[  620], 10.00th=[  668], 20.00th=[  684],
     | 30.00th=[  700], 40.00th=[  700], 50.00th=[  716], 60.00th=[  724],
     | 70.00th=[  740], 80.00th=[  756], 90.00th=[  804], 95.00th=[  868],
     | 99.00th=[ 1004], 99.50th=[ 1096], 99.90th=[ 1608], 99.95th=[ 2960],
     | 99.99th=[17792]
   bw (  KiB/s): min=157370, max=253968, per=98.82%, avg=168814.17, stdev=15047.80, samples=41
   iops        : min=39342, max=63492, avg=42203.44, stdev=3761.97, samples=41
  lat (nsec)   : 750=77.16%, 1000=21.82%
  lat (usec)   : 2=0.93%, 4=0.06%, 10=0.01%, 20=0.01%, 50=0.01%
  lat (usec)   : 100=0.01%
  cpu          : usr=5.06%, sys=94.78%, ctx=351, majf=0, minf=37
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=883533,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=167MiB/s (175MB/s), 167MiB/s-167MiB/s (175MB/s-175MB/s), io=3451MiB (3619MB), run=20689-20689msec

Page tree

TrueNAS on Intel N150 RAIDs fio test