# aller 20 Sekunden für sdb und sdc mit Ausgabe in MB/s iostat -d 20 -p sdb sdc -m
Ich habe zwei Raspi 4 für den ESXi Fling mit VSAN laufen. Der ganze USB/SD Card Kram hat den Nachteil, dass die Karten/Sticks langsam sind und nicht lange halten. Deshalb habe ich mir zwei unterschiedliche Adapter für den Raspi4 zugelegt. Einmal einen Geekworm X825 V2.0 für 2,5“ SATA Disks und einen Geekworm X862 V2.0 Adapter für M.2 NGFF Disks - diesen Adapter will ich mal etwas austesten und hier darstellen.
Vornweg - das native Booten von SSD ist einfach unglaublich schnell. Folgende Linux Version wird für den Test genutzt:
root@pi4test:~# lsb_release -a No LSB modules are available. Distributor ID: Raspbian Description: Raspbian GNU/Linux 11 (bullseye) Release: 11 Codename: bullseye
Ergebnisse Übersicht
| IO Art | Durchsatz | IOs | typ. Latenz |
|---|---|---|---|
| Seq. Write | 104 MB/s | 25300 IOs/s | 20µs |
| Seq. Read | 154 MB/s | 37600 IOs/s | 10µs |
| Seq. ReadWrite | 53,1 RMB/s | 12000 RIOs/s | 10-20µs |
| 53,1 WMB/s | 12000 WIOs/s | ||
| Rand. Write | 11,3 MB/s | 2769 IOs/s | 20 - 50µs |
| Rand. Read | 11,7 MB/s | 2848 IOs/s | 500µs |
| Rand. ReadWrite | 5552 RkB/s | 1355 RIOs/s | 20 - 500µs |
| 5546 WkB/s | 1353 WIOs/s |
Sequential Write:
root@pi4test:~# fio --rw=write --name=testfile --size 16G testfile: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 fio-3.25 Starting 1 process Jobs: 1 (f=1): [W(1)][100.0%][w=91.7MiB/s][w=23.5k IOPS][eta 00m:00s] testfile: (groupid=0, jobs=1): err= 0: pid=1124: Sat Apr 2 11:30:55 2022 write: IOPS=25.3k, BW=98.9MiB/s (104MB/s)(16.0GiB/165685msec); 0 zone resets clat (usec): min=8, max=717579, avg=35.32, stdev=1272.33 lat (usec): min=8, max=717581, avg=36.07, stdev=1272.34 clat percentiles (usec): | 1.00th=[ 10], 5.00th=[ 11], 10.00th=[ 11], 20.00th=[ 11], | 30.00th=[ 11], 40.00th=[ 11], 50.00th=[ 12], 60.00th=[ 12], | 70.00th=[ 12], 80.00th=[ 15], 90.00th=[ 18], 95.00th=[ 20], | 99.00th=[ 29], 99.50th=[ 38], 99.90th=[11994], 99.95th=[13960], | 99.99th=[18482] bw ( KiB/s): min= 4928, max=242056, per=100.00%, avg=101397.10, stdev=43117.26, samples=330 iops : min= 1232, max=60514, avg=25349.28, stdev=10779.32, samples=330 lat (usec) : 10=4.02%, 20=91.45%, 50=4.21%, 100=0.11%, 250=0.05% lat (usec) : 500=0.01%, 750=0.01%, 1000=0.01% lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.13%, 50=0.01% lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01% cpu : usr=7.81%, sys=34.74%, ctx=11204, majf=0, minf=59 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,4194304,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): WRITE: bw=98.9MiB/s (104MB/s), 98.9MiB/s-98.9MiB/s (104MB/s-104MB/s), io=16.0GiB (17.2GB), run=165685-165685msec Disk stats (read/write): sda: ios=1/32713, merge=0/37, ticks=0/8077384, in_queue=8077703, util=98.84%
Sequential Read:
root@pi4test:~# fio --rw=read --name=testfile --size 16G testfile: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 fio-3.25 Starting 1 process Jobs: 1 (f=1): [R(1)][100.0%][r=128MiB/s][r=32.8k IOPS][eta 00m:00s] testfile: (groupid=0, jobs=1): err= 0: pid=1365: Sat Apr 2 12:06:42 2022 read: IOPS=37.6k, BW=147MiB/s (154MB/s)(16.0GiB/111547msec) clat (usec): min=6, max=492235, avg=21.61, stdev=2208.68 lat (usec): min=6, max=492235, avg=22.21, stdev=2208.69 clat percentiles (usec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 9], | 30.00th=[ 9], 40.00th=[ 9], 50.00th=[ 9], 60.00th=[ 9], | 70.00th=[ 10], 80.00th=[ 10], 90.00th=[ 11], 95.00th=[ 12], | 99.00th=[ 15], 99.50th=[ 16], 99.90th=[ 67], 99.95th=[ 260], | 99.99th=[ 1106] bw ( KiB/s): min=24248, max=262653, per=100.00%, avg=161982.49, stdev=72674.80, samples=207 iops : min= 6062, max=65663, avg=40495.63, stdev=18168.66, samples=207 lat (usec) : 10=89.05%, 20=10.66%, 50=0.15%, 100=0.07%, 250=0.02% lat (usec) : 500=0.04%, 750=0.01%, 1000=0.01% lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01% lat (msec) : 250=0.01%, 500=0.01% cpu : usr=8.23%, sys=56.34%, ctx=12287, majf=0, minf=30 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=4194304,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=147MiB/s (154MB/s), 147MiB/s-147MiB/s (154MB/s-154MB/s), io=16.0GiB (17.2GB), run=111547-111547msec Disk stats (read/write): sda: ios=33442/3, merge=3/3, ticks=2496698/287, in_queue=2496987, util=47.90%
Sequential ReadWrite:
root@pi4test:~# fio --rw=readwrite --name=testfile --size 16G testfile: (g=0): rw=rw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 fio-3.25 Starting 1 process Jobs: 1 (f=1): [M(1)][100.0%][r=1617KiB/s,w=1865KiB/s][r=404,w=466 IOPS][eta 00m:00s] testfile: (groupid=0, jobs=1): err= 0: pid=1648: Sat Apr 2 13:15:27 2022 read: IOPS=12.0k, BW=50.7MiB/s (53.1MB/s)(8196MiB/161797msec) clat (usec): min=6, max=1561.3k, avg=44.31, stdev=6185.70 lat (usec): min=6, max=1561.3k, avg=44.92, stdev=6185.70 clat percentiles (usec): | 1.00th=[ 7], 5.00th=[ 9], 10.00th=[ 9], 20.00th=[ 9], | 30.00th=[ 9], 40.00th=[ 9], 50.00th=[ 9], 60.00th=[ 9], | 70.00th=[ 10], 80.00th=[ 10], 90.00th=[ 12], 95.00th=[ 15], | 99.00th=[ 19], 99.50th=[ 21], 99.90th=[ 61], 99.95th=[ 172], | 99.99th=[ 873] bw ( KiB/s): min= 8, max=161920, per=100.00%, avg=72898.53, stdev=42363.84, samples=230 iops : min= 2, max=40480, avg=18224.61, stdev=10591.00, samples=230 write: IOPS=12.0k, BW=50.6MiB/s (53.1MB/s)(8188MiB/161797msec); 0 zone resets clat (usec): min=6, max=267581, avg=22.01, stdev=944.13 lat (usec): min=6, max=267582, avg=22.71, stdev=944.15 clat percentiles (usec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 9], | 30.00th=[ 9], 40.00th=[ 9], 50.00th=[ 9], 60.00th=[ 9], | 70.00th=[ 10], 80.00th=[ 10], 90.00th=[ 12], 95.00th=[ 15], | 99.00th=[ 21], 99.50th=[ 30], 99.90th=[ 178], 99.95th=[13304], | 99.99th=[22152] bw ( KiB/s): min= 56, max=162224, per=100.00%, avg=73784.31, stdev=41711.19, samples=227 iops : min= 14, max=40556, avg=18446.04, stdev=10427.83, samples=227 lat (usec) : 10=83.44%, 20=15.58%, 50=0.82%, 100=0.06%, 250=0.03% lat (usec) : 500=0.01%, 750=0.01%, 1000=0.01% lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.02%, 50=0.01% lat (msec) : 250=0.01%, 500=0.01%, 1000=0.01%, 2000=0.01% cpu : usr=6.77%, sys=35.65%, ctx=8774, majf=0, minf=49 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2098266,2096038,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=50.7MiB/s (53.1MB/s), 50.7MiB/s-50.7MiB/s (53.1MB/s-53.1MB/s), io=8196MiB (8594MB), run=161797-161797msec WRITE: bw=50.6MiB/s (53.1MB/s), 50.6MiB/s-50.6MiB/s (53.1MB/s-53.1MB/s), io=8188MiB (8585MB), run=161797-161797msec Disk stats (read/write): sda: ios=16929/16403, merge=2/38, ticks=3976787/2720784, in_queue=6697684, util=92.86%
Random Write:
root@pi4test:~# fio --rw=randwrite --name=testfile --size 16G testfile: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 fio-3.25 Starting 1 process Jobs: 1 (f=1): [w(1)][100.0%][w=14.5MiB/s][w=3715 IOPS][eta 00m:00s] testfile: (groupid=0, jobs=1): err= 0: pid=1405: Sat Apr 2 12:36:02 2022 write: IOPS=2769, BW=10.8MiB/s (11.3MB/s)(16.0GiB/1514667msec); 0 zone resets clat (usec): min=8, max=5130.9k, avg=352.32, stdev=28060.03 lat (usec): min=9, max=5130.9k, avg=353.44, stdev=28060.04 clat percentiles (usec): | 1.00th=[ 11], 5.00th=[ 11], 10.00th=[ 12], 20.00th=[ 12], | 30.00th=[ 12], 40.00th=[ 14], 50.00th=[ 19], 60.00th=[ 21], | 70.00th=[ 22], 80.00th=[ 24], 90.00th=[ 25], 95.00th=[ 30], | 99.00th=[ 86], 99.50th=[17695], 99.90th=[18744], 99.95th=[19006], | 99.99th=[27919] bw ( KiB/s): min= 8, max=150741, per=100.00%, avg=21022.85, stdev=24157.80, samples=1596 iops : min= 2, max=37685, avg=5255.66, stdev=6039.46, samples=1596 lat (usec) : 10=0.75%, 20=53.96%, 50=43.53%, 100=0.80%, 250=0.12% lat (usec) : 500=0.06%, 750=0.03%, 1000=0.02% lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.70%, 50=0.02% lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01% lat (msec) : 2000=0.01%, >=2000=0.01% cpu : usr=1.56%, sys=5.94%, ctx=43410, majf=0, minf=27 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,4194304,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): WRITE: bw=10.8MiB/s (11.3MB/s), 10.8MiB/s-10.8MiB/s (11.3MB/s-11.3MB/s), io=16.0GiB (17.2GB), run=1514667-1514667msec Disk stats (read/write): sda: ios=23/4041732, merge=0/385, ticks=5863/83180908, in_queue=83193169, util=99.41%
Random Read
root@pi4test:~# fio --rw=randread --name=testfile --size 16G testfile: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 fio-3.25 Starting 1 process Jobs: 1 (f=1): [r(1)][100.0%][r=11.4MiB/s][r=2916 IOPS][eta 00m:00s] testfile: (groupid=0, jobs=1): err= 0: pid=1504: Sat Apr 2 13:09:22 2022 read: IOPS=2848, BW=11.1MiB/s (11.7MB/s)(16.0GiB/1472703msec) clat (usec): min=170, max=74926, avg=337.03, stdev=57.00 lat (usec): min=171, max=74927, avg=338.26, stdev=57.03 clat percentiles (usec): | 1.00th=[ 260], 5.00th=[ 277], 10.00th=[ 289], 20.00th=[ 297], | 30.00th=[ 306], 40.00th=[ 318], 50.00th=[ 326], 60.00th=[ 355], | 70.00th=[ 367], 80.00th=[ 383], 90.00th=[ 396], 95.00th=[ 412], | 99.00th=[ 429], 99.50th=[ 433], 99.90th=[ 449], 99.95th=[ 465], | 99.99th=[ 498] bw ( KiB/s): min= 6072, max=13192, per=100.00%, avg=11479.51, stdev=308.82, samples=2926 iops : min= 1518, max= 3298, avg=2869.57, stdev=77.23, samples=2926 lat (usec) : 250=0.05%, 500=99.94%, 750=0.01%, 1000=0.01% lat (msec) : 2=0.01%, 4=0.01%, 20=0.01%, 100=0.01% cpu : usr=3.30%, sys=24.04%, ctx=4200629, majf=0, minf=25 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=4194304,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=11.1MiB/s (11.7MB/s), 11.1MiB/s-11.1MiB/s (11.7MB/s-11.7MB/s), io=16.0GiB (17.2GB), run=1472703-1472703msec Disk stats (read/write): sda: ios=4194136/52, merge=0/13, ticks=1223682/255, in_queue=1223958, util=99.45%
Random ReadWrite:
root@pi4test:~# fio --rw=randrw --name=testfile --size 16G testfile: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 fio-3.25 Starting 1 process Jobs: 1 (f=1): [m(1)][99.9%][r=4344KiB/s,w=4284KiB/s][r=1086,w=1071 IOPS][eta 00m:01s] testfile: (groupid=0, jobs=1): err= 0: pid=1664: Sat Apr 2 13:43:09 2022 read: IOPS=1355, BW=5422KiB/s (5552kB/s)(8196MiB/1548099msec) clat (usec): min=221, max=669084, avg=662.63, stdev=12047.16 lat (usec): min=222, max=669084, avg=663.74, stdev=12047.19 clat percentiles (usec): | 1.00th=[ 253], 5.00th=[ 273], 10.00th=[ 281], 20.00th=[ 293], | 30.00th=[ 302], 40.00th=[ 314], 50.00th=[ 326], 60.00th=[ 351], | 70.00th=[ 363], 80.00th=[ 375], 90.00th=[ 392], 95.00th=[ 412], | 99.00th=[ 433], 99.50th=[ 461], 99.90th=[ 14615], 99.95th=[371196], | 99.99th=[517997] bw ( KiB/s): min= 7, max=12304, per=100.00%, avg=5621.54, stdev=4267.16, samples=2986 iops : min= 1, max= 3076, avg=1405.32, stdev=1066.79, samples=2986 write: IOPS=1353, BW=5416KiB/s (5546kB/s)(8188MiB/1548099msec); 0 zone resets clat (usec): min=9, max=3560.6k, avg=52.69, stdev=7256.56 lat (usec): min=9, max=3560.6k, avg=53.98, stdev=7256.59 clat percentiles (usec): | 1.00th=[ 12], 5.00th=[ 13], 10.00th=[ 15], 20.00th=[ 20], | 30.00th=[ 20], 40.00th=[ 20], 50.00th=[ 22], 60.00th=[ 23], | 70.00th=[ 23], 80.00th=[ 24], 90.00th=[ 28], 95.00th=[ 33], | 99.00th=[ 46], 99.50th=[ 51], 99.90th=[ 87], 99.95th=[ 120], | 99.99th=[ 1811] bw ( KiB/s): min= 7, max=13016, per=100.00%, avg=6402.52, stdev=3956.59, samples=2619 iops : min= 1, max= 3254, avg=1600.56, stdev=989.15, samples=2619 lat (usec) : 10=0.01%, 20=20.96%, 50=28.74%, 100=0.23%, 250=0.14% lat (usec) : 500=49.74%, 750=0.09%, 1000=0.01% lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.02%, 50=0.01% lat (msec) : 100=0.01%, 250=0.01%, 500=0.02%, 750=0.02%, 1000=0.01% lat (msec) : 2000=0.01%, >=2000=0.01% cpu : usr=2.53%, sys=14.07%, ctx=2114381, majf=0, minf=37 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2098266,2096038,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=5422KiB/s (5552kB/s), 5422KiB/s-5422KiB/s (5552kB/s-5552kB/s), io=8196MiB (8594MB), run=1548099-1548099msec WRITE: bw=5416KiB/s (5546kB/s), 5416KiB/s-5416KiB/s (5546kB/s-5546kB/s), io=8188MiB (8585MB), run=1548099-1548099msec Disk stats (read/write): sda: ios=2097615/2048347, merge=0/326, ticks=1298235/40511328, in_queue=41812500, util=99.77%
Die Evo Plus Serie soll besonders robust und performant sein. Deswegen die gleichen Tests als Vergleich zum SSD Modul.
Die Random Tests habe ich mit verkleinerter Testdatei (nur 1Gb statt 16Gb) machen müssen. Der Random Write Test hat mehr als 11 Stunden Laufzeit bei 16GB hochgerechnet! Durch besseres Caching sind die Daten dadurch evtl. nicht richtig vergleichbar. Aber schon die seq. Tests zeigen deutliche Unterschiede.
Ergebnisse Übersicht
| IO Art | Durchsatz | IOs | typ. Latenz |
|---|---|---|---|
| Seq. Write | 24,4 MB/s | 5948 IOs/s | 20 - 50µs |
| Seq. Read | 42 MB/s | 10800 IOs/s | 20µs |
| Seq. ReadWrite | 15,9 RMB/s | 3877 RIOs/s | 10-20µs |
| 15,9 WMB/s | 3873 WIOs/s | ||
| Rand. Write | 2246 kB/s | 548 IOs/s | 50µs |
| Rand. Read | 5149kB/s | 1257 IOs/s | 500 - 750µs |
| Rand. ReadWrite | 836 RkB/s | 204 RIOs/s | 50 - 750µs |
| 836 WkB/s | 204 WIOs/s |
Sequential Write:
root@pi4test:~# fio --rw=write --name=testfile --size 16G; testfile: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 fio-3.25 Starting 1 process testfile: Laying out IO file (1 file / 16384MiB) Jobs: 1 (f=1): [W(1)][100.0%][w=24.2MiB/s][w=6191 IOPS][eta 00m:00s] testfile: (groupid=0, jobs=1): err= 0: pid=1512: Sat Apr 2 16:06:42 2022 write: IOPS=5948, BW=23.2MiB/s (24.4MB/s)(16.0GiB/705139msec); 0 zone resets clat (usec): min=9, max=4123.6k, avg=162.67, stdev=11836.43 lat (usec): min=10, max=4123.6k, avg=163.63, stdev=11836.44 clat percentiles (usec): | 1.00th=[ 12], 5.00th=[ 12], 10.00th=[ 12], 20.00th=[ 13], | 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 18], 60.00th=[ 22], | 70.00th=[ 23], 80.00th=[ 26], 90.00th=[ 30], 95.00th=[ 34], | 99.00th=[ 50], 99.50th=[ 104], 99.90th=[17433], 99.95th=[18744], | 99.99th=[28181] bw ( KiB/s): min= 8, max=142984, per=100.00%, avg=32273.09, stdev=23835.09, samples=1040 iops : min= 2, max=35746, avg=8068.18, stdev=5958.77, samples=1040 lat (usec) : 10=0.01%, 20=54.59%, 50=44.43%, 100=0.48%, 250=0.02% lat (usec) : 500=0.01%, 750=0.01%, 1000=0.01% lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.43%, 50=0.03% lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 2000=0.01% lat (msec) : >=2000=0.01% cpu : usr=2.78%, sys=11.96%, ctx=32209, majf=0, minf=66 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,4194304,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): WRITE: bw=23.2MiB/s (24.4MB/s), 23.2MiB/s-23.2MiB/s (24.4MB/s-24.4MB/s), io=16.0GiB (17.2GB), run=705139-705139msec Disk stats (read/write): mmcblk0: ios=81/33465, merge=151/1556, ticks=14287/48845811, in_queue=48860098, util=98.46%
Sequential Read:
root@pi4test:~# fio --rw=read --name=testfile --size 16G; testfile: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 fio-3.25 Starting 1 process Jobs: 1 (f=1): [R(1)][99.7%][r=43.0MiB/s][r=11.0k IOPS][eta 00m:01s] testfile: (groupid=0, jobs=1): err= 0: pid=1548: Sat Apr 2 16:13:14 2022 read: IOPS=10.8k, BW=42.0MiB/s (44.1MB/s)(16.0GiB/389870msec) clat (usec): min=5, max=72870, avg=85.88, stdev=588.88 lat (usec): min=6, max=72871, avg=86.83, stdev=588.88 clat percentiles (usec): | 1.00th=[ 8], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 12], | 30.00th=[ 12], 40.00th=[ 12], 50.00th=[ 13], 60.00th=[ 13], | 70.00th=[ 13], 80.00th=[ 13], 90.00th=[ 13], 95.00th=[ 14], | 99.00th=[ 4686], 99.50th=[ 4752], 99.90th=[ 4948], 99.95th=[ 5145], | 99.99th=[ 5276] bw ( KiB/s): min=28640, max=44544, per=100.00%, avg=43925.71, stdev=627.86, samples=764 iops : min= 7160, max=11136, avg=10981.31, stdev=157.00, samples=764 lat (usec) : 10=5.15%, 20=93.15%, 50=0.11%, 100=0.03%, 250=0.01% lat (usec) : 500=0.01%, 750=0.01%, 1000=0.01% lat (msec) : 2=0.01%, 4=0.01%, 10=1.56%, 20=0.01%, 100=0.01% cpu : usr=4.41%, sys=21.28%, ctx=70088, majf=0, minf=30 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=4194304,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=42.0MiB/s (44.1MB/s), 42.0MiB/s-42.0MiB/s (44.1MB/s-44.1MB/s), io=16.0GiB (17.2GB), run=389870-389870msec Disk stats (read/write): mmcblk0: ios=65512/71, merge=0/3, ticks=671153/40874, in_queue=712026, util=98.45%
Sequential ReadWrite:
root@pi4test:~# fio --rw=readwrite --name=testfile --size 16G; testfile: (g=0): rw=rw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 fio-3.25 Starting 1 process Jobs: 1 (f=1): [M(1)][99.6%][eta 00m:02s] testfile: (groupid=0, jobs=1): err= 0: pid=1560: Sat Apr 2 16:22:15 2022 read: IOPS=3877, BW=15.1MiB/s (15.9MB/s)(8196MiB/541118msec) clat (usec): min=5, max=3310.5k, avg=223.88, stdev=12260.46 lat (usec): min=6, max=3310.5k, avg=224.58, stdev=12260.46 clat percentiles (usec): | 1.00th=[ 7], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 8], 40.00th=[ 8], 50.00th=[ 8], 60.00th=[ 8], | 70.00th=[ 9], 80.00th=[ 13], 90.00th=[ 16], 95.00th=[ 19], | 99.00th=[ 4047], 99.50th=[ 4293], 99.90th=[ 4555], 99.95th=[ 5014], | 99.99th=[350225] bw ( KiB/s): min= 7, max=105344, per=100.00%, avg=19911.38, stdev=13472.37, samples=841 iops : min= 1, max=26336, avg=4977.78, stdev=3368.12, samples=841 write: IOPS=3873, BW=15.1MiB/s (15.9MB/s)(8188MiB/541118msec); 0 zone resets clat (usec): min=7, max=3855.8k, avg=22.92, stdev=4011.07 lat (usec): min=8, max=3855.8k, avg=23.70, stdev=4011.07 clat percentiles (usec): | 1.00th=[ 9], 5.00th=[ 10], 10.00th=[ 10], 20.00th=[ 10], | 30.00th=[ 10], 40.00th=[ 10], 50.00th=[ 11], 60.00th=[ 11], | 70.00th=[ 13], 80.00th=[ 18], 90.00th=[ 22], 95.00th=[ 25], | 99.00th=[ 32], 99.50th=[ 38], 99.90th=[ 66], 99.95th=[ 108], | 99.99th=[13304] bw ( KiB/s): min= 7, max=107312, per=100.00%, avg=19889.49, stdev=13495.28, samples=841 iops : min= 1, max=26828, avg=4972.32, stdev=3373.83, samples=841 lat (usec) : 10=58.60%, 20=33.06%, 50=7.47%, 100=0.09%, 250=0.01% lat (usec) : 500=0.01%, 750=0.01%, 1000=0.01% lat (msec) : 2=0.02%, 4=0.19%, 10=0.50%, 20=0.01%, 50=0.01% lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01% lat (msec) : 2000=0.01%, >=2000=0.01% cpu : usr=2.81%, sys=11.54%, ctx=35531, majf=0, minf=46 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=2098266,2096038,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=15.1MiB/s (15.9MB/s), 15.1MiB/s-15.1MiB/s (15.9MB/s-15.9MB/s), io=8196MiB (8594MB), run=541118-541118msec WRITE: bw=15.1MiB/s (15.9MB/s), 15.1MiB/s-15.1MiB/s (15.9MB/s-15.9MB/s), io=8188MiB (8585MB), run=541118-541118msec Disk stats (read/write): mmcblk0: ios=30739/16651, merge=687/127, ticks=638495/11316064, in_queue=11954560, util=98.92%
Random Write:
root@pi4test:~# fio --rw=randwrite --name=testfile --size 1G testfile: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 fio-3.25 Starting 1 process Jobs: 1 (f=1): [w(1)][100.0%][w=3843KiB/s][w=960 IOPS][eta 00m:00s] testfile: (groupid=0, jobs=1): err= 0: pid=822: Sat Apr 2 18:20:04 2022 write: IOPS=548, BW=2194KiB/s (2246kB/s)(1024MiB/477994msec); 0 zone resets clat (usec): min=10, max=10665k, avg=1812.56, stdev=34779.32 lat (usec): min=11, max=10665k, avg=1814.21, stdev=34779.33 clat percentiles (usec): | 1.00th=[ 12], 5.00th=[ 13], 10.00th=[ 19], | 20.00th=[ 27], 30.00th=[ 28], 40.00th=[ 30], | 50.00th=[ 32], 60.00th=[ 35], 70.00th=[ 36], | 80.00th=[ 36], 90.00th=[ 43], 95.00th=[ 52], | 99.00th=[ 58459], 99.50th=[ 68682], 99.90th=[ 98042], | 99.95th=[ 152044], 99.99th=[1199571] bw ( KiB/s): min= 8, max=196686, per=100.00%, avg=2527.76, stdev=6814.11, samples=829 iops : min= 2, max=49171, avg=631.78, stdev=1703.51, samples=829 lat (usec) : 20=10.10%, 50=84.40%, 100=2.80%, 250=0.18%, 500=0.01% lat (usec) : 750=0.01%, 1000=0.01% lat (msec) : 4=0.01%, 20=0.01%, 50=1.35%, 100=1.07%, 250=0.06% lat (msec) : 500=0.01%, 750=0.01%, 1000=0.01%, 2000=0.01%, >=2000=0.01% cpu : usr=0.56%, sys=1.79%, ctx=8529, majf=1, minf=27 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,262144,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): WRITE: bw=2194KiB/s (2246kB/s), 2194KiB/s-2194KiB/s (2246kB/s-2246kB/s), io=1024MiB (1074MB), run=477994-477994msec Disk stats (read/write): mmcblk0: ios=4/177602, merge=0/231, ticks=240/59463236, in_queue=59463477, util=95.64%
Random Read
root@pi4test:~# fio --rw=randread --name=testfile --size 1G testfile: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1 fio-3.25 Starting 1 process Jobs: 1 (f=1): [r(1)][100.0%][r=8740KiB/s][r=2185 IOPS][eta 00m:00s] testfile: (groupid=0, jobs=1): err= 0: pid=845: Sat Apr 2 18:23:36 2022 read: IOPS=1257, BW=5028KiB/s (5149kB/s)(1024MiB/208547msec) clat (usec): min=364, max=1166.1k, avg=565.39, stdev=2278.64 lat (usec): min=365, max=1166.1k, avg=566.64, stdev=2278.64 clat percentiles (usec): | 1.00th=[ 392], 5.00th=[ 408], 10.00th=[ 412], 20.00th=[ 457], | 30.00th=[ 570], 40.00th=[ 578], 50.00th=[ 594], 60.00th=[ 594], | 70.00th=[ 611], 80.00th=[ 635], 90.00th=[ 644], 95.00th=[ 652], | 99.00th=[ 668], 99.50th=[ 676], 99.90th=[ 758], 99.95th=[ 840], | 99.99th=[ 3851] bw ( KiB/s): min= 360, max= 8776, per=100.00%, avg=6948.06, stdev=521.44, samples=301 iops : min= 90, max= 2194, avg=1736.77, stdev=130.29, samples=301 lat (usec) : 500=28.14%, 750=71.76%, 1000=0.07% lat (msec) : 2=0.01%, 4=0.03%, 10=0.01%, 2000=0.01% cpu : usr=1.51%, sys=7.27%, ctx=264364, majf=0, minf=24 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=262144,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=5028KiB/s (5149kB/s), 5028KiB/s-5028KiB/s (5149kB/s-5149kB/s), io=1024MiB (1074MB), run=208547-208547msec Disk stats (read/write): mmcblk0: ios=261731/11075, merge=0/81, ticks=136444/6987958, in_queue=7124403, util=99.22%
Random ReadWrite:
root@pi4test:~# fio --rw=randrw --name=testfile --size 1G testfile: (groupid=0, jobs=1): err= 0: pid=853: Sat Apr 2 18:34:18 2022 read: IOPS=204, BW=816KiB/s (836kB/s)(512MiB/642113msec) clat (usec): min=318, max=15370k, avg=4536.72, stdev=125745.82 lat (usec): min=319, max=15370k, avg=4537.91, stdev=125745.85 clat percentiles (usec): | 1.00th=[ 400], 5.00th=[ 412], 10.00th=[ 424], | 20.00th=[ 441], 30.00th=[ 529], 40.00th=[ 586], | 50.00th=[ 594], 60.00th=[ 603], 70.00th=[ 611], | 80.00th=[ 619], 90.00th=[ 627], 95.00th=[ 635], | 99.00th=[ 701], 99.50th=[ 94897], 99.90th=[1266680], | 99.95th=[2432697], 99.99th=[4731175] bw ( KiB/s): min= 7, max= 8232, per=100.00%, avg=2077.54, stdev=2986.64, samples=505 iops : min= 1, max= 2058, avg=519.32, stdev=746.63, samples=505 write: IOPS=204, BW=817KiB/s (836kB/s)(512MiB/642113msec); 0 zone resets clat (usec): min=11, max=14314k, avg=338.08, stdev=57482.73 lat (usec): min=12, max=14314k, avg=339.48, stdev=57482.74 clat percentiles (usec): | 1.00th=[ 15], 5.00th=[ 22], 10.00th=[ 23], 20.00th=[ 24], | 30.00th=[ 24], 40.00th=[ 25], 50.00th=[ 27], 60.00th=[ 27], | 70.00th=[ 28], 80.00th=[ 29], 90.00th=[ 36], 95.00th=[ 38], | 99.00th=[ 48], 99.50th=[ 57], 99.90th=[ 90], 99.95th=[ 104], | 99.99th=[109577] bw ( KiB/s): min= 7, max= 8520, per=100.00%, avg=2517.12, stdev=3121.97, samples=417 iops : min= 1, max= 2130, avg=629.21, stdev=780.47, samples=417 lat (usec) : 20=0.73%, 50=48.93%, 100=0.33%, 250=0.01%, 500=14.52% lat (usec) : 750=35.06%, 1000=0.04% lat (msec) : 2=0.01%, 4=0.02%, 10=0.01%, 20=0.01%, 50=0.01% lat (msec) : 100=0.15%, 250=0.12%, 500=0.01%, 750=0.01%, 1000=0.01% lat (msec) : 2000=0.03%, >=2000=0.04% cpu : usr=0.38%, sys=1.74%, ctx=131377, majf=0, minf=39 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=131040,131104,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=816KiB/s (836kB/s), 816KiB/s-816KiB/s (836kB/s-836kB/s), io=512MiB (537MB), run=642113-642113msec WRITE: bw=817KiB/s (836kB/s), 817KiB/s-817KiB/s (836kB/s-836kB/s), io=512MiB (537MB), run=642113-642113msec Disk stats (read/write): mmcblk0: ios=131067/107292, merge=30/113, ticks=593905/68646500, in_queue=69240405, util=96.72%
root@pi4test:~# 7z b 7-Zip [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE) LE CPU Freq: 64000000 64000000 64000000 - - - - - 2048000000 RAM size: 7898 MB, # CPU hardware threads: 4 RAM usage: 882 MB, # Benchmark threads: 4 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 4113 361 1107 4001 | 108393 399 2318 9248 23: 3935 375 1070 4010 | 105545 398 2294 9132 24: 3777 376 1080 4062 | 102803 398 2265 9025 25: 3555 371 1095 4060 | 99861 398 2231 8887 ---------------------------------- | ------------------------------ Avr: 371 1088 4033 | 398 2277 9073 Tot: 385 1682 6553 root@pi4test:~# root@pi4test:~# 7z b 7-Zip [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21 p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,32 bits,4 CPUs LE) LE CPU Freq: 32000000 - - - 128000000 - - 1024000000 2048000000 RAM size: 7898 MB, # CPU hardware threads: 4 RAM usage: 882 MB, # Benchmark threads: 4 Compressing | Decompressing Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS 22: 4043 350 1123 3933 | 108442 399 2317 9252 23: 3917 370 1078 3991 | 105547 398 2294 9132 24: 3778 374 1085 4062 | 102708 398 2267 9016 25: 3573 371 1101 4080 | 99778 398 2231 8880 ---------------------------------- | ------------------------------ Avr: 366 1097 4017 | 398 2277 9070 Tot: 382 1687 6543
dd if=/path/urandom of=/dev/sda & pid=$! && sleep 5 && while kill -USR1 $pid; do sleep 5; done
https://dennis-henke.de/2016/05/20/samba-4-domain-controller/
samba-tool domain provision –use-rfc2307 –interactive
net sam rights grant “MS\\Domain Admins” SeDiskOperatorPrivilege -Uadministrator
net rpc rights grant “MS\\Domain Admins” SeDiskOperatorPrivilege -Uadministrator
net rpc rights list accounts -U Administrator