NAS upgrade
NAS
- Login
sudo synogear installiperf3 -s -p 5201
Workstation
- Login
iperf3 -c 192.168.129.3 -p 5201
Plan
NAS
- LAN: 192.168.129.3/24
- P2P: 192.168.130.9/30
Titan
- LAN: 192.168.129.217/24
- P2P: 192.168.130.10/30
Hardware
USB-C 10Gb NIC
enp4s0f4u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 34:c8:d6:b3:08:ac brd ff:ff:ff:ff:ff:ff
altname enx34c8d6b308ac
Baseline
Titan: 1Gb port / 1Gb link NAS: 1Gb port / 1Gb link
$ iperf3 -c 192.168.129.3 -p 5201
Connecting to host 192.168.129.3, port 5201
[ 4] local 192.168.129.217 port 35646 connected to 192.168.129.3 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 4] 0.00-1.00 sec 114 MBytes 955 Mbits/sec 0 355 KBytes
[ 4] 1.00-2.00 sec 112 MBytes 942 Mbits/sec 0 355 KBytes
[ 4] 2.00-3.00 sec 112 MBytes 940 Mbits/sec 0 386 KBytes
[ 4] 3.00-4.00 sec 113 MBytes 946 Mbits/sec 0 386 KBytes
[ 4] 4.00-5.00 sec 112 MBytes 944 Mbits/sec 0 404 KBytes
[ 4] 5.00-6.00 sec 112 MBytes 941 Mbits/sec 0 404 KBytes
[ 4] 6.00-7.00 sec 112 MBytes 936 Mbits/sec 0 424 KBytes
[ 4] 7.00-8.00 sec 113 MBytes 947 Mbits/sec 0 424 KBytes
[ 4] 8.00-9.00 sec 112 MBytes 937 Mbits/sec 0 424 KBytes
[ 4] 9.00-10.00 sec 112 MBytes 944 Mbits/sec 0 424 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 4] 0.00-10.00 sec 1.10 GBytes 943 Mbits/sec 0 sender
[ 4] 0.00-10.00 sec 1.10 GBytes 941 Mbits/sec receiver
Use USB 10Gb on Titan
Titan: 10Gb port / 1Gb link NAS: 1Gb port / 1Gb link
$ iperf3 -c 192.168.129.3 -p 5201
Connecting to host 192.168.129.3, port 5201
[ 4] local 192.168.129.231 port 37496 connected to 192.168.129.3 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 4] 0.00-1.00 sec 114 MBytes 959 Mbits/sec 0 419 KBytes
[ 4] 1.00-2.00 sec 112 MBytes 937 Mbits/sec 0 419 KBytes
[ 4] 2.00-3.00 sec 112 MBytes 944 Mbits/sec 0 440 KBytes
[ 4] 3.00-4.00 sec 112 MBytes 943 Mbits/sec 0 440 KBytes
[ 4] 4.00-5.00 sec 112 MBytes 943 Mbits/sec 0 440 KBytes
[ 4] 5.00-6.00 sec 112 MBytes 937 Mbits/sec 0 440 KBytes
[ 4] 6.00-7.00 sec 113 MBytes 945 Mbits/sec 0 440 KBytes
[ 4] 7.00-8.00 sec 112 MBytes 938 Mbits/sec 0 440 KBytes
[ 4] 8.00-9.00 sec 112 MBytes 944 Mbits/sec 0 440 KBytes
[ 4] 9.00-10.00 sec 112 MBytes 937 Mbits/sec 0 440 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 4] 0.00-10.00 sec 1.10 GBytes 943 Mbits/sec 0 sender
[ 4] 0.00-10.00 sec 1.10 GBytes 941 Mbits/sec receiver
iperf Done.
Install 10Gb on NAS
Titan: 10Gb port / 1Gb link NAS: 10Gb port / 1Gb link
$ iperf3 -c 192.168.130.9 -p 5201
Connecting to host 192.168.130.9, port 5201
[ 4] local 192.168.130.10 port 51120 connected to 192.168.130.9 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 4] 0.00-1.00 sec 564 MBytes 4.73 Gbits/sec 0 1.16 MBytes
[ 4] 1.00-2.00 sec 561 MBytes 4.70 Gbits/sec 0 1.16 MBytes
[ 4] 2.00-3.00 sec 561 MBytes 4.71 Gbits/sec 0 1.23 MBytes
[ 4] 3.00-4.00 sec 562 MBytes 4.71 Gbits/sec 0 1.23 MBytes
[ 4] 4.00-5.00 sec 561 MBytes 4.71 Gbits/sec 0 1.23 MBytes
[ 4] 5.00-6.00 sec 561 MBytes 4.71 Gbits/sec 0 1.23 MBytes
[ 4] 6.00-7.00 sec 561 MBytes 4.71 Gbits/sec 0 1.29 MBytes
[ 4] 7.00-8.00 sec 560 MBytes 4.70 Gbits/sec 0 1.29 MBytes
[ 4] 8.00-9.00 sec 562 MBytes 4.71 Gbits/sec 0 1.29 MBytes
[ 4] 9.00-10.00 sec 561 MBytes 4.70 Gbits/sec 0 1.29 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 4] 0.00-10.00 sec 5.48 GBytes 4.71 Gbits/sec 0 sender
[ 4] 0.00-10.00 sec 5.48 GBytes 4.71 Gbits/sec receiver
iperf Done.
File performance testing
Basic rsync from local NVME to $HOME/tmp/
- Over 1Gb NICs
$ rm -f /home/nfs/dan/tmp/ubuntu-26.04-desktop-amd64.iso && sync && rsync --progress /home/dan.local/iso/ubuntu-26.04-desktop-amd64.iso /home/nfs/dan/tmp/ && sync
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 161.83MB/s 0:00:38 (xfr#1, to-chk=0/1)
- Over 10Gb NICs
$ rm -f /home/nfs/dan/tmp/ubuntu-26.04-desktop-amd64.iso && sync && rsync --progress /home/dan.local/iso/ubuntu-26.04-desktop-amd64.iso /home/nfs/dan/tmp/ && sync
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 225.15MB/s 0:00:27 (xfr#1, to-chk=0:00:27 (xfr#1, to-chk=0/1)
Performance testing
Trying multiple iperf3 connections
Two connections
$ iperf3 -P 2 -p 5201 -c nas
Connecting to host nas, port 5201
[ 4] local 192.168.130.10 port 36980 connected to 192.168.130.9 port 5201
[ 6] local 192.168.130.10 port 36994 connected to 192.168.130.9 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 4] 0.00-1.00 sec 284 MBytes 2.38 Gbits/sec 0 559 KBytes
[ 6] 0.00-1.00 sec 282 MBytes 2.37 Gbits/sec 0 536 KBytes
[SUM] 0.00-1.00 sec 566 MBytes 4.75 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 1.00-2.00 sec 280 MBytes 2.35 Gbits/sec 0 559 KBytes
[ 6] 1.00-2.00 sec 282 MBytes 2.36 Gbits/sec 0 564 KBytes
[SUM] 1.00-2.00 sec 562 MBytes 4.71 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 2.00-3.00 sec 280 MBytes 2.35 Gbits/sec 0 559 KBytes
[ 6] 2.00-3.00 sec 280 MBytes 2.35 Gbits/sec 0 564 KBytes
[SUM] 2.00-3.00 sec 560 MBytes 4.70 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 3.00-4.00 sec 281 MBytes 2.35 Gbits/sec 0 559 KBytes
[ 6] 3.00-4.00 sec 281 MBytes 2.35 Gbits/sec 0 594 KBytes
[SUM] 3.00-4.00 sec 562 MBytes 4.71 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 4.00-5.00 sec 280 MBytes 2.35 Gbits/sec 0 559 KBytes
[ 6] 4.00-5.00 sec 281 MBytes 2.36 Gbits/sec 0 594 KBytes
[SUM] 4.00-5.00 sec 561 MBytes 4.71 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 5.00-6.00 sec 280 MBytes 2.35 Gbits/sec 0 559 KBytes
[ 6] 5.00-6.00 sec 280 MBytes 2.35 Gbits/sec 0 622 KBytes
[SUM] 5.00-6.00 sec 560 MBytes 4.70 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 6.00-7.00 sec 281 MBytes 2.36 Gbits/sec 0 559 KBytes
[ 6] 6.00-7.00 sec 280 MBytes 2.35 Gbits/sec 0 622 KBytes
[SUM] 6.00-7.00 sec 561 MBytes 4.71 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 7.00-8.00 sec 280 MBytes 2.35 Gbits/sec 0 629 KBytes
[ 6] 7.00-8.00 sec 281 MBytes 2.36 Gbits/sec 0 622 KBytes
[SUM] 7.00-8.00 sec 561 MBytes 4.71 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 8.00-9.00 sec 281 MBytes 2.36 Gbits/sec 0 629 KBytes
[ 6] 8.00-9.00 sec 280 MBytes 2.35 Gbits/sec 0 622 KBytes
[SUM] 8.00-9.00 sec 562 MBytes 4.71 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 9.00-10.00 sec 280 MBytes 2.35 Gbits/sec 0 629 KBytes
[ 6] 9.00-10.00 sec 282 MBytes 2.36 Gbits/sec 0 622 KBytes
[SUM] 9.00-10.00 sec 561 MBytes 4.71 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 4] 0.00-10.00 sec 2.74 GBytes 2.36 Gbits/sec 0 sender
[ 4] 0.00-10.00 sec 2.74 GBytes 2.35 Gbits/sec receiver
[ 6] 0.00-10.00 sec 2.74 GBytes 2.36 Gbits/sec 0 sender
[ 6] 0.00-10.00 sec 2.74 GBytes 2.35 Gbits/sec receiver
[SUM] 0.00-10.00 sec 5.49 GBytes 4.71 Gbits/sec 0 sender
[SUM] 0.00-10.00 sec 5.48 GBytes 4.71 Gbits/sec receiver
iperf Done.
Result: Same ~4.7 Gbits/sec performance.
Maybe it needs more cores/threads to see a difference?
Four connections
$ iperf3 -P 4 -p 5201 -c nas
Connecting to host nas, port 5201
[ 4] local 192.168.130.10 port 50562 connected to 192.168.130.9 port 5201
[ 6] local 192.168.130.10 port 50578 connected to 192.168.130.9 port 5201
[ 8] local 192.168.130.10 port 50590 connected to 192.168.130.9 port 5201
[ 10] local 192.168.130.10 port 50596 connected to 192.168.130.9 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 4] 0.00-1.00 sec 141 MBytes 1.18 Gbits/sec 0 324 KBytes
[ 6] 0.00-1.00 sec 142 MBytes 1.19 Gbits/sec 0 322 KBytes
[ 8] 0.00-1.00 sec 142 MBytes 1.19 Gbits/sec 0 321 KBytes
[ 10] 0.00-1.00 sec 142 MBytes 1.19 Gbits/sec 0 307 KBytes
[SUM] 0.00-1.00 sec 567 MBytes 4.75 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 1.00-2.00 sec 140 MBytes 1.18 Gbits/sec 0 324 KBytes
[ 6] 1.00-2.00 sec 140 MBytes 1.18 Gbits/sec 0 322 KBytes
[ 8] 1.00-2.00 sec 140 MBytes 1.18 Gbits/sec 0 321 KBytes
[ 10] 1.00-2.00 sec 140 MBytes 1.17 Gbits/sec 0 322 KBytes
[SUM] 1.00-2.00 sec 560 MBytes 4.70 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 2.00-3.00 sec 140 MBytes 1.18 Gbits/sec 0 324 KBytes
[ 6] 2.00-3.00 sec 140 MBytes 1.18 Gbits/sec 0 338 KBytes
[ 8] 2.00-3.00 sec 141 MBytes 1.18 Gbits/sec 0 321 KBytes
[ 10] 2.00-3.00 sec 141 MBytes 1.18 Gbits/sec 0 322 KBytes
[SUM] 2.00-3.00 sec 562 MBytes 4.72 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 3.00-4.00 sec 141 MBytes 1.18 Gbits/sec 0 341 KBytes
[ 6] 3.00-4.00 sec 140 MBytes 1.18 Gbits/sec 0 356 KBytes
[ 8] 3.00-4.00 sec 140 MBytes 1.18 Gbits/sec 0 355 KBytes
[ 10] 3.00-4.00 sec 140 MBytes 1.17 Gbits/sec 0 338 KBytes
[SUM] 3.00-4.00 sec 561 MBytes 4.71 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 4.00-5.00 sec 140 MBytes 1.17 Gbits/sec 0 341 KBytes
[ 6] 4.00-5.00 sec 141 MBytes 1.18 Gbits/sec 0 356 KBytes
[ 8] 4.00-5.00 sec 141 MBytes 1.18 Gbits/sec 0 355 KBytes
[ 10] 4.00-5.00 sec 140 MBytes 1.18 Gbits/sec 0 338 KBytes
[SUM] 4.00-5.00 sec 562 MBytes 4.71 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 5.00-6.00 sec 140 MBytes 1.18 Gbits/sec 0 341 KBytes
[ 6] 5.00-6.00 sec 140 MBytes 1.18 Gbits/sec 0 373 KBytes
[ 8] 5.00-6.00 sec 140 MBytes 1.18 Gbits/sec 0 373 KBytes
[ 10] 5.00-6.00 sec 141 MBytes 1.18 Gbits/sec 0 338 KBytes
[SUM] 5.00-6.00 sec 562 MBytes 4.71 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 6.00-7.00 sec 140 MBytes 1.17 Gbits/sec 0 341 KBytes
[ 6] 6.00-7.00 sec 140 MBytes 1.18 Gbits/sec 0 373 KBytes
[ 8] 6.00-7.00 sec 140 MBytes 1.18 Gbits/sec 0 373 KBytes
[ 10] 6.00-7.00 sec 140 MBytes 1.18 Gbits/sec 0 338 KBytes
[SUM] 6.00-7.00 sec 560 MBytes 4.70 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 7.00-8.00 sec 140 MBytes 1.18 Gbits/sec 0 341 KBytes
[ 6] 7.00-8.00 sec 140 MBytes 1.18 Gbits/sec 0 373 KBytes
[ 8] 7.00-8.00 sec 140 MBytes 1.17 Gbits/sec 0 373 KBytes
[ 10] 7.00-8.00 sec 140 MBytes 1.18 Gbits/sec 0 338 KBytes
[SUM] 7.00-8.00 sec 561 MBytes 4.70 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 8.00-9.00 sec 140 MBytes 1.18 Gbits/sec 0 341 KBytes
[ 6] 8.00-9.00 sec 141 MBytes 1.18 Gbits/sec 0 373 KBytes
[ 8] 8.00-9.00 sec 141 MBytes 1.18 Gbits/sec 0 373 KBytes
[ 10] 8.00-9.00 sec 141 MBytes 1.18 Gbits/sec 0 338 KBytes
[SUM] 8.00-9.00 sec 562 MBytes 4.72 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 9.00-10.00 sec 141 MBytes 1.18 Gbits/sec 0 434 KBytes
[ 6] 9.00-10.00 sec 140 MBytes 1.18 Gbits/sec 0 373 KBytes
[ 8] 9.00-10.00 sec 140 MBytes 1.17 Gbits/sec 0 373 KBytes
[ 10] 9.00-10.00 sec 140 MBytes 1.18 Gbits/sec 0 338 KBytes
[SUM] 9.00-10.00 sec 561 MBytes 4.70 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 4] 0.00-10.00 sec 1.37 GBytes 1.18 Gbits/sec 0 sender
[ 4] 0.00-10.00 sec 1.37 GBytes 1.18 Gbits/sec receiver
[ 6] 0.00-10.00 sec 1.37 GBytes 1.18 Gbits/sec 0 sender
[ 6] 0.00-10.00 sec 1.37 GBytes 1.18 Gbits/sec receiver
[ 8] 0.00-10.00 sec 1.37 GBytes 1.18 Gbits/sec 0 sender
[ 8] 0.00-10.00 sec 1.37 GBytes 1.18 Gbits/sec receiver
[ 10] 0.00-10.00 sec 1.37 GBytes 1.18 Gbits/sec 0 sender
[ 10] 0.00-10.00 sec 1.37 GBytes 1.18 Gbits/sec receiver
[SUM] 0.00-10.00 sec 5.49 GBytes 4.71 Gbits/sec 0 sender
[SUM] 0.00-10.00 sec 5.48 GBytes 4.71 Gbits/sec receiver
iperf Done.
Result: Same ~4.7 Gbits/sec performance.
Nope. But we'll give more threads a test...
(Insert obligitory "MOAR!!!" image here...)
Ten connections
$ iperf3 -P 10 -p 5201 -c nas
Connecting to host nas, port 5201
[ 4] local 192.168.130.10 port 48174 connected to 192.168.130.9 port 5201
[ 6] local 192.168.130.10 port 48188 connected to 192.168.130.9 port 5201
[ 8] local 192.168.130.10 port 48204 connected to 192.168.130.9 port 5201
[ 10] local 192.168.130.10 port 48208 connected to 192.168.130.9 port 5201
[ 12] local 192.168.130.10 port 48212 connected to 192.168.130.9 port 5201
[ 14] local 192.168.130.10 port 48224 connected to 192.168.130.9 port 5201
[ 16] local 192.168.130.10 port 48228 connected to 192.168.130.9 port 5201
[ 18] local 192.168.130.10 port 48230 connected to 192.168.130.9 port 5201
[ 20] local 192.168.130.10 port 48242 connected to 192.168.130.9 port 5201
[ 22] local 192.168.130.10 port 48250 connected to 192.168.130.9 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 4] 0.00-1.00 sec 56.8 MBytes 476 Mbits/sec 0 161 KBytes
[ 6] 0.00-1.00 sec 56.9 MBytes 477 Mbits/sec 0 161 KBytes
[ 8] 0.00-1.00 sec 56.8 MBytes 476 Mbits/sec 0 144 KBytes
[ 10] 0.00-1.00 sec 56.9 MBytes 477 Mbits/sec 0 161 KBytes
[ 12] 0.00-1.00 sec 56.8 MBytes 476 Mbits/sec 0 161 KBytes
[ 14] 0.00-1.00 sec 56.8 MBytes 476 Mbits/sec 0 161 KBytes
[ 16] 0.00-1.00 sec 56.9 MBytes 477 Mbits/sec 0 161 KBytes
[ 18] 0.00-1.00 sec 56.6 MBytes 475 Mbits/sec 0 156 KBytes
[ 20] 0.00-1.00 sec 56.8 MBytes 476 Mbits/sec 0 163 KBytes
[ 22] 0.00-1.00 sec 56.9 MBytes 477 Mbits/sec 0 161 KBytes
[SUM] 0.00-1.00 sec 568 MBytes 4.76 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 1.00-2.00 sec 56.2 MBytes 472 Mbits/sec 0 161 KBytes
[ 6] 1.00-2.00 sec 55.9 MBytes 469 Mbits/sec 0 161 KBytes
[ 8] 1.00-2.00 sec 56.0 MBytes 470 Mbits/sec 0 144 KBytes
[ 10] 1.00-2.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 12] 1.00-2.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 14] 1.00-2.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 16] 1.00-2.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 18] 1.00-2.00 sec 55.8 MBytes 468 Mbits/sec 0 163 KBytes
[ 20] 1.00-2.00 sec 56.0 MBytes 470 Mbits/sec 0 163 KBytes
[ 22] 1.00-2.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[SUM] 1.00-2.00 sec 560 MBytes 4.70 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 2.00-3.00 sec 56.2 MBytes 472 Mbits/sec 0 161 KBytes
[ 6] 2.00-3.00 sec 55.9 MBytes 469 Mbits/sec 0 161 KBytes
[ 8] 2.00-3.00 sec 56.0 MBytes 470 Mbits/sec 0 144 KBytes
[ 10] 2.00-3.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 12] 2.00-3.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 14] 2.00-3.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 16] 2.00-3.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 18] 2.00-3.00 sec 55.8 MBytes 468 Mbits/sec 0 163 KBytes
[ 20] 2.00-3.00 sec 56.0 MBytes 470 Mbits/sec 0 163 KBytes
[ 22] 2.00-3.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[SUM] 2.00-3.00 sec 562 MBytes 4.72 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 3.00-4.00 sec 56.2 MBytes 472 Mbits/sec 0 161 KBytes
[ 6] 3.00-4.00 sec 55.9 MBytes 469 Mbits/sec 0 161 KBytes
[ 8] 3.00-4.00 sec 56.0 MBytes 470 Mbits/sec 0 144 KBytes
[ 10] 3.00-4.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 12] 3.00-4.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 14] 3.00-4.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 16] 3.00-4.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 18] 3.00-4.00 sec 55.8 MBytes 468 Mbits/sec 0 163 KBytes
[ 20] 3.00-4.00 sec 56.0 MBytes 470 Mbits/sec 0 163 KBytes
[ 22] 3.00-4.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[SUM] 3.00-4.00 sec 561 MBytes 4.71 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 4.00-5.00 sec 56.2 MBytes 472 Mbits/sec 0 161 KBytes
[ 6] 4.00-5.00 sec 55.9 MBytes 469 Mbits/sec 0 161 KBytes
[ 8] 4.00-5.00 sec 56.0 MBytes 470 Mbits/sec 0 144 KBytes
[ 10] 4.00-5.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 12] 4.00-5.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 14] 4.00-5.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 16] 4.00-5.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 18] 4.00-5.00 sec 55.8 MBytes 468 Mbits/sec 0 163 KBytes
[ 20] 4.00-5.00 sec 56.0 MBytes 470 Mbits/sec 0 163 KBytes
[ 22] 4.00-5.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[SUM] 4.00-5.00 sec 562 MBytes 4.72 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 5.00-6.00 sec 56.2 MBytes 472 Mbits/sec 0 161 KBytes
[ 6] 5.00-6.00 sec 55.9 MBytes 469 Mbits/sec 0 161 KBytes
[ 8] 5.00-6.00 sec 56.0 MBytes 470 Mbits/sec 0 144 KBytes
[ 10] 5.00-6.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 12] 5.00-6.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 14] 5.00-6.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 16] 5.00-6.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 18] 5.00-6.00 sec 55.8 MBytes 468 Mbits/sec 0 163 KBytes
[ 20] 5.00-6.00 sec 56.0 MBytes 470 Mbits/sec 0 163 KBytes
[ 22] 5.00-6.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[SUM] 5.00-6.00 sec 561 MBytes 4.71 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 6.00-7.00 sec 56.2 MBytes 472 Mbits/sec 0 161 KBytes
[ 6] 6.00-7.00 sec 55.9 MBytes 469 Mbits/sec 0 161 KBytes
[ 8] 6.00-7.00 sec 56.0 MBytes 470 Mbits/sec 0 144 KBytes
[ 10] 6.00-7.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 12] 6.00-7.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 14] 6.00-7.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 16] 6.00-7.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 18] 6.00-7.00 sec 55.8 MBytes 468 Mbits/sec 0 163 KBytes
[ 20] 6.00-7.00 sec 56.0 MBytes 470 Mbits/sec 0 163 KBytes
[ 22] 6.00-7.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[SUM] 6.00-7.00 sec 562 MBytes 4.72 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 7.00-8.00 sec 56.2 MBytes 472 Mbits/sec 0 161 KBytes
[ 6] 7.00-8.00 sec 55.9 MBytes 469 Mbits/sec 0 161 KBytes
[ 8] 7.00-8.00 sec 56.0 MBytes 470 Mbits/sec 0 144 KBytes
[ 10] 7.00-8.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 12] 7.00-8.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 14] 7.00-8.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 16] 7.00-8.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 18] 7.00-8.00 sec 55.8 MBytes 468 Mbits/sec 0 163 KBytes
[ 20] 7.00-8.00 sec 56.0 MBytes 470 Mbits/sec 0 163 KBytes
[ 22] 7.00-8.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[SUM] 7.00-8.00 sec 561 MBytes 4.71 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 8.00-9.00 sec 56.2 MBytes 472 Mbits/sec 0 161 KBytes
[ 6] 8.00-9.00 sec 55.9 MBytes 469 Mbits/sec 0 161 KBytes
[ 8] 8.00-9.00 sec 56.0 MBytes 470 Mbits/sec 0 144 KBytes
[ 10] 8.00-9.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 12] 8.00-9.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 14] 8.00-9.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 16] 8.00-9.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 18] 8.00-9.00 sec 55.8 MBytes 468 Mbits/sec 0 163 KBytes
[ 20] 8.00-9.00 sec 56.0 MBytes 470 Mbits/sec 0 163 KBytes
[ 22] 8.00-9.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[SUM] 8.00-9.00 sec 562 MBytes 4.72 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 9.00-10.00 sec 56.2 MBytes 472 Mbits/sec 0 161 KBytes
[ 6] 9.00-10.00 sec 55.9 MBytes 469 Mbits/sec 0 161 KBytes
[ 8] 9.00-10.00 sec 56.0 MBytes 470 Mbits/sec 0 144 KBytes
[ 10] 9.00-10.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 12] 9.00-10.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 14] 9.00-10.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 16] 9.00-10.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[ 18] 9.00-10.00 sec 55.8 MBytes 468 Mbits/sec 0 163 KBytes
[ 20] 9.00-10.00 sec 56.0 MBytes 470 Mbits/sec 0 163 KBytes
[ 22] 9.00-10.00 sec 56.0 MBytes 470 Mbits/sec 0 161 KBytes
[SUM] 9.00-10.00 sec 561 MBytes 4.71 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 4] 0.00-10.00 sec 5.49 GBytes 4.71 Gbits/sec 0 sender
[ 4] 0.00-10.00 sec 5.48 GBytes 4.71 Gbits/sec receiver
iperf Done.
Result: Same ~4.7 Gbits/sec performance.
Trying with MTU of 9000
Both ports set to MTU=9000:
Local workstation:
$ ip address show dev enp4s0f4u1
3: enp4s0f4u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc fq_codel state UP group default qlen 1000
link/ether 34:c8:d6:b3:08:ac brd ff:ff:ff:ff:ff:ff
altname enx34c8d6b308ac
inet 192.168.130.10/30 brd 192.168.130.11 scope global noprefixroute enp4s0f4u1
valid_lft forever preferred_lft forever
And on the NAS:
# # ip address show dev eth2
5: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 9000 qdisc mq state DOWN group default qlen 1000
link/ether 90:09:d0:aa:83:d0 brd ff:ff:ff:ff:ff:ff
inet 192.168.130.9/30 brd 192.168.130.11 scope global eth2
valid_lft forever preferred_lft forever
$ iperf3 -p 5201 -c nas
Connecting to host nas, port 5201
[ 4] local 192.168.130.10 port 41914 connected to 192.168.130.9 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 4] 0.00-1.00 sec 507 MBytes 4.25 Gbits/sec 0 935 KBytes
[ 4] 1.00-2.00 sec 508 MBytes 4.26 Gbits/sec 0 987 KBytes
[ 4] 2.00-3.00 sec 505 MBytes 4.24 Gbits/sec 0 987 KBytes
[ 4] 3.00-4.00 sec 501 MBytes 4.20 Gbits/sec 0 1.02 MBytes
[ 4] 4.00-5.00 sec 500 MBytes 4.20 Gbits/sec 0 1.02 MBytes
[ 4] 5.00-6.00 sec 506 MBytes 4.25 Gbits/sec 0 1.07 MBytes
[ 4] 6.00-7.00 sec 502 MBytes 4.21 Gbits/sec 0 1.07 MBytes
[ 4] 7.00-8.00 sec 502 MBytes 4.21 Gbits/sec 0 1.07 MBytes
[ 4] 8.00-9.00 sec 498 MBytes 4.18 Gbits/sec 0 1.64 MBytes
[ 4] 9.00-10.00 sec 496 MBytes 4.15 Gbits/sec 0 1.64 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 4] 0.00-10.00 sec 4.91 GBytes 4.22 Gbits/sec 0 sender
[ 4] 0.00-10.00 sec 4.91 GBytes 4.21 Gbits/sec receiver
Result: Dang, a slight decrease, but possibly due to the larger overall packet size?
BUT!!! The "rsync" seems to be quite a bit faster, but also all over the place when it comes to the overall MB/s:
Write performance - local to NAS:
$ for C in $(seq 1 5) ; do rm -f /home/nfs/dan/tmp/ubuntu-26.04-desktop-amd64.iso && date && sync && rsync --progress /home/dan.local/iso/ubuntu-26.04-desktop-amd64.iso /home/nfs/dan/tmp/ ; done
Sat Oct 31 02:55:27 PM CDT 2026
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 808.13MB/s 0:00:07 (xfr#1, to-chk=0/1)
Sat Oct 31 02:56:02 PM CDT 2026
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 440.86MB/s 0:00:14 (xfr#1, to-chk=0/1)
Sat Oct 31 02:56:57 PM CDT 2026
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 680.04MB/s 0:00:09 (xfr#1, to-chk=0/1)
Sat Oct 31 02:57:46 PM CDT 2026
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 733.48MB/s 0:00:08 (xfr#1, to-chk=0/1)
Sat Oct 31 02:58:37 PM CDT 2026
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 650.65MB/s 0:00:09 (xfr#1, to-chk=0/1)
[dan@titan ~]$
The overall write speed comes out to 662MB/s.
Doing read performance from both locations:
Testing the local read:
$ time (for C in $(seq 1 5) ; do date ; time sha1sum /home/dan.local/iso/ubuntu-26.04-desktop-amd64.iso ; done ; echo Total time:)
Sat Oct 31 03:05:02 PM CDT 2026
9fd52c962c2823dc3d5379b18c189b468b4d6d1b /home/dan.local/iso/ubuntu-26.04-desktop-amd64.iso
real 0m3.271s
user 0m2.841s
sys 0m0.418s
Sat Oct 31 03:05:05 PM CDT 2026
9fd52c962c2823dc3d5379b18c189b468b4d6d1b /home/dan.local/iso/ubuntu-26.04-desktop-amd64.iso
real 0m3.266s
user 0m2.853s
sys 0m0.401s
Sat Oct 31 03:05:09 PM CDT 2026
9fd52c962c2823dc3d5379b18c189b468b4d6d1b /home/dan.local/iso/ubuntu-26.04-desktop-amd64.iso
real 0m3.269s
user 0m2.818s
sys 0m0.438s
Sat Oct 31 03:05:12 PM CDT 2026
9fd52c962c2823dc3d5379b18c189b468b4d6d1b /home/dan.local/iso/ubuntu-26.04-desktop-amd64.iso
real 0m3.269s
user 0m2.819s
sys 0m0.436s
Sat Oct 31 03:05:15 PM CDT 2026
9fd52c962c2823dc3d5379b18c189b468b4d6d1b /home/dan.local/iso/ubuntu-26.04-desktop-amd64.iso
real 0m3.282s
user 0m2.803s
sys 0m0.465s
Total time:
real 0m16.364s
user 0m14.135s
sys 0m2.164s
And now a read over NFS:
$ time (for C in $(seq 1 5) ; do date ; time sha1sum /home/nfs/dan/tmp/ubuntu-26.04-desktop-amd64.iso ; done)
Sat Oct 31 03:03:33 PM CDT 2026
9fd52c962c2823dc3d5379b18c189b468b4d6d1b /home/nfs/dan/tmp/ubuntu-26.04-desktop-amd64.iso
real 0m11.146s
user 0m3.501s
sys 0m0.894s
Sat Oct 31 03:03:44 PM CDT 2026
9fd52c962c2823dc3d5379b18c189b468b4d6d1b /home/nfs/dan/tmp/ubuntu-26.04-desktop-amd64.iso
real 0m3.279s
user 0m2.858s
sys 0m0.406s
Sat Oct 31 03:03:48 PM CDT 2026
9fd52c962c2823dc3d5379b18c189b468b4d6d1b /home/nfs/dan/tmp/ubuntu-26.04-desktop-amd64.iso
real 0m3.279s
user 0m2.816s
sys 0m0.449s
Sat Oct 31 03:03:51 PM CDT 2026
9fd52c962c2823dc3d5379b18c189b468b4d6d1b /home/nfs/dan/tmp/ubuntu-26.04-desktop-amd64.iso
real 0m3.280s
user 0m2.807s
sys 0m0.459s
Sat Oct 31 03:03:54 PM CDT 2026
9fd52c962c2823dc3d5379b18c189b468b4d6d1b /home/nfs/dan/tmp/ubuntu-26.04-desktop-amd64.iso
real 0m3.275s
user 0m2.841s
sys 0m0.420s
Total time:
real 0m24.267s
user 0m14.826s
sys 0m2.636s
Reading five times gives 24.267s over NFS, vs 16.364s local.
Heat problems?
I noticed when I went to re-run another test the speeds were decreasing:
$ time (for C in $(seq 1 5) ; do date ; rm -f /home/nfs/dan/tmp/ubuntu-26.04-desktop-amd64.iso && sync && rsync --progress /home/dan.local/iso/ubuntu-26.04-desktop-amd64.iso /home/nfs/dan/tmp/ ; done ; echo Total time:)
Sat Oct 31 03:10:55 PM CDT 2026
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 449.69MB/s 0:00:13 (xfr#1, to-chk=0/1)
Sat Oct 31 03:11:47 PM CDT 2026
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 289.55MB/s 0:00:21 (xfr#1, to-chk=0/1)
Sat Oct 31 03:12:25 PM CDT 2026
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 307.83MB/s 0:00:20 (xfr#1, to-chk=0/1)
Sat Oct 31 03:13:19 PM CDT 2026
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 251.13MB/s 0:00:24 (xfr#1, to-chk=0/1)
Sat Oct 31 03:14:06 PM CDT 2026
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 267.18MB/s 0:00:23 (xfr#1, to-chk=0/1)
Total time:
real 3m57.811s
user 0m9.038s
sys 0m25.878s
Admittedly, over the past ~10-15 minutes I've been doing a lot of big transfers and the metal case of the USB NIC is a bit warm, but not ultra hot.
I'll let this cool down and try later.
[Many hours later...]
After letting this sit all day while I was at work, I re-ran the speed tests:
$ time (for C in $(seq 1 5) ; do date ; rm -f /home/nfs/dan/tmp/ubuntu-26.04-desktop-amd64.iso && sync && rsync --progress /home/dan.local/iso/ubuntu-26.04-desktop-amd64.iso /home/nfs/dan/tmp/ ; done ; echo Total time:)
Mon Nov 2 06:05:29 PM CST 2026
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 598.01MB/s 0:00:10 (xfr#1, to-chk=0/1)
Mon Nov 2 06:06:15 PM CST 2026
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 503.93MB/s 0:00:12 (xfr#1, to-chk=0/1)
Mon Nov 2 06:07:07 PM CST 2026
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 730.20MB/s 0:00:08 (xfr#1, to-chk=0/1)
Mon Nov 2 06:07:53 PM CST 2026
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 666.13MB/s 0:00:09 (xfr#1, to-chk=0/1)
Mon Nov 2 06:08:42 PM CST 2026
ubuntu-26.04-desktop-amd64.iso
6,518,974,464 100% 542.73MB/s 0:00:11 (xfr#1, to-chk=0/1)
Total time:
real 4m14.499s
user 0m9.099s
sys 0m31.878s
Intersting - the total time (real 4m14.499s) was more but the MB/s were faster in each case...
USB settings
I came across this writeup (https://kohlschuetter.github.io/blog/posts/2026/04/28/rtl10/) and had some interesting things to check.
- Find the USB devices "Bus" and "Device" IDs:
$ sudo lsusb
[...]
Bus 004 Device 002: ID 0bda:815a Realtek Semiconductor Corp. USB 10/100/1G/2.5G/5G/10G LAN
- View the details of that specific Bus:Device identifier:
$ sudo sudo lsusb -v -s 004:002
Bus 004 Device 002: ID 0bda:815a Realtek Semiconductor Corp. USB 10/100/1G/2.5G/5G/10G LAN
Negotiated speed: SuperSpeed+ (10Gbps)
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 3.20
bDeviceClass 0 [unknown]
bDeviceSubClass 0 [unknown]
bDeviceProtocol 0
bMaxPacketSize0 9
idVendor 0x0bda Realtek Semiconductor Corp.
idProduct 0x815a USB 10/100/1G/2.5G/5G/10G LAN
bcdDevice 30.00
iManufacturer 1 Realtek
iProduct 2 USB 10/100/1G/2.5G/5G/10G LAN
iSerial 7 000334C8D6B308AC
bNumConfigurations 3
[...snip...]
Ah-ha moment...
See the Negotiated speed: line? It reports SuperSpeed+.
In the wacky hodge-podge of USB naming standards, that is a USB 3.2 Gen 2×1 (10Gbps).
To support the full 10Gbs bandwidth of the network adapter over USB, the port must provide more to provide some "room" (overhead) for the protocols. The next step up is 20Gbps, or the USB 3.2 Gen 2×2 standards name.
In the end.
Checking my system BIOS and manua confirm it - that is the best I'll get out of this system. The USB chips in my system only support the USB-C 3.2 Gen 2x1 (10Gbps).
So 5Gbps for me it is. 😅
```