alt Hacker NewsI've done a lot of headless/diskless stuff. I haven't done much for years, because my NAS only has gigabit Ethernet ports. I can cascade them and get four Gbps downstream, but it's still painful.
I have recently upgraded my house to 10Gbps Ethernet, with only one room still stuck at gigabit, and unfortunately, it's my main office. I'm working on getting the drop there now (literally, just taking a break here).
Even once I'm done, accessing an iSCSI drive over 10GbE will be 4-8 times slower than a local NVMe drive, but it will sure be a lot better than it was!
Ideally, I could run VMs on the NAS and have great performance, but that's another hardware upgrade...
Replies
Really I wonder how this turns out to be diskless while you're clearly accessing a disk/drive over the network. Shouldn't we refer to this as network boot?
Using a proper NIC (Chelsio) with their iSCSI accelerator will boost your iSCSI performance significantly. Another alternative is Mellanox with RDMA. You need CX4+ for optimal performance over TCP/IP, while the cheap CX3 is excellent with IPoIB. If you have a lot of packet drops and retransmissions, another option for boosting iSCSI performance is getting a network switch with a lot of memory for packet buffering. This helps with incast congestion. There are special switches with gigabytes of memory built for this.
NVMe-oF is the best protocol with least overhead for network drives, with a proper setup you lose only 10-20% latency compared to local disk even with Intel Optane. Throughput should be almost similar.