2022-08-22

Switch to QNAP QXG-10G2SF-CX4 network card

In the last year's review I mentioned about my plan to switch to QNAP QXG-10G2SF-CX4 dual port SFP+ PCIe Gen3 x8 network controller from the Dell 540-BBGS one. The QNAP card is based on Mellanox ConnectX-4 Lx controller, which NetBSD support was the primary reason for the replacement.

Since I have already tested this card on my main machine using Manjaro Linux and NetBSD (which even led to one NetBSD specific bugfix), I was expecting a simple drop-in replacement (I use Artix Linux in my NAS server currently). Unfortunately, it wasn't meant to be, I faced two worth to mention issues.


Dell 540-BBGS and QNAP QXG-10G2SF-CX4

The non-standard LED positions

The first issue is related to non standard LED indicators' positions. Instead of being placed on the side of the ports, they are positioned on the top of the card. Because of this, lights are completely blocked once the card is installed. It is due to the fact that the QNAP card is designed for their own NAS devices, not for normal PCs, where likely those indicators are visible. I am personally not using LED indicators often, but it may be an issue for others. I believe it may be resolvable by drilling holes in place of LED's positions, however I left it unresolved this time (there are 4 LED lights, activity and speed indicators for each port).

In addition to that, I got confused with the low profile brackets. Two of them are included in addition to full size bracket: one is standard PC style bracket, another seems to be either QNAP or some server specific. Since I replaced low profile bracket with full size one to attach it to my main PC, I got surprised by the one I found in the box upon the switch: 1) it is not bent on the "upper" side to fix it to the case 2) it has a small hole for the screw instead of the groove in a bit lower position in comparison. I forgot, that it was also a standard low profile bracket in the package (which I didn't put back to the box), instead I used QNAP specific one. Since my case uses a hinge to fix attached cards instead of screws, I drilled a small hole on this hinge aligned with bracket's hole and used a screw to fix the card. Once I realized that I have low profile bracket for a normal PC, my setup was already complete...

QNAP QXG-10G2SF-CX4 network card with the low profile bracket
QNAP QXG-10G2SF-CX4 with the QNAP specific low profile bracket
Unsupported Mikrotik S+DA001 DAC cable

I do have two types of DAC cables, one is Mikrotik S+DA001 1 meter SFP+ cable, others are Ubiquiti UniFi UDC-3 3 meters cables. I used a mix of one Mikrotik and one Ubiquiti cables in my NAS , since the third one was attached to my main computer. This arrangement was required, because 1 meter was too short to reach my main computer from the switch device, however it could reach the closer placed NAS server. This cable worked well with all my previous network cards, even if Dell was showing warning regarding unsupported modules. Unfortunately, QNAP network card refused to work with it at all and showed an error instead: "port_module:249:(pid 0): Port module event[error]: module 0, Cable error, Unknown identifier". Thus, it is advisable to use either officially supported cables or make sure about their support in the community. Luckily, Ubiquiti cables are supported and work without any warnings. I needed to buy one more Ubiquiti cable to resolve the issue, which costs a bit over 20 euros for 1 meter cable with the price gradually increasing for the longer ones (up to 3 meters).

Summary

If blocked LED indicators are not an issue, the careful cable/module selection is the only important concern before buying this card. Otherwise, the card works well on all major operating systems out of the box without additional firmware packages needed. Software wise it didn't require any configuration changes, since interfaces are named by generic eth0/eth1 names in Linux, same as the Dell card. I only needed to update my MAC address in the router to make sure that the same IP address is assigned by DHCP server. This network controller also enables me for the potential transition to NetBSD in the future, and I do plan some initial testing later this year. Depending on the delayed NetBSD 10 release schedule, the final switch won't likely happen until the middle or even end of the next year though (in case of successful testing). Finally, I plan to add few iperf3 test results in next week or two, but I do expect similar results to the Dell card.

QNAP QXG-10G2SF-CX4 network card
QNAP QXG-10G2SF-CX4