Switch to QNAP QXG-10G2SF-CX4 network card

In the last year's review I mentioned about my plan to switch to QNAP QXG-10G2SF-CX4 dual port SFP+ PCIe Gen3 x8 network controller from the Dell 540-BBGS one. The QNAP card is based on Mellanox ConnectX-4 Lx controller, which NetBSD support was the primary reason for the replacement.

Since I have already tested this card on my main machine using Manjaro Linux and NetBSD (which even led to one NetBSD specific bugfix), I was expecting a simple drop-in replacement (I use Artix Linux in my NAS server currently). Unfortunately, it wasn't meant to be, I faced two worth to mention issues.

Dell 540-BBGS and QNAP QXG-10G2SF-CX4

The non-standard LED positions

The first issue is related to non standard LED indicators' positions. Instead of being placed on the side of the ports, they are positioned on the top of the card. Because of this, lights are completely blocked once the card is installed. It is due to the fact that the QNAP card is designed for their own NAS devices, not for normal PCs, where likely those indicators are visible. I am personally not using LED indicators often, but it may be an issue for others. I believe it may be resolvable by drilling holes in place of LED's positions, however I left it unresolved this time (there are 4 LED lights, activity and speed indicators for each port).

In addition to that, I got confused with the low profile brackets. Two of them are included in addition to full size bracket: one is standard PC style bracket, another seems to be either QNAP or some server specific. Since I replaced low profile bracket with full size one to attach it to my main PC, I got surprised by the one I found in the box upon the switch: 1) it is not bent on the "upper" side to fix it to the case 2) it has a small hole for the screw instead of the groove in a bit lower position in comparison. I forgot, that it was also a standard low profile bracket in the package (which I didn't put back to the box), instead I used QNAP specific one. Since my case uses a hinge to fix attached cards instead of screws, I drilled a small hole on this hinge aligned with bracket's hole and used a screw to fix the card. Once I realized that I have low profile bracket for a normal PC, my setup was already complete...

QNAP QXG-10G2SF-CX4 with the QNAP specific low profile bracket

Unsupported Mikrotik S+DA001 DAC cable

I do have two types of DAC cables, one is Mikrotik S+DA001 1 meter SFP+ cable, others are Ubiquiti UniFi UDC-3 3 meters cables. I used a mix of one Mikrotik and one Ubiquiti cables in my NAS , since the third one was attached to my main computer. This arrangement was required, because 1 meter was too short to reach my main computer from the switch device, however it could reach the closer placed NAS server. This cable worked well with all my previous network cards, even if Dell was showing warning regarding unsupported modules. Unfortunately, QNAP network card refused to work with it at all and showed an error instead: "port_module:249:(pid 0): Port module event[error]: module 0, Cable error, Unknown identifier". Thus, it is advisable to use either officially supported cables or make sure about their support in the community. Luckily, Ubiquiti cables are supported and work without any warnings. I needed to buy one more Ubiquiti cable to resolve the issue, which costs a bit over 20 euros for 1 meter cable with the price gradually increasing for the longer ones (up to 3 meters).

Summary

If blocked LED indicators are not an issue, the careful cable/module selection is the only important concern before buying this card. Otherwise, the card works well on all major operating systems out of the box without additional firmware packages needed. Software wise it didn't require any configuration changes, since interfaces are named by generic eth0/eth1 names in Linux, same as the Dell card. I only needed to update my MAC address in the router to make sure that the same IP address is assigned by DHCP server. This network controller also enables me for the potential transition to NetBSD in the future, and I do plan some initial testing later this year. Depending on the delayed NetBSD 10 release schedule, the final switch won't likely happen until the middle or even end of the next year though (in case of successful testing). Finally, I plan to add few iperf3 test results in next week or two, but I do expect similar results to the Dell card.

QNAP QXG-10G2SF-CX4

2021 (and beyond) in review

This article is incredibly late this year, thus it touches some events from the first half  of 2022 as well. The blog was quite dormant last year due to personal factors, however, they were few interesting developments in hardware/software side, including new purchases.

HP Color LaserJet Pro MFP M183fw

The need for the printer was getting more and more difficult to ignore, because of this I finally decided to buy one last year. Since, I got burned by the inkjet printers in the past, I have been looking for the laser one this time. Preferably, I wanted a multi-function device with a scanner, with options for network connection, including WiFi support. Colour support was a must too. Despite having pretty clear requirements, it proved to be pretty challenging to find one with all those features available. Laser printers tend to be business oriented, and they tend to be big, heavy, as well as expensive. In addition to previous expectations, I was also was trying to ensure that printer supplies will be readily available at least in the near future. There were some attractive options until I have checked availability of toners' cartridges. Eventually I ended up choosing HP Color LaserJet Pro MFP M183fw multi-functional printer/fax for around ~300 euros at the time of purchase. I definitely don't need fax, but other than that, it had all the points covered above. It even has Web UI interface for management. The biggest disadvantage is an old style physical printer management panel with the small screen. It is a bit cumbersome to use, but the dimensions are relatively compact, just slightly bigger than a typical ink-jet multi-functional devices. Windows and Linux were quite well supported, I could print and scan using network connection, however scanning quality on Linux was disappointing, forcing me to use Windows, if detailed scanning is needed. Printing is fine in both OSes though. I didn't manage to make it work in NetBSD (using network), however it was not essential for my case, thus I didn't try too hard. It's neat feature is it's ability to print from the Android phone, however it requires to create and use an HP account, which is honestly in my opinion, an unnecessary prerequisite. What is more, the HP app has issues with login operation, which redirects to the browser first, doesn't authenticate upon returning to the app. Eventually, I found out that only Opera mobile browser allows me to perform the operation. Multiple application updates didn't resolve the issue as of today. In conclusion, I am quite satisfied with the printer despite all the oddities, but some peculiarities can be a show-stopper for others, especially for Linux users. Finding cartridges may also be more difficult than I would like to, but supplies are available to buy the last time I have checked. The device is also affected by CVE-2022-3942 vulnerability without any patches available except recommendation to disable LLMNR feature, which I followed myself.

QNAP Dual-port SFP+ 10GbE network

Last year I bought one more 10Gbit network card. This time it was QNAP QXG-10G2SF-CX4 dual port SFP+ PCIe Gen3 x8 card for around 180 euros at the time of purchase. It is based on Mellanox ConnectX-4 Lx (now Nvidia) network controller, which attracted me to purchase it. This controller is supported by NetBSD and gives me an opportunity  to migrate my NAS server to this OS, as it was originally intended a decade ago :). Not sure if it will really happen soon, but the network card was the major missing piece in the puzzle. I may do some trials this year, but the final decision depends on various performance, stability and compatibility factors. On NetBSD I hit an issue that it wasn't working on current branch (future NetBSD 10 release), though it did on netbsd-9 one. After some investigation, it appeared to be due to my CPU core count, which was not a a power of 2  (6/12 cores/threads). RQ Table size was based on number of cores, though controller requires it to be a power of 2 by reference manual. Thus the patch was applied to ensure the correct number. Currently the network card is not actively used, but I am planning to swap it soon. Dell QLogic 57810 network card will go to a well deserved rest or will be sold/donated.

QNAP QXG-10G2SF-CX4 network controller

WD Blue SSD SN550 NVMe 1TB

The final significant purchase was made early this year in the form of WD Blue SN550 NVMe M.2 SSD.  I bought it for around ~100 euros. SSD doesn't have the most modern specs, nevertheless it is considerably faster than my SATA based SSD, at least on paper. I also preferred bigger capacity over more modern specifications. It is currently used as a major storage for my desktop NetBSD installation and I didn't have any visible issues with it so far. I haven't performed any speed tests though, thus I can't provide any numbers for comparison unfortunately.

WD SN550 NVMe SSD

ClockworkPi DevTerm

One of the most anticipated devices last year was a DevTerm computer. I bought the A04 model by the end of 2020, but this specific configuration appeared to be the last one to be ready for shipment. While RPi CM3 or A06 based models were shipped starting August/September of the last year, A04 shipments started around late 2021, even making some people anxious about that. I received mine in the beginning of this year. However, the whole process didn't turn out perfectly as I wanted it, starting from incorrect tracking info to a broken screen upon arrival. Moreover, I also damaged the WiFi antenna during assembly. Luckily, ClockworkPi team sent me replacements promptly without additional charge! Though I needed to wait again, I finally managed to fully assemble it as a complete working device by the end of January. A04 model is based on AllWinner H6 SoC and comes with pre-installed Clockwork OS (Armbian) Linux distribution. The device is really cute and nice, screen seems to be a bit small, but keyboard is surprisingly usable. I was hoping to use it pretty frequently, however it is not the case as of now. I did a one time attempt to boot NetBSD as well, however unsuccessfully. Likely it should be possible to run (considering that SoC is supported), but probably requires some work on boot configuration and properties, as it is the case for various specific ARM devices. Pre-installed Linux also had some peculiarities, for example, not all cores are enabled by default. It requires small script to enable them. I didn't manage to make HDMI work yet, but I believe this issue was discussed extensively in ClockworkPi forums and possibly solvable. Also I needed to install an app (or script) to change screen brightness with a simple shortcut, otherwise it was usually too low by default. Sound works OK. Battery life with moderate usage is quite short unfortunately, but it can be partially resolved by having at least two pairs of batteries, which can be easily replaced even on the fly. I hope to utilize it more in the future, but only time will tell.

ClockworkPi DevTerm

Biostar FX9830M NAS

At the end of 2020 I upgraded my NAS server to Biostar FX9830M motherboard and throughout the year it served me pretty well. I feel that it is bit too noisy, however, changing the CPU fan is a bit challenging because of an unusual pin connector. Nevertheless, I would like to find some solution to address the problem in the future. Few solutions are in my head, from a bigger computer case to improve airflow to even fan-less solutions. Some of those would need a few creative approach on how to install them. 

Software wise Artix Linux proves to be pretty stable with a major issue being abrupt Linux kernel changes at times. The network interfaces were renamed twice for an unknown reason over a year or so, which was rendering my network aggregation configuration to fail, since it was relying on those names. Another more mysterious issue is likely related to dhcpcd or OpenRC init system in the a way that IP address is not assigned at boot, which is only solved by rerunning dhcpcd service manually (even reboot doesn't help). After that it usually works OK on subsequent reboots (happened twice or three times over a year and half). Finally, once it stopped working because of missing firmware. The solution was to install a separate linux-firmware-bnx2x package, though it wasn't required before. I guess these issues can be partially attributed to rolling release distribution approach with the most up to date kernel and software.

Akasa Cyper SPX case

This year I acquired Akasa Cyper SPX case to accommodate my ICOP VEX2-6427-5C4NE board. This case has openings to install 4 serial ports, which seemed a pretty good choice for me (considering my board has exactly 4 serial headers), but it ended up to be a bit of a disappointment. The inside design has a piece of plastic sticking out, which prevented me from installing serial ports with rectangle shaped frames in the far left and right openings. Thus, I ended up utilizing only the two middle ones. Something more lean and rounded is needed for the edge openings, which I hope to either find them or modify the ones I own in the future. Unfortunately, the Akasa page does not provide a compatibility list or requirements and their customer support is unresponsive.

The board itself is not Mini-ITX, but a much smaller 3.5'' single-board design, because of this I tried to utilize board pads and some glue with limited success. Though I managed to position it successfully, the board is not very firm, making attachment of USB devices a bit challenging.

ICOP VEX2-6427 board placed inside Akasa Cyper SPX case

Other devices 

I did mention some devices on the last review like Pocketbook InkPad Color 7.8 and AsRock DeskMini X300, however I didn't have enough time to do any useful review about them. Pocketbook is quite a nice e-reader device, but little slow, especially with big PDF files which are usually available in colours, limiting its usage. It feels like such devices should have stronger hardware, however it likely brings challenge to keep long battery usage and increases the price. 

DeskMini X300 is definitely a good choice for the mini computer, if limited expansion options are not an issue. CPU support was extended to Ryzen 5000G series over the time, thus it can opt very performant models. In my case, Athlon 200GE CPU is installed and it is virtually silent. Probably major limitation was that initial configuration didn't have WiFi, side USBs and audio jack at back (only at the front). ASRock provides its own sets for all, but they are pretty expensive and hard to get. For side USB/WiFi though third party options can be adapted.

Other than that, I had some smaller purchases, but those were usually required to facilitate certain setup, like mini-PCIe SATA controller with the flat ports instead of 90 degrees ones or more silent Cherry G80-3000N mechanical keyboard.

ASRock X300M-STX board inside DeskMini X300

Work on NetBSD

Last year I was honoured to become an official NetBSD developer, which gave an ability to contribute to the project directly! At the time of writing this review, I already did over 200 commits, however those were mainly non functional changes like misspelling fixes. Nevertheless, I did some bug fixes too. Likely the main achievement was to finally identify why R6040 MAC was failing to work on Vortex86DX3 SoC. It took me countless hours in the span of around three years of debugging, including temporarily acquiring on more DX2 based system, but I managed to find out that it was related to certain speed control register (MDC) being restored to wrong value after reset action. This issue appeared to be the same not only across all BSDs but in Linux as well. Despite the fact, that interface was working up to 5.3 kernel version by luck, it was because Linux didn't check BMCR register on identifying the link state. That changed starting 5.3, thus R6040 MAC stopped working in Linux as well. Fortunately solution was just to restore MDC value back to original value leading to restored functionality. Just recently this change was applied to OpenBSD and FreeBSD as well, thus R6040 should should work properly on VortexDX3 SoC in all BSDs and Linux (I guess FreeBSD driver will be synced to Haiku OS code at some point). It was the first fix, which was actually reapplied to all major operating systems and motivates me to continue on such efforts. Other than that, I did few other fixes, mainly in drivers, I also did various testing/investigation work, wrote some missing documentation. Unfortunately, time constraints prevent me to work on some drivers more seriously, but I still have few projects in my mind, hopefully over the time I will manage to implement them. Nevertheless, I plan to stay an active member.

 

Future developments

I do not plan any major hardware purchases this year, however some upgrades and changes are possible. The main focus will likely to stay on the NetBSD project, which comes closer to branching netbsd-10. This may lead to more testing and debugging work and potentially some help in different areas. I am still trying to debug interrupt issue on Vortex86DX3 systems. On the other hand, I still dream to dig up and revive at least one of my Alpha machines. Hopefully, DevTerm will find more usage as well. Though I don't expect much articles in the next 6 months, I will make an effort not to be completely silent and share more interesting developments.