2020 in review

The beginning of 2021 was very difficult for me and my family. In January I unfortunately lost my father because of COVID-19. This deeply wounded me, but at the same time it was a special and unforgettable moment for us, because my second daughter was born just a few days later, four days to be exact. All of this diverted my focus from IT for a while. Despite that, few months later, I would like to keep tradition to write a yearly review, even if it is late. In 2020 I had some projects like new NAS setup and 10Gbit network hardware. In addition, I also bought ICOP VEX2-6427-5C4NE board which was one of the first DM&P Vortex EX2 SoC based solutions in the market.

ICOP VEX2-6427-5C4NE

VEX2-6427 was one of the first publicly announced boards based on the DM&P Vortex86 EX2 SoC, which intrigued me a bit due to its unusual main/secondary two-core design. In addition to that, I was expecting that it may help me to resolve R6040 Ethernet issue on BSD systems for my other VortexDX3 system. Finally, I was hoping to use it as small firewall system. Unfortunately, it was not meant to be. There were several reasons for that:

• the BIOS is based on SeaBIOS without ACPI support. Initial firmware didn't have any user interface. Though ICOP support provided me a BIOS update with UI configuration, I needed to buy miniPCIe graphics card in order to update and utilize it (I bought IEI IGCME-1300-R10, since it was much cheaper than DM&P VortexVGA solution, mainly due to shipping costs. Also Aspeed AST1300 is theoretically supported by NetBSD, though I haven't tested that yet Update: no, AST1300 does not seem to have DRM support in NetBSD).
  
• The secondary CPU core is not accessible in this board, thus defeating the purpose of the unique SoC design.
• Due to limited CPU performance, PCIe bandwidth is also limited. My original plan was to use 1 Gbit Ethernet on mini-PCIe card, but it failed to meet my performance expectations (usually transfer speed was way below 200MBit/s with Intel or Realtek controller). Similarly mini-PCIe SATA card was also under-performing compared to modern PCs (if I remember correctly, SSD was reaching up to 50MB/s reading performance, way below its capabilities on the modern platforms).
  
• Due to different R6040 PHY model and lack of ACPI support, I couldn't utilize it much to investigate DX3 SoC issue. On the other hand, it worked well on NetBSD without any modification. I submitted a patch to recognize new PHY model but even general PHY driver worked as well.
• NetBSD required manually to fix COM IRQs in the kernel configuration to boot into the system because of IRQ conflicts. Initially between USB and serial port, but later with some miniPCIe cards too. Only with the updated firmware, I was finally able to reassign IRQs to my own needs. Nevertheless, to utilize all COM ports, manual kernel is still needed (similarly, Linux also requires some bootloader configuration changes to properly support all serial ports, if needed). 

I also stumbled into stability issues, however it appeared to be caused by my own mistake. Apparently, PC sets serial port TX signal to 0V during PC boot and it triggers the peer to go to the debugger. In order to resolve that, hw.cnmagic=+++++ needs to be added to /etc/sysctl.conf as a workaround (cnmagic(9)). The signal does not trigger debugger in this case. Because of all these reasons, grand plan for the system ended up unfulfilled. Nevertheless, I am still planning to use it as a serial terminal between different systems and possibly as local pkgsrc packages storage to keep it busy.

During this time I submitted patches to identify new CPU ID and new devices, thus NetBSD should be more verbose about those and attach rdcpcib (PCI-ISA bridge) driver with more DM&P SoC models.

Despite the board failing to meet my expectations, I was pleasantly surprised with ICOP support. It was very prompt, professional and helpful. Would I need to do some professional embedded hardware business, I would be really happy to deal with this company.

Assembled VEX2-6427 board with SanDisk X110 SSD and Delock miniPCe Ethernet

Mikrotik 10 Gbit router

Last year I was slowly moving to 10 Gbit internal network, connecting my NAS, router and main PC, as well as some other devices from time to time. For that purpose I bought a Mikrotik CRS309-1G-8S+ switch. It has 8 SFP+ ports and one 1Gbit port, as well as one serial port. The size is pretty compact, it uses passive cooling with external radiator connected to CPU using heat pipes. The switch is based on dual-core Marvell 98DX8208 800MHz ARMv7 CPU. The device is relatively cheap, but the price has some trade offs: though it is packed with quite a bunch of features, CPU is not very powerful do handle something more than simple configuration. Thus, I also didn't spend much on setting it up beyond bonding interface for my NAS server. Of course, I followed the security advice from their own page to have at least basic protection for the switch itself, but I guess that's it. Also, I've read somewhere that it is not recommended to use RJ45 modules, since they tend to emit lots of heat and may cause SFP+ cage to overheat. It is a bit of a disadvantage with the increasing RJ45 10GBit solutions (myself I am using DAC and fiber cables only) but likely can be partially overcome by custom cooling solutions. The major issue was initial connection to the switch, since even 1Gbit port wasn't configured as DHCP client, making it really challenging to reach from the internal router network. Hopefully, I could access it using serial port, but it took me a while to understand how to configure 1Gbit port to get IP address from my router. Afterwards, it was pretty easy to setup, including enabling web interface. Network aggregation configuration wasn't very intuitive as well, but trial/error approach worked eventually. I good part that Mikrotik RouterOS is updated pretty frequently. There is a choice between stable/testing/development and event long term releases. Thus, updates are the only reason I am connecting to this device.

MikroTik CRS309-1G-8S+IN

Asus XG-C100F

Asus XG-C100F became my third 10GBit SFP+ network adapter. Previously acquired Edimax EN-9320SFP+ and Dell QLogic 57810S are not supported by NetBSD, because of this I couldn't use them in my desktop system. Only Dell card is supported by FreeBSD but the driver is huge and I gave up on trying to port it pretty soon and repurposed it to Linux based NAS system later. Tehuti 4010 based Edimax is not supported by any of BSDs, though PRs for FreeBSD were submitted by the company. I believe it would have been easier to port, would it be accepted, but time constraint forced me to choose easier path with Aquantia based controller. On the time of purchase, the driver was just recently introduced into current branch, eventually it was back-ported to netbsd-9 branch with 9.1 release. On arrival, the card wasn't supported out of the box, since adapter's specific AQC100 MAC controller wasn't yet included in the driver list. Adding device ID made it to be recognized by the driver, but every reboot required to reattach the cable to make it work, which obviously was quite inconvenient. Fortunately, the solution was found quickly by Ryo Shimizu. Thanks to him, adapter is working without any issues now, except that I need to work on some configuration to optimize performance. It is not on par with Linux, especially on uploading which seems to be capsized to 1Gbit for some reason. It is a future project which I will try to base on NetBSD documentation and/or help in the mailing lists and IRC. During that time Linux driver also had a small bug, which was reporting TP (twisted pair) as supported port instead of FIBRE. I submitted a bug, it was fixed pretty soon after (however, it was likely discovered independently from my bug report). Besides that, adapter works well both in NetBSD and Linux, but the latter shows much better performance by default. I am not sure about the current state of OpenBSD and FreeBSD support, it was lacking from default kernel in both the last time I checked. FreeBSD likely has a vendor provided driver, OpenBSD one is in development I believe. 

Asus XG-C100F

NAS project

Following upgrade to 10Gbit network and increasing issues with the Marvel based SATA controller on ADPE4S-PB daughterboard, I eventually decided to upgrade my NAS computer from Jetway JNF76 to much more modern Biostar FX9830M motherboard. I described the process pretty well in my blog towards the end of 2020, thus I won't be repeating them here. However, there are few negative notes which came up over the time. I had at least one overheating issue, once I tried to execute long running software building process. Also, I can't call the system very silent, maybe due to pretty confined environment it is currently placed, but it has a constant background noise. It's not annoying but it is consistently audible. Finally, my active hard drive (non backup one), which is placed at the bottom of the case, seems to operate on quite high temperatures. Because of that, I may need to work on better cooling soon, especially since summer is just around the corner. I believe, even putting the computer on some elevated support may help with that. Other than that, it serves me well, as the system itself is stable and solid. Hopefully, I will manage to improve cooling and noise issues over time to avoid potential hard drive failures.

Biostar FX9830M

Smaller purchases and projects

Most smaller purchases last year were related to failed ICOP project, thus some hardware is unused now. As mentioned previously, IGCME-1300-R10 miniPCIe graphics card was used only to update BIOS and readjusting its settings. For testing purposes I successfully booted MX Linux distribution with it. I also bought two miniPCIe Ethernet cards. One is Realtek 8111F based Delock 2x1Gbit low profile card and one more unbranded Intel i350 based card. Delock has a disadvantage in taller pin type connections, while Intel based one has flat cable between miniPCIe card and backplane card with Ethernet connectors. On another hand, Delock cables are more flexible compared to thick Intel one. Unfortunately, I can't comment much on their performance, since both were performing similarly because of limited board's PCIe bandwidth. If I would have connected ICOP support before buying second card, I wouldn't have bought it. Initially I believed that poor performance is a Realtek controller issue, but it wasn't the case.

IEI IGCME-1300-R10

I also bought a miniPCIe 2xSATA controller (ASmedia ASM1061 based) and SanDISK X110 32GB SSD as a main storage, since SD cards were seemingly limited to very poor I/O performance. I used 2xUSB to SATA 15 pin power cable to power up the SATA SSD itself. This solution worked pretty well but, similar to Ethernet card read/write performance was quite limited. Nevertheless, it still was a tangible improvement over SD cards. I was pleasantly surprised that VEX2-6427 board supported direct booting from this SATA controller, thus SD card was not needed after upgrade.

I also had a chance to buy 8xKMM53616000AK-6 FPM parity SIMM memory modules with the hope to revive Microway PC164 Screamer Alpha board. However, it appeared to be either board or CPU issue. Memory was recognized but nevertheless board was spitting lots of errors on boot. Due to time constraints I abandoned attempts to revive the system for now.

One more small project was triggered by the death of my old FORTRON EPSILON 600W PSU last year. Luckily, I had one more PSU available in my stockpile (non modular), previously used in AMD Opteron 3280 based system, thus I decided to switch to it instead of buying a new one. Unfortunately, its cables were not designed for bottom placement, and most of them were too short to reach their designated connectors on the motherboard. Conveniently, all required extensions were available for purchase in local stores. For 24-pin power and 6-pin GPU cable extensions I bought SilverStone PP07-MBR and PP07-IDE6R red cables. These are extremely nice with individually sleeved wires for protection. For 8-pin power extension cable I acquired simpler Delock 83342 extention cable. Besides solving the short cables issue it also helped me organize them better, since I could utilize case design by wiring them through the right part of the case.

Finally, by the end of the year I bought the Clockwork DevTerm A04 Kit. It will be my first ARM based computer, but its current proposed shipment date is on early June, which is a slight delay since initially it was supposedly to be shipped by the end of March 2021. Hopefully, no more delays will occur but with this current shortage of microelectronics everything is possible. Nevertheless, I feel excited about it and I hope it will become a very useful device.

NetBSD endeavors

Besides mentioned patches for DM&P SoC and related RDC devices, last year wasn't very fruitful for me. R6040 PHY issue on VortexDX3 didn't move forward and I don't expect to work on it this year as well. I have reported some typos, submitted a patch to properly identify VIA PHY with MIIVERBOSE option, fixed small dmesg issue for viadrmums driver (actually reported last year and fixed this year), submitted a bug report on auvia(4) audio device driver, which also was affecting some other audio devices, also had some discussions leading to one bugfix for VIA PadLock engine, updated CodeLite package in pkgsrc to 14 version (and this year to 15). Over the year I attempted to work on porting bxe(4) driver from FreeBSD, but abandoned it, since driver is way too big and my skills on such work are too limited. I've also started some work on improving unfinished unichromefb driver with hopes to progress more this year (but no promises). This year I don't expect to do much as well but the usual bug reporting, small fixes and typos, pkgsrc package updates/fixes are expected. Just recently I updated to NetBSD 9.2 release!

Besides NetBSD I also contributed to Kwort 4.3.5 release which I use irregularly on one of my testing machines. Mainly it contributed to installation process improvements. It is really nice and very minimal CRUX Linux based distribution. Give it a try! I also reviewed previous 4.3.4 version last year.

Summary and 2021

The highlight of 2020 was local network upgrade to 10GBit leading to various new hardware and upgraded NAS system. Interesting and unusual DM&P VortexEX2 based system mainly ended up as a playground last year with some expectations to utilize it more this year. Finally I made a first step to ARMv8 world with Clockwork DevTerm device, which will hopefully arrive pretty soon. In 2021 I already had quite a few purchases. I upgraded my main CPU from Zen to Zen 2. I also updated my mom's computer to ASRock DeskMini X300 with Athlon 200GE CPU. I gave my wife a Pocketbook InkPad Color 7.8'' e-reader as a birthday gift. Finally, I bought multifunction HP LaserJet Pro MFP M183fw printer. All of these devices a worth to be mentioned in the next year's review, but may also have dedicated articles if time permits.

ASRock DeskMini X300 and AMD Athlon 200GE

 

In meantime I wish you all to stay safe and healthy during this difficult period.

Kwort Linux 4.3.4 review

In my endeavor to revisit lightweight Linux distributions I looked back on my old dream to try CRUX Linux distribution. Since my last attempt ended up in failure, I decided to look if there were any CRUX-based distributions around with the simplified installation process. With the help of DistroWatch.com filter I found that only two CRUX-based distributions existed. Between two of them, the Kwort Linux seemed like the one I may have been looking for. There were few uncertainties though:

• It wasn't clear if I would also need to build kernel during installation process similar to CRUX distribution.
• At that time the project homepage was down despite its relative recent release (2019-06-16), casting doubts if it will be maintained in the future.
• I wasn't sure how distribution's kpkg package manager operates, and if it is still maintained. Finally, I didn't know if I would be able to use CRUX ports system alongside with it. I believe together with BSD style init system it makes an essence of CRUX Linux and support for it was one of requirements to CRUX based distribution as well.

Nevertheless, I downloaded the ISO image from linuxtracker.org with the intention to try it out on my VIA EPIA-M900 motherboard later on. The ISO image is around ~917 MB in size. Fortunately, before my attempt, the homepage was partially restored with the documentation available in github (by the time I finished an article, the homepage is fully recovered and updated). It provided answers to some of those uncertainties except maintainability prospects of this small distribution. CRUX ports appeared to be supported to a certain extent, installation process didn't require to recompile the kernel as well. Furthermore, I also checked that kpkg mirrors were still updated regularly. All those factors combined gave a decent reason to try Kwort Linux.

The Kwort Linux did not have a "real" installer so to speak. Once booted, it automatically logins into console and gives a long message on the installation process. It can be always reprinted using "helpinstall" command. All the steps are manual: 1) first step is to partition the hard drive using one of fdisk/cfdisk (for MBR) or gdisk/cgdisk (for GPT) utilities. In my case I didn't need to partition hard drive since I just reused partitions used by previously installed SparkyLinux. 2) format partitions using mkfs. command; 3) mount those partitions to /mnt/install/ for root and /mnt/install/ for other partitions (if any); 4) run pkgsinstall command to install base packages; 5) use jumpOS command to chroot to your newly installed system; 6) finally configure your system by editing /etc/fstab, /etc/rc.conf (vim editor was available), creating users and installing bootloader (lilo or grub2 options are available). After all these steps, system installation is finished and computer can be rebooted.

In my case everything went smoothly and the newly installed system booted successfully. By default, kpkg package manager doesn't have any repositories, so the first command requires to install one. europa.kdb is available after installation and can be installed using kpkg instkdb /root/europa.kdb command. Alternative ctrl-c.kdb mirror was available here. After that, package manager is pretty simple to use kpkg update && kpkg upgrade will upgrade the system. Unfortunately, the package manager doesn't handle dependency management, thus, issues appeared right after the first upgrade. Several applications were failing to start citing missing library files. In all cases, I solved it by reinstalling those libraries. However, the challenge was to find the right package, as at times, few libraries were needed to be reinstalled before application started to work again. Another issue is that kpkg is pretty limited. During the time of writing this article, it had 203 packages only.

To partially circumvent missing packages, CRUX ports can be used. To achieve that, initially you need to install packages from CD image kpkg install /mnt/cdrom/packages_more/ports/*.tar.gz. After that, just refer to CRUX ports documentation. Unfortunately, not everything successfully compiled too and during kpkg upgrade, packages may interfere if versions doesn't match. Nevertheless, I successfully used ports system to install some applications unavailable in kpkg package manager mirrors (like wget). As a last resort, I compiled some packages manually from original repositories (like openchrome video driver).

To run Xorg you just need to run startx command. It did not seem that there's any display manager available as well. The default window manager is Openbox combine with conky and tint2. There are some distribution specific shortcuts like alt+z to lunch urxvt or alt+x to launch the browser. Actually, no alternative to Openbox was available in kpkg, though ports system provided some.

Probably, the final point is that though the kernel binary was provided during installation, no kernel updates are available. So, in order to update it, compiling manually is necessary. The installer provides 4.19.46 version in the ISO image. I successfully updated to 4.19.88 soon after installation.

In conclusion, I can say that Kwort is a really different distribution compared to the mainstream. It requires more manual work and research but in turn it gives one of the leanest systems available, fast boot process and complete control in your hands. It is suitable mainly for the experienced Linux user or the person who is not afraid to tinker with his/her system. It can be a perfect distribution for your secondary PCs, testing and rescue purposes or on dual boot as well. For desktop usage, one should have quite minimal requirements, since running typical heavy weight applications or games may be a challenging task. Here there is a nice response from a distribution creator and developer, along with its features and goals. It also has an IRC channel on OFTC network where you can reach him directly.

Default Kwort Linux 4.3.4 desktop