10 August, 2023

Thin client router part3: Bare metal atom


"Bare Metal Atom" is the name of my Skrillex cover band... OK, maybe that joke's not ready for a revival yet.

In this part of my Dell Wyse 3040 thin client hacking series I'm going to bite the bullet and run bare metal OpenWRT on the 3040. In part 1 of the series I showed all the things I needed to do to run any software other than the inbuilt ThinLinux client software. Then I messed-about with a fairly pointless (but personally satisfying) exercise in running a virtual machine of OpenWRT on it in part 2.

Where we're up to

As previously mentioned, the Z-series Atom chips aren't quite compatible with "fat" x86. These CPUs were targeted at "mobile" applications (phones and tablets), but never really found much power/price/performance success. Anyway, it is possible to compile the Linux kernel with Atom support and that's what I'm going to do.

A few seconds googling led me to a helpful GitHub project by Paul Jobson for an Atom build of OpenWRT. I downloaded his prebuilt v22.03.2 image and started messing around with it.

Getting distracted by Ventoy shenanigans

I've found the easiest way to test out any old x86 boot image or ISO is to just dump it onto a Ventoy USB drive and go from there. Sadly when I try to boot this image using Ventoy,  I get a slightly confusing error about OpenWRT:


Having a look at that help page, it seems to be telling me I need to download an extension containing missing kernel modules and put them into the "/ventoy" directory on the USB device. The instructions are a bit hard to understand, but it seems that the "ext4" images should work as-is now, but "squashfs" images need a script run over them.I tried following these instructions but sadly Ventoy still didn't want to play ball. However I could start one of the official OpenWRT x86 images and I can get as far as the GRUB menu, but it locks-up shortly after that (Again presumably because of the CPU incompatibility).

This had now entered the "too hard" basket, so I gave up and just installed the pre-built Atom image directly onto a spare USB drive. It booted-up just fine, but the USB install is a bit limited because the default behavior is to have the single ethernet port in the "lan" bridge (serving DHCP out to its clients) and it doesn't have any wireless packages installed. So the fact it will boot and run is encouraging. I will now have to build a custom image from source to get the system I want.

Building from source

Paul's README.md briefly outlines the process for building the images by firstly setting-up a new build environment following the OpenWRT guide. (however, note that I'm using Fedora and the current version of it does not include the Perl package by default, so I had to install that in addition). I did this in a new Fedora38 vm to keep it isolated (I could have used a container, but in the early stages, this would be something of a longer-lived "pet", so I used a VM).

I then ran through the git clone/pull/tag steps and checked-out the "v22.03.5" tag (the latest stable version as I wrote this). Following through the outline, at the "make menuconfig" step, I added in a couple of options as "compiled in" features (not modules):

  • Kernel modules → Wireless drivers → kmod-mt7921u - This is the kernel driver I needed build for the target Atom CPU to support the ComFast WiFi dongle. You might want drivers for different WiFi devices (I'll leave that up to you to sort out).
  • Network → WirelessAPD → wpad-wolfssl - I thought that as I was baking it in, I might as well use the "full fat" wpad. This supports access-point (hostapd) and client (supplicant) modes with all types of security (including the enterprise ones). Having SSL allows for WPA3 security and the WolfSSL version (vs. OpenSSL) sounds like it'll be more efficient/performant (Purely a guess based on their marketing). There's relatively heaps of room (both ram and storage) spare on this hardware compared to most WiFi routers, so I don't need to scrimp on package alternatives.

 Then I continued following Paul's setup guidelines (Atom CPU, no serial port, etc.).

Pre-configuring DHCP ≠ WAN

Paul's outline tells you to disable the DHCP server on the "lan" bridge so it doesn't conflict with your existing lan's DHCP server (implicitly assuming that you use a 192.168.1.1/24 lan?) . I don't want to do that for 2 reasons:

  1. I want to use the single physical ethernet port as the "wan" interface, being a DHCP client to the external network.
  2. I want the "lan" bridge to just cover the WiFi interface and it needs to retain its own DHCP server.

 So to that end I want to customise the two config files /etc/config/network and /etc/config/wireless to be set up this way. It's probably possible to tinker with the scripts and defaults that create these files to make them how I want. However it's easier just to make pre-built copies of these files and install them in the build root under "files/etc/config". The OpenWRT build system will just copy this files into the built image for us. The two files are below:

files/etc/config/network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config device
        option name 'br-lan'
        option type 'bridge'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'

config interface 'wan'
        option device 'eth0'
        option proto 'dhcp'

files/etc/config/wireless

config wifi-device 'radio0'
        option type 'mac80211'
        option phy 'phy0'
        option band '5g'
        option htmode 'HE80'

config wifi-iface 'default_radio0'
        option device 'radio0'
        option network 'lan'
        option mode 'ap'
        option ssid 'OpenWrt'
        option encryption 'none'

NOTE: The intention here is to default to an open WiFi access point called "OpenWrt". That will be a massive security hole if you plug it into your private lan and turn it on. Ideally someone using this image would set it up stand-alone first to set a wireless key and root password, then connect it to the private lan. If you want to bake-in a pre-shared key (PSK) and/or different station ID, you should consider changing the last couple of lines of the wireless config to something like:

        option ssid 'RadioFreeTuggeranong' # Station name
        option encryption 'sae-mixed' # WPA2/WPA3 security
        option key 'Sekrit-Key' # Plain-text encryption key

You might chose something like "sae" (WPA3) or "psk2" (WPA2) for the security mode.

Having configured the kernel and options, I can proceed with the build as-per the guide. This prompts for four new options during the build and I just take the defaults.The build proceeds and takes about 30m for the first build and 10m for subsequent builds in my virtualised environment.

Deploying

Once the build is complete, the resultant binaries will be under "bin/targets/x86/64". I'm only interested in combined bootable disk images and the 3040 is EFI capable, so I'll choose one of the "EFI combined" images.

EXT4 or SquashFS?

The big decision is whether to use the EXT4 or SquashFS images. While the 3040 does use NAND nonvolatile storage (that will eventually wear out), the eMMC controller should look after wear-levelling and mapping-out bad blocks. Nevertheless, the SquashFS image has the benefit of being a bit easier to expand to fill the full eMMC device later on - so I'll use that. Both images "work alike" and the EXT4 image is a drop-in replacement if you prefer it. If I really wanted to bubble wrap the SSD, I might consider using the "Extroot" setup with a separate, replaceable USB flash disk to take the write load (I did have a half-hearted attempt at this, but it didn't seem to work for me). To be honest, I don't fully understand how the SquashFS build does some of its magic and there's a whole rabbit hole of flash filesystem research I'd have to go down to understand it (a blog post for another day?).

Install image

We need to be able to run a live OS to be able to write the target OS image to the eMMC device. Pretty much any modern Linux "live CD" image should do, but I've been using Clonezilla running off the abovementioned Ventoy USB drive. You also need access to your install image (e.g. as a file on a second USB drive you can mount at runtime). Then it's just a case of running:

dd if=${SRC_IMAGE} of=/dev/mmcblk0 bs=512

You could append "status=progress" if your version of dd supports it and you want to see it doing stuff. Note also that some OSes might enumerate the eMMC device with a different name, so double-check that first with something like lsblk.

Fix & resize the partitions

Now I have something like a 100M EFI disk image written to an 8G eMMC device. There's also some misconfiguration of the partition sizes and they're in a weird ordering too. I want to fix these issues before going live, so I'll firstly use gdisk to fix the partition numbering and sizes.

I run "gdisk /dev/mmcblk0" and complains about the partition size mismatch (This will be fixed when the partition table is written). I'll also use the "s" ("sort") option to re-index the partitions in the order they appear on the eMMC (i.e. #1 is the EFI partition, #2 is Grub boot and #3 is the root filesystem). Selecting "w" ("write") will prompt to write-out the fixed partition and then exit.

An alternative is using fdisk and entering  "x f w" ("Select the extra-functionality commands, fix the partition ordering and write-out the partition table).

I also used the command line "growpart /dev/mmcblk0 3" (alternatively "parted /dev/mmcblk0 resizepart 3 100%") to expand the root partition to fill the full eMMC device. If you're using the EXT4 image, you'll also need to execute "resize2fs /dev/mmcblk0p3" (Not necessary for the SquashFS image).

Remember that the eMMC device might enumerate with a different device name other than "mmcblk0" in different OS environments. Make sure you use the actual eMMC device name.

Reboot into operation

So I can now reboot into the new environment and there's an open access point called "OpenWrt" NAT-routed to the ethernet port's network. The first thing to do would be to (Using LuCI web interface or console):

  • Set a root password (console: "passwd")
  • Set WiFi security/password (console: "uci set wireless.default_radio0.encryption='sae-mixed'; uci set wireless.default_radio0.key='wifi-password'" - possibly other security modes if you prefer)
  • Change the WiFi access point name (console: "uci set wireless.default_radio0.ssid='ap-name'")
  • Possibly set the console to require a password to log in (console: "uci set system.@system[0].ttylogin='1'")
  • Install available updates (console: "opkg update && opkg install $(opkg list-upgradable|awk '{print $1}')")

NOTE: If you've used any "uci set" commands you'll need to enter "uci commit" to make them persistent.

Once you've done all that, you'd be best to reboot to pick-up all the changes and it's then up to you what other customisations you want.

How does it perform?

Firstly, trying out the two external speed test tools I used before, I can see I'm getting close to line-rate on my ISP's promised 100Mb/s speeds:



So far, so good - but what about local transfers, perferrably using WiFi6 devices to theoretically maximise throughput? For that I'll need to use iperf and make sure I can use a WiFi6 client.

Looking at the OpenWRT radio status, good things seem to be happening:

 

I get a similar report when I get the connection status from a Windows laptop with WiFi6 capability.

A simple test using iperf3 on a laptop in UDP mode going to the 3040 access point yields a fairly consistent 104/105Mb/s:

$ iperf3 -u -c 192.168.1.1
Connecting to host 192.168.1.1, port 5201
[ 5] local 192.168.1.180 port 38593 connected to 10.20.30.3 port 5201
[ ID] Interval Transfer Bitrate Total Datagrams
[ 5] 0.00-1.00 sec 129 KBytes 1.05 Mbits/sec 91
[ 5] 1.00-2.00 sec 127 KBytes 1.04 Mbits/sec 90
[ 5] 2.00-3.00 sec 129 KBytes 1.05 Mbits/sec 91
[ 5] 3.00-4.00 sec 129 KBytes 1.05 Mbits/sec 91
[ 5] 4.00-5.00 sec 127 KBytes 1.04 Mbits/sec 90
[ 5] 5.00-6.00 sec 129 KBytes 1.05 Mbits/sec 91
[ 5] 6.00-7.00 sec 127 KBytes 1.04 Mbits/sec 90
[ 5] 7.00-8.00 sec 129 KBytes 1.05 Mbits/sec 91
[ 5] 8.00-9.00 sec 127 KBytes 1.04 Mbits/sec 90
[ 5] 9.00-10.00 sec 129 KBytes 1.05 Mbits/sec 91
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.00 sec 1.25 MBytes 1.05 Mbits/sec 0.000 ms 0/906 (0%) sender
[ 5] 0.00-10.10 sec 1.25 MBytes 1.04 Mbits/sec 0.212 ms 0/906 (0%) receiver

I also tried using scp to transfer a large file from a USB3 RAID array and usually got about 9.4MB/s, which is on-par with the bit-rates from the iperf tests.

I get identical results if I test going to a server on the WAN side of the 3040, so the Ethernet and routing side of it don't appear to have any effect on throughput. I tried all sorts of devices and also with the 3040 OpenWRT router, another two separate WiFi4 and WiFi 6 routers. They all seemed to be getting similar throughputs, so I'm not sure if I'm "doing it right" or there's really not much benefit of WiFi6 over 4?

Range?

As for range testing, I applied the highly scientific method of "walking with my mobile phone to various approximate distances from the router on various sides and number of intervening walls". It seems to work fairly reliably within about a 10m radius, but much beyond that and my phone reports "No Internet connectivity" even though it can see and access the AP. A WiFi scanner can see the AP on and off up to about 30m, but if I just cradle the phone in my hand (Like I normally do when using it), the signal usually disappears.

Overall not great, but reasonable considering the ComFast WiFi dongle only has two dinky-little fold-up diversity antennas. It also wouldn't surprise me if these are just inactive plastic "placebo" antennas anyway (I have had that in the past with some wireless dongles, but I'd hope the ComFast brand is above that).

Download the binaries

I've built 3 binary images and hopefully Google Drive is a reliable way to host them publicly. All of the images have had their EFI layouts "fixed" and re-ordered the partitions. The volumes just need to be expanded once installed on the 3040's eMMC drive (or some other Atom system, possibly). I've chosen only the combined/EFI images as we want a bootable system and EFI is the most "modern" way of doing it for the 3040. Note that all the images are compressed, so you'll need to uncompress them before installing them. The three images are:

Authored by M0les circa 7:35 AM

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.

Subscribe to: Post Comments (Atom)