Ordinary Men

I was reading Christopher Robert Browning's Ordinary Men recently.

It's a sobering thought that ordinary men can be made to behave so badly. That the inclinations to morality, empathy, compassion and decency are so feeble that they can so easily be overcome.  That aggression and hatred are so easily stirred, to become dominant in so many, with so little resistance, comment or criticism.

What is happening in the United States now is not yet as evil, but it is already far from the civil, decent, compassionate and caring society that made so many in the world envy and long for it. The rule of law and due process seem on the wane, leaving only brute force. Ordinary men, turned brutal and violent and uncaring.

There is some criticism, but not nearly enough to stop it. Barely enough even to slow it. Which makes one wonder how far will it go this time, before there is some real resistance? How many will die? How many will suffer.

There is a long history of violence and destruction. It is far easier to destroy and terrorize than to make a great, civil society. There are very few left who experienced the evil of WWII and they are now too old and feeble to have much influence. To resist the inclination to violence and evil. And history shows us that without such resistance, violence is almost inevitable. The inclination to it is too much a part of human nature.

Civilization requires active support and defense, else it will crumble and decay and be overcome, yet again. The time is now. 

Even ordinary men. 

NZ PC Clearance

I just received a 'new' laptop from NZ PC Clearance. 

I placed the order Friday morning, about 09:00. Promptly received an email confirmation. The order included replacing the SSD so, presumably, they had to open one to put in the new SSD.

Confirmation that the package had been picked up by NZ Post came Monday at about 17:20. That's not bad turnaround.

And it was delivered today, Tuesday morning, before 11:00.

It's a refurbished Lenovo ThinkPad T590 (20N5). i5 processor, 16GB RAM and 1TB SSD. It came with a power adapter. Nothing else in the box. Windows 11 Pro - pre-installed. I haven't put it on the network, but it already wants to install updates, nagging me for a time to install.

Cosmetically, it looks good. Some of the key caps are slightly worn, but no more than I wear them on a new laptop in just a few months. The TrackPoint is a bit discoloured - like black permanent marker on the bottom quarter. But all the little bumps on the surface are fine. And it works well. There's some very slight scuffing around the edges. A bit more than my three year old Dynabook, but not really noticeable, unless I'm looking for them. No defects on the screen that I can see. So, practically speaking: nearly as new condition. I'm very pleased. I was worried it would be all banged up and worn out.

I don't think anyone sells them new any more. I saw a review listing a unit with i7 processor, 16GB RAM and 512GB SSD for 1,800 euros. Another with i5 processor, 8GB RAM and 256GB SSD for US$1,469.99 at Best Buy (sold out - but I guess that's the price they were selling them at new). So, under NZ$900, including the new SSD, shipping and taxes seems a reasonable price.

I hadn't dealt with NZ PC Clearance before and was hesitant about buying a refurbished laptop, but thus far, I'm very pleased. 

Ubuntu VM on Windows 11

Recently I had need of an Ubuntu system for developent, in an environment where Windows is the only operating system allowed on work devices. I proposed a virtual machine running Ubuntu, which was accepted. So, I had to set up a VM running Ubuntu, running on Windows.

For consistency with the production server, we ruled out WSL as an option: without being specific about the differences, documentation of WSL indicates the environment is different from native Linux: it can't run everything.

I had experience with qemu on Linux, so I first set up a qemu VM on Windows. This was easy but performance was poor. Without KVM (not supported by the Windows kernel) it is un-accelerated software emulation. It works fine. It's easy to manage. But a bit slow. The developer team was likely to be frustrated.

Guidance I could find on speeding up qemu said the solution was to enable hyper-v and WSL, run Ubuntu in WSL2 and install and run qemu from this Ubuntu. While this might work, I wasn't inclined to run qemu in WSL2. One of the motivations for developing the VM was to eliminate the configuration work for the developers and I wasn't interested to deal with the issues of developing a WSL2 image export/import, if that is even possible.

But Hyper-v is itself virtualisation. Why not run Ubuntu in a Hyper-V VM directly?

It was easy to install Hyper-V and create the VM, booting from an Ubuntu iso, but network configuration failed: despite the documentation, there was not DHCP service on the default switch. Using sniffer, I could see the DHCP requests from the guest, but there was no response. The interface worked, but had to be brought up manually after each boot. I did manage to find documentation for adding a NAT network: add a private switch and configure that, and it was possible to access the Internet from the VM and access the VM from the host using SSH. But still the VM network had to be brought up manually after each boot. The default switch, even if it worked as documented, was for Internet access from the VM and documented to not allow access from host to guest. The recommended solution was an 'external' switch, but that requires two NICs on the host and an external switch to connect them. Hyper-V networking it truly perverse.

Next option was VirtualBox. It's open source, but provided by Oracle and Oracle has a history of adverse management of open source projects under its control. None the less, given the poor performance of qemu and the nearly useless and undocumented networking of Hyper-V, I gave it a go.

And I was pleasantly surprised. VirtualBox is easy to install and it was easy to create the VM. Networking just worked and it was easy to map ports from the host to the guest to allow SSH, HTTP and file share access from the host to the guest. And, after all was done, performance was significantly better than qemu running on Windows.

So, the final solution is Ubuntu running in a VirtualBox VM. It's easy to export/import. Easy to set up on a developer system: just install VirtualBox an import the VM image. Easy to start/stop and change resources (more memory and CPU for those with more grunty workstations) and easy to change the port mapping if access to additional guest services is required. 

Chinese word segregation with jieba

A while ago I wrote a program to analyse the vocabulary of my Chinese ebooks. It was very simple: it extracted the text and segregated it into words by lookup in the cedict dictionary and ranked them according to the HSK word lists. But it was crude: it found the longest matching word at each position in the text, then moved on. It found words but often not the intended or even common or sensible words.

I thought about getting word frequency data and choosing the words that were more common. But while searching for word frequency data (cedict doesn't have it) I found articles on Chinese word segregation and found that there are many existing programs.

One I kept seeing was jieba. There are many implementations. My program was written in JavaScript on node, so I looked for JavaScript implementations on npmjs.com. There were many but most of them had not been updated for years. I tried a few but most of them didn't even install, let alone work. I assume they did when they were published but node/npm and their other dependencies have changed so much in the meantime that they didn't install or didn't work with current versions.

One difficulty was that most of them had very little documentation and most of it was in Chinese. Not surprising for software that does Chinese word segregation, but my Chinese isn't good enough that I can read it.

I tried a few packages based on cppjieba which were more recently updated than most. It was still challenging to get them to work, but eventually I got one working and it seemed quite nice: good performance and good results.

But I wanted to understand the algorithm and when I started to look through the cpp code I got lost. Too many layers of objects and abstractions, and no documentation or comments describing the data structures or algorithms.

So I looked some more and found a pure JavaScript implementation. It hadn't been updated for 7 years and didn't work, but at least it was a familiar language. It was left half way through being converted from CJS JavaScript to ESM TypeScript - not in a working state.

I had a go at TypeScript but after a few days of dealing with a seemingly endless stream of errors I finally got it to compile and got runtime errors with references into a monolithic blob of transpiled, compacted code. Life is too short to waste it on such nonsense.

So I stripped out all the TypeScript and reverted to simple ESM JavaScript, and in a few minutes I had it working well enough to run the included demos.

I then was able to add logging and trace the code to understand what it was doing.

It is quite simple, actually, like most good algorithms. The implementation I had picked up was based on Python Jieba module as it was about 12 years ago: quite different from current. And the implementation in JavaScript wasn't ideal, but it worked and I was able to understand it and then quickly rewrite various of the wonky bits.

So now I have an all JavaScript implementation that works with current node/npm and I have updated my analysis script to use it. Speed is good: as fast or maybe a bit faster than my previous, simple implementation of word lookup in cedict.

I don't know where the dictionary and weights come from: whatever Python Jieba was using a decade or so ago. From manual segregation of some corpus of books and articles, if I understand correctly.

I found a nice article from people who had used move subtitles to develop lexicon and statistics for more modern Chinese usage. They have done similar for several other languages. It was actually this paper that lead me to Jieba in the first place, as they had used it in their research: cppjieba.

My revised implementation omits the DAG: it builds the segregation route directly from a 'prefix dictionary'. The dictionary is like current Python Jieba but the algorithm for building the route isn't: no DAG. Much quicker.

I only have basic cut functionality, but that's all I need for the moment.

@ig3/node-jieba-js

Dynabook TECRA A50-J annoyance

Today I was running my laptop on battery. Not intentionally. I had unplugged it to move earlier and had neglected to plug it back in when I returned.

I was concentrating on what I was doing, so didn't notice that the battery was running low. No audio alert because I had disabled all the system sounds.

So the battery ran down and the system fairly abruptly shut down. The system is set to shut down when battery falls to 10% of capacity.

I plugged it in within 10 seconds of it shutting down, waited a few seconds then pressed the power button.

The keyboard backlight lit for about a second, as it usually does when powering up. Then nothing.

I waited a few seconds then pressed the power button again, with the same result.

I kept waiting, then trying. Sometimes the keyboard backlight would come on briefly but sometimes not even that.

After a couple of minutes, the keyboard backlight came on briefly, then all went dark but then the screen backlight came on. But no Dynabook logo appeared. I thought it might be doing some sort of POST and waited. After somewhere between 30 and 60 seconds, it shut down again.

I kept waiting and trying, with the keyboard backlight coming on briefly but usually nothing more.

Finally, after about five minutes being plugged in, it finally booted normally.

The whole time it had full power: it was plugged in and powered up. Any reasonable laptop could have run without any battery at all.

But not the Dynabook TECRA A50-J. Evidently, it can't run without a battery, even when it has full power.

People have been making personal computers for almost 50 years now. Making laptops, or at least portable computers with batteries for almost as long. Dynabook wasn't a novice company when it designed and built its TECRA A50-J. But this behaviour, they thought, was acceptable.

I don't think I'll be buying another Dynabook computer. I had a Toshiba Satellite previously. Used it for many years. It was a tank, except for the too delicate case. After about a decade, I finally gave up on it because the last time the mounts for the screen had failed I had potted them up in epoxy and glued the case shut: too many of the clips and screw sockets had failed. When the mounts failed again, I couldn't open it to glue it back together without doing more damage. I also thought it would be nice to have something lighter, with longer battery life.

The Dynabook has been OK. The paint came off many of the keycaps within a few months. But the battery can't be replaced without opening it up and not being able to run without a battery is stupid.

I've only had it for about 3.5 years, but it's time to start looking for a new laptop, before they're all unavoidably infested with Microsoft AI.

Debian 13 on HP 665 G11 laptop

Today I installed Debian 13 on a new HP 665 G11 laptop that came with Windows 11 pre-installed in 1TB SSD. It's not well tested yet but superficially it seems fine.

I created only local Windows 11 accounts on the laptop.

I used Windows Disk Manager to shrink the Windows C drive to make space, reducing it to 250GB.

I disabled BitLocker Device Encryption, which was enabled by default, though I don't think it had encrypted the device yet: I could not export recovery keys. After dire warnings that it would take a long time to decrypt the disk, it was complete in under a minute but, again, I suspect the disks weren't encrypted, pending login to a Windows account.

I downloaded the Debian 13 network install image and copied it to USB using `sudo cp /dev/sda` on another system running Linux.

Getting to the BIOS/Boot menus is a nuisance. Power on and keep pressing Esc or F10 quickly/repeatedly. I don't know when it checks but sometimes it notices and brings up the BIOS/Boot menu. Otherwise it starts Windows 11 again and I had to go round and try again: too many times.

I disabled secure boot. I might try re-enabling it later. I expect the laptop has the new Microsoft keys but haven't confirmed and don't want trouble when the old key expires. I don't need the secure boot security, given the intended use of the laptop.

Then I booted from USB and installed Debian. It took about half an hour, mostly due to my slow Internet connection, making it slow to download all the software. I installed basic system with XFCE desktop.

Now it boots to Debian by default but boot to Windows 11 works.

Wireless network works fine. I haven't explored all the other hardware, bluetooth, power management, etc. but I am optimistic that any problems can be resolved before too long.

It was quick and easy. It took less time to install Debian than for Windows to download all its updates and I don't think it is done yet: whenever I was running Windows 11 my network was overloaded despite me setting download limits on Windows Update for 0.5Mbps background an 1Mbps foreground.

Starting X Server on Debian with systemd, xfce and lightdm

I have been reading about X server keyboard configuration and this lead me to wonder how the X server was started, with what arguments and configuration files, resulting in what keyboard configuration.

There are many posts that describe how to run X, xfce and lightdm but they are almost all in terms of what packages to install or reconfigure before rebooting. None that I found explain what configuration those packages create or the process of booting to bring up the X server and 'greeter' for login.

So I investigated the lightdm source code, finally coming to the following conclusions:

The lightdm package includes a lightdm systemd service which is started at boot, to run the lightdm executable.

The lightdm executable starts the X server: /usr/lib/xorg/Xorg.

After starting the X server, lightdm must start the greeter, which is provided by a separate package. Presumably it is a separate executable that creates windows on the display managed by the X server. But I didn't look for the code that does this.

I still don't know how the session setup after authentication is done but I expect that this can change the keyboard layout. It would be strange if different users were not able to use different keyboard layouts. One might prefer qwerty US English and another dvorak or French or German or whatever. That is, the keyboard layout should configurable as a per-user preference: optionally overriding the system default.

The Xorg command line does not specify the keyboard layout. The layout appears to be specified in /etc/X11/xorg.conf.d/00-keyboard.conf. At least, this seems plausible. I haven't edited it to confirm that it makes a difference or found evidence in logs that it did determine the layout.

There are still many unknowns and uncertain guesses, but I am happy enough with these conclusions to get back to investigation of the keyboard layout itself.