Category: BIOS

Adventures in Virtualization

I currently pay ~$150/mo for my ISP. It’s a legacy commercial account through my local cable provider, and is 50mbit down, and 10mbit up, with a static IP. Gigabit fiber is available for < $100/mo, but without a static IP. The cable provider has “helpfully” suggested I update my account to higher speeds at higher prices, but it occurred to me that there’s no reason my home needs a static IP if I can set up a tunnel from home to my VPS, which also has a static IP. I take my public services and instead of reverse-proxying them to my local router, I proxy them over VPN to my VPS.

The first step of course is updating my VPS to the “latest and greatest” (it’s still running a very svelte, firewalled, and outdated NetBSD-7) but also to get some hopefully improved VPN capability, because while it would be neat to get an IPSEC tunnel running, (possibly over IPv6,) I suspect it will be a lot easier in practice to get the newly available wireguard support working.

My VPS is one of the few who doesn’t mind running NetBSD, and started out with 32-bit PV instances running on a NetBSD dom0. They shifted to Debian Linux as the company grew, but the lowest-spec instances were still 32-bit PV when I first signed up. They didn’t directly advertise them; you had to do a “view source” in the HTML to find the obscured link, but the single vCPU, 1.4G memory, and 18G storage is plenty for my needs as a DNS server, and soon as a VPN server. (The disk space and the physical CPU backing the vCPU have grown over the years.)

Linux dropped 32-bit PV support in Linux 5.9, and Xen in general has been moving towards PVH from PV, since it utilizes hardware features which were introduced after PV, like page table updates. I was a bit surprised to discover my home server running Ubuntu 22.04, kernel 5.15.0, with Xen 4.16.0 supports PVH, but no longer supports 32-bit PV. A freshly compiled NetBSD-9 32-bit PV kernel completely crashed on my VPS, NetBSD-10 32-bit PV kernel got a little farther but crashed with MMU errors, and the lack of support for 32-bit PV on my home server prevents me from doing more extensive debugging. The solution seems to be to update my VPS to 64-bit PV, which makes things more consistent between my VPS and home.

PVH seems the future, though. A directly specified NetBSD-10 GENERIC kernel seemed to boot, but the Xen docs and some searching indicated that there is a version of OVMF which can be used as well? As a UEFI developer who has been hacking on the build system to greatly improve build times as a side-project, this is interesting. It looks like I have to build it from scratch, though, which I do frequently at work, but doing it under NetBSD instead of Linux or Windows (gasp) has some challenges.

Instead of booting UEFI, it looks like there is the possibility of pvhgrub. Ubuntu includes a 32-bit PVH version of grub, but I can’t seem to find a 64-bit version? Maybe since it’s a naked HVM it runs grub2 in 32-bit mode and then flips to 64-bit as part of the boot process? I have no idea how to boot a NetBSD kernel from a grub2> interactive prompt — it seems to reject the magic number of my kernel, but at least it can see the GPT partition and knows about the UFS2 filesystem. Maybe a menu.lst or grub.cfg and it will magically start working.

For now I’m booting a kernel directly from the xl.cfg side and get on to bootstrapping everything. Maybe ye olde pygrub bootloader will still work here?

Plenty of technical baggage for Xeon

Reading through Xeon Bang Per Buck, Nehalem to Broadwell, and came across this gem:

Nehalem is the touchstone against which all Xeons should be measured because this is when Intel got rid of so much technical baggage that had been holding the Xeons back.

Sure, FSB was ditched for QPI, and the memory controllers were brought on-die, but there’s still a heck of a lot of technical baggage being carried around. I started writing this in 2016. It’s now 2020. Do modern Xeons really need to continue carrying around legacy in hardware from the original 8086 and other x86 “mileposts” (80286, 80386, 80486, pentium, pentium pro, etc…)? I bet there’s a Greenlow or Denlow board somewhere in embedded land with ISA slots plumbed in through eSPI, booting DOS 6.

EFI has been shipping since 2000, and Apple started shipping Intel-based Macs with EFI in 2006. Why are we still booting in 16-bit real mode? Why did it take so long for option ROMs for modern on-board ethernet on Intel reference hardware (and PCSD’s commercial versions) to be re-coded for EFI in the intervening decade? (Whitley finally dropped real-mode opROM support.) Add a software-based 16-bit emulator for legacy option ROMs optionally loaded during DXE phase? (DEC had a MIPS emulator in turbochannel Alphas for running turbochannel option ROMs. Sun and other openfirmware systems went one better and used architecture-independent FORTH bytecode.)

The necessity for direct hardware support of legacy 32-bit support is questionable — BIOS claims it needs 32-bit to meet code-size requirements, but the code is not exactly tidy or concise. There seems to be a vicious circle between IBVs and BIOS developers unwilling to update crufty code, with long-aged code bases suffering from a tragedy of the commons, with no clear stewardship or short-term monetary payout for cleaning things up. I do have a few BIOS buddies who grok this and are trying to do what they can, but there’s a lot of momentum in technical debt that’s coming up on two decades old in a company that has historically viewed software as an enabler for hardware.

Surely 32-bit backwards compatibility could be handled at the OS and software layer, not the hardware? IA64 bungled implementation of IA32 backwards compatibility by not making it performant, but that doesn’t make it an inherently bad idea.

Don’t get me started on SMM… or maybe that’s a rant for another post.

(posted from the depths of my drafts years after it was started…)