Ubuntu Linux installation to a USB external drive with EFI boot

Note: this is a tech person's guide, for sharing with other tech folk. I'm not attempting to make this completely newbie friendly - but I'd be happy to see its points used in something that is.

What is the Purpose?

The goal here is to create a fully-functional forever-updateable system. So the two things that this is not:

it is not just a “live boot”;

this is not just something to then use to install Linux onto the internal hard drive of a computer.

Rather, this is Linux ready to plug into the USB port of any computer that can be told to boot from that. Every time you boot this system, it will inspect and deal with the hardware that it finds.

This way, the external USB drive becomes your "system", and the computer is just the commodity used at the time. Depending on your use cases:

a physically small drive is easy to hide for security;

a drive can be more portable than travelling with a computer;

a drive can be cloned for backup with instant usabilty;

drive capacity is a simple purchasing choice - I currently use 1TB drives;

it's quite feasible to make lots of these e.g. 16GB flash memory thumb drives are handy;

stating the obvious a little, being Free Open Source Software, there are no license counts to worry about.

While I'll try to explain the principles, it is however just a description of how I make such a USB drive using, and for use on, my fairly regular, slightly-old laptop.

Overview

Why write this, when there are plenty of guides already published online?

Well, there are two things that make my process different to those I've seen online. These are:

I perform the install with no drive present in the computer other than the target drive;

I re-install the boot manager with a special command. sudo grub-install --removable

The reasons for those two distinctions, are:

there's no chance of anything being installed to anything but the target drive;

the EFI boot that this new drive will enact, will not affect anything other than itself, even when doing later updates to the kernel and boot setup.

I learned the necessity of this the hard way, by building setups that would cause problems when the next kernel update happened. Back when I was doing MBR booting, achieving this was quite easy. It was EFI that changed everything, which is to say: complicated everything. I read a lot about EFI and tried a lot of things to no avail. Then I find out about the --removable option for installing grub.

It's now history, but I was doing this external USB full Linux for years, before computers changed from 32bit and MBR to 64bit and EFI.

Sequence in Detail

Here is my sequence in its full form. It's from my own notes, that I expect to re-use, years apart so I don't want to have to try remembering the little practical tips that I've learned are useful.

Before I do anything, I look at my list of potential computer names and choose a new one.

the USB drive always comes as NTFS so that has to go first. Boot an existing Linux, plug the new drive in, run GParted and have it create a new partition table (as GPT).

while I'm there, download a current Xubuntu installer image and burn that to a DVD+RW and power down.

Turn the laptop upside down, open a panel with a screwdriver then unplug the internal hard drive, put the panel back on, turn back over.

Boot with the DVD, let it do an integrity check of itself - have a coffee and come back later. This integrity check would only be done the first time using that disc and is really only for the logic that it is better to check it once than later suffer consequences. The reality is that errors at this point are very unlikely nowadays. After the integrity check, let the disc boot into the "live" session.

Plug in the new drive and start the installer process.

As the only drive is the new USB drive I let the installer do what it wants. (It's really for this lack of complication + risk that I'd removed the internal drive.)

When the installer asks, I put in my chosen new name for the "machine/computer", I give my standard name for an admin account. (I don't happen to use "admin" but that's an obvious good-enough choice.)

The installation runs its course, and when done, prompts for a reboot which also ejects the DVD.

I let it reboot.

I log in to the admin account.

I open a terminal and put in the command to re-install GRUB in "removable" mode (see below). This is vitally important for a USB booting drive as it will ensure it uses the EFI controls from the drive not the host hardware.

Do a reboot to be sure nothing went wrong with GRUB.

Shutdown and put the internal hard drive back in.

Because I tend to be both cautious and pedantic, I boot the internal drive to be sure it's still ok.

Reboot again, this time with the USB drive plugged in.

Check the mounted partitions to confirm that the ESP (EFI System Partition) being used is the USB one, not the one on the internal hard drive.

The Special Command

With only one drive present, the special command is just

Comments

While that might seem a lot, I have put in all the steps that experience has taught me to do - and it's all quite familiar by now. It also doesn't take very long. If it is slow that's because I usually use an optical disc in my process. When I've had to use a machine with no optical drive the installation step goes much faster - thumb drive beats a disc! But there is quite some more mucking about to prepare a boot thumb drive than a DVD, and I'm lazy.

The distro I install these days is the latest Xubuntu LTS 64bit Desktop. As far as I know the process is identical for any of the other 'buntu variants. How much it's the same for other main distros I can't say, but I'd be surprised that they'd be any different if they also use GRUB.

Most of us who use Linux will have a stock set of things we do after installation. There are many of these given as articles online. I also have such a set of things that I do, but that's really another topic.

The only part where those stock things-to-do intersect with this guide, is that I usually do my check of which ESP partition is in use by looking with the Gnome Disk Utility. That used to be trivial, but it’s not part of the default Xubuntu install, so I go through a few of my other usual post-install customisations to get it. There will of course be a command line method for doing that check, I just haven't needed to choose one.

Do you really need to unplug the internal drive?

Well no, but not doing so does introduce potential mishaps. I prefer a process that doesn't requite my careful attention to specifics.

There are two broad options should you leave the internal drive present.

use firmware (aka BIOS) settings to have the drive seem to not be there;

add a parameter to the sudo grub-install --removable command to enforce which ESP it will direct to.

That second point is because if the internal drive is present, then there will be two ESP partitions present.

In all honesty, I haven't had the patience to experiment with the latter in order to gain certainty about it. Ditto for trusting the boundaries of firmware settings and booted drivers.

Indeed, I don't really trust the standard 'buntu installer to reliably ensure that the external drive will even get an ESP partition - at least, without being explicitly told by a custom partitioning step.

Even if/when I want to manually set the partitions, the process is still all clearer with the internal drive definitely absent. It seems a small hassle to unplug it.

Appendix

Because I cheated with some simplistic remarks, here are some caveats and extra details.

Live boot Linux with Persistence

Actually many of the methods used for Live boot Linux setups now allow for some kind of persistence. I had problems with the early attempts at these and have never revisited them. Hence I don't really have an opinion about them.

Firmware and Boot Order Variations

The issue here is that computers vary widely in how they allow for control of the booting order and setting of defaults. For this article, I considered that out of scope.

However I will say that for my own computer, I was able to use efibootmgr to reset its default EFI order so that I can just plug in my drive then power up to automatically boot into my roaming Linux. For the other computer in the house, that technique made no difference - and it needs a manual process for every boot to avoid going into Windows.

MBR vs EFI, 64bit vs 32bit

These aspects get complicated quickly. For this article, I've just assumed to only apply it to the current combination of 64bit computers, GPT partitioning and EFI booting.

But I have been using external USB Linux since 32 bit CPUs were the norm. There's been periods when I made setups that could be taken to and used on either 32bit or 64bit computers. It is very difficult to do a perfect setup, that will boot a 32bit Linux on a 32bit computer, but when used on a 64bit computer will boot into a 64bit Linux.

Instead if there is a need to use a system across all types, it's mainly better to just make it the old combination of 32bit computers, MBR partitioning and booting, with a 32bit Linux. Even with this, many modern computers will require some firmware setting changes to enable "Legacy" booting and/or turning of so-called "secure" boot.

p.s. I do want to credit the terrific comprehensive annotations about EFI at The rEFInd Boot Manager by Roderick W. Smith and while it wasn't there that I found the crucial GRUB installation trick, his notes meant that when I did, I understood the ESP puzzle.

Interaction with Windows

I've made no mention of having the booted Linux interact with the storage of Windows on a computer's internal drive. I simply have no need for that, and mainly see it as an anti-feature.

However, further in the past I would use these kinds of boots to backup/recover/repair the internal Windows system. When I noticed that Microsoft were having Windows shut down with the drive in some kind of locked state, I gave up dealing with it. Life is too short to be decoding Microsoft's machinations, so I just don't.

See my other article

Except that I haven't posted it yet, I wrote a wiki page at my place of work so the text exists but will need adapting for a broader audience. When I've posted it, I'll return here to include a link.