Reinstalling your remote debian server
This article is about reinstalling your remote server. Maybe you made a mess of your machine, your development machine is taken by sales as a demo server, your webserver gets a second life as a mailserver. Sometimes you just want to start over. Everything would be easy if you could just put the debian etch cd into the cdrom drive and access the console. But you can't. Your server is in a datacenter and you don't trust the goons walkin' around there to know the difference between the eject button and the power switch.
The procedure we are going to follow in highlights:
- Inspect our current configuration
- Make space on disk
- Copy root partition
- Create our new root disk
This document is to give you a general idea of how to do this. This is NOT a cookbook. I'm assuming you understand everything we do here, installed debian servers before. There is a lot of stuff in this document that could ruin your setup. Think for yourself. If you see a command you haven't used before: Do research, and build a virtual box on your own machine and test it's workings.
If you wreck your server: Don't blame me, blame yourself.
Inspect our current configuration
Before you do any work on this server and shut it down, you should make sure it is not used by anyone. The things we are going to do to your machine are not easily reversed. Also we need to reboot your current configuration twice. You want to make very sure that your server is able to boot! I've seen old machines with an incorrectly configured grub or lilo. If you machine has an uptime of more than a few months you want to thoroughly check:
- The kernel installed should be the running kernel (at least you know that one is working and has correct hardware support) compare the uname -a output to your grub or lilo config.
- The lilo or grub configuration correctly specifies the kernel as default.
- Lilo or grub is installed properly in the MBR
- Your /etc/network/interfaces file should contain the network settings you are running on now.
- /etc/rc2.d contains a link to the ssh server start script
The ultimate configuration check is a reboot. You should really do that at this point. That is: before you make any modifications to it's configuration. If things go bad here you can always blame the idiot who screwed up the configuration in the first place (and the sales guys might actually believe you).
Make space on disk
Now it's time for a cleanup. The goal here is to remove all partitions (including swap) except the root partition. On this partition there should be your entire current debian configuration. Make sure you remove old partitions from /etc/fstab accordingly. This should free enough space on disk to copy this single partition later on.
We need a continuous block of free space at least be the size of your current root partition. In the free space, create a new partition using fdisk. There is no need to build a filesystem on it (format the partition) because we are going to copy your partition in the next step.
After all this cleaning up and changing the partition table with fdisk, you should reboot to let the kernel re-read the partition table.
Copy root partition
You have booted your server twice already, and it still seems to work? You must be very lucky! Get your lottery tickets soon!
Now we copy our root partition to the partition we created in the previous step. I'm assuming here that your sda1 is the original root partition, and sda2 is the new partition you created in the previous step. We are using a 1MB copy-buffer here.
dd if=/dev/sda1 of=/dev/sda2 bs=1M
When you're done copying, mount the disk to make some small changes:
mount /dev/sda2 /mnt/sda2
First we are going to alter your boot loader. In this case I'm assuming you use grub. For lilo, follow the same general idea:
Change root=/dev/sda1 to root=/dev/sda2 on every line starting with 'kernel'
Then tell grub to use /dev/sda2 to start:
grub-install --root-directory=/mnt/sda2 /dev/sda
Now, we have built a MBR so your system nows to find the grub loader on /dev/sda2. that grub loader is configured to boot using kernel files and the such from the /dev/sda2 disk. The kernel will boot and load the /etc/fstab file to mount the root filesystem. Make sure it contains /dev/sda2 as '/' instead of sda1.
In the fstab file, change the reference /dev/sda1 to /dev/sda2.
Everything should be set, keep your fingers crossed and reboot again!
After the reboot when you have access to the machine again, type
You should see /dev/sda2 mounted as your root filesystem (/)
Create our new root disk
Because our root filesystem is now /dev/sda2 disk /dev/sda1 is not used anymore. We can reformat and build our new machine on it. We use reiserfs here because it supports online resizing. You need this if you want to have one big disk-size partition. If you are happy with having multiple disks, you could use any filesystem that suits your needs.
mkfs -t reiserfs /dev/sda1
mount /dev/sda1 /mnt/sda1
After creating the disk and mouting it, we use debootstrap to bootstrap a new clean debian system on it:
aptitude install debootstrap
debootstrap etch /mnt/sda1
On a new debian system the network interfaces are not configured. I copy the one on sda2 to your new install and then edit it (if you feel like it)
cp /etc/network/interfaces /mnt/sda1/etc/network/interfaces
On a new debian system the security update repository is not used by default. Add a deb line to etc/apt/sources.list using the following command:
echo "deb http://security.debian.org/ stable/updates main contrib non-free" >> /mnt/sda1/etc/apt/sources.list
You now created a minimal debian system in /mnt/sda1, on a reiserfs filesystem. There still is some important stuff missing in our new root partition. Most important: a kernel and boot loader. To install those we chroot into the new filesystem and use aptitude to install them. We use the amd64 kernel here, choose the one that is applicable for your system.
aptitude install linux-image-2.6-amd64 grub
grub-install --recheck /dev/sda
If you want remote access to your server you'd probably want a ssh server running as well:
aptitude install openssh-server
If you need other stuff to boot (kernel modules or other specific stuff) this is the time to install them.
Now there's the ultimate test: Will it blend? ehh boot...
While you are rebooting your machine this is probably a good time to tell you: I didn't get it right the first time. I had to go to the machine to manually get it started. I made all these stupid mistakes: I forgot the /etc/network/interfaces and ssh server. I assumed stuff...
Things might well break at your end. Feel free to improve this document by adding a comment!