	    Dolly - A program to clone disks / partitions
	    ---------------------------------------------

			     Version 0.2
			     7 July 1999
		   Felix Rauch <rauch@inf.ethz.ch>


This document describes the program "dolly", its purpose and the
format of the required config-file.


Purpose
-------

Dolly is used to clone the installation of one machine to (possibly
many) other machines. It can distribute image-files (even gnu-zipped),
partitions or whole hard disk drives to other partitions or hard disk
drives. As it forms a "virtual TCP ring" to distribute data, it works
best with fast switched networks (we were able to clone a 2 GB Windows
NT partition to 15 machines in our cluster over Gigabit Ethernet in
less than 4 minutes).

As dolly clones whole partitions block-wise it works for most
filesystems. We used it to clone partitions of the following type:
Linux, Windows NT, Oberon, Solaris (most of our machines have multi
boot setups). We have a small (additional) Linux installation on all
of our machines or use a small one-floppy-disk-linux (e.g. muLinux) to
do the cloning.


How it works
------------

Setting up or upgrading a cluster of PCs typically leads to the
problem that many machines need the exact same files. There are
different approaches to distribute the setup of one "master" machine
to all the other machines in the cluster. Our approach is not
sophisticated, but simple and fast (at least for fast switched
networks). We send the data around in a "virtual TCP ring" from the
server to all the clients which store tha received data on their local
disks.

One machine is the master and distributes the data to the others. The
master can be a machine of the cluster or some other machine (in the
current version of dolly it should be the same architecture
though). It stores the image of the partition or disk to be cloned or
has the partition on a local disk. The server should be on a fast
switched network (as all the other machines too) for fast cloning.

All other machines are clients. They receive the data from the ring,
store it to the local disk and send it to the next machine in the ring.

The cloning process is depicted in the following two figures. Usually
there are more than two clients, but you get the idea:

      +--------+  +----------+ +----------+
      | Master |  | Client 1 | | Client 2 |
      +----+---+  +---|------+ +----+-----+
            \         |            /
             \    +---+----+      /
              +---+ Switch |-----+
                  +--------+

        Cloning process, physical network


     +--------+  Data   +----------+  Data  +----------+
     | Master |-------->| Client 1 |------->| Client 2 |
     +--------+         +----------+        +----------+
         ^                   |                   |
         | Data              | Data              | Data
         |                   V                   V
      +------+            +------+            +------+
      | Disk |            | Disk |            | Disk |
      +------+            +------+            +------+

     Cloning process, virtual network with TCP connections


We choose this method instead of a multicast scheme because it is
simple to implement, doesn't require the need to write a reliable
multicast protocoll and works quite well. One could also use the
master as and NFS server and copy the data to each client, but this
puts quite a high load on the server and makes it ot the
bottleneck. Further, it would not be possible to directly clone
partitions from one machine to some others with any filesystem in the
partition.


Different cloning possibilities
-------------------------------

There are different possibilities to clone your master machine:

- You already have an image of the partition which you want to clone
  on your master (raw or compressed). In this case you need Linux
  (some other UNIX might also work, but we haven't tested that yet) on
  your master and a Linux on each client.

- You want to clone a partition which is on a local disk of your
  master. In this case you need Linux (or probably another UNIX, we
  haven't tried that) on your master as well as on all the clients.
  You can use any Linux installation as long as it's not the one you
  want to clone (i.e. you can not the clone the Linux which you are
  currently running in. See the warning below).

- You want to clone a whole disk including all the partitions. In this
  case you either need a second disk on all machines where your Linux
  used for the cloning process runs on (not the one you want to clone)
  or you need a small one-floppy-disk-Linux which you boot on all
  machines. In the later case you also need dolly on all machines
  (copy it to your floppy disk or mount it with NFS) and the
  config-file on the master.

WARNING: You can NOT clone an OS which is currently in use. That's why
         we have a small second Linux installation on all of our
         machines so that we can boot this to clone our regular Linux
         partition.


Configuration file
------------------

You need a configuration file for the cloning process. Its format is
strict, but easy. It contains the following entries (note that the
order of the entries is fix):
(The text after "Syntax:" explains the syntax of the entry, the lines
following "EG:" are example lines)

1. The file/partition you want to clone, preceeded by the keywords
   "infile" or "compressed infile" in case of a compressed image.
   This file or partitions needs to be available on the master only.
   Dolly will warn you if you try to use a compressed infile which
   does not end with ".gz". The compressed keyword is important so
   that the master can inform the clients when they have to use gunzip
   before writing a file.
   Syntax: [compressed] infile <input file or device>
   EG: compressed infile /images/cloneimages/sda10_WinNTRes.gz
   EG: infile /dev/sda10

2. The file or partition you want to write (usually its a partition,
   but you can also write to a file) after the keyword "outfile". This
   file needs to be available on the clients only.
   Syntax: outfile <output file or device>
   EG: outfile /dev/sda10

3. After the keyword "server" follows the hostname of the server (or
   master). This is required for the last machine in the ring to be
   able to send the end-acknowledge back to the server.
   Syntax: server <master machine>
   EG: server cluster-master

4. This entry has the keyword "firstclient" followed by the hostname
   of the first client in the ring. You should use the hostname of the
   machine here, not the name of the interface where you want to
   connect.
   Syntax: firstclient <name of first machine>
   EG: firstclient cluster-1

5. This entry has the keyword "lastclient" followed by the hostname of
   the last client in the ring. You should use the hostname of the
   machine here, not the name of the interface where you want to
   connect.
   Syntax: lastclient <name of last machine>
   EG: lastclient cluster-9

6. This entry specifies how many clients are in the ring. The keyword
   is "clients" followed by the actual number of clients. This number
   does not include the master.
   Syntax: clients <number of clients>
   EG: clients 9

7. The following lines contain the interface-names of the client
   machines. The number of machines must match the above number of
   clients (see 6.). You should use the name of the interface on
   which the machines will receive the data.   
   Syntax: <name of client 1>
           <name of client 2>
           [...]
           <name of client n>
   EG: cluster-1-giga
       cluster-2-giga
       [...]
       cluster-9-giga

8. The last entry in the config file consists of the keyword
   "endconfig" and marks the end of the configuration file.
   Syntax: endconfig
   EG: endconfig


Dolly options
-------------

Dolly has a few options which are explained here:

  -v
    This switches to verbose mode in which dolly prints out a little
    bit more information. This option is recommended if you want to
    know what's going on during cloning and it might be helpful during
    debugging.

  -s
    This option specifies the server machine and should only be used
    on the master. Dolly will warn you if the config file specifies
    another master than the machine on which this option is set.
    This option must be secified before the "-f" option!

  -c
    With this option it is possible to specify the uncompressed size
    of a compressed file. It's only needed for performance statistics
    at the end of a cloning process and not important if you are not
    interested in the statistics.

  -f <config file>
    This option is used to select the config file for this cloning
    process. This option makes only sense on the master machine and
    the configuration file must exist on the master.


Starting the process
--------------------

To start the cloning, you need to start dolly on each machine. It is
recommended to start it with the "-v" (verbose) option. The order in
which you start the programs on the master and the clients doesn't
matter. You must give the "-s" (server) option on exactly one machine
(the master).

When the machines have found each other and the ring is completed, the
cloning starts. Dolly will print some progress information every
10 MBytes.

