The syncopt script and its associated work practices
are yet another approach to the standard sysadmin problem
of keeping multiple machines' software installations
up to date.
It is independent of the vendor's packaging scheme
and thus you can use either your vendor's system or syncopt, or both!
Also see the manual entry: syncopt.1.html.
The core notion is that the install is done once on a central machine and then syncopt takes care of attaching that to every client, often via cron.
Like most such solutions, syncopt has to achieve a few goals:
As implemented, syncopt achieves this with the following advantages:
It presumes you have a ``large'' main machine which can have an instance of everything installed on it and that your other machines generally have mostly-local core installs of the OS and main packages and probably run most of the optional stuff from the main server via NFS or other network filesystem.
We will approach the last criterion first, since it's crucial to having things work smoothly in the long run. The vendor namespace tends to be /usr and its children - /usr/bin, etc. So, we elect not to install there. Likewise, the ``user namespace'' or ``local customisation namespace'' tends to be /usr/local and its children. Although our users could use a /opt style scheme, /usr/local is familiar and has its own conventions.
Accordingly, I've taken a leaf out of Sun's book http://www.sun.com/
and used /opt
for my syncopt scheme.
This doesn't actually fly in the face of Sun,
as Solaris packages are installed with names
of the form [A-Z]+[a-z0-9]+.
Syncopt generally uses names of the form [a-z][a-z0-9]*-version,
so we can play happily in Sun's field without conflict.
That decision made, here is how we have things arranged at my workplace:
Under that directory is a subdirectory called common
for the architecture independent stuff
(config file, image/icon archives, scripts etc)
and a subdirectory matching the $ARCH environment variable
for the architecture dependent stuff
(pretty well everything that gets compiled).
Here's the directory listing from ours:
% ls -la /u/syncopt/.
total 48
drwxr-xr-x 8 root bin 8192 Oct 12 14:45 .
drwxr-xr-x 14 root root 8192 Oct 11 15:05 ..
drwxrwsr-x 17 root technic 8192 Oct 14 11:56 common
drwxrwsr-x 5 cameron technic 8192 Sep 3 11:55 freebsd.x86.freebsd
drwxrwsr-x 2 root geeks 96 Jul 20 2000 redhat.ppc.linux
drwxrwsr-x 184 cameron geeks 8192 Oct 15 10:11 redhat.x86.linux
drwxrwsr-x 155 root geeks 8192 Sep 20 11:36 sun.sparc.solaris
drwxrwsr-x 4 root technic 96 Mar 13 2000 sun.sparc.sunos
The purpose of the syncopt script itself is to make the /opt directory correctly configured with respect to /u/syncopt.
The core algorithm is simple and conservative:
for every subdirectory found in /u/syncopt/arch
or /u/syncopt/common,
check the matching name in /opt.
If it is missing,
make it a symlink to the matching central item.
If it is a directory, ensure the contents are an exact
match for the central item,
using rsync(1) http://rsync.samba.org/.
This behaviour can be overridden with the /opt/.syncopt configuration file as described below.
As a consequence, to make an instance of something local to a client machine (let's call it pkg, release version), remove the local symlink:
$ rm /opt/pkg-version
Make a stub directory:
$ mkdir /opt/pkg-version
Alternatively, just edit the .syncopt file and add this line:
pkg-version local
Run syncopt:
$ syncopt -x
That syncs everything. You can just do the new package like this:
$ syncopt -x pkg pkg-version
which syncs the generic (unversioned) link and the version specific local directory.
To make a once-local copy remote, remove the local copy:
$ rm -rf /opt/pkg-version
Also, if you edited the .syncopt file as above, remove that line.
Run syncopt:
$ syncopt -x pkg pkg-version
To set up a new client's /opt directory after a fresh install:
$ mkdir /opt # if necessary
$ syncopt -x
The behaviour deduced from the presence or absence of a directory can be overridden with the /opt/.syncopt file, which contains line of the form:
pkg version
to make version the default package version on this particular machine, or
pkg local
to force a package to be local on this machine, or
pkg-version local
to make a particular version local, or
pkg nosync
to not run syncopt on it at all.
A scheme such as this is naturally not useful without things to install this way.
For very small packages (usually only a single command and matching manual entry) it's often not worth bothering with /opt, instead installing them in the traditional way with the executable in /opt/bin and the manual in /opt/man and so forth.
For larger packages (netscape, emacs, vmware, the pbmtools, elm, mh, etc) the /opt comes into its own.
When building the package from source or installing a binary distribution, tell the package to install in /opt/package-version.
Usually version is the release version,
but occasionally I tack extra info into it, such as the build platform
where $ARCH is too vague.
For example, I use redhat.x86.linux for RedHat Linux platforms.
Unfortunately, these are not always binary compatible
and so I might make the version be version-rh9 to indicate
it was built on RedHat 9.
In this way I can choose, say, a -rh7 build for RedHat 7 boxes.
For packages built with GNU autoconf
[http://sourceware.cygnus.com/autoconf/]
this usually is as simple as adding the --prefix option
to the configure run:
$ ./configure --prefix=/opt/package-version
Do the build as normal.
If it is successful,
set your umask to 2 for the install.
This presumes a group exists with write privileges in /u/syncopt
(ours is called ``geeks''),
then make the master directory and link it to the local /opt:
$ umask 2
$ mkdir /u/syncopt/arch/pkg-version
$ ln -s /u/syncopt/arch/pkg-version /opt/.
then install as normal:
$ make install
Then adjust the permissions on the master to prevent accidental damage in the future:
$ cd /opt/pkg-version # should take you into /u/syncopt
$ chmod -R a-w .
If that goes well, and this is to be the ``default'' version of the package, add the generic symlink:
$ cd /u/syncopt/arch
$ rm -f pkg
$ ln -s pkg-version pkg
Then run syncopt
or just make the same symlink in the local /opt by hand.
In this way you can now talk about a generic /opt/package
in shell scripts, /opt/pkg/bin in $PATH
and so forth
without having these things know about the version.
This has the added advantage that upgrades are done by installing the new version and switching the symlink in /u/syncopt. This way you can keep multiple versions around without conflict, which is often quite handy with unstable or experimental upgrades or for legacy uses.
It is quite important to never mention /u/syncopt outside of this scheme - by having the package and everything which uses it believe firmly in /opt, making things local or shuffling versions ``just works''.
Having done the central install in this way, saying:
$ mkdir /opt
$ syncopt -x
on every client machine suffices to finish things off. This can be put in a nightly cron job on each machine if you desire.
It all seemed so easy, didn't it? Well, I've glossed over a few issues. Let's peel back the paint:
Everything you've installed in /opt
will most likely not be in your users' $PATH
or $MANPATH,
so they won't be able to simply type the name of the package
and have it work.
There are three main approaches to correcting this situation:
$PATH/etc/profile
to insert the relevant bits into your users'
environment variables, eg:
PATH=$PATH:/opt/package/bin
export PATH
To keep central control of this kind of thing, here we keep this stuff a file in /opt/config/shell and simply source that from the /etc/profile. That way, aside from the initial edit of /etc/profile on each client machine (to add the source line), future package installs need only involve editing the central file in /u/syncopt/common/config/shell and rerunning syncopt.
/opt/bin
which inserts the /opt/package/bin
directory at the front of the $PATH
and then execs the real executable.
In this way the main package is accessible as normal
and the general utility names do not pollute the default
command namespace.
For example, the my elm wrapper:
#!/bin/sh
PATH=/opt/elm/bin:$PATH
MANPATH=/opt/elm/man:$MANPATH
export PATH MANPATH
exec /opt/elm/bin/elm ${1+"$@"}
It is installed as /opt/bin/elm.
Note: users who truly wish the package's utilities in their default $PATH
can naturally add /opt/package/bin
to their own path in their .profile file.
/opt/bin
and add symlinks to the executable(s) and matching symlinks for
the manual entries, eg:
$ cd /opt/bin
$ ln -s /opt/netscape/netscape
$ ln -s /opt/netscape-4.79/netscape netscape-4.79
or
$ cd /opt/bin
$ ln -s /opt/netpbm/bin/* .
$ cd /opt/netpbm-10.17/bin
$ for bin in *
> do ln -s /opt/netpbm-10.17/bin/$bin /opt/bin/$bin-10.17
> done
Notice that we make both unversioned and version links so that users can run the default app or a specific version if needed.
Some packages include their dynamic components in the install.
For example,
NNTPCache [http://www.nntpcache.org/]
includes a var subdirectory
to hold its news spool.
My approach with these is to make a matching
/var/package subdirectory
(or /var/spool/package is appropriate)
on the client machines which will run the package,
and to make the var subdirectory
in the install directory a symlink to /var/package.
This means that the package can continue to believe in its ``normal''
install structure
and still correctly access the local, dynamic files.
It also protects those files from syncopt by shifting
them out of the /opt area.
if there's significant structure to the ``var'' directory I tend to move it sideways in the install tree:
$ cd /u/syncopt/redhat.x86.linux/nntpcache-3.0.1
$ mv var var.dist
$ ln -s /var/spool/nntpcache var
Then on any client machine that will actually run it:
$ mkdir /var/spool/nntpcache
$ rsync -avHP /opt/nntpcache/var.dist/. /var/spool/nntpcache/.
and it's ready to go.
RedHat [http://www.redhat.com/]'s RPM [http://www.rpm.org/].
Debian [http://www.debian.org/.debian's apt [http://www.debian.org/doc/manuals/apt-howto/index].
Sun Solaris [http://www.sun.com/software/solaris/]' pkgadd [http://docs.sun.com/ab2/coll.47.8/SYSADV1/%40Ab2PageView/25509].