Hints for Creating and Installing UNIX Software

Introduction

Generating and installing software on the UNIX systems should be done in a similar, consistent fashion. This allows for ease of maintenance and upgrades, broader support between software staff members, and higher levels of security. This document will describe the process of introducing a new software package to the UNIX environment, starting from downloading the source code to the announcement to the mail list.

Building the application

Each section listed below describes the basic steps. Each step will have a full description of the process. But as each package is different, each step should be considered to be only a guideline. Adjust each step, as needed, when software packages have special needs.

To show an example during each step of the process, we will use the application “pcre”. “pcre” (Perl-compatible regular expressions) is a package that is fairly simple to download, compile and install. Compiling and installing will usually produce a few platform dependent executable programs, some include files, some libraries, some manual pages and some platform independent shared files. This example will create the package for Red Hat Enterprise Linux 5.

Preparing the software

First we need to find where the software source code resides, and what is the current stable version.

For the software application pcre, we need to find the source repository for the software. Usually a software has a web site of its own, and in this case, the web site is PCRE - Perl Compatible Regular Expressions.

Also, usually, there will be previous versions of the software in the /usr/src/local directory that include a “SRC” file. Use this file to store the path to the software download so others will have a clue where it was downloaded previously.

After checking the web site for pcre, the current version of the software is at version “7.8”.

Creating the source directory

Once a software application is identified, find a place in the source code directory to store the source code, any related support files and building scripts. Place all of the source code in the directory /usr/src/local and name the source code directory package-version.

For example, setting up the product pcre version 7.8, create a directory and set the modes for general ECN staff access:

iceberg$ cd /usr/src/local
iceberg$ mkdir pcre-7.8
iceberg$ cd pcre-7.8
iceberg$ chmod 775 .
iceberg$ chgrp ecnstaff .
iceberg$ chmod g+s .
iceberg$ 

Downloading the source code

Download the source into the source code directory. If a PGP signature is available for verifying the source code against tampering, be sure to download that file too. Next, verify the source code file with the signature file.

iceberg$ gpg --verify pcre-7.8.tar.gz.sig pcre-7.8.tar.gz
gpg: Signature made Fri 05 Sep 2008 12:41:58 PM EDT using RSA key ID FB0F43D8
gpg: Good signature from "Philip Hazel <ph10@hermes.cam.ac.uk>"
gpg:                 aka "Philip Hazel <ph10@cam.ac.uk>"
gpg:                 aka "Philip Hazel <ph10@cus.cam.ac.uk>"
iceberg$ 

Also, save the download location for the source code into a file named SRC. This will help with getting newer versions of the source code in the future.

iceberg$ cat >SRC
ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/
iceberg$ 

Finally, make sure all the source files are writable by the ecnstaff group.

iceberg$ chmod g+w *
iceberg$ ls -l
total 1154
-rw-rw-r-- 1 cs ecnstaff 1168513 Jan  9 09:12 pcre-7.8.tar.gz
-rw-rw-r-- 1 cs ecnstaff     287 Jan  9 09:12 pcre-7.8.tar.gz.sig
-rw-rw-r-- 1 cs ecnstaff      55 Jan  9 09:17 SRC
iceberg$ 

Picking the installation path

When the final package is installed, it will first be installed in the /opt directory. So for this example, the /opt directory for pcre will be:

/opt/pcre/7.8

Creating the installation directory

During the installation process, the files will be deposited into the /opt directory. We need to create the directory structure ahead of time so the installation process will complete without getting permission errors.

iceberg# cd /opt
iceberg# mkdir pcre
iceberg# cd pcre
iceberg# mkdir 7.8
iceberg# cd 7.8
iceberg# 

Next, temporarily set the owner of the directory so you may write to the files as a non-privileged user.

iceberg# chown cs:ecnstaff .
iceberg# 

Creating a build script

Create a script that will build the source code into the final program. It will be the build script that will hold all of the knowledge on how the executable is built and what kind of pain is needed to build new versions of the executable in the future.

The build script has four sections:

  1. Sets up the build environment: stopping on errors, setting the umask, setting the PATH variable
  2. Removed the old version and extracts out a new version
  3. Patches and configures the source code
  4. Builds the source code into an executable

The name of the build script will be BUILD, BUILD-platform, or BUILD-platform-bits, depending on the need. If the build script will need to build differently between Solaris and Red Hat Enterprise Linux, create two scripts, one for each platform. If the build script will need to build 32-bit and 64-bit version differently, include the bit count.

Some example script names are:

  • BUILD - (builds anywhere)
  • BUILD-SOLARIS - (builds on Solaris)
  • BUILD-LINUX - (builds on Linux)
  • BUILD-SOLARIS-32 - (builds a 32-bit version on Solaris)
  • BUILD-LINUX-64 - (builds a 64-bit version on Linux)

We will tend to build only the 32-bit version of software. Usually this works, but one example that does not work is when the application links against libraries when only the 64-bit library exists and the 32-bit version of the same library does not. In that case, build both 32-bit and 64-bit versions of the application, and install each on their corresponding host.

This script will be called BUILD.

Note: the script will create the programs so that the final installation will be in the /opt directory, along with the package name and version number.

In this example, here is the build script.

#!/bin/sh

#  Set the build environment
set -e
umask 002
PATH=/usr/local/bin:/usr/opt/bin:/usr/bin:/bin
export PATH

#  Remove old source and extract fresh copy
rm -rf pcre-7.8
tar xfz pcre-7.8.tar.gz
cd pcre-7.8

#  Patch and configure
./configure \
        --prefix=/opt/pcre/7.8

#  Build
make

Note: This method of extracting out a fresh copy of the source code is prone to disaster if more than once instance of the build script is running on different platforms. Good advice is to run only one build a time.

Building the application

Build the source code directory. Cross your fingers.

iceberg$ ./BUILD
...
mv -f .deps/pcre_stringpiece_unittest.Tpo .deps/pcre_stringpiece_unittest.Po
/bin/sh ./libtool --tag=CXX   --mode=link g++  -O2   -o pcre_stringpiece_uni
ttest pcre_stringpiece_unittest.o libpcrecpp.la 
g++ -O2 -o .libs/pcre_stringpiece_unittest pcre_stringpiece_unittest.o  ./.l
ibs/libpcrecpp.so /usr/src/local/pcre-7.8/pcre-7.8/.libs/libpcre.so  -Wl,--r
path -Wl,/opt/pcre/7.8/lib
creating pcre_stringpiece_unittest
make[1]: Leaving directory `/system/src/local/pcre-7.8/pcre-7.8'
iceberg$ 

Installing the application

Once the build is complete and seems to be okay, install the application into the /opt directory. By installing the programs into the /opt directory, using your own non-privileged account, keeps the installation from scribbling all over the system disk in case of errors or bad code.

Create an installation script INSTALL like the build script. If there will be special installation commands depending on the platform or number of bits, use a naming convention like the BUILD script name (see example script names above).

#!/bin/sh

#  Set the environment
set -e
umask 002
PATH=/usr/local/bin:/usr/opt/bin:/usr/bin:/bin
export PATH

#  Set the build directory
cd pcre-7.8

#  Install
make install

Install the application. Cross your fingers again.

iceberg$ ./INSTALL
...
 /usr/bin/install -c -m 644 'libpcre.pc' '/opt/pcre/7.8/lib/pkgconfig/libpcr
e.pc'
 /usr/bin/install -c -m 644 'libpcrecpp.pc' '/opt/pcre/7.8/lib/pkgconfig/lib
pcrecpp.pc'
make[1]: Leaving directory `/system/src/local/pcre-7.8/pcre-7.8'
iceberg$ 

Setting the final modes

Once the code is in the /opt directory, reset the modes of the files and directories away from your own user-identifier. This set of commands will do the job.

iceberg# cd /opt/pcre/7.8
iceberg# find . -type d -exec chmod g-s '{}' ';'
iceberg# find . -exec chmod go-rwx '{}' ';'
iceberg# find . -type d -exec chmod 755 '{}' ';'
iceberg# find . -type f -perm -400 -exec chmod go+r '{}' ';'
iceberg# find . -type f -perm -100 -exec chmod go+x '{}' ';'
iceberg# find . -exec chown bin:bin '{}' ';'
iceberg# 

Build the distribution package

Once the application is built into the /opt directory, the next step is to package the application for distribution to all hosts.

The application will be built into a platform package, rpm packages for Linux or pkg packages for Solaris, and stored into a package repository area. Instructions for installation will be placed into an igor controlled configuration file. The configuration file will update overnight, and each host will install the package. References for executables and libraries will be placed into commonly reference directories so users can use the new software.

The method to create an rpm Linux or pkg Solaris package is unified so that these installations will work for either platform.

Creating a packaging directory

The packaging directory is located at /usr/src/local/packages. There are (currently) two directories, one for Solaris 10 and one for Red Hat Enterprise Linux 5.2.

  • sparc-sunos5.10 - (Solaris 10)
  • x86-redhat_e5 - (Red Hat Enterprise Linux 5)

Enter the directory for the platform. Next, create a package directory for the application. Use the same method as used for creating the source code directory. In this case, create a directory for pcre with version number 7.8.

iceberg$ mkdir pcre-7.8
iceberg$ cd pcre-7.8
iceberg$ chmod 775 .
iceberg$ chgrp ecnstaff .
iceberg$ chmod g+s .
iceberg$ 

Create the package script

Create a package creation script called PKGINFO.

The PKGINFO file will contain all the information necessary to create the package installation files (either type of rpm or pkg) that will install the application into /opt. Also, will create additional package files that will install all of the symbolic links pointing to the /opt directory from places like /usr/local, /usr/opt, /usr/old or /usr/new.

The big feature of the PKGINFO file is to selectively create the symbolic links in the application. Not all files in the application need to be placed into the /usr area. By allowing the PKGINFO to select the files by regular expression, we can control the set of symbolic links in a smart way that can adjust as new versions of the software are developed.

Other features include adding additional symbolic links to adjust the presentation to the user, and to build service scripts to support the application that will automatically install or remove as the application is updated.

For this example, we will create a straightforward package info file. The package info file has a required section, plus some optional sections.

In the required section is the package description:

  1. $pkg_package - Name of the application
  2. $pkg_version - Version of the application
  3. $pkg_instance - Instance number
  4. $pkg_dir - /opt directory path
  5. $pkg_comment - Description of package

Then one or more optional sections. Almost always used is:

  1. @pkg_build_in - Where to build the symbolic links

Next mostly used will be:

  1. @pkg_skip_links - Symbolic links to skip
  2. %pkg_add_links - Symbolic links to add

The following are used to add menu entries for the package:

  1. %menu_progs - Define menu entries and link them to program files
  2. @menu_tags - Define which ECN submenu(s) will contain the entries 

Optional menu entry information:

  1. %menu_terminal - Use when a program runs in a text based environment (e.g. pine)
  2. %menu_icon - Define an icon to be displayed on the menu item

These will be used for more complex packages:

  1. $pkg_warning - Display a warning message
  2. %pkg_preinstall - Script to run before installing
  3. %pkg_postinstall - Script to run after installing
  4. %pkg_preremove - Script to run before removing
  5. %pkg_postremove - Script to run after removing
  6. $prelink - Define as non-zero for packages that replace system libraries

For the example package pcre, where is the completed PKGINFO file:

$pkg_package = 'pcre';
$pkg_version = '7.8';
$pkg_instance = 1;
$pkg_dir = '/opt/pcre';
$pkg_comment = 'pcre - Perl-compatible regular expressions';
$prelink = 1;

@pkg_build_in = (
	'/usr/new',
        '/usr/local',
	'/usr/old'
);

The pkg_instance variable describes the instance number of a package. Should a package need to be recompiled or updated, but the version number is the same, then we use the instance number to tell the host systems to update their package. The instance number starts at 1 and is incremented each time the package is re-made.

The @pkg_build_in variable describes the one or more places that the package will create symbolic links. It can have any of the four places for an application:

  1. /usr/new
  2. /usr/local
  3. /usr/opt
  4. /usr/old

Note: /usr/local and /usr/opt are mutually exclusive. Both /usr/local and /usr/opt produce the same reference to the “current” version of the package.

The typical life cycle of a package is:

/usr/new → /usr/local → /usr/old
/usr/new → /usr/opt → /usr/old

Because there are multiple packages, and the packages are split between the application installing into /opt and the symbolic links, we are able to create a method of moving the packages between /usr/new, /usr/opt and /usr/old without having to add or remove the application from /opt.

Since pcre replaces system libraries, $prelink is set to a non-zero value to add it to the prelink search list.

In the case of pcre, we can create all four packages, (/opt package plus the three symbolic link packages), then only install the /opt and /usr/local package at this time.

Create the packages

Once the package info file is ready, create the package scripts. Running the package command will create all the files that will build the final rpm or pkg files used by the installer. The set of files built by the package command will be different between Solaris and Linux.

Run the package command. Usually there is no output when successful. When complete, there will be a new directory. For Solaris, the new directory will be called pkg. For Linux, the new directory will either be i386 or x86_64 depending on the platform.

iceberg$ ../package
iceberg$ ls
i386  PKGINFO
iceberg$ 

Go into the new directory. In the directory will be a script for the binary (/opt) package and one script for each of the places for the symbolic links. In additional, there may be some other files to support the scripts.

iceberg$ cd i386
iceberg$ ls
local_rpmmk  new_rpmmk  old_rpmmk  opt_rpmmk
local.spec   new.spec   old.spec   opt.spec
iceberg$ 

Next, run each of the scripts. If successful, each script will produce an rpm or pkg file that will be used for installation.

Note: Solaris builds the result into a file ending in .tgz - Linux will end in .rpm.

iceberg$ ./opt_rpmmk
Building target platforms: i386
Building for target i386
Processing files: ECNpcreb-7.8-1
Checking for unpackaged file(s): /usr/lib/rpm/check-files /tmp/pk18573/RPM
Wrote: /tmp/pk18573/RPM/RPMS/i386/ECNpcreb-7.8-1.i386.rpm
Command completed successfully
iceberg$ ./local_rpmmk
Building target platforms: i386
Building for target i386
Processing files: ECNpcrel-7.8-1
Checking for unpackaged file(s): /usr/lib/rpm/check-files /tmp/pk18600/RPM
Wrote: /tmp/pk18600/RPM/RPMS/i386/ECNpcrel-7.8-1.i386.rpm
Command completed successfully
...
iceberg$ 

Adding the package to igor

One the package files are created, the package will need adding to the installation configuration file so each host will be able to download and install the new application. In order to add the lines needed to igor configuration file, there is a utility command called make-igor.

Go back one directory, to the one with the PKGINFO file, and run the command make-igor in the parent directory.

iceberg$ ../make-igor
@@
@@  pcre - Perl-compatible regular expressions
@@
install ECNpcreb 7.8 1 ECNpcreb-7.8-1.i386.rpm
install ECNpcren 7.8 1 ECNpcren-7.8-1.i386.rpm
install ECNpcrel 7.8 1 ECNpcrel-7.8-1.i386.rpm
install ECNpcreo 7.8 1 ECNpcreo-7.8-1.i386.rpm
iceberg$ 

Note: The name of the package is based on the pattern starting with “ECN”, followed by the name of the package, followed by a single character identifier to indicate the package type.

The type identifiers are:

  • The letter “b” is the /opt package,
  • The letter “n” is the /usr/new package,
  • The letter “l” is the /usr/local package,
  • The letter “p” is the /usr/opt package,
  • The letter “o” is the /usr/old package.

The next step is to add these lines to the package configuration file that is under igor control. Go to Harbor and edit one of the package configuration files.

  • For Solaris, edit /usr/share/adm/config/os/SunOS/5.10/packages
  • For Linux, edit /usr/share/adm/config/os/Linux/5.2/packages

Take the lines produced by the make-igor command and add them to the file. The packages are sorted in alphabetical order, so find an appropriate place. Once the lines are added to the file, we need to comment out the entries we won't use now but will use in the future. Because we're installing the symbolic link package in the /usr/local directory, we need to comment out the lines for the /usr/new and /usr/old packages. Once the editing is complete, it will look like this:

@@
@@  pcre - Perl-compatible regular expressions
@@
install ECNpcreb 7.8 1 ECNpcreb-7.8-1.i386.rpm
@@install ECNpcren 7.8 1 ECNpcren-7.8-1.i386.rpm
install ECNpcrel 7.8 1 ECNpcrel-7.8-1.i386.rpm
@@install ECNpcreo 7.8 1 ECNpcreo-7.8-1.i386.rpm

Copy packages to repository

In order to install an application, the package files need to be placed into the package repository directory /package/repository. Any host installing packages will go to the repository to download the files. The repository directory is replicated across multiple package servers so that the loaded is distributed more evenly.

In the package directory, go back one directory, the one with the directory pcre-7.8. In this directory, there is a command to copy the package files to the repository called make-links. Run the command. Since the files are copying to the main package server, Dock, the command may prompt for your password.

iceberg$ ./make-links
find: ./intel_fc-10.1.008/x86_64-wrong: Permission denied
find: ./intel_cc-10.1.008/x86_64-wrong: Permission denied
Password: guess
building file list ... done
./
ECNpcreb-7.8-1.i386.rpm
ECNpcrel-7.8-1.i386.rpm
ECNpcren-7.8-1.i386.rpm
ECNpcreo-7.8-1.i386.rpm

sent 681369 bytes  received 114 bytes  104843.54 bytes/sec
total size is 877617221  speedup is 1287.81
iceberg$ 

Note: It's not unusual to see some permission errors. So long as the package files were updated, it is okay to ignore the errors.

It is helpful to run the following command. It makes sure that if a deploy is started after you have updated the packages file in igor that it won't hang because the files aren't in a repository used by the host being deployed:


/home/pier/a/cs/bin/update-pkgs 

 

Test install

We need to test that the installation will go correctly. The first step is to clear away the source development directory so that the package installation has a clean start. The source code was installed into the /opt/pcre directory. Either delete the directory (the preferred method) or move it to a new name. If moving to a new name, be sure to clean up the mess after the test install is complete.

iceberg# rm -rf /opt/pcre
iceberg# 

Next, copy the new packages configuration file from Harbor.

harbor# cd /usr/share/adm/config/os/Linux/5.2
harbor# rsync -av packages iceberg:`pwd`
building file list ... done
packages

sent 25114 bytes  received 42 bytes  50312.00 bytes/sec
total size is 25017  speedup is 0.99
harbor# 

Next, run the igor command to update the packages.

iceberg# igor packages
INFO: Installing ECNpcreb version 7.8 revision 1
INFO: Installing ECNpcrel version 7.8 revision 1
iceberg# 

If nothing complains, check to see if the build seems to have set files up in the right places. Is the /opt directory there? For “bin” commands, is the /usr/local/bin symbolic links there and pointing to the right place?

iceberg$ ls /opt/pcre/7.8
bin  include  lib  share
iceberg$ ls /opt/pcre/7.8/bin
pcre-config  pcregrep  pcretest
iceberg$ ls -l /usr/local/bin/pcregrep
lrwxrwxrwx 1 root root 30 Jan 13 09:29 /usr/local/bin/pcregrep -> /opt/pcre
/current/bin/pcregrep
iceberg$ 

Announce

Now once you've tested the application and become satisfied that the application is ready to use, make an announcement on the syslog mailing list. Be sure to list the following information in the announcement text:

  1. When will the software be available - now or overnight?
  2. What is the application name and version?
  3. What platforms on which will the application install?
  4. What is the application?
  5. Why is the application installing - new or upgrade?
  6. Are there any additional notes about the application?
  7. What package files are affected by the announcement?

Here's a sample announcement text:

After tonight's distribution, pcre version 7.8 will install or upgrade on all Solaris 10 and Red Hat Enterprise Linux 5.2 hosts. "pcre" is the Perl Compatible Regular Expressions library and executables.

This installation will make the "pcre" package available on Red Hat Enterprise Linux - it was already available on Solaris 10. The Solaris 10 version is an upgrade to match the version on Red Hat Enterprise Linux.

Package additions:
ECNpcre version 7.8 revision 1 (Linux)

Package changes:
ECNpcre version 7.8 revision 1 (Solaris)

Done!

You're done! Simple, wasn't it?

Advanced topics

The previous section was for creating and distributing a package should the package following a simple method, and that the package pretty much works the first time. Not all packages are that simple. In the following sections are advanced packaging topics that might need reference should a package need to be re
made because or an error, need distributing immediately, or need special features - such as those with an initialization script.

Adding menu entries.

Menu entries will only be added to 'current' versions (built in /usr/opt or /usr/local), and
can be added by defining the following variables in PKGINFO:

  • %menu_progs

This is a hash that contains the string to be shown on the menu and the program to
be run when the menu item is chosen.  The executable program can be designated
as either a relative name (relative to /opt/program/version), or as an absolute
path (path begins with '/').

Example:
%menu_progs = (

'Matlab' => 'ecn/matlab.wrapper',
'Matlab Ver. x.y' => '/opt/matlab/x.y/ecn/matlab.wrapper'

);

  • @menu_tags

This is an array of tags that determine which site menu(s) will contain the entries.
The tags are strings designating the site(s) (i.e. 'aae', 'abe', 'ie', etc.).  The menu
entries will appear only in the listed site submenus.  The submenus are chosen by
the host's igor SITE setting.
A special tag, 'all', will place the item in the 'ECN Applications' submenu on all hosts.
Menu items with site specific tags will appear in the '<site_name> Site Applications'
submenu (e.g. 'AAE Site Applications').  In general, you should not include a package
in both an individual site and 'all'.

  • %menu_terminal (Optional)

This designates whether or not a terminal window is required by the application.  Some
command line style applications may be launched from the menu interface.
To designate that a terminal window is to be used, this should be set to 'True'.  If it is
not defined, it will default to 'False' and no terminal window will be started.
This is a hash so that it can be designated per program.

Example:
%menu_terminal = (

'Pine' => 'True',
'Pico' => 'True'

);

  •  %menu_icon (Optional)

This is used to define an icon to be displayed on the menu item.  The value is the absolute
path to the icon file to be displayed.  It will default to NULL and no icon will be displayed
if it is not defined.  It is a hash so that it can be designated per program.

Example:
%menu_icon = (

'Kile' => '/opt/kile/2.0.3/share/icons/hicolor/16x16/apps/kile.png'

);

Removing packages by hand.

During the package development, if the package was installed but had some kind of error, the package may need to be removed from the system by hand. Errors may happen even if the source code builds and the installation script installs correctly, but the package itself create a problem such as a conflicting file between this package and a previously installed package.

The igor script will remove a package, but if development of the application is on-going, it is usually easier to remove the package by hand. Removing the package is done differently between Linux and Solaris, but the only difference is the command.

  • To remove a Linux package, use the command rpm -e followed by each package name,
  • To remove a Solaris package, use the command pkgrm followed by each package name.

Note: It is probably best to remove all the packages at once, and to remove the /opt package last.

An example package removal command is:

iceberg# rpm -e ECNpcrel ECNpcreb
iceberg# 

Re-running igor

Igor caches the list of packages that wants to install on a host. If you've removed the package by hand, igor won't reinstall the package without help - it thinks the packages are up-to-date. Once way to fix this is to remove the package configuration file stored by igor. This will force igor to evaluate all the packages again to make sure they are up-to-date.

iceberg# igor packages
iceberg# rm /etc/packages.conf
rm: remove regular file `/etc/packages.conf'? y
iceberg# igor packages
INFO: Installing ECNpcreb version 7.8 revision 1
INFO: Installing ECNpcrel version 7.8 revision 1
copy: WARNING: Igor controlled file disappeared since last run: /etc/packages.conf
iceberg# 

Replicating the repository to other package servers by hand

Should a newly create package need installing immediately, there will be a problem when a non-central-ECN host is requested to install the package.

The package repository is distributed across package disk servers. When the package was updated in the repository, the update only occurred on the main package disk server, Dock. Before pushing out a newly package application, update all of the repository mirrors.

One way to accomplish an update is to run the packages “rdist” command. Unfortunately, this command will update all of the package disk contents and will take much to long to complete. We need to update only the repository portion of the package &ldquob” disk.

In order to do a hand update of the repository from the package “b” disk, you'll need to get the list of package disk mirror hosts. On the server Dock, go to the directory /var/rdist/packages. At the top of the file dock is a macro for to package &ldquob” disk called PKGB_SERVERS. Same the list of host.

Next, create a distribution file. The file, call it my-pkg, should have the following commands, replacing the list of hosts with the PKGB_SERVERS macro.

(
/export/package/b/Repository
) -> (
list of hosts
)
install -R;
except_pat(/RCS\$);

Next, run the following command to distribute the files:

/usr/local/etc/rdist -onumchkowner,numchkgroup -fmy-pkg

Once the repository directory is up-to-date on all package servers, continue with the package update to all hosts.

Deciding when to add or skip links.

Sometimes it is desirable to skip certain portions of the application files when creating links in the /usr/xxx directory. Reasons to skip some of the links may be:

  • Reduce the number of links,
  • Alter link paths,
  • Remove unnecessary links that users won't need,
  • Combine whole directory trees into one link,
  • Remove conflicts between applications.

Some applications create huge number of files. Vendor applications, for example, are stored on the package disk, and we only need to create one command to refer the user to when they want to run the application.

An example, Abaqus, only needs one program installed into the bin to get things started. After that, the application refers to the package directory. If we let the package command create a link for each file, it would create over 18,000 links.

For Abaqus, we only need one executable. To accomplish this, the PKGINFO file needs to skip every file, then create one link to the start-up script. Also, in reference to “altering link paths”, The Abaqus executable is in the different place than the link itself.

To request that the package skip all links, we use the regular expression . (a single dot character) that matches all files. The package info file would have the following rule:

@pkg_skip_links = (
        '.'
);

Next, we need to add back a link to the executable. The package info file add links in the form “from” → “to”. For Abaqus, we need to have a bin command available called abaqus. But the link will point to a location in the /opt directory that is under a different path, called ecn/bin/abaqus. The package info file would have the following rule:

%pkg_add_links = (
        'bin/abaqus' => 'ecn/bin/abaqus'
);

In the add links command, the source and destination are relative paths if the path does not begin with a / (slash character). If this application is installed into /usr/opt, then the link will be created as /usr/opt/bin/abaqus. Having a relative path will allow the packaging command to create absolute paths different between all the different version of the link packages.

Some packages may create conflicts if one package creates a file that is different than another package. If there are cases where a conflict will result, careful analysis will be needed to keep two packages from interfering with each other, or causing the package to fail to install. In these cases, it's usually possible to omit the conflicting file from one or more packages - sometimes all packages. Add skip link commands to the package info file as needed.

Another use of skipping and adding links is to combine whole directory trees. If an application creates a subdirectory in the directory structure that has a unique prefix, it may be possible to remove the subdirectory and make a single link.

An example is the application freetype. In the directory structure for /opt/freetype/2.1.10, there is a set of include files under the directory include/freetype2. On their own, they would create 86 links into the /usr/local/include/freetype2 directory. Because of the subdirectory that we placed under the include, we can skip then relink directory as one link. The package info file would look like this:

#  Remove the directory from the set of links
@pkg_skip_links = (
        'include/freetype2'
);

#  Add back a link to the directory
%pkg_add_links = (
        'include/freetype2' => 'include/freetype2'
);

Adding tags for special handling

Occasionally, conditions exist that require special handling of an installation.  Some examples are package dependencies or unwanted interaction with other packages.  RPM provides a mechanism for handling many of these with tags.  Tags are added to the package's .spec file by using the %pkg_tags hash.  The hash keys are comprised of the tag labels, while the values associated with the keys contain the value for that label.

As an example, to automatically handle a package that contains a version number in its package name, the following tag will automatically obsolete the previous package when the new package is installed.  In this example, the package being built is 'Package124', and its previous versino is named 'Package123'.

%pkg_tags = (
        'Obsoletes' => 'Package123'
);

This will result in the following tag being added to Package124's .spec file:

Obsoletes: Package123

Any valid RPM tag can be specified in this manner.

System services

One of the most often use of advanced application packaging is the ability to create an application that has a system start/stop script. The start/stop script is placed in /etc/init.d and will be called when the system boots or shuts down. In order to use a start/stop script, several changes need to happen, such as adding the start/stop script to the package directory and installing/removing the script to the system when the package is installed/removed.

A start/stop script is an shell script that is called when the system starts or stops, or if the service needs to be restarted. The shell script is called with an argument to indicate if the service is starting (by calling the script with the argument start), or if the service is stopping (by calling the script with the argument stop), or restarting (by calling the script with the argument restart). start and stop commands are required - restart is optional but usually needed.

When package an application that uses a start/stop script, place the script in the package directory under the directory etc/rc. As an example, we'll use the “Condor” application. In the case of Condor, the start/stop script will be located in /opt/condor/version/etc/rc/condor.

Adding the start/stop script to the package directory structure is not enough. We need to install the start/stop script when the package is added to the system, and to uninstall the start/stop script when the package is removed from the system. We do that by adding installation and removal commands to the package info file.

There are two types of install and two types of remove script that can be called by the package. They can be before or after the package is installed or removed. In the case of adding a system service, we need to add the start/stop script to the system after the install (postinstall) and we need to remove the start/stop script to the system before the remove (preremove). We can accomplish these steps by creating a postinstall script and a preremove script.

The script we need to install the start/stop script into the system for Condor will look like this. The name of the script will be called postinstall.

Note: The postinstall and preremove scripts will be different between Linux and Solaris. This is an example script for Linux.

#!/bin/sh

#
#  Condor installation script
#

#  Name of service
service=condor

#  Install service
chkconfig --add $service

Also, the script we need to remove the start/stop script from the system for Condor will look like this. The name of the script will be called preremove.

#!/bin/sh

#
#  Condor removal script
#

#  Name of service
service=condor

#  Stop service
service $service stop

#  Disable service
chkconfig $service off

#  Remove service
chkconfig --del $service

#  Success
exit 0

It is important to make the removal script exit with a zero status. If, for example, the service is not installed (for some reason) when we remove the package. We want the script to attempt to remove the service, but continue to run successfully if it does not remove.

Note: If any of the “preinstall”, “postinstall”, “preremove” or “postremove” scripts are set to be executables, then the package command package will run the script and use the output for making the package. If the scripts are not executable, then the scripts are used as-is. This allows the Solaris version of the postinstall file create a dynamic script.

Next, add all of the files and scripts to the package info file. This will tell the package command to add start/stop script as a link to the file, plus add the install/remove command scripts to the package so the system is updated. Each script will correspond to the symbolic link package that is set to build. For Condor, this is the set of commands.

%pkg_add_links = (
        '/etc/rc.d/init.d/condor' => 'etc/rc/condor',
);

%pkg_postinstall = (
        '/usr/local' => './postinstall'
);

%pkg_preremove = (
        '/usr/local' => './preremove'
);

Because the Solaris version is significantly different, here are the differences should we build Condor for Solaris.

The postinstall script looks like this, and is an executable script. Note how this script executes to build another script.

#!/bin/sh

#  Build postinstall script
cat <<'XXX'
#!/bin/sh

#
#  Condor installation script
#

#  Setup for temporary files
XMLFILE=/tmp/condor-tcp.xml
trap 'rm -f $XMLFILE' 0 1 2 15

#  Add apache service
cat >$XMLFILE <<'ZZZ'
XXX

#  Process Internet service entry
../inet-svc

#  Part 2
cat <<'XXX'
ZZZ
/usr/sbin/svccfg import $XMLFILE
XXX

The preremove script looks like this. This script is not executable - it is used as-is.

#!/bin/sh

#
#  Condor removal script
#

#  Disable service
/usr/sbin/svcadm disable -s svc:/network/condor:default

#  Remove service
/usr/sbin/svccfg delete svc:/network/condor

#  Success
exit 0

And finally, the package info file is nearly the same as Linux. Only the location of the start/stop script is different with Solaris.

%pkg_add_links = (
        'lib/svc/method/condor' => 'etc/rc/condor'
);

%pkg_postinstall = (
        '/usr/local' => './postinstall'
);

%pkg_preremove = (
        '/usr/local' => './preremove'
);

32-bit vs. 64 bit.

More often on Red Hat Enterprise Linux running in 64-bit mode, applications may need to be compiled twice. Once for 32-bit, and once for 64-bit. We can get away with running only the 32-bit version on both 32-bit and 64-bit machines so long as the application references shared libraries that fully exist on the 64-bit platform.

This is not always the case. When creating the 32-bit package and installing it on a 64-bit machine, check to make sure all of the application calls work. If a program references a 32-bit shared library that is not available on the 64-bit platform, it may be necessary to create two sets of packages - one set for 32-bit hosts and one set for 64-bit hosts.

Usually it is possible to create the packages with the same steps as above without changes, including the PKGINFO file. When it comes time to package an application with the information in the PKGINFO using the package command, the results will be stored in a new subdirectory called x86_64. Normal 32-bit applications will create the subdirectory i386. Be sure to pick the right subdirectory to enter before issuing the package building commands.

Adding the list of packages to igor will also need to have the package installation commands separated out by architecture. In the example of Condor, there are two sets of installation files, surrounded by igor @if statements. The command lines for Condor looks like this:

@@
@@  condor - High-throughput network computing environment
@@
@if ARCH == "x86_64"
install ECNcondorb 7.0.4 2 ECNcondorb-7.0.4-2.x86_64.rpm
install ECNcondorl 7.0.4 2 ECNcondorl-7.0.4-2.x86_64.rpm
@endif
@if ARCH == "i386"
install ECNcondorb 7.0.4 2 ECNcondorb-7.0.4-2.i386.rpm
install ECNcondorl 7.0.4 2 ECNcondorl-7.0.4-2.i386.rpm
@endif

Notice the architecture type in the RPM filename - they should match the @if/@endif command block.

Moving applications from new → local → old.

During semester breaks, it is customary to cycle new version of applications into the default path, and the older version into the old path. We can make this change easier if the entire lifecycle of package installation files are made ahead of time.

In the case of our pcre package, we can move the current version to an old version by adjusting the igor commands in the package configuration file. If we start with a set of package installation command lines of:

@@
@@  pcre - Perl-compatible regular expressions
@@
install ECNpcreb 7.8 1 ECNpcreb-7.8-1.i386.rpm
@@install ECNpcren 7.8 1 ECNpcren-7.8-1.i386.rpm
install ECNpcrel 7.8 1 ECNpcrel-7.8-1.i386.rpm
@@install ECNpcreo 7.8 1 ECNpcreo-7.8-1.i386.rpm

We only need to remove the /usr/local package of symbolic links and add in the /usr/old package. Change the script to this:

@@
@@  pcre - Perl-compatible regular expressions
@@
install ECNpcreb 7.8 1 ECNpcreb-7.8-1.i386.rpm
@@install ECNpcren 7.8 1 ECNpcren-7.8-1.i386.rpm
uninstall ECNpcrel 7.8 1 ECNpcrel-7.8-1.i386.rpm
install ECNpcreo 7.8 1 ECNpcreo-7.8-1.i386.rpm

The next time a system updates its packages, the binary package will not change, the /usr/local package will be removed , the the /usr/old package will install. We need to remove the /usr/local explicitly.

Updating a package

The igor script that installs and removes packages knows how to upgrade a package implicitly. That is, if a package of version x is installed, and the package configuration file says that version x+1 should be installed instead, the package x will be upgraded. This elevates the need to explicitly add uninstall commands. There is no harm in adding explicit uninstall commands if you want to make sure a package is removed.

Package name conflicts

Because of the way Solaris and Linux handles base package names, it is sometimes necessary to uniquely identify a package name by a version tag. Yuch!

An example of this problem is when multiple versions of a package is needed. For example, if we want to run pcre for both version 7.7 and 7.8, we would have to create a separate base name. We cannot have a package named ECNpcreb that creates the directory path /opt/pcre/7.7 at the same time we have a package named ECNpcreb that creates the directory path /opt/pcre/7.8. Unfortunately we have to name the base package separately. We can adjust the package name on the new version and merge it into the package configuration file along with the older version.

This example creates two different versions of pcre. The newer version 7.8 in /usr/local and the older version 7.7 in /usr/old. Since the older version is already named pcre we need to adjust the name of the newer version.

@@
@@  pcre - Perl-compatible regular expressions
@@
install ECNpcreb 7.7 1 ECNpcreb-7.7-1.i386.rpm
install ECNpcreo 7.7 1 ECNpcreo-7.8-1.i386.rpm
install ECNpcre78b 7.8 1 ECNpcreb-7.8-1.i386.rpm
install ECNpcre78l 7.8 1 ECNpcrel-7.8-1.i386.rpm

Last modified: 2015/04/28 11:27:17.944018 GMT-4 by joseph.n.howell.1
Created: 2009/01/13 14:21:47.588000 US/Eastern by curtis.f.smith.1.

Categories

Search

Type in a few keywords describing what information you are looking for in the text box below.

Admin Options: Edit this Document