  Building Software Packages for Linux
  Mendel Leo Cooper <mailto:thegrendel@theriver.com>
  http://personal.riverusers.com/~thegrendel/
  v1.52, 27 December 1997

  This is a comprehensive guide to building "generic" UNIX software dis
  tributions under Linux.

  1.  Introduction

  Many software packages for the various flavors of UNIX, including
  Linux, are distributed as compressed archives of source files.  The
  same package may be "built" to run on different target machines, and
  this saves the author of the software from having to produce multiple
  distributions. A single distribution of a software package may thus
  end up running, in various incarnations, on an Intel box, a DEC Alpha,
  a RISC workstation, or even a mainframe.  Unfortunately, this puts the
  responsibility of actually "building" the software on the end user,
  the de facto "system administrator", the fellow sitting at the
  keyboard...  you.  Take heart, though, the process is not nearly as
  terrifying or mysterious as it seems, as this guide will demonstrate.

  2.  Getting Started

  You have downloaded or otherwise acquired a software package.  Most
  likely it is archived (tarred) and compressed (gzipped), in .tar.gz or
  .tgz form. First copy it to a working directory. Then untar and gunzip
  it. The appropriate command for this is tar xzvf filename, where
  filename is the name of the software file, of course.  The de-
  archiving process will usually install the appropriate files in
  subdirectories it will create.  Note that if the package name has a .Z
  suffix, it will require uncompress PACKAGENAME, then tar xvf
  PACKAGENAME rather than the above procedure.

  Sometimes the archived file must be untarred and installed from the
  user's home directory, or perhaps in a certain other directory, as
  specified in the package's config info.  Should you get an error
  message attempting to untar it, this may be the reason. Read the
  package docs, especially the README and/or Install files, if present,
  and edit the config files and/or Makefiles as necessary, consistent
  with the installation instructions. Note that you would not ordinarily
  alter the Imake file, since this could have unforseen consequences.
  Some software packages permit automating this process by running make
  install to emplace the binaries in the appropriate system areas.

  Occasionally, you may need to update or incorporate bug fixes into the
  unarchived source files using a patch file that lists the changes.
  The doc files and/or README file will inform you should this be the
  case. The normal syntax for invoking Larry Wall's powerful patch
  utility is patch < patchfile.

  You may now proceed to the build stage of the process.

  3.  Using Make

  The Makefile is the key to the build process. In its simplest form, a
  Makefile is a script for compiling or building the "binaries", the
  executable portions of a package. The Makefile can also provide a
  means of updating a software package without having to recompile every
  single source file in it, but that is a different story (or a
  different article).

  At some point, the Makefile launches cc or gcc. This is actually a
  preprocessor, a C (or C++) compiler, and a linker, invoked in that
  order.  This process converts the source into the binaries, the actual
  executables.

  Invoking make usually involves just typing make. This generally builds
  all the necessary executable files for the package in question.
  However, make can also do other tasks, such as installing the files in
  their proper directories (make install)and removing stale object files
  (make clean).  Running make -n permits previewing the build process,
  as it prints out all the commands that would be triggered by a make,
  without actually executing them.

  Only the simplest software uses a generic Makefile. More complex
  installations require tailoring the Makefile according to the location
  of libraries, include files, and resources on your particular machine.
  This is especially the case when the build needs the X11 libraries to
  install. Imake and xmkmf accomplish this task.

  An Imakefile is, to quote the man page, a "template" Makefile. The
  imake utility constructs a Makefile appropriate for your system from
  the Imakefile. In almost all cases, however, you would run xmkmf, a
  shell script that invokes imake, a front end for it.  Check the README
  or INSTALL file included in the software archive for specific
  instructions.  Read the imake and xmkmf man pages for a more detailed
  analysis of the procedure..

  Be aware that xmkmf and make may need to be invoked as root,
  especially when doing a make install to move the binaries over to the
  /usr/bin or /usr/local/bin directories.  Using make as an ordinary
  user without root privileges will likely result in write access denied
  error messages because you lack write permission to system
  directories. Check also that the binaries created have the proper
  execute permissions for you and any other appropriate users.

  Invoking xmkmf uses the Imake file to build a new Makefile appropriate
  for your system. You would normally invoke xmkmf with the -a argument,
  to automatically do a make Makefiles, make includes, and make depend.
  This sets the variables and defines the library locations for the
  compiler and linker.  Sometimes, there will be no Imake file, instead
  there will be an INSTALL or configure script that will accomplish this
  purpose. Note that if you run configure, it should be invoked as
  ./configure to ensure that the correct configure script in the current
  directory is called. In most cases, the README file included with the
  distribution will explain the install procedure.

  It is usually a good idea to visually inspect the Makefile that xmkmf
  or one of the install scripts builds. The Makefile will normally be
  correct for your system, but you may occasionally be required to
  "tweak" it or correct errors manually.

  Your general installation procedure will therefore be:

    Read the README file and other applicable docs.

    Run xmkmf -a, or the INSTALL or configure script.

    Check the Makefile.

    If necessary, run make clean, make Makefiles, make includes, and
     make depend.

    Run make.

    Check file permissions.

    If necessary, run make install.

  4.  Troubleshooting

  If xmkmf and/or make succeeded without errors, you may proceed to the
  ``next section''.  However, in "real life", few things work right the
  first time.  This is when your resourcefulness is put to the test.

  4.1.  Link Errors

    Suppose make fails with a Link error: -lX11: No such file or
     directory, even after xmkmf has been invoked. This may mean that
     the Imake file was not set up properly. Check the first part of the
     Makefile for lines such as:

       LIB=            -L/usr/X11/lib
       INCLUDE=        -I/usr/X11/include/X11
       LIBS=           -lX11 -lc -lm

  The -L and -I switches tell the compiler and linker where to look for
  the library and include files, respectively. In this example, the X11
  libraries should be in the /usr/X11/lib directory, and the X11 include
  files should be in the /usr/X11/include/X11 directory. If this is
  incorrect for your machine, make the necessary changes to the Makefile
  and try the make again.

    In a very few cases, running ldconfig as root may be the solution:

     # /etc/ldconfig -n /lib will update the shared library symbolic
     links. This should not be necessary under normal circumstances.

    Yet another thing to try if xmkmf fails is the following script:

                make -DUseInstalled -I/usr/X386/lib/X11/config

    Sometimes the source needs the older release X11R5 libraries to
     build.  If you have the R5 libs in /usr/X11R6/lib (you were given
     the option of having them when first installing Linux), then you
     need only ensure that you have the links that the software needs to
     build.  The R5 libs are named libX11.so.3.1.0, libXaw.so.3.1.0, and
     libXt.so.3.1.0. You generally need links, such as libX11.so.3 ->
     libX11.so.3.1.0. Possibly the software will also need a link of the
     form libX11.so -> libX11.so.3.1.0.  Of course, to create a
     "missing" link, use the command ln -s libX11.so.3.1.0 libX11.so, as
     root.

    Some packages will require you to install updated versions of one
     or more libraries. For example, the StarOffice suite from
     StarDivision GmbH is notorious for needing a libc version 5.4.4 or
     greater.  As root, you would need to copy one or more libraries to
     the appropriate directories, remove the old libraries, then reset
     the symbolic links.

     Caution: Exercise extreme care in this, as you can render your
     system nonfunctional if you screw up.

     You can usually find updated libraries at Sunsite
     <ftp://sunsite.unc.edu>.

  4.2.  Other Problems

    An installed Perl or shell script gives you a No such file or
     directory error message. In this case, check the file permissions
     to make sure the file is executable and check the file header to
     ascertain whether the shell or program invoked by the script is in
     the place specified.  For example, the scrip may begin with:

       #!/usr/local/bin/perl

  If Perl is in fact installed in your /usr/bin directory instead of the
  /usr/local/bin one, then the script will not run.  There are two meth
  ods of correcting this. The script file header may be changed to
  #!/usr/bin/perl, or a symbolic link to the correct directory may be
  added, ln -s /usr/bin/perl /usr/local/bin/perl.

    Some X11 software requires the Motif libraries to build.  The
     standard Linux distributions do not have the Motif libraries
     installed, and at present Motif costs an extra $100-$200 (though
     the freeware Lesstif <http://www.lesstif.org/> also works in many
     cases). If you need Motif to build a certain package, but lack the
     Motif libraries, it may be possible to obtain statically linked
     binaries. Static linking incorporates the library routines in the
     binaries themselves.  This results in much larger binary files, but
     the code will run on systems lacking the libraries.

    Running a configure script creates a strange Makefile, one
     seemingly unrelated to the package you are attempting to build.
     This means the wrong configure ran, one found somewhere else in
     your path. Always invoke configure as ./configure to prevent this.

    Some programs require having setuid root, in order to run with root
     privileges. The command to implement this is chmod u+s filename, as
     root (note that the program must already be owned by root). This
     has the effect of setting the setuid bit in the file permissions.
     This issue comes up when the program accesses the system hardware,
     such as a modem or CD ROM drive, or when the SVGA libs are invoked
     from console mode, as in one particularly notorious emulation
     package. If a program works when run by root, but gives access
     denied error messages to an ordinary user, suspect this as the
     cause.

     Warning: A program with setuid as root may pose a security risk to
     your system. The program runs with root privileges and thus has the
     potential for doing significant damage. Make certain that you know
     what the program does, by looking at the source if necessary,
     before setting the setuid bit.

  4.3.  Tweaking and fine tuning

  You may wish to examine the Makefile to make certain that the best
  compilation options for your system are invoked. For example, setting
  the -O2 flag chooses the highest level of optimization and the -fomit-
  frame-pointer flag results in a smaller binary (though debugging will
  then be disabled). Do not play around with this unless you know what
  you are doing, and in any case, until after a trial build works.

  4.4.  Where to go for more help

  In my experience, perhaps 25% of applications build "right out of the
  box". Another 50% or so can be "persuaded" to build with an effort
  ranging from trivial to herculean. That still means a significant
  number of packages will not build no matter what. Even then, the Intel
  ELF and/or a.out binaries for these might possibly be found at Sunsite
  <ftp://sunsite.unc.edu>, the TSX-11 archive <ftp://tsx-11.mit.edu> or
  other places.  Perhaps the author of the software can supply the
  binaries compiled for your particular flavor of machine.

  Note that if you obtain precompiled binaries, you will need to check
  for compatibility with your system:

    The binaries must run on your hardware (i.e., Intel x86).

    The binaries must be compatible with your kernel (i.e., a.out or
     ELF).

    Your libraries must be up to date.

  If all else fails, you may find help in the appropriate newsgroups,
  such as comp.os.linux.x or comp.os.linux.development.  Once in a
  while, though, you are just plain out of luck, but hey, it was fun
  trying.

  5.  Final Steps

  Read the software package documentation to determine whether certain
  environmental variables need setting (in .bashrc or .cshrc) and if the
  .Xdefaults and .Xresources files need customizing.

  There may be an applications default file, usually named Xfoo.ad in
  the original Xfoo distribution. If so, edit the Xfoo.ad file to
  customize it for your machine, then rename (mv) it Xfoo and install it
  in the /usr/lib/X11/app-defaults directory, as root.  Failure to do
  this may cause the software to behave strangely or even refuse to run.

  Most software packages come with one or more preformatted man pages.
  As root, copy the Xfoo.man file to the appropriate /usr/man directory
  (man1 - man9), and rename it accordingly.  For example, if Xfoo.man
  ends up in /usr/man/man4, it should be renamed Xfoo.4 (mv Xfoo.man
  Xfoo.4).  By convention, user commands go in man1, games in man6, and
  administration packages in man8 (see the man docs for more details).
  Of course, you may deviate from this on your own system, if you like.

  Some packages will not install the binaries in the appropriate system
  directories, that is, they are missing the install option in the
  Makefile. Should this be the case, you can install the binaries
  manually by copying the binaries to the usr/local/bin directory, as
  root.

  Note that some or all of the above procedures should, in most cases,
  be handled automatically by a make install. If so, the README or
  INSTALL doc file will specify this.

  6.  First Example: Xscrabble

  Matt Chapman's Xscrabble seemed like a program that would be
  interesting to have, since I happen to be an avid Scrabble[TM] player.
  I downloaded it, uncompressed it,  and built it following the
  procedure in the README file:

            xmkmf
            make Makefiles
            make includes
            make

  Of course it did not work...

  ______________________________________________________________________
  gcc -o xscrab -O2 -O -L/usr/X11R6/lib
  init.o xinit.o misc.o moves.o cmove.o main.o xutils.o mess.o popup.o
  widgets.o display.o user.o CircPerc.o
  -lXaw -lXmu -lXExExt -lXext -lX11 -lXt -lSM -lICE -lXExExt -lXext -lX11
  -lXpm -L../Xc -lXc

  BarGraf.o(.text+0xe7): undefined reference to `XtAddConverter'
  BarGraf.o(.text+0x29a): undefined reference to `XSetClipMask'
  BarGraf.o(.text+0x2ff): undefined reference to `XSetClipRectangles'
  BarGraf.o(.text+0x375): undefined reference to `XDrawString'
  BarGraf.o(.text+0x3e7): undefined reference to `XDrawLine'
  etc.
  etc.
  etc...
  ______________________________________________________________________

  I enquired about this in the comp.os.linux.x newsgroup, and someone
  kindly pointed out that apparently the Xt, Xaw, Xmu, and X11 libs were
  not being found at the link stage. Hmmm...

  There were two main Makefiles, and the one in the src directory caught
  my interest. One line in the Makefile defined LOCAL_LIBS as:
  LOCAL_LIBS = $(XAWLIB) $(XMULIB) $(XTOOLLIB) $(XLIB) Here were
  references to the libs not being found by the linker.

  Looking for the next reference to LOCAL_LIBS, I saw on line 495 of
  that Makefile:

             $(CCLINK) -o $@ $(LDOPTIONS) $(OBJS) $(LOCAL_LIBS) $(LDLIBS)
       $(EXTRA_LOAD_FLAGS)

  Now what were these LDLIBS?

             LDLIBS = $(LDPOSTLIB) $(THREADS_LIBS) $(SYS_LIBRARIES)
       $(EXTRA_LIBRARIES)

  The SYS_LIBRARIES were:

        SYS_LIBRARIES = -lXpm -L../Xc -lXc

  Yes! Here were the missing libraries.

  Possibly the linker needed to see the LDLIBS before the LOCAL_LIBS...
  So, the first thing to try was to modify the Makefile by transposing
  the $(LOCAL_LIBS) and $(LDLIBS) on line 495, so it would now read:

               $(CCLINK) -o $@ $(LDOPTIONS) $(OBJS) $(LDLIBS) $(LOCAL_LIBS)
       $(EXTRA_LOAD_FLAGS)                          ^^^^^^^^^^^^^^^^^^^^^^^

  I tried running make again with the above change, and lo and behold,
  it worked this time. Of course,  Xscrabble still needed some fine
  tuning and twiddling, such as renaming the dictionary and commenting
  out some assert statements in one of the source files, but since then
  it has provided me with many hours of pleasure.

  You may e-mail Matt Chapman <mailto:matt@belgarath.demon.co.uk>, and
  download Xscrabble from his home page
  <http://www.belgarath.demon.co.uk/programs/index.html>.

  ______________________________________________________________________
         Scrabble is a registered trademark of the Milton Bradley Co., Inc.
  ______________________________________________________________________

  7.  Second Example: Xloadimage

  This example poses an easier problem. The xloadimage program seemed a
  useful addition to my set of graphic tools.  I copied the xloadi41.gz
  file directly from the source directory on the CD included with the
  excellent ``X User Tools'' book, by Mui and Quercia. As expected, tar
  xzvf unarchives the files.  The make, however, produces a nasty-
  looking error and terminates.

  ______________________________________________________________________
  gcc -c -O -fstrength-reduce -finline-functions -fforce-mem
  -fforce-addr -DSYSV  -I/usr/X11R6/include
  -DSYSPATHFILE=\"/usr/lib/X11/Xloadimage\" mcidas.c

  In file included from /usr/include/stdlib.h:32,
                   from image.h:23,
                   from xloadimage.h:15,
                   from mcidas.c:7:
  /usr/lib/gcc-lib/i486-linux/2.6.3/include/stddef.h:215:
  conflicting types for `wchar_t'
  /usr/X11R6/include/X11/Xlib.h:74: previous declaration of
  `wchar_t'
  make[1]: *** [mcidas.o] Error 1
  make[1]: Leaving directory
  `/home/thegrendel/tst/xloadimage.4.1'
  make: *** [default] Error 2
  ______________________________________________________________________

  The error message contains the essential clue.

  Looking at the file image.h, line 23...

  ______________________________________________________________________
         #include <stdlib.h>
  ______________________________________________________________________

  Aha, somewhere in the source for xloadimage, wchar_t has been rede
  fined from what was specified in the standard include file, stdlib.h.
  Let us first try commenting out line 23 in image.h, as perhaps the
  stdlib.h include is not, after all, necessary.

  At this point, the build proceeds without any fatal errors. The
  xloadimage program functions correctly now.

  8.  Third Example: Fortune

  This final example requires some knowledge of C programming. The
  majority of Linux software is written in C, and learning at least a
  little bit of C would certainly be an asset for anyone serious about
  software installation.

  The notorious fortune program displays up a humorous saying, a
  "fortune cookie", every time Linux boots up. Unfortunately (pun
  intended), attempting to build fortune on a Red Hat distribution with
  a 2.0.30 kernel generates fatal errors.

  ______________________________________________________________________
  ~/fortune# make all

  gcc -O2 -Wall -fomit-frame-pointer -pipe   -c fortune.c -o
  fortune.o
  fortune.c: In function `add_dir':
  fortune.c:551: structure has no member named `d_namlen'
  fortune.c:553: structure has no member named `d_namlen'
  make[1]: *** [fortune.o] Error 1
  make[1]: Leaving directory `/home/thegrendel/for/fortune/fortune'
  make: *** [fortune-bin] Error 2
  ______________________________________________________________________

  Looking at fortune.c, the pertinent lines are these.

  ______________________________________________________________________
     if (dirent->d_namlen == 0)
              continue;
          name = copy(dirent->d_name, dirent->d_namlen);
  ______________________________________________________________________

  We need to find the structure dirent, but it is not declared in the
  fortune.c file, nor does a grep dirent show it in any of the other
  source files. However, at the top of fortune.c, there is the following
  line.

  ______________________________________________________________________
  #include <dirent.h>
  ______________________________________________________________________

  This appears to be a system library include file, therefore, the
  logical place to look for dirent.h is in /usr/include.  Indeed, there
  does exist a dirent.h file in /usr/include, but that file does not
  contain the declaration of the dirent structure.  There is, however, a
  reference to another dirent.h file.

  ______________________________________________________________________
  #include <linux/dirent.h>
  ______________________________________________________________________

  At last, going to /usr/include/linux/dirent.h, we find the structure
  declaration we need.

  ______________________________________________________________________
  struct dirent {
          long            d_ino;
          __kernel_off_t  d_off;
          unsigned short  d_reclen;
          char            d_name[256]; /* We must not include
  limits.h! */
  };
  ______________________________________________________________________

  Sure enough, the structure declaration contains no d_namelen, but
  there are a couple of "candidates" for its equivalent. The most likely
  of these is d_reclen, since this structure member probably represents
  the length of something and it is a short integer.  The other
  possibility, d_ino, could be an inode number, judging by its name and
  type. As a matter of fact, we are probably dealing with a "directory
  entry" structure, and these elements represent attributes of a file,
  its name, inode, and length (in blocks).  This would seem to validate
  our guess.

  Let us edit the file fortune.c, and change the two d_namelen
  references in lines 551 and 553 to d_reclen.  Try a make all again.
  Success. It builds without errors. We can now get our "cheap thrills"
  from fortune.

  9.  Where to Find Source Archives

  Now that you are eager to use your newly acquired knowledge to add
  utilities and other goodies to your system, you may find them online
  at the Linux Applications and Utilities Page
  <http://www.redhat.com/linux-info/linux-app-list/linapps.html>, or on
  one of the very reasonably priced CD ROM archives by Red Hat
  <http://www.redhat.com/>, InfoMagic <mailto:orders@infomagic.com>, and
  others.

  A comprehensive repository of source code is the comp sources UNIX
  archive <ftp://ftp.vix.com/pub/usenet/comp.sources.unix/>.

  Much UNIX source code is posted on the alt.sources newsgroup. If you
  are looking for particular source code packages, you may post on the
  related alt.sources.wanted newsgroup.  Another good place to check is
  the comp.os.linux.announce newsgroup.  To get on the Unix sources
  <mailto:unix-sources@pa.dec.com> mailing list, send a subscribe
  message there.

  Archives for the alt.sources newsgroup are at the following ftp sites:

    ftp.sterling.com/usenet/alt.sources/
     <ftp://ftp.sterling.com/usenet/alt.sources/>

    wuarchive.wustl.edu/usenet/alt.sources/articles
     <ftp://wuarchive.wustl.edu/usenet/alt.sources/articles>

    src.doc.ic.ac.uk/usenet/alt.sources/articles
     <ftp://src.doc.ic.ac.uk/usenet/alt.sources/articles>

  10.  Final Words

  To sum up, persistence makes all the difference  (and a high
  frustration threshold certainly helps). As in all endeavors, learning
  from mistakes is critically important.  Each misstep, every failure
  contributes to the body of knowledge that will lead to mastery of the
  art of building software.

  11.  References and Further Reading

  BORLAND C++ TOOLS AND UTILITIES GUIDE, Borland International, 1992,
  pp. 9-42.
  [One of the manuals distributed with Borland C++, ver. 3.1. Gives
  a fairly good intro to make syntax and concepts, using Borland's
  crippled implementation for DOS.]

  DuBois, Paul: SOFTWARE PORTABILITY WITH IMAKE, O'Reilly and Associates,
  1996, ISBN 1-56592-226-3.
  [This is reputed to be the definitive imake reference, though I did not
  have it available when writing this article.]

  Frisch, Aeleen: ESSENTIAL SYSTEM ADMINISTRATION, O'Reilly and
  Associates, 1995, ISBN 1-56592-127-5.
  [This otherwise excellent sys admin handbook has only sketchy coverage
  of software building.]

  Lehey, Greg: PORTING UNIX SOFTWARE, O'Reilly and Associates, 1995, ISBN
  1-56592-126-7.

  Mui, Linda and Valerie Quercia: X USER TOOLS, O'Reilly and Associates,
  1994, ISBN 1-56592-019-8, pp. 734-760.

  Oram, Andrew and Steve Talbott: MANAGING PROJECTS WITH MAKE, O'Reilly
  and Associates, 1991, ISBN 0-937175-90-0.

  Peek, Jerry and Tim O'Reilly and Mike Loukides: UNIX POWER TOOLS,
  O'Reilly and Associates / Random House, 1997, ISBN 1-56592-260-3.
  [A wonderful source of ideas, and tons of utilities you may end up
  building from the source code, using the methods discussed in
  this article.]

  Stallman, Richard M. and Roland McGrath: GNU MAKE, Free Software
  Foundation, 1995, ISBN 1-882114-78-7.
  [Should be required reading.]

  Welsh, Matt and Lar Kaufman: RUNNING LINUX, O'Reilly and Associates,
  1996, ISBN 1-56592-151-8.
  [Still the best overall Linux reference, though lacking in depth
  in some areas.]

  And, of course, the man pages for make, imake, xmkmf, gcc, ldconfig,
  gzip, tar, and patch.

