==============================================================================
                              XPK version 0.1.4                              

       Copyright (C) 1999-2000, Vesa Halttunen (a.k.a. Vesuri/dA JoRMaS)

            vesuri@jormas.com - http://www.jormas.com/~vesuri/xpk
==============================================================================

ABOUT
=====

XPK is an interfacing standard between application programs and packer
libraries. Every XPK application program can pack data with any XPK packer.

The XPK system consists of a master library (libxpkmaster.so) and several
(un)packer sublibraries (libxpkXXXX.so). Application programs only use the
master library directly: the master library takes care of loading and using
the sublibraries. Each sublibrary implements one type of compression. There
are different libraries for different types of data. When unpacking the
applications do not need to know which library was used to pack the data -
the appropriate library needs to be installed.

This port is not complete and is under development. Only a few sublibraries
are currently available and not all of them work perfectly.

Vesa Halttunen has only ported the XPK library from Amiga. Please send
feedback concerning the UNIX port ONLY to him. Please do not blame him for any
design flaws in the API. The author of the original Amiga version is aware of
them and is willing to help if someone wants to redesign XPK to better reflect
the modern trends. It may be a good idea, however, to finish off with a
properly working port of the original Amiga library so that former Amiga users
can easily import all their XPK packed stuff into new environments.

Some things have been done in a very Amiga-specific way. This means weird type
casting and storage size assumptions. These may cause problems on some
architectures.

The first UNIX version was created by Vesa Halttunen in December 1999. The
original Amiga distribution was created by Dirk Stcker in October 1996.

The concept, the most stuff and documentations of version 2.x distribution
were made by Bryan Ford and Urban Dominik Mller. A lot of bug fixes made
Christian von Roques. Vesa Halttunen takes care of the UNIX
version and Dirk Stcker continues the Amiga XPK development. The other persons
mentioned above have no time to do further support.

These documents were converted from the Amiga version. If something important
has been deleted please tell the author. Error reports and newer docs or
corrected contact addresses are welcome too. Some of them are probably
outdated.

If you have UNIXified versions of sub libraries or other enhancements please
contact the author!

TESTING
=======

This package has been tested in the following environments:

Operating system        Architecture(s)   Status
-------------------------------------------------------------------
Linux 2.x (glibc 2.1)   ix86, ppc, m68k   Compiles and works
Solaris 2.7             sparc, ix86       Compiles and works
AIX 4.3.2               ppc               Compiles and works
IRIX 6.5                mips              Compiles and works
Tru64 UNIX 4.0F         alpha             Compiles but doesn't work
HP-UX 11.00             hppa              Does not compile

XPK - A STANDARD FOR DATA COMPRESSION
=====================================

Motivation
----------
  * Many programs that should offer data compression (e.g. HD backup
    utilities) do not.
  * Many programs that offer data compression use old, slow,
    inefficient or inappropriate algorithm.
  * Most programs that offer data compression offer just one algorithm
    and you are stuck with that one.
  * Many good packers are not used by any application program and have
    no good user interface.
  * The decompression of all files packed with existing packers
    requires knowledge about the packer used for compression.
  * The existing compression programs are either slow or have a low
    compression factor.
    
The solution to all these problems is XPK.

Overview
--------
The XPK standard is to data compression what xpr is to file
transmission. It consists of three layers:
  * Level 2: The application/XPK interface for archives
  * Level 1: The application/XPK interface for files
  * Level 0: The XPK/packer interface
    
In addition, there is an optional standard XPK file format.

All parts of the XPK standard are implemented in shared libraries.
There is one master library for archive level access, one master
library for file level access, and one library for each compression
algorithm.

level 2                    libxpkarchive
                           |            |
                           V            |
level 1               libxpkmaster      |
                      |    |     |      V
level 0 type 3        |    |     |   xarZOO
                      |    |     V
        type 2        |    |  xexXPWP
                      V    V
        type 1   xpkNUKE xpkENCO

Level 0 libraries
-----------------
All level 0 libraries offer the same functions. They are very small.
Typical calls are: "Tell me what you can", "Compress this chunk of
memory to another chunk of memory", and "Decompress this chunk of
memory to another chunk of memory". These libs are very limited, their
functionality is expanded by libxpkmaster. No one would want to talk
to a sub library directly.

The Level 1 Library
-------------------
Offers functions like "Compress this file to that chunk of memory
using that algorithm". All combinations permitted: Mem to mem, file to
file, mem to file, decompression and compression. Asynchronous packing
possible. Very convenient tag based caller interface. Determines
automatically out which sub library to use for decompression. Returns
detailed error messages.

The Level 2 Library
-------------------
Offers archiving functions like "add this file to that archive" or
"show me the contents of that archive".

Overriding
----------
It is planned, that libraries of a lower level can offer higher level
functions. They should be able to override the automatic functionality
expansion by the higher level library. xpkmaster.library, for example,
enforces the use of the XPK standard file format. It should be
possible to override this by a sub library. Therefore an new library
interface will be created, the xex libraries.

The XPK File Format
-------------------
Offers checksums, chunks (important when seeks on a compressed file
become necessary) and automatic handling by the libxpkmaster. This
means that any new packer that can only pack mem to mem has its own
file format immediately. And most important: The name of the packer
library is contained in the file. Therefore, copying a new sub library
to libpath is all you have to do to install a new packer (easily done
in installation scripts); libxpkmaster recognizes the new file type
immediately. No changes to libxpkmaster or the application programs
necessary. In case the XPK file format is not used, the introduction
of a new packer requires a change the libxpkmaster.
   
Typical Applications
--------------------
A few examples for applications that could use XPK:
  * A GTK based archiver interface that can deal with all archivers
  * A shell based file compressor/decompressor [xPack, xPK, xUp]
  * A hard disk backup utility that stores compressed data
  * A tool to write compressed images of devices to files
  * A 'more' program with automatic decompression
  * A DTP program that stores its fonts in compressed format
  * A network protocol with built in data compression for slow
    connections
  * A hypertext utility that allows all data to be compressed
       
...plus many more we do not even need to think about yet.
 
Conclusion
----------
XPK would increase the usefulness and flexibility of both application
and compression programs while improving their user friendliness at
the same time. The best way to establish this standard would be to
distribute it with every UNIX distribution :).

THIS VERSION
============

Not implemented in this version:
  * xpkprefs (allows the user to define which sublibrary the xpkmaster library
              automatically uses when packing different kinds of files)
  * passwords (the code is in but totally untested due to lack of
               crypto-enabled sublibraries)
  * xfdmaster.library support (needs an xfdmaster.library UNIX port first)
  * xBench (XPK sublibrary benchmarking tool)

Stuff completely removed from the Amiga version:
  * xScan (uses the Amiga file comments not available on UNIX)

Known problems:

On AIX libtool doesn't create a symbolic link 'libname.so' to the actual
shared library. Currently libxpkmaster searches for libxpkXXXX.so instead
of libxpkXXXX.so.v (v being the version number) so this link needs to be
made by hand:

ln -s libxpkRLEN.so.0 libxpkRLEN.so

DOCUMENTATION
=============

The documentation for the supplied sublibraries and the documentation for the
included programs are available. The library API documentation is only
available in the Amiga autodoc format for now. The autodocs are in the doc
directory of the source archive. When using them you should note the following
type conversions:

CHANGES BETWEEN THE UNIX AND THE AMIGA VERSION
==============================================

The following type conversions have been used:

  ULONG  -> unsigned int
  LONG   -> int
  UWORD  -> unsigned short
  WORD   -> short
  UBYTE  -> unsigned char (or char when appropriate)
  BYTE   -> char
  APTR   -> pointer to something
  BPTR   -> pointer to something
  STRPTR -> char *

In struct XpkPackerList the xpk_Packer entry was changed from
char xpl_Packer[MAXPACKERS][6] to unsigned int xpl_Packer[MAXPACKERS]
because the Amiga version actually wrote an unsigned int to the character
array assuming that the bytes are ordered like they are on Amiga. This
ridiculous idea didn't quite work on Solaris nor on IRIX so I changed the
behaviour a bit.

EXAMPLES
========

Check out the example tools in the shell directory to see how to use the
library. Documentation for the tools is available in the AmigaGuide file but
they should be pretty straight forward. Try

xPK -m RLEN README
xPK -e README

ADDRESSES RELATED TO XPK
========================

Please remember that some of these addresses may be false. Do not
blame if you do not get any answer. If you get newer information,
please contact the author.

UNIX version [http://www.jormas.com/~vesuri/xpk/]
  * Vesa Halttunen <vesuri@jormas.com> [http://www.jormas.com/~vesuri/]
     
Amiga version [http://www.amigaworld.com/support/xpkmaster]
  * Dirk Stcker <stoecker@amigaworld.com> [http://home.pages.de/~Gremlin/]
  * Christian von Roques <roques@pond.sub.org>
  * Urban Dominik Mller <umueller@indiac.relog.ch>
  * Bryan Ford <bryan.ford@m.cc.utah.edu>
