Installation Instructions
*************************

I assume you are familiar with the GNU build system, so I push generic
instructions below (search for "Original Installation Instructions".)

File locations and other macros are defined in config_names.h and some of them
have equivalent --enable-* configure options. For example, in order to build
debug versions without disturbing the running system, I do like so:

	./configure CFLAGS="-O0 -g -DIPQBDB_DEBUG" \
		--enable-database-prefix=$PWD/testdb     \
		--enable-pcre-file=$PWD/testdb/pcre.conf  \
		--enable-option-file=$PWD/testdb/ipqbdb.popt

The ibd-config utility displays those values.

You should not build the package as root. In particular, make check will fail
if you run it as root. That way, make check only checks basic program workings,
including regular expressions and database, but not iptables. You need root
privileges to run ibd-judge, and also to install it.

Having suitable configuration helps invoking each program with fewer options
and some consistency. See also "Setting popt options" below.

Upgrading
=========
Version 1.01 is compatible with version 1.0. However, you have to stop the
running daemons before `make install'. If you don't disable the iptables rule
you'll be unreachable until you restart the (new) ibd-junge. Restart syslog as
needed after restarting ibd-parse.

Versions 1.* are not compatible with earlier release 0.1 --that's the reason
for bumping the major version number. The simplest solution is to uninstall the
old version completely, which includes

   * deleting the iptables rules that NQUEUE to ipqbdbd,
   * remove the ipqbdbd executable (its name is ibd-judge in v1.0),
   * delete the existing database files (by default /var/lib/ipqbdb/*.db).

(An alternative solution is to adjust paths and queue numbers so as to keep
both versions installed at the same time.)
 
After compile and install are done, you should complete the installation
by carrying out the relevant configuration. I split that into three sections:
iptables, databases, and popt.


Testing iptables
================

You're somewhat familiar with iptables, aren't you? If not, Arno's IPtables-
firewall[1] and Shorewall[2] are two tools that build customized scripts that
can be used as an example of how to structure an iptables configuration. For
the purpose of making examples, let's assume you have a chain named "my_server"
that is the last step in the filter table, e.g. you might call it for a web
server like so:

iptables -A previous -p tcp --syn -m multiport --dports 13,80,443 -j my_server

Port 13 is open. Most inetd version can serve the daytime tcp service
internally, and it is quite handy to run telnet to port 13, for the first one
of the two tests we'll do. The actual my_server chain may just consist of, say,

iptables -A my_server -m limit --limit 10/second --limit-burst 40 -j ACCEPT
iptables -A my_server -m limit --limit 5/hour --limit-burst 5 -j\
	LOG --log-level $LOG_LEVEL --log-prefix "syn limit:"
iptables -A my_server -j REJECT

In such environment, we prepend a rule for testing like so:

server# iptables -I my_server -p tcp --destination-port 13 -j NFQUEUE

If all goes well, xt_NFQUEUE will be loaded (check it with lsmod), and if you
try and telnet to port 13, your connection times out. Now run ibd-judge with -D
and -v flags for no-daemon verbose debugging. Trying again a connection should
now work, and ibd-judge reports the address is not in the database. Run ibd-ban
for that address from a different terminal. Using higher verbosity (-v is the
same as -v 0) you see more data, such as the integer probability and the delta
after a rand() tossing, if the verdict is BLOCK, you may see "BLOCK <ip> again"
lines, for automatic retries that the remote software does automatically.
Notice what ibd-del -Lv displays. After some time, the address is rehabilitated.
Give Ctrl-C to quit ipqbdbd and, again, you won't be able to connect, until you
either run ipqbdbd or delete the relevant NFQUEUE rule.

If the above test has worked, you can delete the testing rule and set up a
definitive one. E.g.

iptables -A my_server -m limit --limit 10/second --limit-burst 40 -j NFQUEUE

Notice that the syn flood limit above allows to calculate the maximum number of
records in the block database(s). If each of the 10 connections per seconds
results in a record, I can add up to 864,000 records in one day.

Now for stopping established connections. The rule above implicitly implied
queue 0, which is also ibd-judge's default. In this case, NFQUEUE is equivalent
to either ACCEPT or DROP, according to the verdict. Now, run

   ibd-judge [OPTIONS] Q0S Q2DM4

(--no-daemon and --verbose=4 may be suitable options for a test.) Q0S --take
queue 0 and look at the source address-- is what we have done earlier. Q2DM4 is
for taking queue 2, look at the destination address, and mark the packet with
value 4 in case it's found guilty. You want to look at the destination address
when you issue NQUEUE from the OUTPUT hook. The M in the queue argument tells
to mark and ACCEPT the packet, so it won't be seen again on the same table. The
raw table runs at a higher priority, thus you can do like so:

iptables -t raw -A OUTPUT -p tcp ! --syn -m multiport --sports $MY_TCP -j NFQUEUE --queue-num 2
iptables -A OUTPUT -p tcp ! --syn -m mark --mark 4 -j REJECT --reject-with tcp-reset

The second rule will load xt_mark.

Those can be definitive rules or test ones, depending on $MY_TCP. For tests,
you may want to put additional match options that restrict the rules to your
test packets. And you need a service that you can telnet manually, with fairly
long timeouts. While connected, ban yourself. The server's response packet will
then be marked and rejected. The server will get a tcp-reset, so it can cleanly
dismiss the connection. The client won't get any response and will eventually
time out.

If the above testing has been positive, you should add scripts that set up
iptables and run the daemons. More on this below.

Note that none of these programs use child processes, except for detaching on
startup. You can safely use killall -e ibd-parse ibd-judge from a script.

If you build your kernel with CONFIG_NETFILTER_DEBUG your dmesg will be filled
with "nf_hook: Verdict = QUEUE." So, don't enable that.

See also:
[1] Arno's IPtables-firewall http://rocky.eld.leidenuniv.nl/
[2] Shorewall http://shorewall.net/


Filling the databases
=====================

Review your log files, catching IPs that youd'like to block. Use an editor
to create the IPQBDB_PCRE_FILE file. You have to match the log lines with
Perl 5 Compatible Regular Expression (pcre) expressions, e.g.:

   /LOGIN FAILED, user=\S* ip=\[<HOST>]/ * "dictionary attack" 5

The format is:

   /regex/  I  "reason"  [initial-count]  [filename]  [decay]

Blank lines and lines whose first non-blank char is a # (hash) are ignored.
The six fields of germane lines are explained in the following six paragraphs.
A - (minus) can be used as a placeholder for optional fields.

Around regex you may use whatever delimiter (not only slashes). See
man pcresyntax for help. Like fail2ban, you may use the <HOST> macro;
unlike fail2ban it is replaced by (?P<IP>([0-9]{1,3}\.){3}[0-9]{1,3})
that only matches IPv4 addresses, and is named IP (not host).

The I field should be a single * (star) if you use <HOST>, otherwise it should
be a - (minus) if you use a named subexpression, otherwise it should be the
number of the subexpression that delivers the IPv4 address.

The reason field will be searched and possibly inserted in a description table,
if it's not a numeric key already. (Although using description numbers may get
an imperceivable efficiency gain, if you do that you'll have to prepare a
script that restores the descriptions in the given order; not recommended.)

The initial-count is the number of times after the first that you want the IP
to be caught before reaching 100% probability of being blocked. In one phrase,
the lower the initial-count, the sooner the IP will be blocked. It can range
from 0 to the number of bits of the RAND_MAX value (defined by the C library.)
The initial count only affects the first time an IP is caught. Thereafter the
probability doubles each time. However, if an IP is caught issuing an initial
count of, say, 1 --a 50% probability--, but its current probability is lower
than the half of that --12.50%--, then the highest one is set. In the latter
case, the reason is possibly changed too.

filename is the database file where records from matching expressions are
inserted or updated. Prefix and suffix will be added. ibd-parse checks that
all databases are in the same directory, and writes the socket there.

The decay field specifies how many seconds are needed for the probability
to halve. The higher the decay, the longer the IP will be kept blocked.
If you had read config_names.h, you'd know that there is a threshold,
IPQBDB_PROBABILITY_THRESHOLD, that triggers the addition of a fixed amount
of seconds, IPQBDB_DECAY_ADDEND_ON_THRESHOLD, to the decay when passed
upward. That way, an IP that oscillates frequently, gets rehabilitated
slower and slower.

The last four fields have exactly the same meaning in ibd-ban.

Before running ibd-parse, make sure the directory of IPQBDB_DATABASE_PREFIX
exists and has the permissions you want. If you didn't change it, the prefix
is /var/lib/ipqbdb. However, you don't have to be root for running ibd-parse,
and if you run it with, say, --db-descr=~/my-tests/descr, while no filename
in the pcre file specifies paths, ibd-parse will use that directory. It will
also contain the socket, and files named __db.00?. They are the Berkeley db
Environment. You may want to adjust bdb behavior by writing a DB_CONFIG file
in that directory. Using DB_CONFIG you may set values such as cache size, page
size, etcetera. See bdb documentation[1]. Ipqbdb uses the Concurrent Data Store
(CDB) model, that supports deadlock-free, multiple-readers/single-writer
semantics, but not record or region locking. (See READMEconcurrency.)

To test pcre expressions, run ibd-parse with the -D and -v options, for
no-daemon verbose debugging. The verbose output will display some values for
each expression, such as the highest back reference and the size of a compiled
pattern, that may be useful if performance is a issue. See also the pcreperform
man page.

After all expressions are compiled, check, on a different terminal window
that the socket file has been created, and give it some stuff, e.g.

  cat /var/log/mail.log > ibd-parse.sock
  
Check the previous terminal window for results. You may send SIGUSR1 to a
running ibd-parse process to get the number of matches and errors for each
expression. There should be no error! (If the program is daemonized, it will
write on daemon.log.) You may send it SIGHUP to re-read its config file.
Ctrl-C, SIGINT, or SIGTERM terminate it cleanly.

To install ibd-parse properly, you should configure your syslog.conf so that
it writes the log lines you need to the ibd-parse.sock named fifo. Sysklogd,
for example, the stock debian logger, wants a | (pipe) before the filename.
Try and address only the facilities and severities that are relevant for the
lines that you want to catch. It may deliver more performance to monitor
different facilities using different processes listening on different sockets
with less expression each, because for each line of log ibd-parse tries every
expression until one possibly matches. If you want to run it as a different
user, just make sure it can read config files and write the databases. In
addition, ibd-parse should create its socket before syslogd starts.

You can run ibd-del -Lv to list the contents of a block database; ibd-del is
for deleting entries, but it requires the --del flag to actually do that. Old
records should be deleted to avoid wasting space, and it is up to you to decide
what is the amount of time that makes a record obsolete and schedule ibd-del
execution with the appropriate permissions and options, e.g.

  ibd-del --min-time=4d --max-records=4M --del

will delete all records that have not been updated for at least four days. By
specifying --max-records you allow ibd-del to enlarge the min-time given, as
long as the number of records is not exceeded. When records are deleted from
block.db, the relevant counters are added to the currently ascribed reason in
descr.db. Use ibd-del --stats -v to display the latter counters. The verbose
(-v) option enables column headers in ibd-del's output for --ls and --stats.

If you need to block IPs from your own scripts, you don't have to write a log
line to ibd-parse: ibd-ban if just for that. You can specify the same values
by giving the relevant command line options. In addition, by not specifying an
IP address, you can use ibd-ban to populate descr.db.

You'll need to give it proper permissions, e.g. make it suid by

   server# chmod u+s /usr/local/bin/ipb-ban

where ownership of the databases match that of the executables.

You can use the white.db to twist the decay value for specific IPs. The
ibd-white utility reads from stdin lines formatted like so:

   192.0.2.1                0.0
   192.0.2.10-192.0.2.20  194.5
   192.0.2.8/25            32
  
**_WARNING_** although the utility accepts IP, range, or CIDR notation, it
inserts single records in the database. Hence DON'T SET WIDE RANGES or your
database will blow up to unmanageable sizes. This state of affairs may be
bettered in a further ipqbdb release.

The decay value in white.db overrides the decay that would be inserted from
either ibd-ban or ibd-parse. If the decay is 0, the record will not even do it
to the block database. However, the decay from white.db doesn't have to be
shorter: it can be used to worsen the situation for the remote host.

To recap this section,
* write the ipqbdb pcre configuration,
* check the work directory (/var/lib/ipqbdb),
* schedule execution of ibd-del,
* start ibd-parse early in the boot sequence (before syslog),
* start ibd-judge as soon as the firewall is configured (see previous section),
* adjust syslog.conf to write to ibd-parse's named fifo,
* review scripts that control blocking, calling ibd-ban as needed, and
* checkout suid flags and ownership of the executables (ibd-ban) if needed.

See also:
[1] DB_CONFIG
http://www.oracle.com/technology/documentation/berkeley-db/db/ref/env/db_config.html


Setting popt options
====================
The option-file, by default /etc/ipqbdb.popt, can be used to define program-
specific options: it consists of an arbitrary number of lines formatted like
this:

   <program> alias <newoption> <expansion>

See man popt for further details. For example, ibd-ban default values can be
configured in ipqbdb.popt with lines like this:

   ibd-ban alias --my-opt --reason="client is ugly" --initial-decay=86400

You can then check whether it works by looking at the output of

   ibd-ban --my-opt --help

If you use that to change databases, keep in mind that all the .db files that
a program works with must live in the same directory. While ibd-parse enforces
this restriction, ibd-ban and ibd-del can be fooled by giving wrong arguments.


Original Installation Instructions
**********************************

Copyright (C) 1994, 1995, 1996, 1999, 2000, 2001, 2002, 2004, 2005,
2006 Free Software Foundation, Inc.

This file is free documentation; the Free Software Foundation gives
unlimited permission to copy, distribute and modify it.

Basic Installation
==================

Briefly, the shell commands `./configure; make; make install' should
configure, build, and install this package.  The following
more-detailed instructions are generic; see the `README' file for
instructions specific to this package.

   The `configure' shell script attempts to guess correct values for
various system-dependent variables used during compilation.  It uses
those values to create a `Makefile' in each directory of the package.
It may also create one or more `.h' files containing system-dependent
definitions.  Finally, it creates a shell script `config.status' that
you can run in the future to recreate the current configuration, and a
file `config.log' containing compiler output (useful mainly for
debugging `configure').

   It can also use an optional file (typically called `config.cache'
and enabled with `--cache-file=config.cache' or simply `-C') that saves
the results of its tests to speed up reconfiguring.  Caching is
disabled by default to prevent problems with accidental use of stale
cache files.

   If you need to do unusual things to compile the package, please try
to figure out how `configure' could check whether to do them, and mail
diffs or instructions to the address given in the `README' so they can
be considered for the next release.  If you are using the cache, and at
some point `config.cache' contains results you don't want to keep, you
may remove or edit it.

   The file `configure.ac' (or `configure.in') is used to create
`configure' by a program called `autoconf'.  You need `configure.ac' if
you want to change it or regenerate `configure' using a newer version
of `autoconf'.

The simplest way to compile this package is:

  1. `cd' to the directory containing the package's source code and type
     `./configure' to configure the package for your system.

     Running `configure' might take a while.  While running, it prints
     some messages telling which features it is checking for.

  2. Type `make' to compile the package.

  3. Optionally, type `make check' to run any self-tests that come with
     the package.

  4. Type `make install' to install the programs and any data files and
     documentation.

  5. You can remove the program binaries and object files from the
     source code directory by typing `make clean'.  To also remove the
     files that `configure' created (so you can compile the package for
     a different kind of computer), type `make distclean'.  There is
     also a `make maintainer-clean' target, but that is intended mainly
     for the package's developers.  If you use it, you may have to get
     all sorts of other programs in order to regenerate files that came
     with the distribution.

Compilers and Options
=====================

Some systems require unusual options for compilation or linking that the
`configure' script does not know about.  Run `./configure --help' for
details on some of the pertinent environment variables.

   You can give `configure' initial values for configuration parameters
by setting variables in the command line or in the environment.  Here
is an example:

     ./configure CC=c99 CFLAGS=-g LIBS=-lposix

   *Note Defining Variables::, for more details.

Compiling For Multiple Architectures
====================================

You can compile the package for more than one kind of computer at the
same time, by placing the object files for each architecture in their
own directory.  To do this, you can use GNU `make'.  `cd' to the
directory where you want the object files and executables to go and run
the `configure' script.  `configure' automatically checks for the
source code in the directory that `configure' is in and in `..'.

   With a non-GNU `make', it is safer to compile the package for one
architecture at a time in the source code directory.  After you have
installed the package for one architecture, use `make distclean' before
reconfiguring for another architecture.

Installation Names
==================

By default, `make install' installs the package's commands under
`/usr/local/bin', include files under `/usr/local/include', etc.  You
can specify an installation prefix other than `/usr/local' by giving
`configure' the option `--prefix=PREFIX'.

   You can specify separate installation prefixes for
architecture-specific files and architecture-independent files.  If you
pass the option `--exec-prefix=PREFIX' to `configure', the package uses
PREFIX as the prefix for installing programs and libraries.
Documentation and other data files still use the regular prefix.

   In addition, if you use an unusual directory layout you can give
options like `--bindir=DIR' to specify different values for particular
kinds of files.  Run `configure --help' for a list of the directories
you can set and what kinds of files go in them.

   If the package supports it, you can cause programs to be installed
with an extra prefix or suffix on their names by giving `configure' the
option `--program-prefix=PREFIX' or `--program-suffix=SUFFIX'.

Optional Features
=================

Some packages pay attention to `--enable-FEATURE' options to
`configure', where FEATURE indicates an optional part of the package.
They may also pay attention to `--with-PACKAGE' options, where PACKAGE
is something like `gnu-as' or `x' (for the X Window System).  The
`README' should mention any `--enable-' and `--with-' options that the
package recognizes.

   For packages that use the X Window System, `configure' can usually
find the X include and library files automatically, but if it doesn't,
you can use the `configure' options `--x-includes=DIR' and
`--x-libraries=DIR' to specify their locations.

Specifying the System Type
==========================

There may be some features `configure' cannot figure out automatically,
but needs to determine by the type of machine the package will run on.
Usually, assuming the package is built to be run on the _same_
architectures, `configure' can figure that out, but if it prints a
message saying it cannot guess the machine type, give it the
`--build=TYPE' option.  TYPE can either be a short name for the system
type, such as `sun4', or a canonical name which has the form:

     CPU-COMPANY-SYSTEM

where SYSTEM can have one of these forms:

     OS KERNEL-OS

   See the file `config.sub' for the possible values of each field.  If
`config.sub' isn't included in this package, then this package doesn't
need to know the machine type.

   If you are _building_ compiler tools for cross-compiling, you should
use the option `--target=TYPE' to select the type of system they will
produce code for.

   If you want to _use_ a cross compiler, that generates code for a
platform different from the build platform, you should specify the
"host" platform (i.e., that on which the generated programs will
eventually be run) with `--host=TYPE'.

Sharing Defaults
================

If you want to set default values for `configure' scripts to share, you
can create a site shell script called `config.site' that gives default
values for variables like `CC', `cache_file', and `prefix'.
`configure' looks for `PREFIX/share/config.site' if it exists, then
`PREFIX/etc/config.site' if it exists.  Or, you can set the
`CONFIG_SITE' environment variable to the location of the site script.
A warning: not all `configure' scripts look for a site script.

Defining Variables
==================

Variables not defined in a site shell script can be set in the
environment passed to `configure'.  However, some packages may run
configure again during the build, and the customized values of these
variables may be lost.  In order to avoid this problem, you should set
them in the `configure' command line, using `VAR=value'.  For example:

     ./configure CC=/usr/local2/bin/gcc

causes the specified `gcc' to be used as the C compiler (unless it is
overridden in the site shell script).

Unfortunately, this technique does not work for `CONFIG_SHELL' due to
an Autoconf bug.  Until the bug is fixed you can use this workaround:

     CONFIG_SHELL=/bin/bash /bin/bash ./configure CONFIG_SHELL=/bin/bash

`configure' Invocation
======================

`configure' recognizes the following options to control how it operates.

`--help'
`-h'
     Print a summary of the options to `configure', and exit.

`--version'
`-V'
     Print the version of Autoconf used to generate the `configure'
     script, and exit.

`--cache-file=FILE'
     Enable the cache: use and save the results of the tests in FILE,
     traditionally `config.cache'.  FILE defaults to `/dev/null' to
     disable caching.

`--config-cache'
`-C'
     Alias for `--cache-file=config.cache'.

`--quiet'
`--silent'
`-q'
     Do not print messages saying which checks are being made.  To
     suppress all normal output, redirect it to `/dev/null' (any error
     messages will still be shown).

`--srcdir=DIR'
     Look for the package's source code in directory DIR.  Usually
     `configure' can determine that directory automatically.

`configure' also accepts some other, not widely useful, options.  Run
`configure --help' for more details.

