NAME

  qdb - maintain and query CPAN's update history


SYNOPSIS

  Usage: qdb [-v] [-q] [-d] [-T] [-h] [-t] [-i] [-c conf]
  option v : be verbose
  option q : be quiet
  option d : show debug info
  option T : show trace info
  option h : show help ; exit
  option t : test the config, using port+1 ; exit
  option i : interactive ; don't fork the daemon
  option c : use config $conf ; default [ qdb.conf, /etc/qdb/conf]


DESCRIPTION

At the moment, CPAN's instant mirroring uses RECENT-files, which contain CPAN's update history. The idea is to maintain the update history in an online database, which instant mirroring clients can query for new events. This would obsolete the RECENT-files and simplify rrr-server and instant mirroring clients.

Program qdb stops/starts a server daemon thats maintains a database containing CPAN's update history.

The update history is a list of tuples (events)

  ⟦ id, type, path, time ⟧

where

Server qdb listens on port 22722, and accepts simple databse queries :

  q first_id          # the id of the first (oldest) event
  q last_id           # the id of the last (most recent) event
  q from $n           # the oldest 1000 events, with id >= $n
  q from $n limit $x  # same ; max $x results

This information would enable an instant mirroring client to track CPAN updates ; it would obsolete CPAN's RECENT-files.

At the moment, qdb can add new events to the database is two ways :

  1. The server can accept new ⟦type,path,time⟧ events from an authorised client ; for instance rrr-server, rrr-client, or iim.

  2. The server can periodicly extract new events from the RECENT-files in the local CPAN tree.

This is quite flexible. We could, for instance, have the situation where :


OPTIONS

-q

be quiet ; only errors

-v

be verbose ; show all actions

-d

show debug info ; internals

-T

trace ; show all debug info

-h

show help ; exit

-t

test config, using $port + 1 ; exit

-i

interactive ; don't fork the daemon

-c

use config file ; default : qdb.conf, /etc/qdb/conf


CONFIG FILE

location

The default locations of the config file are :

syntax

A config file looks like this :

  +--------------------------------------------------
  |# lines that start with '#' are comment
  |# blank lines are ignored too
  |# tabs are replaced by a space
  |
  |# the config entries are 'key' and 'value' pairs
  |# a 'key' begins in column 1
  |# the 'value' is the rest of the line
  |somekey  part1 part2 part3 ...
  |otherkey part1 part2 part3 ...
  |
  |# indented lines are glued
  |# the next three lines mean 'somekey part1 part2 part3'
  |somekey part1
  |  part2
  |  part3
  +--------------------------------------------------

config entries

In the config, an interval-spec specifies an interval (or time-span) ; it can be given in seconds (as in 22 or 22s), minutes [m], hours [h], days [d] and/or weeks [w].

The interval-specs can be combined in any order :

  dw      # a day and a week
  7d+24h  # same thing
  w-0.5h  # a week minus half an hour
  hm6     # 3666 seconds
cpan /path/to/cpan

Specify the (absolute) path of your CPAN directory.

port port

Specify a number ; the default is :

  port 22722

The server listens for connections on port $port.

permit allow/deny-list

Option permit defines which IPs may access (connect to) the qdb server.

  allow/deny-list :: ❪ allow-list ∥ deny-list ❫ ...
  allow-list :: 'allow' ❪ hostname ∥ netblock ❫ ...
  deny-list  :: 'deny'  ❪ hostname ∥ netblock ❫ ...
  netblock   :: IP ∥ CIDR

The first allow or deny rule that matches a given IP determines the result. If no rules match then access is denied.

Examples :

  # permit all hosts
  permit allow 0.0.0.0/32
  # permit the 131.211-network
  permit allow 131.211.0.0/16
  # permit all hosts except the 131.211-network
  permit deny 131.211.0.0/16 allow 0.0.0.0/32
max_idle interval-spec

Specify the interval after which the server drops an idle connection. The default is five minutes :

  max_idle 5m
ival_next_rrr interval-spec

As an option, the server can periodicly glean new events from CPAN's RECENT-files. The default is 0, meaning : don't look at the RECENT-files.

  ival_next_rrr 0

If/when ival_next_rrr is non-zero, qdb inits $epoch from $database.meta.epoch ; a fatal error occurs if $database.meta.epoch is not set (zero or undefined).

$database.meta.epoch is set by program qdb-fsck when it initialises the database.

secret $secret

Specify the server's secret. There is no default.

The secret is shared between the server and some client. The client uses $secret to authenticate itself. Only authenticated clients may send history-update commands to the server.

server $server

Applications qdb-send and qdb-fsck (with -s) send new events to a qdb-server. The default is

  server localhost
db_init $bool

Specify how the database must be initialised. The default is :

  db_init 0

For a root server, specify :

  db_init root

This option is only used by qdb-fsck.

loglvl Silent | Quiet | Terse | Verbose | Debug | Trace

Specify a log-level ; the default is

  loglvl Terse
rotate num interval-spec

Specify parameters for log rotation ; the default is

  rotate 8 1d

On start-up, and after interval-spec seconds, qdb will rotate its logfile (/var/log/qdb/qdb.log), saving num files.

logdir dir

Specify the directory where the logfiles live. The default is :

  logdir /var/log/qdb
vardir dir

Specify a qdb keeps it files. The default is :

  vardir /var/qdb
rundir dir

Specify a run-dir. The default is :

  rundir /var/run/qdb

This is where the daemon keeps its files.

lckdir dir

Specify a lock-dir. The default is :

  lckdir /var/lock/subsys


Server

By default, the server listens on port 22722, accepting connections from permitted hosts. It expects commands from the list below.

By default, each command returns a (json) hash like { cmd ⇒ .., err ⇒ .., res ⇒ ... } If $err is not empty, $res is undefined. In PRETTY-mode, the result is human-readable, and not specified.

To send a command to the server, use qdb-send :

  % qdb-send -i

... or use netcat (nc) :

  % echo <command> | nc localhost 22722
  % echo <command> | nc <server-hostname> 22722

Server commands & results

Server queries & results

q first_id

The result is the $id of the first (oldest) event.

q last_id

The result is the $id of the last (most recent) event.

q from $n ⟦ limit $x

The result is a (ordered) list of the oldest events with $id >= $n. At most 1000 (or $x) events are returned.

q path $path

The result is the last event pertaining to $path.

q stat $path
q lstat $path

The result is [ stat "/path/to/cpan/$path" ] ; likewise for lstat.

q readlink $path

The result is readlink "/path/to/cpan/$path".


INSTALL

Requirements

Installation


SIGNALS

  HUP  : qdb reloads  ; qdb re-reads the config ; re-inits client and server
  USR1 : qdb re-execs ; qdb stops with END { exec $PROGRAM_NAME, @ARGV }


FILES

  /etc/qdb/conf        qdb-configuration
  /etc/init.d/qdb      qdb start/stop script
  /var/lock/subsys/qdb touched on qdb-startup ; removed on stop
  /var/log/qdb         logfiles
  /var/qdb/data.lite   sqlite database with event-history and meta-data
  /var/run/qdb/qdb.lck lock-file
  /var/run/qdb/qdb.pid pid-file
  /var/run/qdb/qdb.stp qdb stop-secret ; used in STOP-command


SEE ALSO

qdb-fsck(1), qdb-send(1), qdb-cgi(1)


LICENSE

You may distribute under the terms of either the GNU General Public License or the Artistic License, as specified in the Perl 5.10.0 README file.


AUTHOR

qdb © 2015-2017 Henk P. Penning - All rights reserved ; qdb-0.04-p12 - Sat Jan 21 10:42:19 2017