DCVSUPD(8) FreeBSD System Manager's Manual DCVSUPD(8)
NAME
dcvsupd -- network distribution server for DCVS repositories
SYNOPSIS
dcvsupd [-efv] [-A addr] [-b base] [-c collPath] [-C maxClients] [-l log]
[-o embedded-client-command] [-p port] [-P range] [-s scanDir]
[-Z comprLevel]
DESCRIPTION
dcvsupd is the server program for the CVSup network distribution package.
For information about the companion client program, see dcvsup(1).
In normal usage, dcvsupd should be started with the `-C maxClients'
option. Unless -f is specified, it runs as a background daemon, fielding
connection requests from remote clients. For each connection, dcvsupd
forks a child process to serve the files requested by the client.
The following options are supported:
-A addr Specifies a local address (dotted quad or hostname) on which
the server will accept connections. This may be useful on
hosts which have multiple IP addresses.
-b base Uses base as the base directory for the configuration files.
The default is /usr/local/etc/cvsup.
-c collPath
Searches the specified directories for information about the
collections being offered. collPath contains of one or more
directory names, separated by colons. Non-absolute pathnames
are interpreted relative to the base directory. The default
search path is `sup'.
-C maxClients
Limits the number of simultaneous clients to maxClients.
Clients beyond the specified maximum are politely refused
service.
If this option is not specified, dcvsupd serves one client in
the foreground and then exits.
-e Suppresses the redirection of the standard output and stan-
dard error when running as a daemon and logging to a syslog
facility. Otherwise, these descriptors are redirected to
/dev/null. This option is useful for capturing any panic
messages which might be emitted in the rare event of a crash
of the server. Such messages are very helpful for debugging,
but there is no reliable way to send them to syslog. A rec-
ommended way to use the option is like this:
dcvsupd -e ... >>/var/tmp/dcvsupd.out 2>&1
assuming that the command line syntax of sh(1) is used.
-f When -C maxClients is specified, causes dcvsupd to stay in
the foreground instead of becoming a background daemon.
-l log Directs log messages to log. If log is of the form @facility
(e.g., `@local0') then logging is done via syslog to the
indicated facility. Otherwise, log is treated as the name of
a log file. If the file already exists, new messages are
appended to its end.
For each client served, at least two messages are logged.
The first message identifies the client by username and host-
name. The last message reports the success or failure of the
update and gives the total network I/O in Kbytes (1K = 1024).
Additional messages may be emitted to report errors or other
noteworthy conditions.
-o embedded-client-command
Executes the given command every time a network-client
requests a reversion of the replication's direction. Thus, a
client is beeing embedded, while the remote client embeds a
server. Among others, option -o should always be given as an
option to an embedded client.
-p port Listens for connections on an alternate TCP port. The
default port is 5999. When not in passive mode, the server
also uses the next lower port to establish a second connec-
tion (called the data connection) back to the client.
-P range Sets the range of server TCP ports to be used for the data
connection, when in passive mode. Range may be a single
integer or a range given as `lo-hi'.
-s scanDir Enables mirror mode, and specifies the directory under which
the scan files can be found. If it is not an absolute path-
name, scanDir is interpreted relative to the base directory.
See RUNNING A CVSup MIRROR SITE, below.
-v Prints the version number and exits, without serving any
clients.
-Z comprLevel
Sets the compression level to comprLevel. The compression
level must be between 0 and 9. A level of 0 means no com-
pression, while 9 provides maximum compression. The default
level is 1. Higher levels provide relatively little improve-
ment, at a rather high cost in CPU cycles.
DCVS
CVSup is used as underlying replication mechanism for DCVS. CVSup has
been extended to know about dcvs_config(5) files and their semantics,
refuse to serve deltas and tags it has not been allowed to serve, and the
ability to reuse the TCP connection for reverse syncronisation (push and
pull with one TCP connection). Several bugs that would just show up for
multi-site replication have been corrected.
PREPARING A FILE COLLECTION REPOSITORY
The file collections which dcvsupd makes available to clients are
described by various configuration files. The configuration files reside
under the directory base/collDir, where base is the directory specified
by the -b base command line option, or /usr/local/etc/cvsup by default.
collDir is any of the directories specified with the -c option, or `sup'
by default.
Each collection's configuration files are kept in a separate subdirectory
of base/collDir, named after the collection itself. For example, the
configuration for the `src-base' collection would be found in
base/collDir/src-base. Within the collection's subdirectory, there
should be a releases file and a list file. The releases file contains
one line per release. The first word of each line is the name of the
release, e.g., `cvs'. That must be followed by the following phrases, in
any order:
list=file Specifies the name of the list file, relative to the collec-
tion's subdirectory. The list file is described below.
prefix=directory
Specifies where the files making up the collection are to be
found. If directory is not an absolute pathname, it is
interpreted relative to the base directory base. If there is
no prefix clause, the prefix defaults to base.
keywordprefix=directory
Specifies a ``pseudo-prefix'' which is used only for expand-
ing the absolute pathnames for the `$Header$' and `$Source$'
RCS keywords. The directory should normally be the absolute
pathname of the CVS repository on the machine containing the
master copy of the repository. The use of keywordprefix
ensures that CVSup will expand the RCS keywords identically
on all machines, even though the repositories may reside in
different directories.
super=collection
Specifies the immediate super-collection of the current col-
lection. In large distributions, the collections are often
organized in a hierarchical manner. At the top of the hier-
archy is a collection containing all of the distributed
files. At the next level are several sub-collections, each
containing a subset of the files. Each sub-collection may in
turn have further sub-collections, and so forth. The super
keyword specifies the collection's parent in such a hierar-
chical arrangement.
This keyword is optional. If it is omitted, dcvsupd assumes
nothing about the relationship between the current collection
and the other available collections.
Information from the super keywords is used for finding the
appropriate scan files when the server is running as a mirror
site. See RUNNING A CVSup MIRROR SITE.
nocheckrcs Disables the comparison of MD5 checksums for updated RCS
files. Checksum mismatches for RCS files may not be meaning-
ful, since a given logical RCS file can have many different
textual renditions.
norcs Disables special processing for RCS files. They will be
treated the same as other files.
norsync Disables the use of the rsync algorithm. Note: this keyword
is deprecated in the releases file. Use norsync or rnorsync
in the list file instead. (See below.)
Unrecognized phrases are accepted but ignored, for backward compatibility
with the sup package. Note that even though dcvsupd often serves only a
single release, the releases file is still required.
The list file specifies the details of how to upgrade the client's ver-
sion of the collection. Each line contains a single command. Blank
lines are ignored, as are lines beginning with `#'. Any filenames speci-
fied are taken as relative to the prefix directory given in the releases
file.
Many of the list file commands accept file name patterns as arguments.
These are similar to the patterns accepted by sh(1), and may include the
wild card constructs `*', `'?, and `[...]'. With the exception of
omitany patterns, any slash characters in file names must be matched
explicitly by slash characters in the pattern. Leading periods in file
names are not treated specially. For example, the pattern `*' matches
the filename `.profile'.
upgrade pattern ...
All files and directories matching any of the given patterns
will be included in the list of files to be upgraded. If a
directory name is matched, it recursively includes all files
and subdirectories within it.
always pattern ...
This command is the same as upgrade, except that it overrides
any omitany commands.
omitany pattern ...
Files or directories matching any of the given patterns will
be omitted from the upgrade. If a directory name is matched,
then it and all files and subdirectories beneath it are omit-
ted.
The patterns for omitany are interpreted differently than
other patterns. For normal patterns, a slash character in a
pathname must be matched explicitly by a slash character in
the pattern. For omitany patterns, slashes are treated the
same as other characters. Thus the pattern `*.c' will match
any pathname ending in `.c', including, for example,
`foo/bar/lam.c'.
symlink pattern ...
Symbolic links matching any of the given patterns will be
upgraded as symbolic links, rather than as the files they
refer to. Otherwise, symbolic links are followed and their
target files are sent to the client.
rsymlink pattern ...
This command is similar to symlink, but if a directory is
matched, all symbolic links beneath it in the tree are
treated as matched.
norsync pattern ...
The rsync algorithm will not be used for updating files
matching any of the given patterns. This is useful for log
files, where CVSup's ``append'' optimization is more effec-
tive than the rsync algorithm.
rnorsync pattern ...
This command is similar to norsync, but if a directory is
matched, all files beneath it in the tree are treated as
matched.
execute command (pattern ...)
The given command will be executed by the client whenever a
file matching one of the patterns is successfully updated.
The command comprises all words up to the first (`' charac-
ter. Any occurrences of the string `%s' are replaced by the
pathname of the updated file on the client host. Occurrences
of `%%' are replaced by `%'. The command is executed by
passing it to /bin/sh.
There may be multiple patterns, separated by white space.
They are interpreted relative to the prefix directory. Each
pattern should be written to match the appropriate files as
they exist on the server. For example, the `,v' suffix of an
RCS file name must be matched, even though that suffix will
not be present on the client if checkout mode is in effect.
Slashes in file names must be matched by explicit slashes in
the pattern. CVS `Attic' directories are implicitly included
in the matching process, and should not be specified directly
in the patterns. A matching file will be found whether it is
in the Attic or not.
If an execute statement matches a directory, its command is
executed if the directory is created for the first time, or
if its attributes are changed. The command is executed when
ascending out of the directory, i.e., after its files and
subdirectories have been processed.
If multiple execute statements match a file, all of the asso-
ciated commands are executed in sequence.
For security reasons, the client may choose to ignore all
execute statements.
Unrecognized commands are accepted but ignored, for backward compatibil-
ity with sup.
RUNNING A CVSup MIRROR SITE
A mirror is a server which obtains and updates its files from a master
site by means of CVSup, and redistributes them via CVSup to other sites.
Mirror sites are commonly used in large projects to spread the load among
a number of servers. The files being distributed originate at a master
site. Each mirror site updates its own copies of the files periodically
from there. Clients in turn obtain their updates from any of the mirror
sites.
dcvsupd has a special mode of operation for mirror sites that dramati-
cally increases its efficiency. This mode is enabled by the -s scanDir
option on the command line. Without the -s option, dcvsupd makes a full
file tree traversal over the files in each requested collection, perform-
ing a stat(2) system call on every file. It does this for each client
that connects to it, on the assumption that any of the files could change
at any time. Such a traversal imposes a heavy seek load on the disks
containing the files, and limits the number of clients that can be served
simultaneously.
On a mirror site, the files in the collections are known to change only
when new versions of them come in via CVSup. The -s option allows
dcvsupd to take advantage of this property to completely avoid the file
tree traversals. This reduces the disk load on the server by orders of
magnitude. In place of the file tree traversal, dcvsupd gets the neces-
sary information about the files in the collections by reading scan
files. The scan files are created by the dcvsup client when it updates
the files on the mirror site from the originals at the master site. In
DCVSUP(1), these files are referred to as list files. Both names refer
to the same files. Each time dcvsupd serves a client, it finds the scan
files created by the most recent update from the master site. Thus the
server always has fully up-to-date information about the files in the
collections, without the need to perform a file tree traversal.
The -s option is followed by a directory name which specifies where the
scan files can be found. It is normally a subdirectory of the base
directory, and it must match the location where the dcvsup client main-
tains its list files. By default, dcvsup locates these files under the
sup subdirectory of the base directory. To match this, dcvsupd should be
run with `-s sup'. If dcvsup's list file location is changed from the
default using the -c option, then dcvsupd's scan directory must be
changed the same way. There is no default for the -s option. If it is
not given explicitly on the command line, no scan files are used.
There does not need to be a scan file for every collection. dcvsupd
first looks for the scan file for the collection requested by the client.
If that scan file does not exist, dcvsupd tries the scan files for each
successive super-collection, and uses the first one it finds. (See the
description of the super keyword in PREPARING A FILE COLLECTION
REPOSITORY for details.) If no suitable scan file is located, dcvsupd
falls back on performing a file tree traversal.
ACCESS CONTROL
Access to the server is unrestricted by default, but there is a reason-
ably flexible mechanism for limiting access based on the IP addresses of
connecting clients. It is enabled by placing a set of rules into the
access file base/dcvsupd.access. The access file is a text file with one
rule per line. Comments begin with `#' and extend to the end of the
line. White space is ignored except where it is needed to separate adja-
cent tokens. Blank lines are ignored.
Each rule consists of the following components:
· A flag indicating whether the rule is a permit rule, an authenticate
rule, or a deny rule. The flag is expressed as a single character:
`+' means permit, `*' means authenticate, and `-' means deny.
· An IP address to compare with the client's IP address to determine
whether the rule applies to the client. This may be expressed either
as a numeric IP address or as a host name. Numeric addresses consist
of 1 to 4 octet values, separated by dots. If fewer than 4 octets
are specified, the trailing octets are assumed to contain 0.
Host names are converted to numeric addresses when they are read. If
a host has multiple addresses, a separate rule is added for each
address. This may or may not have the desired effect.
Host names should be used with caution. A name that is slow to
resolve can bog down the server significantly.
· A matching mask to be ANDed with the IP addresses of the rule and the
client before comparing the addresses. This mask is specified as `/'
followed by an integer giving the number of high-order 1s in the
mask. For example, `/24' specifies a mask of 0xffffff00. The
matching mask is optional; if omitted, it defaults to `/32'.
· A counting mask that determines how the clients that match the rule
are counted. (See below.) It is specified the same way as the
matching mask. The counting mask is optional; if omitted, it
defaults to the same value as the matching mask.
· A limit specifying the maximum number of matching clients allowed at
the same time. This is specified as a decimal integer, preceded by
white space to separate it from the preceding component. The limit
is optional. If omitted, it defaults to 0 for a deny rule, or to
infinity for a permit rule.
When a client connects to the server, its IP address is checked against
successive rules in sequence. Each rule is processed as follows:
1. The IP address of the rule is compared with the IP address of the
client, after ANDing each address with the matching mask. If the
addresses do not match, the rule is ignored.
2. The IP addresses of all other currently connected clients are com-
pared with the IP address of the connecting client, after ANDing
each address with the counting mask. If the number of matching
clients (not counting the connecting client) is less than the limit,
then the rule succeeds. Otherwise, the rule fails.
3. If the rule is a permit rule and it succeeded, the client is allowed
access, and the rest of the rules are ignored.
4. If the rule is an authenticate rule and it succeeded, the server
attempts to verify the client's identity using a challenge-response
protocol (see AUTHENTICATION, below). Access is granted or denied
based on the outcome of authentication. The rest of the rules are
ignored.
5. If the rule is a deny rule and it failed, the client is denied
access, and the rest of the rules are ignored.
6. Otherwise, processing continues with the next rule.
There is an implicit authenticate rule at the end of the list which
matches any IP address. Thus, if the processing reaches the end of the
list of rules without having allowed or denied access, access is con-
trolled by the authentication mechanism.
Here are some examples illustrating how the rules are commonly used.
-spam.cyberpromo.com
Deny all access from a specific host.
+mirror.FreeBSD.org
Permit unlimited access from a specific host.
-user.FreeBSD.org 1
Limit a specific host to at most 1 connection at a time.
-198.211.214/24
Deny all access from hosts in a specific class C address block.
-198.211.214/24 3
Allow at most 3 connections total from the hosts in a specific class C
address block.
-198.211.214/24/32 3
Allow at most 3 connections from each of the hosts in a specific class C
address block.
Note the difference between the previous two examples. The first example
imposes a per-network limit, while the second example imposes a per-host
limit. The difference is in the counting mask. The 24 bit mask in the
first example produces a single counter that is shared by all of the
hosts in the specified address block. The 32 bit mask in this example
produces a separate counter for each host.
-0.0.0/0/24 1
Allow no more than 1 connection at a time from each block of 256
addresses.
*0.0.0.0/0
For all clients, use authentication to decide whether access is allowed.
When updating the access file, it is not necessary to halt the server.
But the file should be copied for editing, and then the new version
should be moved atomically into place. There is no need to signal the
server after updating the file. The server will notice that the file has
been touched, and will rescan it automatically. In addition, the server
rescans the file every 3 hours to keep up with DNS changes that might
affect the resolved addresses of host names.
Syntax errors in individual rules are logged, and the offending rules are
ignored. Host name lookup failures are also logged.
AUTHENTICATION
CVSup implements an authentication mechanism which can be used to control
access to the server. It uses a challenge-response protocol which is
immune to packet sniffing and replay attacks. No passwords are sent over
the network in either direction. Both the client and the server can
independently verify the identities of each other.
Authentication of the client is invoked by a successful authenticate rule
in the base/cvsupd.access file, or by ``falling off the end'' of the
file. If there is no cvsupd.access file, clients are not authenticated.
The file base/cvsupd.passwd holds the information used for performing
authentication. This file contains a record for each client who is
allowed access to the server. Each record occupies one line in the file.
Lines beginning with `#' are ignored, as are lines containing only white
space. White space is significant everywhere else in the file. Fields
are separated by `:' characters.
The first record of the file is special. It specifies the identity of
the server itself. This server record has the form:
serverName:privateKey
ServerName is the canonical name of the server, e.g.,
`CVSup.FreeBSD.ORG'. This name is sent to the client, which uses it to
choose an appropriate client name and shared secret for authentication.
The name is case-insensitive.
PrivateKey is any string of characters except `:'. When the server gen-
erates random challenges to send to the client, it uses privateKey to
make the challenges harder to guess. Challenges are random and quite
unpredictable in any case, so the privateKey isn't really very important.
It can be left empty if desired, but the `:' that precedes it must be
present.
All of the remaining records in the file correspond to individual
clients. Each client record has the following form:
clientName:sharedSecret:class:comment
All fields must be present even if some of them are empty. ClientName is
the name of the client to which the record applies. By convention, e-
mail addresses are used for all client names, e.g.,
`BillyJoe@FreeBSD.ORG'. Client names are case-insensitive.
SharedSecret is a secret string of characters known only to the client
and the server. It is generated from a password chosen by the client,
using the cvpasswd utility. The client proves its identity to the server
(and vice versa) by demonstrating that it knows the sharedSecret. A
client's access may be disabled by changing its sharedSecret field to
`*'.
The shared secret is never sent across the network, nor can it be used to
find out the client's password. However, given the shared secret, a mod-
ified version of dcvsup could impersonate the client. Thus, care must be
taken to ensure that the cvsupd.passwd file is readable only by the
server.
Class is reserved for future use. It should be empty.
Comment contains any additional information about the client that might
be useful to the server administrator. For example, it may contain the
client's full name and other contact information.
When updating the cvsupd.passwd file, it is not necessary to halt the
server. But the file should be copied for editing, and then the new ver-
sion should be moved atomically into place. There is no need to signal
the server after updating the file.
Syntax errors in individual records of the cvsupd.passwd file are logged,
and the offending records are ignored.
HOW ACCESS CONTROL AND AUTHENTICATION INTERACT
Here is a summary of the interactions between the access control and
authentication mechanisms. The key principle is that access control
takes place first. The outcome of access control determines whether
authentication is performed too.
1. If there is no cvsupd.access file, then all clients are granted
access. No authentication is done, even if cvsupd.passwd exists.
2. If the cvsupd.access file exists but is empty, all clients are sub-
jected to authentication. If cvsupd.passwd does not exist, nobody
can access the server.
3. If cvsupd.access exists and has some rules in it, but there is no
cvsupd.passwd file, then successful authenticate rules cause access
to be denied. Access is still granted to those who match successful
permit rules. Falling off the end of the cvsupd.access file results
in denial of access.
4. If both the cvsupd.access and cvsupd.passwd files exist, then:
· Successful permit rules cause access to be granted without
authentication.
· Successful authenticate rules cause authentication to be per-
formed. Access is granted or denied based on the outcome of
that. Falling off the end of the cvsupd.access file is included
in this case.
· Failing deny rules cause access to be denied.
RCS KEYWORD EXPANSION
In checkout mode, CVSup expands RCS keywords as described in co(1). It
expands all of the standard keywords, and also the non-standard
`$CVSHeader$' keyword. This expands the same as `$Header$', except that
the RCS file's pathname is expressed relative to the prefix directory,
rather than as an absolute pathname. The prefix is assumed to be the
root of the CVS repository.
It is also possible to define aliases for the standard RCS keywords, and
to selectively enable or disable the recognition of individual keywords.
These properties are controlled on a repository-wide basis by directives
in a file named prefix/CVSROOT/options. Each directive occupies one line
of the file. Comments begin with `#' and extend to the end of line.
Blank lines are ignored. The syntax is ridiculous, for historical rea-
sons.
To define a keyword alias, use a line of the form:
tag=alias[=keyword]
For example:
tag=FreeBSD=CVSHeader
defines a new RCS keyword `$FreeBSD$', which expands the same as
`$CVSHeader$'. If the second `=' and the keyword are omitted, the key-
word defaults to `Id'.
To disable all but certain selected keywords, use a line of the form:
tagexpand=ikeyword[,...]
For example:
tagexpand=iFreeBSD,Id
disables the expansion of all keywords except `$FreeBSD$' and `$Id$'.
The leading `i' stands for ``include''.
To enable all but certain selected keywords, use a line of the form:
tagexpand=ekeyword[,...]
For example:
tagexpand=eFreeBSD,Id
enables the expansion of all keywords except `$FreeBSD$' and `$Id$'. The
leading `e' stands for ``exclude''.
SHUTDOWN
If there exists a file base/cvsupd.HALT that is newer than the time when
the server was started, then the server will reject all new incoming con-
nection requests. Clients which had already started will run to comple-
tion, but no new ones will be accepted. This mechanism is awkward and
weak, and will probably be changed in a future release.
SECURITY
dcvsupd does not create or write any files, except for its log file if
one is specified on the command line. There is thus little risk that
dcvsupd can be subverted into damaging the system on which it is running.
A more likely risk is that dcvsupd might be spoofed into sending out
files that are not intended to be publicly distributed. dcvsupd is very
careful to prevent this from happening. Nevertheless, for maximum pro-
tection it is a good idea to run dcvsupd as a completely unprivileged
user analogous to `nobody', serving only files that are readable by
everyone.
CVSup has no provision for encrypting the data sent across the network.
If secrecy is desired then the connection can be tunneled through ssh.
FILES
/usr/local/etc/cvsup Default base directory.
sup Default collDir subdirectory.
base/collDir/collection/releases
Releases file.
base/collDir/collection/list List file.
base/cvsupd.HALT Shutdown file.
base/cvsupd.access Access control file.
base/cvsupd.passwd Authentication password file.
prefix/CVSROOT/options RCS keyword configuration file.
SEE ALSO
DCVS(7), dcvs(1), cvsupadm(1), dcvs_config(5), co(1), cvpasswd(1),
cvs(1), cvsup(1).
http://www.cvsup.org/
http://www.elegosoft.com/dcvs
AUTHORS
John Polstra <jdp@polstra.com>.
Elego Software Solutions GmbH (http://www.elegosoft.com).
LEGALITIES
CVSup is a registered trademark of John D. Polstra.
BUGS
An RCS file is not recognized as such unless its name ends with `,v'.
Any directory named `Attic' is assumed to be a CVS Attic, and is treated
specially.
FreeBSD January 1, 2002 FreeBSD