Courier Authentication Library

Table of Contents

The authpwd authentication module
The authshadow authentication module
The authpam authentication module
The authpipe authentication module
The authpipe protocol
The authuserdb authentication module
A brief userdb primer
A simple userdb setup
Large virtual domain farm
Beyond userdb
The authvchkpw authentication module
The authmysql authentication module
The authpgsql authentication module
The authldap authentication module
authcustom
Account options
Running authtest
Changing account passwords
Authentication internals
FILES
SEE ALSO

This library is used for two purposes:

1. Read the name of a mail account. Determine the local account's home directory, and system userid and groupid.

2. Read an account name, and a password. If valid, determine the account's home directory, system userid, and groupid.

The term "authentication" is used in the following documentation to refer to either one of these two functions. The library contains several alternative authentication modules to choose from, described below.

The configuration file @authdaemonrc@ contains several settings. The most important of them are:

The authpwd authentication module

This modules obtains account information and passwords from the /etc/passwd file.

Note

This module doesn't actually read the /etc/passwd file, it uses the C library's getpw() functions. The C library implementation could use any mechanism to obtain the equivalent information.

The authshadow authentication module

This module is a version of the authpwd module that reads passwords from /etc/shadow (the C library's getsp() functions).

The authpam authentication module

This modules uses the system's PAM library (pluggable authentication modules) for authentication. This is, essentially, a way to use existing PAM modules for authentication. Note, however, that the authenticated account's home directory, userid and groupid are still read from the /etc/passwd file, since PAM functionality is limited to validating account passwords.

Note

Not all PAM modules are compatible with Courier's authentication library. PAM modules that make use of PAM's session functions, or authentication token functions, like pam_krb5 will not work with Courier.

Note

Additional configuration steps will be required to set up the PAM library to authenticate Courier's services. Courier's IMAP and POP3 servers, for example, require that the “imap” and “pop3” PAM service to be configured.

The specific configuration steps differ from system to system. Consult the system documentation for more information. It might be tempting to throw in a towel and use authshadow or authpwd if you cannot figure out how to install PAM support, however that is not advisable. It is highly recommended to use authpam wherever the PAM library is available.

The exact configuration procedure depends on the PAM implementation. Most PAM libraries use configuration files in the /etc/pam.d directory. Therefore, it will be necessary to install the configuration files /etc/pam.d/imap and /etc/pam.d/pop3. Similarly, Courier's webmail server, SqWebMail, uses /etc/pam.d/webmail, and its optional calendar component uses /etc/pam.d/webmail. Courier-MTA's authenticated SMTP component uses the /etc/pam.d/smtp service.

In nearly all cases all these configuration files will specify an identical PAM library configuration for all services. The exact configuration details are site-specific. Here's an example of a PAM configuration file for a recent version of the most common PAM library:

auth       required     pam_nologin.so
auth       required     pam_stack.so service=system-auth
account    required     pam_stack.so service=system-auth
session    required     pam_stack.so service=system-auth

Again, the actual configuration is site specific. Examine the contents of existing configuration files in /etc/pam.d for similar services (if there's /etc/pam.d/ppp it's often a good example to follow) in order to derive the correct setup for Courier.

Older PAM libraries use a single configuration file, usually /etc/pam.conf. Append Courier-specific PAM settings to this configuration file, again using settings for existing services as a guide. For example:

imap  auth    required        pam_unix.so      try_first_pass
imap  account required        pam_unix.so
imap  session required        pam_permit.so
pop3  auth    required        pam_unix.so      try_first_pass
pop3  account required        pam_unix.so

Some PAM libraries use pam_pwdb.so instead of pam_unix.so; consult the PAM library's documentation for more information.

The authpipe authentication module

This is a generic plug-in module that runs an external script, or a program, in response to authentication requests.

The external program reads from stdin and writes to stdout. It can be persistent and handle many authentication requests. Only one request will be sent to it at a time; each authdaemon process starts its own copy of the external script.

The location of the external program is set by the --with-pipeprog configure option, which defaults to @sysconfdir@/authlib/authProg. A sample program is included in the courier-authlib source.

The authpipe protocol

authpipe uses the same protocol as authdaemon clients use to communicate with authdaemond.

There are four possible requests: PRE, AUTH, PASSWD and ENUMERATE. Apart from AUTH, each request is a single line terminated by newline.

PRE . authservice username <newline>

Look up data for an account. authservice identifies the service the user is trying to use - e.g. pop3, imap, webmail etc.

If the account exists, return the account data as a series of ATTR=value newline-terminated lines, followed by a period on a line of its own. Valid attributes are:

      USERNAME=username         -- system account which owns mailbox (name)
      UID=uid                   -- system account which owns mailbox (numeric uid)
      GID=gid                   -- numeric groupid
      HOME=homedir              -- home directory
      ADDRESS=addr              -- e-mail address
      NAME=name                 -- full name
      MAILDIR=maildir           -- Maildir relative to home directory
      QUOTA=quota               -- quota string: maxbytesS,maxfilesC
      PASSWD=cryptpasswd        -- encrypted password
      PASSWD2=plainpasswd       -- plain text password
      OPTIONS=acctoptions       -- option1=val1,option2=val2,...
      .
      

Of these, it is mandatory to return ADDRESS, HOME, GID, and either UID or USERNAME; the others are optional.

If the account is not known, return FAIL<newline>. If there is a temporary failure, such as a database being down, authProg should terminate (thereby closing stdin/stdout) without sending any response. authdaemon will restart the pipe module for the next request, thus ensuring it is properly reinitialized.

AUTH len<newline>len-bytes

Validate a login attempt. The AUTH line is followed by len-bytes of authentication data, which does not necessarily end with a newline. The currently defined authentication requests are:

    login \n username \n password [\n]         -- plaintext login
    cram-md5 \n challenge \n response [\n]     -- base-64 encoded challenge and response
    cram-sha1 \n challenge \n response [\n]    -- ditto
    cram-sha256 \n challenge \n response [\n]  -- ditto
    

In the case of success, return the complete set of account parameters in the same format as PRE, ending with a period on a line of its own. In the case of failure (e.g. username does not exist, password wrong, unsupported authentication type), return FAIL<newline>. If there is a temporary failure, such as a database being down, authProg should terminate without sending any response.

Note: if the user provides a plaintext password and authenticates successfully, then you can return it as PASSWD2 (plain text password) even if the database contains an encrypted password. This is useful when using the POP3/IMAP proxy functions of courier-imap.

PASSWD service<tab> username<tab> oldpasswd<tab> newpasswd<tab> <newline>

Request a password change for the given account: validate that the oldpassword is correct, and if so, change it to the newpassword.

Reply: the string for success, or FAIL<newline> for a data error (e.g. no such account, old password wrong, new password not acceptable). In the case of a temporary failure, such as a database being down, authProg should terminate without sending any response.

ENUMERATE <newline>

Return a list of all accounts, one per line in the following format, ending with a period on a line of its own:

    username \t uid \t gid \t homedir \t maildir \t options \n
    .
    

If your module does not support the ENUMERATE command then return just a period on a line of its own (which will still allow enumeration data from other modules to be returned). In the case of a temporary failure, such as a database being down or an error occuring mid-way through returning account data, authProg should terminate before sending the terminating period.

The authuserdb authentication module

This module uses a GDBM or a DB-based userdb(8) database. This module also incorporates userdb-based challenge-response authentication implementation that was done by a separate authcram module in previous versions of the Courier authentication library.

@sysconfdir@/authlib/userdb is a plain file that can be edited with any text editor. The file contains a list of account names, and their pertinent information. @sysconfdir@/authlib/userdb may alternatively be a directory containing plain text files, which are effectively concatenated together to form the actual list of accounts. The makeuserdb script compiles the account information into a GDBM or DB database file, which can be quickly looked up.

@sysconfdir@/authlib/userdb is loosely equivalent in function to /etc/passwd and /etc/shadow, and contain analous information: account name, its numeric userid and groupid, home directory, and passwords. @sysconfdir@/authlib/userdb also contains additional Courier-specific metadata, such as account quotas and other account-specific settings. @sysconfdir@/authlib/userdb files can also be maintained by custom-written Perl scripts, instead of being edited by hand.

@sysconfdir@/authlib/userdb allows creation of virtual mail accounts that do not have a corresponding login account -- virtual mail accounts that can share the same, reserved, system userid. @sysconfdir@/authlib/userdb can also be used to completely supersede /etc/passwd. With many accounts it can be quite a drain to have to continuously linearly scan /etc/passwd in order to look up an account. Instead, a fast database lookup can retrieve the same information from the database file. Review the included manual pages, starting with userdb(8), for more information.

A brief userdb primer

userdb is a way to implement many virtual mailboxes - many mailboxes that do not have to have a separate system userid allocated for each one, and there is no system login associated with each mailbox. userdb uses a database for mapping virtual addresses to physical maildirs. It should be scalable to thousands of mailboxes. It can also be used to replace linear searches of /etc/passwd with a database lookup, see pw2userdb(8).

Note - you still MUST use some valid system userid and groupid that is shared by all virtual mailboxes. Instead of allocating a single userid and groupid per each mailbox, the same userid and groupid is used for all of them.

This is a rough overview of using userdb. For additional information, read userdb(8) and makeuserdb(8). All the scripts will be installed in @sbindir@, so look for them there.

The best way to describe how userdb works is to try to create one virtual mail account. As mentioned before, virtual mailboxes still need one system account to be used for uid/gid purposes. Let's call this system account "vmail".

A simple userdb setup

This approach should be used if you do not have many virtual mailboxes. It's very simple, but quickly becomes cumbersome if you administer many virtual mailboxes.

Create an empty @userdb@ file:

# cp /dev/null @userdb@
# chmod 700 @userdb@

@userdb@ must have 700 permissions, since it will contain passwords.

Now, run the script pw2userdb, as root. This script converts the contents of /etc/passwd to the @userdb@ format (including the contents of /etc/shadow, this is why permissions on @userdb@ must be 700). This script is usually used where you want to convert a very large /etc/passwd to @userdb@. userdb applications can now use a fast userdb database instead of a linear scan of /etc/passwd in order to look up system accounts. However, you probably don't want to use this feature right now, so what you want to do is take the output of pw2userdb, and find the entry for the vmail account that you created earlier. Look for a line that starts with 'vmail' followed by tab, followed by familiar fields from /etc/passwd. Save the output of pw2userdb in a temporary file, edit it, and remove everything except the line containing vmail, and the very next line, which is a special entry that maps vmail's userid back to the vmail record.

Here's what you might find in the output of pw2userdb:

vmail   uid=1012|gid=1012|home=/home/vmail|systempw=*
1012=   vmail

The actual numerical values and the home directory location may vary. Save these two lines as @userdb@, and set the permissions on @userdb@ to 700:

$ chmod 700 @userdb@

Now, with that out of the way, let's really create a virtual account. In this example we'll create a virtual mailbox for 'john@example.com'.

# su vmail
$ cd ~vmail
$ mkdir john-example
$ maildirmake john-example/Maildir
$ exit
#

You may need to specify a full path to your maildirmake program. The end result is that you created $HOME/john-example in vmail's account, which can be thought of as a “virtual home directory” for “john@example.com”, that contains the account's maildir mailbox.

Now, let's connect the dots here, and create an entry in @userdb@ for john@example.com:

# userdb "john@example.com" set home=/home/vmail/john-example \
                                uid=UUU gid=GGG

This command runs the script named userdb , which is installed, by default in @sbindir@. Replace UUU and GGG with the userid and groupid of the vmail account. If you now look in @userdb@, you will see that a new record for “john@example.com” has been appended to the end of the file.

One more detail: we need to set the IMAP password for this mailbox:

# userdbpw | userdb "john@example.com" set imappw

On most modern Linux and BSD distributions, you can specify the -md5 option to userdbpw, in order to use MD5 password hashes, instead of crypt. The traditional password function allows passwords only up to 8 characters long; everything in excess is ignored. The newer MD5 passwords, now supported by most modern systems, allow longer passwords.

Use "systempw" instead of "imappw" if you would like to use the same password for the POP3 server, and for all other services. The "imappw" field is only checked by the IMAP server. If not defined, "systempw" is used instead. The field pop3pw is checked only by Courier's POP3 server. If it is not defined the POP3 server will check systempw too.

Finally, compile the database:

# makeuserdb

This command creates the actual database, @userdb@.dat and @userdb@shadow.dat from the plain text file @userdb@. Courier will now start accepting logins to this mailbox. Adding and removing mailboxes can be done while Courier is running.

Courier reads @userdb@.dat and @userdb@shadow.dat only. The plain text source, @userdb@ is not read by Courier itself. Changes take effect only when makeuserdb runs.

Large virtual domain farm

The previous approach used a single flat file, @userdb@. This will work for up to a couple of hundred accounts. An slightly different approach can scale to thousands of domains and mailboxes.

Instead of creating a @userdb@ file, create a subdirectory:

# mkdir @userdb@
# chmod 700 @userdb@

Now, create @userdb@/default, containing pw2userdb's output for the vmail account, as previously described.

This time, you probably want to create all mailboxes for the same domain in a separate subdirectory:

# su - vmail
$ cd ~vmail
$ mkdir -p domains/example-com
$ mkdir domains/example-com/john
$ maildirmake domains/example-com/john
$ exit

The idea is that all the maildirs for @example.com will now be found in ~vmail/domains/example-com. All maildirs for domain.org will be in ~vmail/domains/domain.org. The actual layout and naming conventions are entirely up to you to define.

Here's how configure @userdb@:

$ userdb "example-com/john@example.com" set \
             home=/home/vmail/domains/example-com/john \
             uid=UUU gid=GGG

This creates the file @userdb@/example-com (the first parameter to the userdb command), and appends a record named "john@example.com". You will store all userdb entries for @example.com in the file @userdb@/example-com. All entries for @domain.org will be maintained in @userdb@/domain-org, and so on.

$ userdbpw | userdb "example-com/john@example.com" set imappw

This sets the IMAP access password for this account. Finally:

$ makeuserdb

Beyond userdb

userdb is a simple, straightforward solution that scales to a couple of thousand of mail accounts, depending on the hardware. Beyond that, one of database-based modules will need to be used, such as authldap, authmysql, authpgsql. Since userdb is maintained as plain text files that are easily parsed by a script, migrating data from userdb will not be difficult.

The authvchkpw authentication module

This module uses the vpopmail/vchkpw library for authentication.

The authmysql authentication module

This module reads the list of mail accounts and passwords from a table in a MySQL database. The @authmysqlrc@ configuration file defines the particular details regarding the MySQL database and the schema of the mail account table.

The authpgsql authentication module

This module reads the list of mail accounts and passwords from a table in a PostgreSQL database. The @authpgsqlrc@ configuration file defines the particular details regarding the PostgreSQL database and the schema of the mail account table.

The authldap authentication module

This module reads the list of mail accounts and passwords from an LDAP directory. The @authldaprc@ configuration file defines the particular details regarding the LDAP directory layout.

A suggested LDAP schema can be found in the file authldap.schema, which is included in Courier authentication library's source code, and may be installed on your system.

authcustom

This is a do-nothing module where custom authentication code can be added. This authentication module is just a stub that doesn't really do anything. It's purpose is to serve as a placeholder where custom authentication code can be easily implement.

Account options

The authentication library has a facility for keep arbitrary “name=value”-type settings, called “options”, for individual accounts. This feature is only available with userdb, LDAP, MySQL, and PostgresSQL modules. Individual account options are not supported with system-based authentication modules (password/shadow files, or PAM).

See the auth_generic(3) for a description of option names used by various Courier packages. Other applications can make up names for their own settings, and use them in the same way.

Account options are specified via the authentication modules in the following manner:

userdb

Use the userdb command to set a field called "options". Example:

userdb user1@example.com set options=disableimap=1,sharedgroup=44
makeuserdb

The option text string here is "disableimap=1,sharedgroup=44". It specifies two options.

LDAP

Account options are defined by the LDAP_AUXOPTIONS setting in the authldaprc configuration file. LDAP_AUXOPTIONS consists of a comma-separated list of "attribute=setting". "attribute" is the name of an LDAP attribute, and "setting" is the corresponding account setting name. The value of the attribute becomes the value of the setting. Unless you value your sanity, the names of LDAP attributes should be the same as the actual setting names (in which case "=setting" may be dropped and LDAP_AUXOPTIONS becomes a simple comma-separated list of supported settings), but they don't have to be.

LDAP_AUXOPTIONS is nothing more than a simple mapping of LDAP attributes to account settings. A LDAP_AUXOPTIONS of "shared=sharedgroup,disableimap" means that the LDAP attribute called "shared" contains the "sharedgroup" setting, as described previously; and an LDAP attribute of disableimap contains the setting of the same name.

MySQL, and PostgreSQL

Account options are defined by MYSQL_AUXOPTIONS_FIELD or POSTGRESQL_AUXOPTIONS_FIELD, in its corresponding configuration file. In the most simplest case, add a character field to the database, and put the field name into the MYSQL_AUXOPTIONS_FIELD or POSTGRESQL_AUXOPTIONS_FIELD configuration file setting. For each account, the character field should contain the literal option string. Yes, you'll just put "shared=sharedgroup,disableimap" literally, in that field.

Fortunately, there is a cleaner way to do this, which avoid driving a database designer batty. Keep in mind that the contents of MYSQL_AUXOPTIONS_FIELD/POSTGRESQL_AUXOPTIONS_FIELD are simply inserted directly into the SQL query that fetches the account information. Both MySQL and PostgreSQL have a rich SQL that can be used to manufacture a suitable option string from plain, garden-variety, database fields. That is, you may define individual table fields like "disableimap", and "disablepop3", then provide a suitable (albeit ugly) SQL fragment that combines them together into the expected option string. An example of such an SQL string is provided in the comments portion of the configuration file.

Note

When using the alternative custom query option, the option string is the last field that the custom SQL query should return.

Running authtest

The authtest command may be used to verify that the authentication library is working:

authtest userid
authtest userid password
authtest userid password newpassword
authenumerate

Running authtest with one argument should display the selected account's home directory, userid, groupid, and other related data. The second argument to authtest, if supplied, specifies the account's password. The two argument form of authtest validates the password, and displays an indication whether the given password is valid, or not. The three argument form of the authtest command attemps to change the account's password. The second argument is the old password, the third argument is the new password.

See README.authdebug.html for more information.

Changing account passwords

For the virtual domain modules (authldap, authmysql, authpgsql and friends) changing the login is a no-brainer. The tricky situation is when SqWebMail uses system passwords to log in (the authpwd, authshadow, or authpam authentication module). Different systems use different ways to keep login passwords. Many systems use the traditional /etc/passwd and /etc/shadow files. Other systems use a binary database; other systems use NIS. And on some systems the password file lookup library is a wrapper that goes against an external LDAP directory, or a database. For maximum compatibility, SqWebMail changes login passwords by running the passwd command. This is the traditinal *nix command that changes login passwords. passwd is an interactive command. It's normally run from a terminal. SqWebMail uses an expect script - as mentioned in the introduction - to answer interactive prompts from passwd. The expect script expects to get a plain, garden-variety, passwd command, which acts something like this:

     # passwd
     Changing password for luser
     (current) UNIX password:         (old password typed here)
     New UNIX password:               (new password typed here)
     Retype new UNIX password:        (new password retyped here)
     passwd: all authentication tokens updated successfully
     #

Systems that use a passwd command with very different prompts may find that the default expect script will fail. In which case it will be necessary to tweak the expect script to match the prompts from the system's passwd command.

Modern systems use a passwd command that rejects "bad" passwords - passwords that are based on dictionary words, are too short, or are obvious for other reasons. When testing the ability to change system passwords be sure to use randomly-generated gibberish for the test passwords. Otherwise, the default expect script will actually be working, but you won't be the wiser. For security reasons, the actual messages from passwd will not be shown by.

The expect script is installed as /usr/local/libexec/courier-authlib/authsystem.passwd (assuming default options to the configure script).

Authentication internals

The following structure describes an authentication module:

Example 1. struct authstaticinfo

struct authstaticinfo {
	const char *auth_name;
	int (*auth_func)(const char *, const char *, char *, int,
			 int (*)(struct authinfo *, void *),
			 void *);
	int (*auth_prefunc)(const char *, const char *,
			    int (*)(struct authinfo *, void *),
			    void *);
	void (*auth_cleanupfunc)();
	int (*auth_changepwd)(const char *, /* service */
			      const char *, /* userid */
			      const char *, /* oldpassword */
			      const char *); /* new password */

	void (*auth_idle)();
	/* Not null - gets called every 5 mins when we're idle */

	void (*auth_enumerate)( void(*cb_func)(const char *name,
					       uid_t uid,
					       gid_t gid,
					       const char *homedir,
					       const char *maildir,
					       void *void_arg),
				void *void_arg);
	} ;

An authentication module is a shared library that defines a single function called “courier_auth_NAME_init”, where “NAME” is the name of the authentication module. The shared library does not need to export any other symbols, this is the only function that needs to be exported. The function returns a pointer to the authstaticinfo structure. For example, the relevant code from the authmysql module is:

Example 2. authmysql

static struct authstaticinfo authmysql_info={
	"authmysql",
	auth_mysql,
	auth_mysql_pre,
	auth_mysql_cleanup,
	auth_mysql_changepw,
	auth_mysql_cleanup,
	auth_mysql_enumerate};


struct authstaticinfo *courier_authmysql_init()
{
	return &authmysql_info;
}

auth_func points to a function that handles an authentication request. The function is invoked as follows:

Example 3. auth_func

int result=auth_func(const char *service, const char *authtype,
			const char *authdata,
			int (*callback_func)(struct authinfo *, void *),
			void *callback_arg);

service” is the name of the service being authenticated, such as “imap” or “pop3”. “authtype” defines the authentication format, and “authdata” is the actual authentication request.

Two authentication formats are defined at this time. The “authtype” string is set to one of the following strings:

login

Tradition userid/password authentication. authdata points to a string that consists of: the userid; a newline character; the password; a final newline character.

cram-md5”, or “cram-sha1

Challenge/response authentication. authdata points to a string that consists of: the base64-encoded challenge; a newline character; the base64-encoded response string; and a final newline character. Furthermore, the base64-encoded response string consists of: the login ID, a space character, and the response as a hexadecimal string (yes, base64-encoding of the response string is not strictly necessary).

Note

Not all authentication modules may implement all authentication formats. An authentication module that does not implement a particular authentication format should handle it the same way as an invalid login ID.

The authentication function should return a negative value if the login ID is invalid. The authentication library will try the next authentication module.

The authentication function should return a positive value if the login ID is valid, but the password is invalid. The authentication library will not try any more authentication modules.

Otherwise, the authentication module should call the callback_func function, and return the same value that's returned by this function.

The authentication module should pass through callback_arg to the callback function as a second argument. The first argument is a pointer to the authinfo structure, which is described in detail in the auth_generic(3) manual page. The authentication module is responsible for allocating this structure. After the callback function returns this structure can be deallocated. The authentication module initializes the following fields:

auth_pre_func points to a function that obtains account information. The function is invoked as follows:

Example 4. auth_pre_func

int auth_pre_func(const char *user, const char *service,
                  int (*callback)(struct authinfo *, void *), void *arg);

This function does the same thing as “auth_func” except that the password is not actually verified. If the account exists, the callback function is invoked with the same callback arguments.

auth_cleanup_func points to a function that will be invoked just before the authentication module is uninstalled, giving it the opportunity for some last-minute cleanup.

auth_idle points to a function that will be invoked when no authentication requests are received for a couple of minutes, giving the authentication module an opportunity to close any database connections, so that they do not get shut down by the server, for inactivity, resulting in an error the next time an authentication request is received.

auth_changepwd points to a function that will be invoked to change a password on an account, as follows.

Example 5. auth_changepwd

int auth_changepwd(const char *service, const char *user,
			const char *oldpw, const char *newpw);

service is the name of the service whose password is to be changed (such as “imap” or “pop3”). auth_changepwd should return 0 if the password was changed succesfully, a negative value if user is invalid (the next authentication module will be tried), or a positive value if the password change request failed (no more modules will be tried).

auth_enumerate points to a function that enumerates the list of all login IDs known to the authentication module. The first argument auth_enumerate is a callback function. auth_enumerate invokes the callback function once for each login ID, supplying the login ID, the userid, groupid, home directory and maildir as arguments. The last argument to the callback function is passed through from the second argument to auth_enumerate.

After enumerating all login IDs auth_enumerate calls the callback function one last time, with a NULL pointer for the login ID, then returns. If an error is encountered while enumerating the login IDs, auth_enumerate terminates without invoking the callback function with a NULL login ID.

FILES

@authdaemonrc@ - authdaemond configuration file

@authldaprc@ - authldap configuration file

@authmysqlrc@ - authmysql configuration file

@authpgsqlrc@ - authpgsql configuration file

SEE ALSO

courier(8), userdb(8)