Exim's interfaces to mail filtering Philip Hazel Copyright (c) 2014 University of Cambridge Revision 4.92 10 Feb 2019 PH ------------------------------------------------------------------------------- TABLE OF CONTENTS 1. Forwarding and filtering in Exim 1.1. Introduction 1.2. Filter operation 1.3. Testing a new filter file 1.4. Installing a filter file 1.5. Testing an installed filter file 1.6. Details of filtering commands 2. Sieve filter files 2.1. Recognition of Sieve filters 2.2. Saving to specified folders 2.3. Strings containing header names 2.4. Exists test with empty list of headers 2.5. Header test with invalid MIME encoding in header 2.6. Address test for multiple addresses per header 2.7. Semantics of keep 2.8. Semantics of fileinto 2.9. Semantics of redirect 2.10. String arguments 2.11. Number units 2.12. RFC compliance 3. Exim filter files 3.1. Format of Exim filter files 3.2. Data values in filter commands 3.3. String expansion 3.4. Some useful general variables 3.5. Header variables 3.6. User variables 3.7. Current directory 3.8. Significant deliveries 3.9. Filter commands 3.10. The add command 3.11. The deliver command 3.12. The save command 3.13. The pipe command 3.14. Mail commands 3.15. Logging commands 3.16. The finish command 3.17. The testprint command 3.18. The fail command 3.19. The freeze command 3.20. The headers command 3.21. Obeying commands conditionally 3.22. String testing conditions 3.23. Numeric testing conditions 3.24. Testing for significant deliveries 3.25. Testing for error messages 3.26. Testing a list of addresses 3.27. Testing for personal mail 3.28. Alias addresses for the personal condition 3.29. Details of the personal condition 3.30. Testing delivery status 3.31. Multiple personal mailboxes 3.32. Ignoring delivery errors 3.33. Examples of Exim filter commands =============================================================================== 1. FORWARDING AND FILTERING IN EXIM This document describes the user interfaces to Exim's in-built mail filtering facilities, and is copyright (c) University of Cambridge 2014. It corresponds to Exim version 4.92. 1.1 Introduction ---------------- Most Unix mail transfer agents (programs that deliver mail) permit individual users to specify automatic forwarding of their mail, usually by placing a list of forwarding addresses in a file called .forward in their home directories. Exim extends this facility by allowing the forwarding instructions to be a set of rules rather than just a list of addresses, in effect providing ".forward with conditions". Operating the set of rules is called filtering, and the file that contains them is called a filter file. Exim supports two different kinds of filter file. An Exim filter contains instructions in a format that is unique to Exim. A Sieve filter contains instructions in the Sieve format that is defined by RFC 3028. As this is a standard format, Sieve filter files may already be familiar to some users. Sieve files should also be portable between different environments. However, the Exim filtering facility contains more features (such as variable expansion), and better integration with the host environment (such as the use of external processes and pipes). The choice of which kind of filter to use can be left to the end-user, provided that the system administrator has configured Exim appropriately for both kinds of filter. However, if interoperability is important, Sieve is the only choice. The ability to use filtering or traditional forwarding has to be enabled by the system administrator, and some of the individual facilities can be separately enabled or disabled. A local document should be provided to describe exactly what has been enabled. In the absence of this, consult your system administrator. This document describes how to use a filter file and the format of its contents. It is intended for use by end-users. Both Sieve filters and Exim filters are covered. However, for Sieve filters, only issues that relate to the Exim implementation are discussed, since Sieve itself is described elsewhere. The contents of traditional .forward files are not described here. They normally contain just a list of addresses, file names, or pipe commands, separated by commas or newlines, but other types of item are also available. The full details can be found in the chapter on the redirect router in the Exim specification, which also describes how the system administrator can set up and control the use of filtering. 1.2 Filter operation -------------------- It is important to realize that, in Exim, no deliveries are actually made while a filter or traditional .forward file is being processed. Running a filter or processing a traditional .forward file sets up future delivery operations, but does not carry them out. The result of filter or .forward file processing is a list of destinations to which a message should be delivered. The deliveries themselves take place later, along with all other deliveries for the message. This means that it is not possible to test for successful deliveries while filtering. It also means that any duplicate addresses that are generated are dropped, because Exim never delivers the same message to the same address more than once. 1.3 Testing a new filter file ----------------------------- Filter files, especially the more complicated ones, should always be tested, as it is easy to make mistakes. Exim provides a facility for preliminary testing of a filter file before installing it. This tests the syntax of the file and its basic operation, and can also be used with traditional .forward files. Because a filter can do tests on the content of messages, a test message is required. Suppose you have a new filter file called myfilter and a test message in a file called test-message. Assuming that Exim is installed with the conventional path name /usr/sbin/sendmail (some operating systems use /usr/lib/ sendmail), the following command can be used: /usr/sbin/sendmail -bf myfilter to e.g. add 2 to n3 There are 10 user variables of this type, with names $n0 - $n9. Their values can be obtained by the normal expansion syntax (for example $n3) in other commands. At the start of filtering, these variables all contain zero. Both arguments of the add command are expanded before use, making it possible to add variables to each other. Subtraction can be obtained by adding negative numbers. 3.11 The deliver command ------------------------ deliver e.g. deliver "Dr Livingstone " This command provides a forwarding operation. The delivery that it sets up is significant unless the command is preceded by "unseen" (see section 3.8). The message is sent on to the given address, exactly as happens if the address had appeared in a traditional .forward file. If you want to deliver the message to a number of different addresses, you can use more than one deliver command (each one may have only one address). However, duplicate addresses are discarded. To deliver a copy of the message to your normal mailbox, your login name can be given as the address. Once an address has been processed by the filtering mechanism, an identical generated address will not be so processed again, so doing this does not cause a loop. However, if you have a mail alias, you should not refer to it here. For example, if the mail address L.Gulliver is aliased to lg303 then all references in Gulliver's .forward file should be to lg303. A reference to the alias will not work for messages that are addressed to that alias, since, like .forward file processing, aliasing is performed only once on an address, in order to avoid looping. Following the new address, an optional second address, preceded by "errors_to" may appear. This changes the address to which delivery errors on the forwarded message will be sent. Instead of going to the message's original sender, they go to this new address. For ordinary users, the only value that is permitted for this address is the user whose filter file is being processed. For example, the user lg303 whose mailbox is in the domain lilliput.example could have a filter file that contains deliver jon@elsewhere.example errors_to lg303@lilliput.example Clearly, using this feature makes sense only in situations where not all messages are being forwarded. In particular, bounce messages must not be forwarded in this way, as this is likely to create a mail loop if something goes wrong. 3.12 The save command --------------------- save e.g. save $home/mail/bookfolder This command specifies that a copy of the message is to be appended to the given file (that is, the file is to be used as a mail folder). The delivery that save sets up is significant unless the command is preceded by "unseen" (see section 3.8). More than one save command may be obeyed; each one causes a copy of the message to be written to its argument file, provided they are different (duplicate save commands are ignored). If the file name does not start with a / character, the contents of the $home variable are prepended, unless it is empty, or the system administrator has disabled this feature. In conventional configurations, this variable is normally set in a user filter to the user's home directory, but the system administrator may set it to some other path. In some configurations, $home may be unset, or prepending may be disabled, in which case a non-absolute path name may be generated. Such configurations convert this to an absolute path when the delivery takes place. In a system filter, $home is never set. The user must of course have permission to write to the file, and the writing of the file takes place in a process that is running as the user, under the user's primary group. Any secondary groups to which the user may belong are not normally taken into account, though the system administrator can configure Exim to set them up. In addition, the ability to use this command at all is controlled by the system administrator - it may be forbidden on some systems. An optional mode value may be given after the file name. The value for the mode is interpreted as an octal number, even if it does not begin with a zero. For example: save /some/folder 640 This makes it possible for users to override the system-wide mode setting for file deliveries, which is normally 600. If an existing file does not have the correct mode, it is changed. An alternative form of delivery may be enabled on your system, in which each message is delivered into a new file in a given directory. If this is the case, this functionality can be requested by giving the directory name terminated by a slash after the save command, for example save separated/messages/ There are several different formats for such deliveries; check with your system administrator or local documentation to find out which (if any) are available on your system. If this functionality is not enabled, the use of a path name ending in a slash causes an error. 3.13 The pipe command --------------------- pipe e.g. pipe "$home/bin/countmail $sender_address" This command specifies that the message is to be delivered to the specified command using a pipe. The delivery that it sets up is significant unless the command is preceded by "unseen" (see section 3.8). Remember, however, that no deliveries are done while the filter is being processed. All deliveries happen later on. Therefore, the result of running the pipe is not available to the filter. When the deliveries are done, a separate process is run, and a copy of the message is passed on its standard input. The process runs as the user, under the user's primary group. Any secondary groups to which the user may belong are not normally taken into account, though the system administrator can configure Exim to set them up. More than one pipe command may appear; each one causes a copy of the message to be written to its argument pipe, provided they are different (duplicate pipe commands are ignored). When the time comes to transport the message, the command supplied to pipe is split up by Exim into a command name and a number of arguments. These are delimited by white space except for arguments enclosed in double quotes, in which case backslash is interpreted as an escape, or in single quotes, in which case no escaping is recognized. Note that as the whole command is normally supplied in double quotes, a second level of quoting is required for internal double quotes. For example: pipe "$home/myscript \"size is $message_size\"" String expansion is performed on the separate components after the line has been split up, and the command is then run directly by Exim; it is not run under a shell. Therefore, substitution cannot change the number of arguments, nor can quotes, backslashes or other shell metacharacters in variables cause confusion. Documentation for some programs that are normally run via this kind of pipe often suggest that the command should start with IFS=" " This is a shell command, and should not be present in Exim filter files, since it does not normally run the command under a shell. However, there is an option that the administrator can set to cause a shell to be used. In this case, the entire command is expanded as a single string and passed to the shell for interpretation. It is recommended that this be avoided if at all possible, since it can lead to problems when inserted variables contain shell metacharacters. The default PATH set up for the command is determined by the system administrator, usually containing at least /bin and /usr/bin so that common commands are available without having to specify an absolute file name. However, it is possible for the system administrator to restrict the pipe facility so that the command name must not contain any / characters, and must be found in one of the directories in the configured PATH. It is also possible for the system administrator to lock out the use of the pipe command altogether. When the command is run, a number of environment variables are set up. The complete list for pipe deliveries may be found in the Exim reference manual. Those that may be useful for pipe deliveries from user filter files are: DOMAIN the domain of the address HOME your home directory LOCAL_PART see below LOCAL_PART_PREFIX see below LOCAL_PART_SUFFIX see below LOGNAME your login name MESSAGE_ID the unique id of the message PATH the command search path RECIPIENT the complete recipient address SENDER the sender of the message SHELL /bin/sh USER see below LOCAL_PART, LOGNAME, and USER are all set to the same value, namely, your login id. LOCAL_PART_PREFIX and LOCAL_PART_SUFFIX may be set if Exim is configured to recognize prefixes or suffixes in the local parts of addresses. For example, a message addressed to pat-suf2@domain.example may cause the filter for user pat to be run. If this sets up a pipe delivery, LOCAL_PART_SUFFIX is "-suf2" when the pipe command runs. The system administrator has to configure Exim specially for this feature to be available. If you run a command that is a shell script, be very careful in your use of data from the incoming message in the commands in your script. RFC 2822 is very generous in the characters that are permitted to appear in mail addresses, and in particular, an address may begin with a vertical bar or a slash. For this reason you should always use quotes round any arguments that involve data from the message, like this: /some/command '$SENDER' so that inserted shell meta-characters do not cause unwanted effects. Remember that, as was explained earlier, the pipe command is not run at the time the filter file is interpreted. The filter just defines what deliveries are required for one particular addressee of a message. The deliveries themselves happen later, once Exim has decided everything that needs to be done for the message. A consequence of this is that you cannot inspect the return code from the pipe command from within the filter. Nevertheless, the code returned by the command is important, because Exim uses it to decide whether the delivery has succeeded or failed. The command should return a zero completion code if all has gone well. Most non-zero codes are treated by Exim as indicating a failure of the pipe. This is treated as a delivery failure, causing the message to be returned to its sender. However, there are some completion codes that are treated as temporary errors. The message remains on Exim's spool disk, and the delivery is tried again later, though it will ultimately time out if the delivery failures go on too long. The completion codes to which this applies can be specified by the system administrator; the default values are 73 and 75. The pipe command should not normally write anything to its standard output or standard error file descriptors. If it does, whatever is written is normally returned to the sender of the message as a delivery error, though this action can be varied by the system administrator. 3.14 Mail commands ------------------ There are two commands that cause the creation of a new mail message, neither of which count as a significant delivery unless the command is preceded by the word "seen" (see section 3.8). This is a powerful facility, but it should be used with care, because of the danger of creating infinite sequences of messages. The system administrator can forbid the use of these commands altogether. To help prevent runaway message sequences, these commands have no effect when the incoming message is a bounce (delivery error) message, and messages sent by this means are treated as if they were reporting delivery errors. Thus, they should never themselves cause a bounce message to be returned. The basic mail-sending command is mail [to ] [cc ] [bcc ] [from
] [reply_to
] [subject ] [extra_headers ] [text ] [[expand] file ] [return message] [log ] [once ] [once_repeat