Home page Services provided Software available Site licenses Systems status Local Documentation Windows 2000 Reporting problems Links Contact information


Table of contents

        1.0 Document id
            1.1 General
            1.2 What is Procmail?
            1.3 Abbreviations and thanks
            1.4 Version information
            1.5 Document layout and maintenance
            1.6 About presented recipes
            1.7 Variables used in recipes
            1.8 About "useless use of cat award"

        2.0 UBE in Internet
            2.1 Terms used and foreword
            2.2 UBE strategies
            2.3 UBE and bouncing message back
            2.4 UBE and "I don't mind" attitude
            2.5 We need a law against UBE

        3.0 Anti-UBE pointers
            3.1 NoCEM, CAUCE and others
            3.2 General Filtering pages (more than procmail)
            3.3 Junk email and spam
            3.4 Comprehensive list of spammers
            3.5 Misc pointers
            3.6 Questionable UBE stop services
            3.7 UBE related newsgroups or mailing lists
            3.8 Software: adcomplain -- Perl junk email rport
            3.9 Software: Ricochet -- Perl junk email rport
            3.10 Software: yell -- perl
            3.11 Software: RBL lookup tool -- C
            3.12 Software: mapSoN
            3.13 Software: spamgard
            3.14 Software: Spam Be Gone
            3.15 Software: ifile - Perl
            3.16 Software: ClearMail

        4.0 Procmail pointers
            4.1 Where to get procmail binary
            4.2 Where is procmail developed
            4.3 About procmail's Y2K compliance
            4.4 Procmail mailing lists
            4.5 Procmail recipe modules and faqs
            4.6 Procmail mode for Emacs
            4.7 Procmail module list
            4.8 Where to get Procmail code and modules
            4.9 Procmail code to filter UBE

        5.0 Dry run testing
            5.1 What is dry run testing
            5.2 Why the From field is not okay after dry run
            5.3 Getting default value of a procmail variable

        6.0 Things to remember
            6.1 Get the newest procmail
            6.2 Csh's tilde is not supported
            6.3 Be sure to write the recipe start right
            6.4 Always set SHELL
            6.5 Check and set PATH
            6.6 Keep the log on all the time
            6.7 Never add a trailing slash for directories
            6.8 Remember what term DELIVERED means
            6.9 Beware putting comment in wrong place
            6.10 Brace placement
            6.11 Local lockfile usage
            6.12 Global lockfile
            6.13 Gee, where do I put all those ! * $ ??
            6.14 Sending automatic reply, use X-loop header
            6.15 Avoid extra shell layer (check command for SHELLMETAS)
            6.16 Think what shell commands you use
            6.17 Using absolute path when calling a shell program
            6.18 Disabling recipe temporarily
            6.19 Keep message backup, no matter what
            6.20 Order of the procmail recipes

        7.0 Procmail flags
            7.1 The order of the flags
            7.2 Flag w and recipe with |
            7.3 Flag w, lockfile and recipe with |
            7.4 Flag f and w together
            7.5 Flags h and b
            7.6 Flag h and sinking to /dev/null
            7.7 Flag i and pipe flag f
            7.8 Flag r
            7.9 Flag c's background
            7.10 Flag c before nested block forks a child
            7.11 Flag c and understanding possible forking penalty
            7.12 Flags before nested block
            7.13 Flags aAeE tutorial

        8.0 Matching and regexps
            8.1 Philosophy of abstraction in regexps
            8.2 Matches are not case sensitive
            8.3 Procmail uses multiline matches
            8.4 Headers are folded before matching
            8.5 Improving Space-Tab syndrome
            8.6 Handling exclamation character
            8.7 Rules for generating a character class
            8.8 Matching space at the end of condition
            8.9 Beware leading backslash
            8.10 Correct use of TO Macro
            8.11 Procmail's regexp engine
            8.12 Procmail and egrep differences
            8.13 Undesrtanding procmail's minimal matching (stingy vs. greedy)
            8.14 Explaining \/ and ()\/
            8.15 Explaning  ^^ and ^
            8.16 ANDing traditionally
            8.17 ORing traditionally
            8.18 ORing and score recipe
            8.19 ORing by using De Morgan rules

        9.0 Variables
            9.1 Setting and unsetting variables
            9.2 Variable initialisation and sh syntax
            9.3 Testing variables
            9.4 What does $\VAR mean?
            9.5 Common pitfalls when using variables
            9.6 Quoting: Using single or double quotes
            9.7 Quoting: Passing values to an external program
            9.8 Passing values from an external program
            9.9 Incrementing a variable by a value N
            9.10 Comparing values
            9.11 Strings: How many characters are there in a given string?
            9.12 Strings: How to strip trailing newline.
            9.13 Strings: deriving the last N characters of a string.
            9.14 Strings: Getting partial matches from a string.
            9.15 Strings: Procmail string manipulation example
            9.16 How to raise a flag if the message was filed
            9.17 Dollar sign in condition lines.
            9.18 Finding mysterious foo variable
            9.19 Storing code to variable
            9.20 Getting headers into a variable.
            9.21 Converting value to lowercase

        10.0 Suggestions and miscellaneous
            10.1 Speeding up procmail
            10.2 See the procmail installation's examples
            10.3 Printing statistics of your incoming mail
            10.4 Storing UBE mailboxes outside of quota
            10.5 Using first 5-30 lines from the message
            10.6 Using cat or echo in scripts?
            10.7 How to run an extra shell command as a side effect?
            10.8 Forcing "ok" return status from shell script
            10.9 Make your own .procmailrc available to others
            10.10 Using dates efficiently
            10.11 Keep simple header log
            10.12 Gzipping messages
            10.13 Emergency stop for your .procmailrc

        11.0 Scoring
            11.1 Using scores by an example
            11.2 Brief Score tutorial
            11.3 Score's scope
            11.4 Counting length of a string
            11.5 Counting lines in a message (Adding Lines: header)
            11.6 Determining if body is longer than header
            11.7 Matching last Received header
            11.8 How to add Content-Length header
            11.9 Testing message size or number of lines
            11.10 Counting commas with recursive includerc

        12.0 Formail usage
            12.1 Fetching fields with formail -x
            12.2 Always use formail's -rt switch
            12.3 Using -rt and rewriting the From address
            12.4 Formail -rt and Resent-From header
            12.5 Quoting the message
            12.6 Without quoting the message
            12.7 How to include headers and body to the reply message
            12.8 Adding text to the beginning of message
            12.9 Adding text to the end of message
            12.10 How to truncate headers (save filing space)
            12.11 Adding extra headers from file
            12.12 Splitting digest
            12.13 Mailbox: Splitting to individual files
            12.14 Mailbox: Extracting all From addresses from mailbox
            12.15 Mailbox: Applying procmail recipe on whole mailbox
            12.16 Mailbox: run series of commands for each mail (split mailbox)
            12.17 Option -D and cache
            12.18 Option -D and message-id in the body
            12.19 Reducing formail calls (conditionally adding fields)
            12.20 Formail -A -a options
            12.21 Formail -e -s options

        13.0 Saving mailing list messages
            13.1 Using subroutine pm-jalist.rc to detect mailing lists
            13.2 Using plus addressing foo+bar@address.com
            13.3 Using RFC comment trick for additional information
            13.4 Simple mailing list handling
            13.5 Archiving according to TO
            13.6 Using Return-Path to detect mailing lists

        14.0 Procmail, MIME and HTML
            14.1 Mime Bibliography
            14.2 Mime notes
            14.3 Software to deal with mime or html
            14.4 Mime content type application/ms-tnef
            14.5 Trapping html mime messages
            14.6 Complaining about html messages
            14.7 Converting HTML body to plain text
            14.8 Getting rid of unwanted mime attachments (html, vcard)
            14.9 Sending contents of a html page in plain text to someone

        15.0 Simple recipe examples
            15.1 Saving: MH folders -- numbered messages
            15.2 Saving: to monthly folders
            15.3 Modifying: Filtering basics
            15.4 Modifying: Squeezing empty lines around message body
            15.5 Modifying: shuffling headers always to same order
            15.6 Service: Auto answerer to empty messages
            15.7 Service: File server -- send fileas as attachments upon request
            15.8 Service: Ping responder
            15.9 Service: simple vacation with procmail
            15.10 Service: vacation code example
            15.11 Service: Auto-forwarding
            15.12 Service: forward only specific messages
            15.13 Service: Making digests
            15.14 Kill: killing advertisement headers and footers
            15.15 Kill: simple killfile recipe with procmail
            15.16 Kill: duplicate messages
            15.17 Kill: spam filter with simple recipes
            15.18 Kill: (un)subscribe messages
            15.19 Time: Once a day cron-like job
            15.20 Time: Running a recipe at a given time
            15.21 Time: Triggering email and using cron
            15.22 Decoding: Uudecode
            15.23 Decoding: MIME
            15.24 How to send commands in the message's body
            15.25 Matching two words on a line, but not one
            15.26 How to define personal XX macros?
            15.27 How to change subject by body match
            15.28 How to change Subject according to some other header
            15.29 How to call program with parameters

        16.0 Miscellaneous recipes
            16.1 Matching valid Message-Id header
            16.2 Sending two files in a message
            16.3 Excessive quoting of message
            16.4 Sending message to pager in chunks
            16.5 Playing particular sound when message arrives
            16.6 Combining multiple Original-Cc and Original-To headers
            16.7 Forwarding sensitive messages in encrypted format

        17.0 Procmail and PGP
            17.1 Decrypt pgp messages automatically
            17.2 Getkeys from keyserver
            17.3 Auto grab incoming pgp keys

        18.0 Includerc usage
            18.1 Using: multiple rc files
            18.2 Using: You can call rc file conditionally
            18.3 Autoloading an rc file
            18.4 Making: naming of the rc file
            18.5 Making: Using namespace when saving procmail variables
            18.6 Making: Public and private variables in rc file
            18.7 The rules of thumb for constructing general purpose rc file
            18.8 An includerc skeleton

        19.0 Mailing list server
            19.1 Mailing list server pointers
            19.2 Simple Mailing list server

        20.0 Common troubles
            20.1 Procmail modes: normal, delivery, and mailfilter.
            20.2 Procmail as sendmail Mlocal mail filtering device
            20.3 Procmail doesn't pass 8bit characters
            20.4 My ISP isn't very interested in installing procmail
            20.5 My ISP has systemwide procmailrc; is this a good idea?
            20.6 Procmail changes mailbox and directory permissions
            20.7 Changing mbox permission during compilation to 660
            20.8 The .forward file must be real file
            20.9 Using .forward if procmail already is LDA
            20.10 Mail should be put in the mailqueue if write fails
            20.11 Qmail: how to make it work with procmail
            20.12 Qmail: Procmail looks file from /var/spool/mail only
            20.13 Qmail: patch to procmail 3.11pre7 to work with Maildirs
            20.14 AFS: How to use Procmail when HOME is in AFS cell
            20.15 Help, some idiot sent my address to 30 mailing lists
            20.16 Help, Procmail beeps and prints to my console
            20.17 Help, procmail dumps mail to console
            20.18 Help, corrupted From_ line in mailbox
            20.19 Directing user's mail to HOME instead of /var/spool/
            20.20 NFS mounting /var/mail is a good way to get bad performance
            20.21 I can't see the sendmail's response in LOGFILE
            20.22 Compiling procmail and choosing locking scheme
            20.23 Forwarding lot of mail causes heavy load
            20.24 What happens to mail if MDA Procmail fails
            20.25 Procmail reads entire 90Mb message into memory
            20.26 Help, procmail uses occasionally huge chunk of memory
            20.27 Procmail signalled out of memory in my verbose log
            20.28 Variables DEFAULT and ORGMAIL
            20.29 When DEFAULT cannot be mailed to
            20.30 Variable DROPPRIVS
            20.31 Variable HOME
            20.32 Variable HOST
            20.33 Variable LINEBUF
            20.34 Variable LOG and LOGFILE
            20.35 Variable TRAP
            20.36 Variable UMASK
            20.37 UMSAK and permissions
            20.38 Performance difference between backtick and "|" recipe
            20.39 Procmail's temporary file names while writing file out
            20.40 Parameter $@
            20.41 Procmail variables are null terminated (detecting null string)
            20.42 FROM_DAEMON TO and TO_ and case-sensitiveness
            20.43 TO_ macro deciphered
            20.44 TO_ macro and RFC 822
            20.45 FROM_DAEMON deciphered

        21.0 Technical matters
            21.1 List of exit codes
            21.2 List of precedence codes
            21.3 Sendmail and -t
            21.4 RFC822 Reply-To and formail problem with multiple recipients
            21.5 Procmail and IMAP server
            21.6 Machine which processes mail
            21.7 Compiling procmail and MAILSPOOLHOME

        22.0 Smartlist
            22.1 MLM RFC
            22.2 Other mailing list software
            22.3 SmartList code (mailing list implementation with procmail)
            22.4 Installation trouble: getparams
            22.5 Accepting mail only from users in whitelist(s)

        23.0 Additional procmail or MUA software
            23.1 Comstat to handle multiple mailboxes
            23.2 Elm and pgp support (Mutt)
            23.3 MH sites

        24.0 Additional procmail software for Emacs
            24.1 What is Emacs
            24.2 Emacs and procmail mode and Lint
            24.3 Emacs and lining up backslashes
            24.4 Emacs and browsing mailbox files
            24.5 Emacs and live-mode.el
            24.6 Emacs and font-lock.el

        25.0 Procmail, Emacs and Gnus
            25.1 Gnus pointers
            25.2 Why use procmail with Gnus
            25.3 Setting up gnus for procmail - Basics
            25.4 Gnus for procmail - More gnus
            25.5 Emacs and Gnus -- Fiddling with spool files
            25.6 Gnus and article snippets
            25.7 Emacs GNUS - POP - Procmail

        26.0 RFC, Request for comments
            26.1 RFCs and their jurisdiction (munged Addresses)
            26.2 Comments about addresses munging
            26.3 RFC and valid email address characters
            26.4 RFC and login-name@fdqn
            26.5 RFCs and message's signature
            26.6 RFC and using MIME in usenet newsgroups
            26.7 Some RFC Pointers

        27.0 Introduction to E-mail Headers
            27.1 To find out more about email (Resources)
            27.2 Lecture by Alan Stebbens
            27.3 Applied to received messages
            27.4 Bcc lecture by Alan Stebbens
            27.5 Bcc lecture by Philip Guenther

        28.0 Message's headers
            28.1 What is correct From address syntax
            28.2 What's that X-UIDL header?
            28.3 What is that first From_ header?
            28.4 Message-Id header
            28.5 Received header
            28.6 Return-Path
            28.7 Errors-To
            28.8 X-Subscription-Info
            28.9 Reply-To header
            28.10 Mail-Copies-To header
            28.11 Mail-Followup-To and Reply-To-Personal headers
            28.12 Content-Length header and From_ specification
            28.13 Moral about CC copies in usenet

        29.0 Other interesting code
            29.1 Misc email related pointers
            29.2 Expire mail pointers
            29.3 Usenet News related pointers
            29.4 Code: Perl Extract procmail man pages from 3.11pre7.tar.gz
            29.5 Code: Sh remove matching lines from file

1.0 Document id

    1.1 General

        .@(#) $Id: pm-tips.txt,v 1.76 1999/11/10 13:55:12 Jari Aalto Exp $
        .$Keywords: procmail sendmail formail mail UBE UCE spam filter $
        .$URL: http://www.procmail.org/jari/ $
        .$Contactid:  $
        .$FileServer: send mail to Contactid with subject "send help" $
        .$UrlLinksLastChecked: 1999-04-30 $

        .@(#) This is a procmail tips page: a collection of procmail recipes,
        .@(#) instructions, howtos. The document also contains URL pointers to
        .@(#) the procmail mailing list and sites that fight against Internet
        .@(#) UBE. You will also find many other interesting subjects that
        .@(#) discuss about internet email: headers, mime and RFCs. There is
        .@(#) also lot of room dedicated to Emacs and Gnus, simply because
        .@(#) those are the best tools you find from Unix to deal with your
        .@(#) mail and news reading. And I happen to know Emacs quite well.

        .@(#) The tips are compiled from the procmail discussion list,
        .@(#) from comp.mail.misc and from the author's own experiences with
        .@(#) procmail.

        This document does not intend to teach you the basics of procmail;
        instead you have to be familiar with the procmail man pages
        already. You may want to read *Nancy's* and *Era's* procmail faq
        pages before this page. Especially Era's link page contains an
        excellent collection of useful procmail links and pointers to unix
        programs that deal with email (eg. Perl *MHonArc* Email hyperarchiver
        at http://www.oac.uci.edu/indiv/ehood/mhonarc.html). If you find
        errors or things to improve in this document, please go ahead and
        send mail to [jari].

        Author's homepage is behind these redirections links. Please keep these
        in your bookmark list, not the absolute addresses, because the sutes may
        move. These link should point always to the correct location:

            http://poboxes.com/jari.aalto/  eg. homepage.html
            http://home.eu.org/~jari/

        If you want to have automatic notification whenever this page changes,
        please visit the link below. To get nicely formatted netmind messages,
        see procmail module `pm-janetmind.rc'.

            http://minder.netmind.com/

        If a mentioned URL is not alive, you may still be able to
        successfully find it using the ftp search located at
        http://ftpsearch.ntnu.no/

    1.2 What is Procmail?

        [faq] Procmail is a mail processing utility, which can help you
        filter your mail, sort incoming mail according to sender, Subject
        line, length of message, keywords in the message, etc, implement an
        ftp-by-mail server, and much more. Procmail is also a complete
        drop-in replacement for your MDA. (If this doesn't mean anything to
        you, you may not want to know.)

        Procmail runs under Unix. See Infinite Ink's Mail Filtering and
        Robots page for information about related utilities for various other
        platforms, and competing Unix programs, too (there aren't that many
        of either).

    1.3 Abbreviations and thanks

        People and documents, abbreviations referred to, tokens used,
        are in no particular order.

        [stephen] Stephen R. van den Berg, Author of Procmail Last heard
        from stephen 1997-08 in procmail mailing list by using address
        . Later 1998 due to his regular work activities and
        lack of time he nominated Philip Guenther to the head of Procmail
        development.

        .[aaron]    Aaron Schrab        
        .[alan]     Alan K. Stebbens    
        .[dan]      Daniel Smith        
        .[david]    David W. Tamkin     
        .[ed]       Edward J. Sabol     
        .[elijah]   Eli the Bearded     
        .[hal]      Hal Wine            
        .[jari]     Jari Aalto          
        .[philip]   Philip Guenther     
        .[richard]  Richard Kabel       
        .[sean]     Sean B. Straw       
        .[timothy]  Timothy J Luoma     
        .[walter]   Walter Dnes         

        .[faq]      Procmail FAQ        j1era+pr@iki.fi
        .[manual]   Quote from some procmail manual page

        o   PM-L, Procmail mailing list
        o   FAQ-L, Faq Maintainers mailing list
            http://www.landfield.com/faq-maintainers/faq-server/
            http://lists.consensus.com/scripts/lyris.pl?visit=faq-maintainers
            http://www.qucis.queensu.ca/FAQs/FAQaid/
        o   DING-L, Emacs Gnus mail/newsreader mailing list (ding).
            http://www.gnus.org/
        o   <> Text has been rephrased or something was added which
            does not exist in original message.

        I also thank following people

        o   Era Eriksson proof read the v1.12 and sent corrections.
        o   Karl E. Vogel 
            sent numerous new anti-spam links to be added to the document.
        o   John Gianni  send some nice recipes: one is now
            in the procmail module list and the other ideas I have added to
            this tips file.
        o   Tim Potter  had a spare moment with v1.27 and
            sent lot of spelling corrections. Thank you.
        o    took 1.48 and sent a huge
            55k patch to correct many English language typos. Thank you
            very much Guido.
        o   1998-10-28 Richard Kabel   sent massive patch
            to correct language and provided excellent improvement comments.
        o   1999-01-08 Steven Alexander  thought that
            a small perl script would help me to fix spelling mistakes more
            easily. The script has been much better correction program that
            simple patches. Thank you.
        o   1999-06-16 Mark Seiden  Did a enermous work to
            proofread the v1.74. He sent a massive 105k with many editorial
            corrections. My wholeheart thanks to you Mark.

    1.4 Version information

        Here is version and file size log of the text file, which gives you
        some estimate how often you should update your copy.

            v1.01   1997-09-13  46 (k)
            v1.05   1997-09-14  53
            v1.5    1997-09-16  76
            v1.6    1997-09-18  94
            v1.8    1997-10-01  127
            v1.9    1997-10-11  142
            v1.10   1997-10-13  181  archive file 1995-10's tips included
            v1.13   1997-11-08  218  Era's correction suggestions.
            v1.14   1997-11-25  260
            v1.17   1997-12-09  343  up till archive 1996-07 now included
            v1.24   1997-12-30  415  up till 1996-12 is now included
            v1.29   1998-01-30  429  "regexp" section rewrite.
            v1.31   1998-03-10  469  Better ordering: ORing rules discussed
            v1.32   1998-03-23  471  All recipes checked (by eye)
            v1.34   1998-04-02  488  ORing and supreme scoring added
            v1.36   1998-04-03  493  Includerc rewritten, plus addressing
            v1.41   1998-06-17  510  How to disable recipe quickly with
            v1.44   1998-06-19  516  Detecting mailing lists with pm-jalist.rc
            v1.45   1998-06-23  521  All recipes checked by eye. Many fixes.
            v1.46   1998-06-24  526  Added live urls to procmail archive
            v1.49   1998-08-10  529  Guido.Van.Hoeck's 55k patch applied
            v1.51   1998-08-18  541  Small changes. MIME notes
            v1.52   1998-08-24  553  Flag c forking study, procmail wish list
            v1.53   1998-08-24  554  Procmail doesn't pass 8bit characters
            v1.55   1998-08-29  565  Fetching fields with formail -x
            v1.57   1998-10-06  575  PLUS addr. Convert HTML body to text
            v1.58   1998-10-12  583  SmartList and other MLM software discussed
            v1.60   1998-10-21  591  UMASK, .forward if procmail already is LDA
            v1.63   1998-10-30  595  Richard's english correction patch
            v1.64   1998-11-26  602  More Richard's comments integrated
            v1.66   1998-12-14  578  Philip took care of bugs/patches listing
            v1.67   1998-01-07  579  Eli's procmail recipes in module section
            v1.68   1998-01-29  587  Added "Lua" language pointer
            v1.69   1999-02-23  590  RFC and using MIME in usenet postings
            v1.70   1999-02-26  592  procmail's Y2K compliance
            v1.71   1999-03-29  597  Ricochet -- Perl script to fight UBE
            v1.72   1999-04-21  597  Links corrected
            v1.74   1999-04-26  599  document moved to www.procmail.org
            v2.0    1999-10-01  602k Mark Seiden's patch applied. Now under CVS.

    1.5 Document layout and maintenance

        This document is maintained in plain text format with Emacs and my
        text formatting package *tinytf.el* (automatic TOC and indentation
        control). Funny marks or indentation are in the text
        version so that the Perl text-to-html
        filter `t2html.pl' can be used. See more about this at:
        http://poboxes.com/jari.aalto/t2html.html

        Text version of this file was converted into HTML with command:

            % perl5.004_04 t2html.pl                                        \
              --html-frame                                                  \
              --title   "Procmail tips page"                                \
              --author  "Jari Aalto"                                        \
              --email   jari.aalto@poboxes.com                              \
              --meta-keywords "procmail, sendmail, mail, filter, faq, ube"  \
              --meta-description "Procmail tips page"                       \
              --base     http://www.procmail.org/jari                       \
              --document http://www.procmail.org/jari                       \
              --url      http://www.procmail.org/jari                       \
              --html-body LANG=en                                           \
              --Out                                                         \
              pm-tips.txt

        Please also familiarise yourself with unix what(1) and GNU RCS
        ident(1), if you have those commands in your system. It is
        important that you mark interesting text to these tools so that
        someone can get an overview of your supplied files

            % what  FILES       - Print @( # ) tags
            % ident FILES       - Print $ $ keywords

       Sending improvements

        Because I'm not English speaking, I regret the bad language I may have
        used in this document. If you have any time, 5-10 minutes to find some
        spelling mistake or misuse of the English verbs, please go ahead and
        send me a patch to correct the wording. The preferred way to send
        corrections to this document is as diff(1) output. Here's how to make
        corrections send them to me:

        The diff option -u is only available in GNU diff, please try to
        send the -u diff if possible. If you don't have -u option, use -c
        switch.

            %   cp pm-tips.txt pm-tips.txt.orig

            ... load the pm-tips.txt to your text editor
            ... edit the file and save
            ... Print the version number first

            %   what pm-tips.txt    > pm-tips.txt.diff  # see man what(1)
            %   diff -u -bw pm-tips.txt.orig pm-tips.txt >> pm-tips.txt.diff

            ...Send content of pm-tips.txt.diff to document maintainer.

    1.6 About presented recipes

        The recipes presented here are collected from the net and procmail
        archives. I have tried my best to keep the recipes as original as
        possible, but I have generalised the examples when necessary. If
        some recipe doesn't work as announced, please a) send note to
        [jari] b) send email to procmail mailing list and ask how to
        correct it. I will watch the procmail list and I'll replace any
        faulty recipe with correct one.

        Sometimes I have taken the liberty to use a simple dot(.) in
        regular expressions, where the right, pedantic way would have
        been to use an escaped dot. If you want to be very strict, you
        should use the escaped dot where applicable.

            [free hand version]     [pedantic version]
            :0                      :0
            * match.this.site       * match\.this\.site

        Procmail also accepts assignments without quotes, like this:

            var = value
            num = 1
            dir = /var/mail

        But I have adopted a style, where literal strings are assigned with
        double quotes:

            var = "value"

        because the procmail code checker then won't warn you about missing
        dollar-sign, which you might have very well forgotten. Emacs fon-lock,
        a syntax highlighting package also displays double quoted string
        in color.

            #   If you do this...

            var = value

            #   then it is in fact not clear what was intended:

            var = "value"   # Did you mean:  literal assignment?
            var = $value    # Did you mean: variable assignment?

        Recipe flags are also _not_ stuck together, because for me the
        visual distinction of `:0' and `flags' is a valuable one.

            [Erm, all stuck]        [I like this better]
            :0ABDc:                :0 A BDc:

    1.7 Variables used in recipes

        These are part of the procmail module *pm-javar.rc* and are used in
        recipes.

            # Pure newline; typical usage: LOG = "$NL message $NL"

            NL = "
            "

        Refer to "improving Space-Tab syndrome" section for more details

            WSPC    = "     "               # whitespace: space + tab

            SPC     = "[$WSPC]"             # Regexp: space + tab
            SPCL    = "($SPC|$)"            # whitespace + linefeed: spc/tab/nl
            NSPC    = "[^$WSPC]"            # negation

            s       = $SPC                  # shortname: like perl -- \s
            d       = "[0-9]"               # A digit -- Perl \d
            w       = "[0-9a-z_A-Z]"        # A word  -- Perl \w
            W       = "[^0-9a-z_A-Z]"       # A word  -- Perl \W
            a       = "[a-zA-Z]"            # A word, only alphabetic chars

        Writing recipes is now a little easier and may look more clear.

            *$ Header:$s+$d+$s+$d           # Matches "Header: 11 12"

        _SUPREME_ = 9876543210, is the highest score value that causes
        procmail to bail out. [david] Actually the maximum is 2147483647,
        but 9876543210 is easier to remember/type and will function just as
        well.

        _PMSRC_ = Procmail includerc code directory, where *rc files
        reside. Anywhere you want it to be: usually $HOME/pm or
        $HOME/.procmail. Here you keep the procmail files, logfiles and
        includerc scripts. You can also use the synonym _PMDIR_.

        _SPOOL_ = Directory where your procmail delivers the categorized
        messages. Like mailing lists:

            list.procmail, list.lyx-users, list.emacs, list.elm

        and work mail:

            work.announcements, work.lab, work.doc, work.customer

        and your private message:

            mail.usenet, mail.private, mail.default, mail.perl

        and unimportant messages

            junk.daemon, junk.cron, junk.ube

        If you read the procmail-delivered files directly, this directory
        is usually $HOME/Mail or $HOME/mail. If you use some other software
        that reads these files as mail spool files (like Emacs Gnus), then
        this directory is typically ~/Mail/spool/ or similar.

        _MY_XLOOP_ = Used to prevent resending messages that have already
        been handled. Typically `$LOGNAME@$HOST', but this can be any user
        chosen string. Make it it unique to your address. In this document
        the definition is:

            MY_XLOOP = "X-Loop: $LOGNAME@$HOST"

        _SENDMAIL_ = Program to deliver composed mail. Usually standard
        Unix `sendmail', but it must have some switches with it. See man
        page for more. We use following definition in scripts:

            SENDMAIL = "sendmail -oi -t"

        _NICE_ = In a Unix environment you can lower the scheduling priority
        wth nice(1). If you are conscious of how many external processes you
        launch for each piece of mail it would be nice to lower the
        priority of such processes. You may see in this tips file that
        external processes are called with `NICE' enabled:

            :0 w                # same as nice -10 script.pls
            | $NICE script.pl

        _IS_ functions; eg. IS_EXIST is defined as "test -e" and so on.
        The definition of _IS_ functions are system-dependent.
        E.g. On Irix the "-e" option is not recognized and
        the nearest equivalent is "test -r". All _IS_ functions
        are defined in the `pm-javar.rc' module.

    1.8 About "useless use of cat award"

        Randal Schwartz, a well-known Perl programmer and Perl book writer,
        started giving emmy rewards for the "useless use of cat command"
        whenever someone wrote examples without token "<". Like this:

            % cat file.name.this | wc -l

        Instead he insisted that the call should have been written like this,
        which saves the pipe. (Never mind that `wc' can read the file
        directly; this is an example.)

            % wc -l < file.name.this

        I stick my opinion in this soup and you're free to disagree. When
        you see the shell commands used in this document, they are written
        so that they can be read from left to right: The "<" is in my
        opinion difficult to understand. As an example, I think that:

            % cmd1 < file1 | cmd2 > file2

        is less clear than my preferred way of writing such commands:

            % cat file1 | cmd1 | cmd2  > file2

        And now to the purist side: Is saving one pipe process so important?
        Let me see, I use a 2Meg file in this test:

            % time sh -c "cat some-file-name-is-here | time wc -l"
            0.29u 0.11s 0:00.47 85.1%

            % time sh -c "wc -l < some-file-name-is-here"
            0.27u 0.05s 0:00.39 82.0%

        There is not much difference, and this 2Meg file is not typical at
        all. The files typically used are many times smaller. The nitpicking is
        therefore pointless. Another reason why I use "left to right
        pipe writing": when you recall the command in csh, you can edit the
        last command's arguments easily. If you used the "<" token, tapping
        keyboard is _much_ more tedious (try changing wc command's option
        above). Oh yeah, you can write like this to get the command to the
        right, but that's even more obscure.

            % < some-file-name-is-here wc -l

        Dallman Ross  also mentioned that csh users can
        replace any word in the previous command by use of caret(^) editing
        commands, like this:

                % cat some-file-name-is-here | time wc -l
                % ^some-file-name-is-here^new-file^
                --> cat new-file | time wc -l

        Ahem, so there, I got it off my chest...

2.0 UBE in Internet

    2.1 Terms used and foreword

        [Part of this has been excerpted from the Email Abuse Faq]

        ._UBE_ = Unsolicited Bulk Email
        ._UCE_ = (subset of UBE) Unsolicited Commercial Email

        _Spam_ = Spam describes a particular kind of Usenet posting (and
        canned spiced ham), but is now often used to describe many kinds of
        inappropriate activities, including some email-related events. It
        is technically incorrect to use "spam" to describe email abuse,
        although attempting to correct the practice would amount to tilting
        at windmills.

        _Spam_ = definition by Erik Beckjord. "Some people decide that Spam
        is anything you decide you want to ban if you can't handle the
        intellectual load on a list." Remember, not to be confused with
        real spam, which is unwanted bulk mail.

        People are nowadays seeking a cure which will stop
        or handle UBE. That can be easily done with procmail (under your
        control) and with sendmail (by your sysadm). In order to select the
        right strategy against UBE messages, you should read this section
        and then decide how you will be using your procmail to deal with it.

    2.2 UBE strategies

        [Excerpted from the Email Abuse Faq]

       4g. I asked to be "removed" - guess what? I got another U*E

        Not surprisingly, many UBE outfits treat a "remove" request as
        evidence that the address is "live"; a "remove" request to some
        bulk emailers will actually guarantee that they will send more to
        you. For many others, the remove procedure does not work, either by
        chance or design. At this point perhaps you're starting to get a
        feel for the type of people with whom you are dealing.

        Also, getting removed doesn't keep you from being added the next
        time they mine for addresses, nor will it get you off other copies
        of the list that have been sold or traded to others. In summary,
        there is no evidence of "remove" requests being an effective way to
        stop UBE.

       4h. I asked to be "removed" - guess what? The message bounced

        Probably the remove procedure was false. Any remove procedure that
        tells you to send remove requests to AOL, CompuServe, Prodigy,
        Hotmail, or Juno is certainly false. The bulk emailers are an
        unpopular lot; they forge headers, inject messages into open SMTP
        ports, use temporary accounts, and pull other stunts to avoid the
        tirade of complaints that follow every mailing.

    2.3 UBE and bouncing message back

          Has anyone found that bouncing spam does any good at all?

        _Note:_ There are several program packages out there that
        can with a high degree of success (but not 100%) trace back a
        spam even if some headers are faked. This will not help you against
        spam houses (which don't care) but will speed you telling
        the sysadmins of an open relay. Such tools need human interaction for
        proper working. See pointers to them in this document later.

        Examine the messages by hand first and feed them to automatic
        complain script. See pointers in this document later.

        [sean] I had a whole policy message written up that would be sent
        out to spammers. Nothing but a waste of my resources. Most return
        paths are either completely bogus, or end up bouncing pretty damn
        soon after the spam, which just brings you more junk to deal with.

        Instead, I choose to send messages occasionally to administrators
        and upline providers of domains which spew.  "Agreement by action"
        is one of the legal standards I like to use (for "should you
        continue to send mail to me, that constitutes acceptance of the
        terms herein").

        InterNIC recently 1997-07 removed the root files for .com, .org,
        and .net (I think) from access at their ftp server. Too many
        spammers were using them for the purpose of generating mailing
        lists. Access to the files now requires an assigned FTP account
        from InterNIC. When I get a domain-style spam, I immediately do a
        whois to get DNS info on the domain, then grep the root files to
        obtain a list of domains serviced by the same DNS. If they appear
        spammy (as spam domains tend to), I add these to a list of domains
        to filter (egrep) in my primary domain-based ruleset. Works for
        me, though the list is getting big.

        [Kimmo Jaskari ] Another good reason is
        that all those bounces, which get ignored by the spammer/recipient
        anyway, still take up needless bandwith on the net. The spam is bad
        enough for that, bouncing it back with some more stuff added is just
        plain silly. You become part of the problem rather than the solution.
        If the bounce even gets to the spammer, the spammer drops it on the
        floor unseen.

        [1998-11-03 PM-L Mark Shaw ] Jari:
        "Autoresponder is bad idea. You need more better heuristics than
        what procmail can do. The UBE messages really need human
        inspection before you send them out, otherwise you may have to
        apologise from lot of people eg if the complaint was mistakenly
        sent off to some mailing list or wrong address." Mark: Having
        originally set up my anti-spam recipes to be autoresponders, I
        absolutely agree with this. I recall one morning when my strongly-
        worded no-spam message went out to *everyone* who sent me email for
        several hours..... *** shudder ***

    2.4 UBE and "I don't mind" attitude

          ...whenever you see a spam you don't want, hit the delete key and
          move on. Grow up and get a life, folks. The spams just don't
          bother me. Why the hell does everyone have to go up in arms
          everytime someone sends a spam? Spams are harmless! Spams even
          sometime are interesting and/or useful!

         [Responses from thread in procmail mailing list 1995-10 to
         "FREE 1 yr. Magazine" spam.]

       [Soren Dayton ]

        The simplest reason against UBE is that it is rude. It costs some
        people money to get email on some commercial services. This is
        fundamentally different than junk snail mail for this reason and
        too much spam can prevent people from getting mail (mailboxes can
        fill up).  So it is both an intrusion into my life _and_ it can
        conceivably end in me either loosing money or loosing mail (which is
        far more important).  It is a burden on the receiver _far_ beyond
        just hitting the delete key.

       [Mark Seiden ]

        people who are able to monitor the incoming machines of one of the
        larger online services (like me) can see a sizeable increase in
        system load average and volume directly resulting from spams. this
        competition for fixed resources inevitably translates to reduced
        service for "first class" mail.

        It is impossible to engineer a mail system that can cope with an
        unlimited amount of abuse. this is in addition to the difficulties
        of doing so on a fixed price economic model, and the difficulties
        of keeping up with the successful rapid expansion of the population
        to be served.

        Even if you, an individual, aren't charged anything per piece of
        mail, there are costs borne by your service provider per piece of
        mail, and these are *somehow* passed on to you. (They've calculated
        an average across their entire user population to come up with a
        "monthly cost of Internet mail".)

        Spamsters and bulk mailers are not at all concerned about
        efficiency. as proof of that, many of them are not even courteous
        enough to supply a proper return address, so they can prune their
        lists of undeliverable mail. all they care about is getting their
        message across without their paying anything whatsoever for that
        service.

        Watch how this will inevitably translate into increased costs for
        you, the consumer, unless we change the mechanisms by which bulk
        mail is delivered as well as putting an appropriate economic model
        in place.

       [Steve Simmons ]

        If you tolerate spamming, it will only get worse. Spamming has been
        stopped again and again. Almost without exception, the spammers
        have been tracked down and, via one means or another, have been
        convinced to stop spamming.

        Spams are harmless? I've already seen the 'Magazine Sub' message
        10 or 12 times. I have a low bandwidth line. If I continue to
        tolerate spamming, I will pay a very real penalty in performance as
        tens, then thousands of spammers do it. Not to mention the
        personal time involved in taking care of the crap.

        Don't think that the time involved is significant? Just wait. My
        wife and I are fairly generous with our time and money. As a
        result, we were getting an average of five telephone calls *per
        night* asking for money for various causes. A year ago, I adopted
        a new policy -- I will not under any circumstances give money to a
        caller, and will only consider it upon written solicitation. I
        ask them to put me on their `do not call list'.  If they do
        *anything else* to continue the conversation, I hang up on them.

        My wife opposed this, and we agreed to disagree -- if they ask for
        her, they get her. If they ask for me, they get my speech. After
        a year, she is getting 2-3 calls per night and I'm getting one or
        two a week.

        My point here is that individual action *does* get re-action from
        the mailers. For them, I copy their internet providers on my
        complaints and call their Better Business Bureau. It works.

        If one does this politely and consistently, 98% of the spammers
        will stop. The remaining 2% will discover that they're in a
        different world from direct mail or telephone solicitation. Their
        mailboxes will be overloaded with complaints (when it takes a
        single keystroke to invoke your complain macro, you're very likely
        to complain).  Then their suppliers mailboxes will be overloaded
        with complaints. The free magazine folks, who've been hiding
        behind false ids and forging mail, will find that they're on the
        wrong side of the law. I'm considering contacting their local
        legal officials and urging them to investigate, because it sure
        looks like fraud to me (read `Consumer Reports' for a similar case
        by surface mail).  Should a few more like this come in, I *will*
        contact their legal authorities. We have their fax number; it's
        all we need to find them.

       [Carl Payne ]

        Um, I don't know about you or anyone else here, but this cutesy,
        "it's-okay-by-me" spam has been circulated under half a dozen
        different user names and "domains" on as many mailing lists. It's
        obvious to me the sender is trying to make people pissed off--how
        can he possibly think someone will buy that crap, and why does he
        think it's okay to send 19 and 20K files over a billion groups?

        AFAIC, it has to stop. Now. I'm tired of the spam, I'm tired of
        the "Who cares" attitude about spam, I'm tired of ISPs letting
        people spam, I'm tired of the jetwash of spam, and I'm tired of the
        bleedinghearts that say, "Golly, just ignore it, and it'll go
        away."

        I've got news for you all: when this method of spamming becomes the
        preferred method of "marketing" on the internet, and people like us
        are the bad guys because we're not allowing such litter to fly
        across the fiber, you will care. You will say something, most
        probably, "Why didn't we do something about this sooner?"

        The guy in the next cube from you, who's paying a per-message
        charge through his ISP, is probably going, "Dammit, over three
        dollars this month on mail I've itemized as being spam."  While
        that doesn't seem like a lot, I revert to my earlier statement: if
        this becomes the preferred method, his bill (and yours) will go up,
        and everyone will wonder why it's too out of control to do anything
        about.

        Spam has the letters *m-a-s* in it, which en Espanol, means "more."
        I say no. Not only no, but hell no. And, I refuse to be told that
        my thinking is out of line just because I don't want my mailbox
        flooded. Do something now. Do anything now. But, don't be quiet
        and listen to anything that sounds like an endorsement of litter

       [Wolfgang Weisselberg ]

        Worse is that it costs a spammer very little to spam, say, 2
        million addresses with 5KB:

        o   5 hours unattended time online
        o   phone costs
        o   a 'free x hours'-CD or a provider looking the othher way i.e.
            something between $0 and $500 (an expensive provider)

        It costs all recipients:

        o   on an average of 5 seconds per UCE to decide that, indeed, it
            is one: 115.7 *DAYS* (2777.8 hours) of mailchecking (at $7.5/h
            that is just $20833 --- excluding all taxes and so on!)
        o   379.5 hours (15.8 days) download time (multiply with your local
            phone costs and remember that in most places even in-city calls
            cost by the minute)
        o   the same time as online time (multiply by your provider costs)
        o   indirect costs (more HDs for the provider (9.5 GB), faster
            connections for all the spams, more transmission costs (9.5
            GB), faster machines, ...

        I can send you the complete calculation if you like :-)

        Now, if UCE becomes more common ... how many businesses are
        connected to the Internet? Say that every business spamms once
        every 10 years, and that they are well distributed over the time.

            Number_of_businesses / 3650 = UCE's iniciated per day
            UCE's iniciated per day * 2_000_000 (or more)
                        / number of email addresses
            = UCEs in your mailbox

        Guess we are going to need T1's to just get all our mail. And a few
        100 secretaries as well. Wave good-bye to usable email.

    2.5 We need a law against UBE

          Ray Everett-Church , Attorney/Online Consultant
          Co-Founder & Congressional Liaison Coalition Against Unsolicited
          Commercial Email; article 1997-12 in remailer politics mailing
          list

        In developing what eventually became the Smith Bill, CAUCE
        discussed this rather extensively among our drafting committee. The
        bill gives a cause of action against the advertiser, not any of the
        pathways taken between you and them. This is consistent with the
        interpretation of the fax law (and many other laws for that matter)
        wherein the advertiser -- not the advertiser's agent -- is
        responsible for the act committed.

        As for the single UCE versus bulk issue, the general consensus has
        been that while a single piece of spam does not do much damage, it
        is fundamentally no less a cost shift than 10 identical messages,
        or 100, or 1000, or a million. The only difference is that the
        costs being shifted are greater and greater. We discussed many cut
        off points... would 50 spams be acceptable? 25? 10? One really well
        crafted, hand written, heartfelt and personalized spam be
        permissible? And in the end we felt like we were discussion angels
        on the heads of pins.

        While virtually nobody's system will crash because of one piece of
        spam (although George Nemeyer had trouble with three or four pieces
        as I recall), what is the ultimate difference if you only get one
        piece from each of 15 different advertisers a day? If one spam is
        ok, but two are bad, what is the interval... a day, a week?
        Enforcement depends on knowing when the threshold is crossed.

        So here's a scenario: you receive three spams from what is,
        unbeknownst to you, the same person (one advertising weightloss
        pills from WeightLoss Associates at PO Box 1, one for an MLM from
        MLM Company at PO Box 2, and Bee Pollen from Pollen Partnership at
        PO Box 3). Each were individually crafted and appeared to be mailed
        only to you.

        Under the scenario above, if the law permits one spam, will you
        sue?

        Would you risk suing one or all of them, gambling that they sent
        the spam to anyone other than you (or whatever the threshold is...
        10, 25, 50)? Would you risk suing one or all of them on the chance
        that they were somehow related? What if there was a chance that
        you'd find out that the three companies were really different? What
        if you did sue and found that they were owned by the same person,
        but were legally organized separate entities and were therefore
        each entitled to one spam a piece?

        In short... if one spam is permitted, it could make enforcement
        incredibly cumbersome, difficult and unlikely, and would present
        spammers with many reasons to violate the law knowing the odds of a
        suit and successful enforcement are greatly reduced. While bulk
        spam is really bad on many levels, whether it's parsed out in very
        small volumes makes little or no difference to the ultimate
        recipients as far as the diminished utility, cost, and annoyance.

        We need a clear, bright line. And the Smith Bill is that.

3.0 Anti-UBE pointers

    3.1 NoCEM, CAUCE and others

       "NoCEM"
        http://www.cm.org/

       "Dougal's NoCeM-E"
        http://advicom.net/~dougal/antispam/
        ... Dougal is sysadm for an ISP. His page has wealth of information
        about Anti-SPAM Tools. You also find his mailing list for NoCeM-E.

       "The Coalition Against Unsolicited Commercial Email (CAUCE)"
        http://www.cauce.org/faq.html
        ...The Problem: Unsolicited commercial email, more commonly known as
        "spam", is a growing problem on the Internet. If you've used the
        Internet for any length of time, you've probably received
        solicitations via email to purchase products or services.

        A Solution: A group of Internet users who are fed up with spam have
        formed a coalition whose purpose is to amend 47 USC 227, the
        section of U.S. law that bans "junk faxing", so that it will cover
        electronic mail as well.

       "Teergrubing against Spam"
        http://www.iks-jena.de/mitarb/lutz/usenet/teergrube.en.html
        ...`Teergrubing' It's German and means Tar-Pit. Once you have been
        stuck you can't get out. ...slow down internet connections in order
        to stop UBE abuse. Several hundred teergrubes are able to block
        spamming worldwide without blocking any e-mail. How do I start: If
        you are the admin of a MX host, install a teergrube.

       "Obtuse smtpd for UNIX"
       http://www.obtuse.com/smtpd.html
        Main (configurable) features:

        o   deny unauthorized relay (no more relay rape!)
        o   permit selective relay exceptions (eg. UUCP downstream)
        o   regex() filtering [block those spamming dialins!]
        o   deny access for no MX, no PTR, etc.
        o   defeat % hack
        o   support MAPS, ORBS, DUL, IMRSS, etc RBLs plus your local RBL
        o   support exception list for domains for which you will accept mail
        o   support selective tarpit'ing on refused connections
        o   individually configurable rejection messages
        o   precedence and override ordering
        o   informative log summary scripts

       "Lot of good articles about spam"
        http://www.sun.com/sunworldonline/swol-12-1997/swol-12-spam.html

        "(anti-spam Law) US Representative Chris Smith's statement on junk
        e-mail"
        http://www.sun.com/sunworldonline/swol-08-1997/swol-08-junkemail.html
        ...considerable variation in the approaches at the federal level,
        and state legislation varies widely as well. Professor David Sorkin
        of John Marshall Law School, who summarized and provided links to
        the major spam-related lawsuits noted above, also provides status
        summaries and links to state and federal legislation

       "Select email court cases -- Lots of them"
        http://www.jmls.edu/cyber/cases/spam.html
        America Online, Inc. v. Cyber Promotions, Inc.,
        Compuserve Inc. v. Cyber Promotions, Inc., etc.

       "Anti-Spam Directory of Information and Resources"
        http://www.ao.net/waytosuccess/nospam.html
       "Forum for Responsible and Ethical E-mail (FREE)"
        http://www.ybecker.net/

       "Ethical Marketing Using FREE Resources"
        http://www.ao.net/waytosuccess/index.html

    3.2 General Filtering pages (more than procmail)

       "Nancy McGough - Mail Filtering FAQ"
        http://ssil.uoregon.edu/~trenton/autopage/page7547.html
        http://www.ii.com/internet/faqs/launchers/mail/filtering-faq/

       "Information Filtering Resources"
        http://www.ee.umd.edu/medlab/filter/ Doug Oard 
        ...This page lists all known internet-accessible information
        filtering resources.

    3.3 Junk email and spam

       "Spam FAQ"
        ftp://rtfm.mit.edu/pub/usenet/alt.spam/
        http://www.cs.ruu.nl/wais/html/na-dir/net-abuse-faq/spam-faq.html

       "The email abuse FAQ"
        http://members.aol.com/emailfaq/emailfaq.html
        What is UBE, UCE, EMP, MMF, MLM, Spam, it is all explained here.

       "Get that spammer -- A VERY GOOD LINK"
        http://kryten.eng.monash.edu.au/gspam.html
        ...All about Spam; traceroute, netabuse etc. Full of links and docs

       "Whois"
        http://www.networksolutions.com/cgi-bin/whois/whois/

       "Advertising on Usenet: How To Do It, How Not To Do It"
        ftp://rtfm.mit.edu/pub/usenet/advertising/

       "Dealing with Junk Email"
        http://www.mcs.com/~jcr/junkemaildeal.html
        ...What you should do (and not do) when you have been victimized by
        a junk emailer. This document teaches you how to read headers in
        order to trace the origin of junk email, and includes detailed
        examples to show you how it is done. Headers are designed for
        computers to read, not people, so they can be a little hard to
        follow. Therefore, I hereby grant permission to print or
        electronically save a copy of this page on your local machine for
        your personal use while tracing junk email. Please check back for
        updates and corrections, though.

        o   What Not To Do: Stuff that doesn't work
        o   What to do: effective techniques, including how to trace junk
            email back to its source
        o   Stay Calm (take a deep breath...)
        o   Stay Mad (don't get discouraged)
        o   How to identify the sender and who gives them Internet access
        o   Who to complain to, abuse addresses, online services
        o   What to say and how to say it, effective complaining

       "How to fight back."
        http://www.oeonline.com/~edog/spamstop.html

        .   Look at the header of the advertising message. Find the
            "Message-ID" line. (You might have to tell your e-mail program to
            display this.)
        .   The words after the @ sign are the sender's real--not
            faked--Internet Service Provider, or ISP. (Spammers often try to
            disguise their address, but the Message-ID is a good clue.)
        .   Write a complaint to the postmaster of that ISP, similar to the
            one below. (If the ISP is junkmail.com, then let
            postmaster@junkmail.com hear from you.)

       "Practical Tools to Boycott Spam"
        http://spam.abuse.net/spam/
        ...We have been actively engaged in fighting spam for years. Recent
        events, including pending court battles, prompt us to present this
        page to the public. Fight spam to keep the Internet useful for
        everyone.

        o   Filtering mail to your personal account
        o   Blocking spam email for an entire site
        o   Blocking Usenet spam for an entire site
        o   Blocking IP connectivity from spam sites
        o   Other tools and techniques for limiting spam
        o   Sample Acceptable Use Policy statements for ISPs

       "Spam -- stop that!"
        http://com.primenet.com/spamking/buyerbeware.html

       "The Campaign to stop junk email web site"
        http://www.mcs.com/~jcr/junkmail.html
        ...we will attempt to teach victims and potential victims (that's
        everyone with an email address) the most effective methods of
        prevention and retribution.

       "news.admin.net-abuse.* Homepage"
        Timothy M. Skirvin 
        http://www.ews.uiuc.edu/~tskirvin/home/nana/

       "The automated spamhandler beta information heap."
        http://www.halcyon.com/natew/

       "Preventing relaying in Sendmail"
        ...This package adds two independent features to sendmail,
        access control and relay control. They will be described here
        simultaneously, but you can elect to include support for only one
        of them (either one) on your mail server. Access control lets you
        deny access to the server based on the senders envelope address or
        his IP address. Relay control lets you decide who gets to relay
        email through your server.
        ftp://ftp.xyzzy.no/sendmail/access.tar.Z

       "Anti-Spam Provisions in Sendmail 8.8"
        http://www.sendmail.org/antispam.html
        http://maps.vix.com/tsi/
        http://www.informatik.uni-kiel.de/%7Eca/email/check.html#check_rcpt

        o   Preventing relaying through your SMTP port
        o   Refuse mail from selected hosts
        o   Restrict mail acceptance from certain users to avoid mailbombing

        [1998-06-15 PM-L walter] Somebody's starting to exploit a hole in
        sendmail 8.8, where giving a HELO longer than 1024 bytes causes
        buffer overflow, and all following "Received:" headers are lost. If
        it's done off a relay, we have no clue who sent it. There may be a
        more elegant solution, but here's a quick-n-dirty procmail filter
        for this stunt...

       "Blocking Email"
        http://www.nepean.uws.edu.au/users/david/pe/blockmail.html

        o   Do you or your users, receive "junk email" (aka., "spam")
        o   Do you have Sendmail R8.8.5 running at your site?
        o   Would you like to block known "junk email" senders' addresses?

        Now you can - and there's no need to patch any source code, either.
        Take advantage of Sendmail's check_mail rule, to see if the
        sender's address is a member of a nominated "class" - drawn from
        the contents of the named file. Additional information and links:

        o   Prospective Addresses/Domains to Block
        o   Limiting Unsolicited Commercial Email
        o   EFF "Net Abuse and Spamming" Archive
        o   [U.S.] Court Lets AOL Block Email
        o   Anti-Spam HOWTO
        o   Net Abuse FAQ
        o   Figuring out Fake Email & Posts
        o   Fight Unwanted Email
        o   Unsolicited Junk Email - Bad for Business
        o   Fight Unsolicited Email and Mailing
        o   Yahoo's Junk Email Resources
        o   jmfilter
        o   Complaints Addresses at U.S. ISPs
        o   news.admin.net-abuse.* Homepage
        o   Processing Mail With ProcMail
        o   Panix's rc.shared ProcMail Configuration
        o   ProcMail Workshop
        o   Email Self Defence
        o   The SPAM-L mailing list

       "US Federal Trade Commission"
        http://www.ftc.gov/
        ...staff publicized the Commission's UCE mailbox, "uce@ftc.gov,"
        and invited consumers to forward their UCE to it. spam complaints
        

       "Spam Spade Web based tracking tool"
        http://www.blighty.com/
        ...Figuring out forged headers and verifying IP addresses and
        whois information.

       "Misc"
        http://www.junkbusters.com/
        http://www.well.com/~jbremson/spam
        http://www.wolfenet.com/~jhardin/procmail-security.html

    3.4 Comprehensive list of spammers

       "Against Spam -- The garbage collecting."
        http://www.spam-archive.org/
        To support this archive please forward email spam to
        . Everybody is invited to bounce Mail-Spam
        he/she has got to this list. This is a mailing list to distribute
        actual spam-eMail. All incoming mail will be checked by subject and
        from/sender-address wether it has already been distributed or not.
        No discussions in this list. To discuss about this list please
        subscribe to .

        To subscribe to _blacklist-update_  mailing list
        TO:   
        BODY: subscribe blacklist-update you@somewhere.com
        Mail  to discuss about blacklist if
        your name is on it. (maintained by Axel Zinser )
        Get the updated blacklist from
        ftp://ftp.spam-archive.org/spam/blacklist/

    3.5 Misc pointers

          Is there a way to block local users from spamming other sites?
          Maybe somehow force sentmail to read a rc file that would maybe
          then grab the from field and see if the user exists on the system
          or not. Or run it through some sort of filters.

        [philip] You can and should do this purely in sendmail. I ended up
        crafting a check_from ruleset that verifies that the envelope
        sender address is either a) not local; b) a local user; or c) a
        local alias. At the time I did this mainly to force people to
        configure their Eudora clients so they didn't say "Return Address:
        yourname@gac.edu" but it also covers the outgoing bogus source
        address spam case. For those interested in this kinda thing I've
        (just) put it up for FTP:

            ftp://ftp.gac.edu/pub/guenther/

       "IBM's Secure Mailer -- open source"
        http://www.postfix.org/

          [1998-12-15 PM-L Matthew McGehrin ] The
          official project is known as 'IBM's Secure Mailer'. The
          unofficial codename was Vmailer, but they had to rename that, to
          Postfix to agree with the lawyers. I should know, I have been
          alpha testing this mailer for the past year, and it so blazing
          fast, its amazing. It's faster and simplier to use than sendmail,
          and also faster and more secure than qmail. It works fine with
          procmail. (look in my headers). set
          "mailbox_command=/usr/bin/procmail" in /etc/postfix/main.cf

          [1998-12-15 PM-L Liviu Daia ] it has
          explicit hooks for both procmail and RBL. In fact it's incredibly
          easy to setup, I got it compiled and configured (with an actually
          usable configuration) in about 15 minutes after downloading it.
          Adding masquerading and a virtual domain took another 2 minutes.
          :-) You should really give it a try, it's faster than QMail and
          _much_ faster than sendmail. So far, I'm quite impressed.

       "Qmail"
        http://pobox.com/~djb/qmail.html
        http://www.qmail.org/

       "Sendmail"
        http://www.sendmail.org/

       "Fetchmail -- old pop3 replacement"
        ftp://ftp.ccil.org/pub/esr/
        http://www.ccil.org/~esr/
        http://www.tuxedo.org/~esr/fetchmail/

       "Maildrop filter utility"
        http://www.geocities.com/SiliconValley/Peaks/5799/maildrop.README.html
        ...Alternative to procmail

       "Lua"
        http://www.tecgraf.puc-rio.br/lua/ 
        [possible replacement for procmail language] ... *Lua* is a
        programming language originally designed for extending
        applications, but also frequently used as a general-purpose,
        stand-alone language. Lua combines simple procedural syntax
        (similar to Pascal) with powerful data description constructs based
        on associative arrays and extensible semantics. Lua is dynamically
        typed, interpreted from bytecodes, and has automatic memory
        management with garbage collection, making it ideal for
        configuration, scripting, and rapid prototyping.

    3.6 Questionable UBE stop services

       "IEMMC: Internet E-Mail Marketing Council Formed 1997-03"

        The IEMMC was formed to provide an industry wide trade association
        for the purpose of promoting responsible e-mail marketing, and to
        establish an industry standard code of procedures and ethics which
        will internally regulate and govern the commercial e-mail marketing
        industry....Under this system, all e-mail of a commercial,
        unsolicited nature must pass through a universal filtration system
        which will block the sending of any and all commercial e-mail to the
        address on the list. Bulk e-mailers will be required to join the
        organization

        Others have commented that:

          ...IEMMC is a joke. you are probably not doing yourself any favors

          ...Don't take that IEMMC seriously! Many people registered with
          them and got as many or even more spam as before. After all,
          Cyberpromo (the operator of IEMMC) knows that the registered
          addresses will be valid for some time, so they can use and sell
          this valuable list to other junk mailers.

       "Spammer blacklist"
        http://www.netchem.com
        ... Dear Sir/Madam, Your email address may be on
        many spammers' lists. We are compiling a *remove* list. Forward the
        original junk to 

       "No Junk E-Mail database"
        http://pages.ripco.com:8080/~glr/nojunk.html
        ...We will help stop unwanted email to you..the list is submitted to
        us, and those addresses that appear in the "do not mail" list are
        removed and the "cleaned" list is returned

    3.7 UBE related newsgroups or mailing lists

        alt.kill.spammers
        alt.hackers.malicous
        alt.2600

        [1997-08-13 alt.privacy.anon-server by anonymous poster] Proper
        etiquette demands you contact their ISP. However, if the ISP are
        not interested in helping you, you should consider a posting in
        alt.kill.spammers (or even alt.hackers.malicous or alt.2600) - give
        as many details as you can about the spammer.

        A certain spam-provider targeted the alt.hackers.malicious
        newsgroup. Not the most sensible thing to do. The ISPs IPs were
        found, their MX host was hacked. All their DNS entries was
        published on alt.2600 (so that everyone could add filters to ignore
        all mail from this company). Oh yeah, their password file also made
        it to the group! The ISP then posted a complaint to alt.2600, much
        to the enjoyment of everyone who took part. That host basically
        died a horrible death. I'm pretty sure that not many people are
        going to lose any sleep over this! I might as well mention that the
        ISP's complaint mentioned that their "freedom" was being
        abused. hehehe. Most of these postings can be seen in dejanews
        or altavista archives of usenet.

       "SPAM-L mailing list and Doug Muth's Page"
        http://www.claws-and-paws.com/spam-l/
        ... "The SPAM-L FAQ" - A FAQ for SPAM-L, an anti-spam mailing list.
        This FAQ discusses how to join the list and what to post there, AND
        it also delves into the technical aspects of spam. For instance,
        the various kinds of forgeries seen in spams are discussed here,
        along with information on how to recognise them. If you hate spam,
        this is something worth checking out... "TheGoodsites List" - I
        maintain this list, which is part of the Spam Boycott, to show
        which Internet providers out there act responsibly when dealing
        with spam. If you're looking for an ISP and want to know where they
        stand on spam, this is the list for you.

        Send an email message to 
        with the words "subscribe SPAM-L  " in the
        body of the message (no quotes). f you would like to contact the
        owner, the convention is the same as with all listserv lists. Just
        send e-mail to 

    3.8 Software: the net abuse page

        Scott Hazen Mueller 
        http://spam.abuse.net/spam/tools/

    3.9 Software: adcomplain -- Perl junk email rport

        
        http://www.rdrop.com/users/billmc/adcomplain.html

        Adcomplain runs under Unix, Windows-NT, and Windows-95. Adcomplain
        is a tool for reporting inappropriate commercial e-mail and usenet
        postings, as well as chain letters and "make money fast" postings.

        It automatically analyzes the message, composes an abuse report,
        and mails the report to the offender's internet service provider.
        The report is displayed for your approval prior to mailing.
        Adcomplain can be invoked from the command line or automatically
        from many news and mail readers.

        #todo: url missing

          [a user happy user reports] ...About 95% of all cases can be
          traced correctly --- unless they come from a known spamhouse;
          where complaining to them would not do much good anyway. Mailing
          lists with strange Received-Headers also can present problems in
          tracing

    3.10 Software: Ricochet -- Perl junk email rport

        http://www.vipul.net/ricochet/
         Vipul Ved Prakash

	MailingLi´st:  with subject 
	"subscribe"

        A lot of unsolicited email goes unreported because tracing the
        origins of a possibly forged mail and finding the right people to
        report to is complicated and time-consuming. Ricochet, a smart net
        agent, automates this process. It traces the names and add resses
        of the systems where the spam originated from along with the
        servers that provide domain name resolution services to these
        systems (in most cases their ISPs). Then it collects/generates a
        list of email addresses of tech/billing/admin/abuse contacts of
        these system and mails them a complaint and a copy of the spam.
        Detailed description of its workings can be found in the README
        file that comes with the package.

    3.11 Software: yell -- perl

        ftp://ftp.netcom.com/pub/bo/bobmacd/yell  (57k)
        Bob MacDowell 

        yell - auto-responds to "spam" e-mail. Scans for site names, e-mail
        addresses and Web site names and sends appropriate messages to
        users, postmasters and Webmasters.

    3.12 Software: ifile - Perl

        http://www.cs.cmu.edu/~jr6b/ifile/
        Jason Daniel Rennie 

        ...ifile is different from other mail filtering programs in
        three major ways: 1) ifile does not require you to generate a set
        of rules in order to successfully filter mail 2) ifile uses the
        entire content of messages for filtering purposes 3) ifile learns
        as you move incorrectly filtered messages to new mailboxes ifile is
        not dependent upon any specific mail system and should be adaptable
        to any mail system which allows an outside program to perform mail
        filtering. Currently, ifile has been adapted to the MH and EXMH
        mail systems.

    3.13 Software: RBL lookup tool -- C

        [1997-12-04 PM-L Edward S. Marshall ]

          ...rblcheck is a lightweight C program for doing checks against
          Paul Vixie's Blackhole List. It works well in conjunction with
          Procmail for filtering unwanted bulk email (under QMail, for
          example, you can invoke it with the value of the environment
          variable TCPREMOTEIP). rblcheck is extremely simple:

            % rblcheck 1.2.3.4

          where 1.2.3.4 is the IP address you want to check.

        This is a quick note to announce the availability of a new tool for
        using Paul Vixie's RBL blacklist (see http://maps.vix.com/rbl/ for
        more information about the blacklist itself, if you don't already
        know). Most tools which use the blacklist block email on a
        site-wide basis. For many networks, this treads on both the ideals
        of the administration, and on the perceived freedoms of the end
        user.

        Personally, I don't care either way. :-)

        This tool was to fill the need I personally had to reject mail,
        since one of the systems I receive mail through cannot, for various
        political reasons, implement the available RBL filters on a
        site-wide basis.

        rblcheck is a simple tool meant to be used from procmail and
        other personal filtering systems under UNIX in the absence of a
        site-wide filter, as an alternative to imposing site-wide
        restrictions, or as a means of imposing restrictions on systems
        that cannot support the existing RBL filter patches.

        Simply put: you hand it an IP address, and it determines if the IP
        is in the RBL filter, providing the caller with a positive or
        negative response. With the package, a sample procmail recipe is
        provided, and examples of using it under QMail and Sendmail are
        given.

        .http://maps.vix.com/rbl/
        .http://www.isc.org/bind.html            The official home page
        .http://www.xnet.com/~emarshal/rblcheck/

        It has only been tested under Linux 2.x and Solaris 2.5.1. Success
        stories, patches, questions, suggestions, and flames can be
        directed to me at .

        [PM-L Aaron Schrab ] Here is my rbl
        setup, but, this depends both upon the format of the Received:
        lines, and the way that mail passes through your mail system.

        I currently grab the IP address from the first Received: header
        inserted by my ISP (I'm a sysadmin at the ISP, so I have a good
        knowledge of how mail gets passed around internally). Here's the
        recipe that I use.

            # if there's a Received: header from one of these servers, it's
            # (probably) the right one

            BACKUPSERVER    = "([yz]\.mx\.execpc\.com)"
            VIRTSERVER      = "(vm[0-9]+\.mx\.execpc\.com)"
            LOCALSERVER     = "([abc]\.mx\.execpc\.com)"

            # Match a header containing:
            #   Received:  []) by 

            :0
            * $ $SUPREME^0 ^Received:.*\[\/[0-9.]+\]\)$s+by$s+${BACKUPSERVER}
            * $ $SUPREME^0 ^Received:.*\[\/[0-9.]+\]\)$s+by$s+${VIRTSERVER}
            * $ $SUPREME^0 ^Received:.*\[\/[0-9.]+\]\)$s+by$s+${LOCALSERVER}
            {
                IP = $MATCH

                # trim it down to just the IP address

                :0
                * IP ?? ^^\/[0-9.]+
                {
                    IP = $MATCH

                    :0 W
                    * ! ? /home/aarons/bin/rblcheck -q $IP
                    {
                        SPAM = "$SPAM $IP is rbl'd$NL"
                    }
                }
            }

          It seems to be a procmail issue with letting the IP info
          from sendmail pass through to the rblcheck program. I have not
          been able to find anyone using rblcheck successfully with
          procmail as a delivery agent...

        [1998-03-26 PM-L Edward S. Marshall  ] This is a
        standard problem; you should be able to change the invocation of
        procmail the same way as the example (run env, which in turn runs
        procmail). Make sure that there is a '-p' argument passed to
        procmail; this preserves the environment you're constructing with
        env (newer sendmail revisions sanitize the environment for you, so
        that's not really an issue).

        If you're still having troubles, make sure you're using the latest
        incarnation of rblcheck, with the latest supplied procmail recipe;
        earlier revisions had rather insidious bugs.

        [1998-03-26 PM-L Xavier Beaudouin (kiwi) ] Also it
        seems that sendmail 8.9.0Beta3 has builtin rules for
        rbl.maps.vix.com. This is somewhat really efficient. I use it with
        sendmail 8.8.8 and tcpwrapper every day and there is about 80%
        spam rejected. Sounds very good. In your /etc/hosts.allow just add
        the following lines :

            sendmail: ALL: spawn /usr/local/bin/rblcheck -q %a && \
                        exec /usr/sbin/sendmail -bs || /bin/echo \\
              "469 Connection refused. You are in my Black List !!!\r\b\r\n"
              && \
              (safe_finger -l @%h 2>&1 | /bin/mail -s "%d-%h %u" root)

        In your /etc/inetd.conf just add this line :

            smtp stream tcp nowait root  /usr/sbin/tcpd  \
                 /usr/sbin/sendmail  -bs

        And check that your sendmail is _not_ working as a daemon. That's
        all. Also if you have huge queue you can add a /usr/sbin/sendmail -q
        in the root crontab... This should help to send some waiting
        messages. I think we can use this to wait for official 8.9.0
        sendmail since there is some cf/feature/rbl.m4 there.

        [timothy] ...I think there's a much more efficient way to do
        this: you can compile sendmail -DTCPWRAPPERS and let it run as a
        daemon

    3.14 Software: mapSoN

          Note: You can do exactly the same as below with procmail with one
          of the listed procmail modules: pm-jacookie.rc. See the code.

       "mapSoN (NoSpam backwards) -- The no spam utility"
        http://mapson.gmd.de/
        ftp://ftp.gmd.de/gmd/mapson/

        Most spam filtering tools I've seen so far are based on procmail, or
        a similar tool, and use a list of keywords or addresses to drop
        unwanted junk mail. While this might be nice to filter mail from
        known spam domains like "cyberpromo.com", it won't catch faked
        headers.

        mapSoN must be installed as filter program for your incoming mail,
        usually by adding an appropriate entry to your $HOME/.forward file.
        This means that mapSoN will get all your incoming mail and it will
        decide whether or not to actually deliver it to your mailbox.

        .   First of all, an user defined ruleset is checked against the
            mail. If any keywords or patterns match, the mail will be dealt
            with according to your wishes. This is useful to drop some
            sender's mail completely, or to sort mail into different mail
            folders.
        o   If no rule matches the mail, mapSoN will check whether the mail
            is a reply to an e-mail you sent, or whether it is a reply to a
            USENET posting of yours. If it is, the mail will always be
            delivered.
        o   If no signs of a reply-mail can be found, mapSoN will check
            whether the sender stated in the From: header has sent you mail
            before. If he has, the mail will pass. If this is the first time
            you receive an e-mail from this address, though, mapSoN will
            delay the delivery of the mail and spool it in your home
            directory. Then it will send a short notice to the address the
            mail comes from, which may look like this:

            From: Peter Simons 
            To: never_mailed@me.before
            Subject: [mapSoN] Request for Confirmation

            mapSoN-Confirm-Cookie: 

        The person who tried to contact you will then reply to this
        "request for confirmation", citing the cookie stated in the mail.
        When your mapSoN receives this confirmation mail, it will deliver
        the spooled mail into your folder. Furthermore, the address will be
        added to the database, so that mail from this person will pass
        directly in future.

        If no confirmation mail arrives within a certain time, mapSoN can
        either delete the spooled mails, or send them to a special folder,
        or whatever you prefer.

    3.15 Software: spamgard

        [similar to MapSon]
        ftp://ftp.netcom.com/pub/wj/wje/release/sg-howto

        ...sppamgard(tm) screens from your e-mail unsolicited bulk mail. It
        does this in a way that you only have to change things if you have
        a new person from whom you _do_ want to receive mail; you don't
        have to change things every time a spamster thinks of a new trick
        to pull, or a new spamster comes along. And spamgard(tm) is
        designed so that those who aren't in your "Good Guys" list can get
        mail to you anyway until you put them there. The instructions for
        them to get mail to you are simple and newbie-tested, but will
        still keep out bulk mail. If you're on a mailing list you _want_ to
        be on, there are provisions for accepting all mail from a set of
        mailing lists that you specify.

    3.16 Software: Spam Be Gone

       "Spam Be Gone"
        http://www.internz.com/SpamBeGone/
        ...uses machine learning and artificial intelligence technologies
        to examine incoming mail messages and determine their
        priority... is more than just a Spam filter, it's a general purpose
        mail message prioritiser. You train the system, telling it which
        are good, and which are bad messages. As Spam Be Gone! learns it
        becomes customised for each individual user.

         PM-L W. Wesley Groleau  comments:

          .> They only distribute binaries, and I'm paranoid. Anyone able to
          .> convince me it's not really a Trojan Horse to collect addresses of
          .> spam-haters or something even worse?

          I did some sleuthing. I am 95% convinced that SpamBeGone is not
          a front or cover for any spammer(s). To protect the author's
          privacy, I won't say why I'm convinced or how I got the info.
          Sorry. If you're paranoid like me, you'll have to do your own
          sleuthing before you use it.

          I'm also convinced SpamBeGone's theory is sound. I won't judge
          the implementation until I've used it for a while.

         PM-L R Lindberg & E Winnie  comments:

          I have to agree with the recent comments about Spam Be Gone, I
          found it tends to be inaccurate. I first set it up about a week
          ago, followed the directions and trained it on several (15 to 20)
          messages. One from each list we get, and the remainder from my
          logs of SPAM messages.

          The first day it missed about half the SPAM, and nailed about 1/3
          of the real messages. So I tuned the key-words a bit, trained it
          on about 100 more SPAMs and trained it on all the good messages
          it nailed. Since then it has nailed every SPAM received, however
          the second day it nailed about 20% of the good messages, which I
          then trained it to like. Since then it has been nailing about
          10% of the good messages, despite continual training. I also
          added every list to the address book, and it still nails posts
          from this list, and my wife's lace list.

          I even went through my entire log of SPAM and trained it on every
          one that didn't come out a 5 (bad). Being the kind of person I
          am, I also checked after I trained it, and found four SPAMs, the
          despite my training it that they were bad (5) came out as not so
          bad (4). I don't dare kill 4's as far too much of my mail (like
          this list) ends up as 4's.

          For me, this program is not ready for prime time. If the comments
          are correct that it only learns on Subject and From headers, it's
          not even worth trying. Since lists use the TO and CC headers to
          be identified, and there are several excellent other headers
          (X-Advertisement comes to mind) that would be assests for killing
          SPAM.

    3.17 Software: ClearMail

        http://www.clearmail.com/ 1998-08-27
        Scott R Carter 

        ClearMail offers individuals some very strong control over spam
        through a quite unique concept. The software includes Procmail,
        Perl and C code. System Requirements include:

        ClearMail helps to control spam by allowing a user to classify
        e-mail as high or low priority based on an Address Book or "White
        List" of known senders. Unknown senders can also send high priority
        mail by including a special Mail Key (token) in their message
        (initial message from unknown sender without valid Key results in a
        bounceback message with instructions).

        What makes ClearMail different from similar concepts is that
        spammers are not able to easily obtain the Mail Key to bypass the `
        system because it is conveyed as an image.

        o   Unix operating system
        o   Shell accounts for users
        o   Individual .forward, .procmailrc files
        o   Sendmail
        o   Procmail
        o   Perl
        o   Public Web server

    3.18 Software: TinyGnus - Emacs Gnus plug-in

        
        http://poboxes.com/jari.aalto/ema-tiny.html

        Platform: win32 and Unix Emacs versions.

        *TinyGnus* Is Emacs lisp extension package that integrated directly
        to Gnus mail/newsreaders. It includes simple but efective UBE
        fighting hotkeys that make it possible to complain bunch of UBE
        messages a once. Features:

        o   USER MUST DECIDE WHICH IS *ube* MAIL.
        o   User selects messages that are ube with Gnus select commands.
        o   Hotkey C-c ' u examines messages' headers and runs `nslookup'
            for each Received header to determine *abuse* *spam* and
            *postmaster* addresses where to send the complaint.

4.0 Procmail pointers

    4.1 Where to get procmail binary

        ftp://ftp.informatik.rwth-aachen.de/pub/packages/procmail/
        On-Line manual: http://www.voicenet.com/~dfma/intro.html

    4.2 Where is procmail developed

        Philip Guenther  is currently taking care of and
        coordinating procmail bug fixes. Please send any procmail bugs to
        the mailing list or to . The development mailing
        list is running SmarList at . Further
        patch and bug info can be found at:

            http://www.gac.edu/~guenther/procmail/todo.html
            http://www.gac.edu/~guenther/procmail/warts.html

        Newest Procmail code:

            http://www.procmail.org/
            ftp://ftp.procmail.org/

    4.3 About procmail's Y2K compliance

        Please consult Philip Guenther  for more up to date
        details. Philip is the Procmail maintainer currently.

        [1998-09-23 Bennett Todd  in Message-Id:
        <19980923164230.C30594@fcmc.com>] Well, from a simple ogle of the
        grep over the sources, it looks like there may be a Y2038 problem
        in the autoconf test code: unsigned otimet = time(). And another,
        possibly less likely to express itself, in formail.c: unsigned long
        h1 = time(). Those could express themselves when 32-bit signed
        time_t wraps; long before then the time_t define should have been
        changed to something that is bigger, even if it's "long long". The
        above type-mixes may fail to profit from a suitably redefined
        time_t, and so may overflow on 2038.

        I don't see any Y2K problems, though. And email headers use
        four-digit years pretty consistently, so that should all be cool.
        This estimation doesn't constitute an in-depth Y2k audit of
        procmail, but the source code to procmail is ... kinda dense for
        in-depth auditing.

        [1998-09-25 Bennett Todd Message-Id:
        <19980925093902.B12428@fcmc.com>] As I see it there are at least
        three measures that a whole email system, taken in aggregate, could
        use for Y2K checking. First, capture a vast cross-section of
        traffic and make sure no email software is using 2-digit years. I
        don't recall having seen any, but it's still worth checking.
        Second, generate a load of traffic with 2000 and 2001 dates and
        shove it through all the channels. And third, run all the systems
        end-to-end with their system clocks rolling over the millenium.

    4.4 Procmail mailing lists

        Traffic in this list is about 5-20 messages per day. Do not join
        if you can't handle that much traffic. The list is run by SmartList,
        which is a procmail-based list management and distribution package.

        ._MailingList_: questions/answers 
        .subscription requests 
        .digest request 

       To get off the procmail mailing list

        To get off the list: send a message to *procmail-request* with:

            unsubscribe user@domain         in the subject line
            unsubscribe                     first line in the body

        If that fails, try email to
         (purportedly that should
        go to a person). See also the original subscriptions message that
        you will received http://www.iki.fi/~era/procmail/welcome.txt

    4.5 Procmail recipe modules and faqs

        Procmail is discussed in usenet newsgroup *comp.mail.misc*.

       "Procmail archive"
        ftp://ftp.informatik.rwth-aachen.de:/pub/packages/procmail/
        Articles from procmail mailing list: covers from 1994-08 to 1995-05
        (A .gz file: ~2Meg when uncompressed)

        And latest articles can be found here, hosted by Achim Bohnet
        Covers from 1995-10 to the present day.
        . The www page has nice search capabilities.
        http://www.rosat.mpe-garching.mpg.de/mailing-lists/procmail/
        http://www.rosat.mpe-garching.mpg.de/~ach/exmh/archive/procmail/

       "Era's Procmail faq"
        http://www.iki.fi/~era/procmail/mini-faq.html
        http://www.dcs.ed.ac.uk/~procmail/faq/          [mirror]
        Also available by email, the ITEM can be: links.html, mini-faq.html,
        procmail-faq

            To: 
            Subject: send ITEM

       "Era's Procmail Link collections"
        http://www.iki.fi/~era/procmail/links.html
        ...A page full of good links to the world of procmail

       "Catherine's Getting Started With Procmail"
        http://shell3.ba.best.com/~ariel/nospam/proctut.shtml
        This is a quick tutorial intended to get a procmail neophyte
        started using procmail with as little trouble and fuss as possible.

       "Joe Gross's short Procmail tutorial"
        http://www.procmail.net/ 
        ...Using procmail and a
        feature of ph you can set up your own mailing list without
        needing root on your own machine.

       "Unix manpages"
        http://www.xs4all.nl/~pater/manpages/
        ...If you don't have procmail manpages at hand, check this site.
        It contains a wealth of Unix related manpages online.
         Jeroen Paternostre

    4.6 Procmail mode for Emacs

        If you use Emacs, please download the Procmail
        programming mode, `tinypm.el'. Lint is included in there and it can
        auto-correct mistakes on the fly. You can get it from the mentioned
        _uta_ ftp site. Here is an example of its output:

            *** 1997-11-24 22:13 (pm.lint) 3.11pre7 tinypm.el 1.80
            cd /users/jaalto/junk/
            pm.lint:010: Warning, no right hand variable found. ([$`']
            pm.lint:055: Pedantic, flag orer style is not standard `hW:'
            pm.lint:060: Warning, message dropped to folder, you need lock.
            pm.lint:062: Warning, recipe with "|" may need `w' flag.
            pm.lint:073: Warning, Formail used but no `f' flag found.

    4.7 Procmail module list

       Where to get the modules

        The UBE stop procmail modules are not listed here. See pointers in
        "procmail code" section later.

        o   All pm-ja*.rc modules are in Jari's procmail kit.
            The Procmail code library page is at
            http://www.procmail.org/jari/pm-code.html

        o   Other modules are by Alan Stebbens http://reality.sgi.com/aks/

        o   1998-12-08 Eli the Bearded <*@qz.to> announced in
            comp.mail.misc that he had made his procmail modules available
            at http://www.qz.to/eli/src/procmail/. You may find
            interesting procmail code there but the modules themselves are not
            general purpose *plug-in* modules that you could use right
            away. Some functionality included:

                Inline decoding of MIME text attachments        (rc.mime-decode)
                Cleansing of obscure "Re:" formats in subject   (rc.pre-list)
                Nifty autoresponder                             (rc.qz-2)
                Sophisticated dupicate email catching           (rc.dupes)
                Example of using my mail bouncer                (rc.lists-out)
                Detection of some classes of autoreplies        (rc.daemon)
                Various junk mail filtering                     (rc.filter)
                Daily log files                                 (rc.vars)

       Terminology

        *subroutine* = A piece of code that gets something in `INPUT' and
        responds with `OUTPUT'. Subroutine is not message specific.

        *recipe* = A piece of code that is somewhat self contained:
        It reads something from the message or does something
        according to matches in message. Recipe may be message-specific.

       Foreword to using modules

        In the module listing, some of the modules are recipes and some can
        be considered subroutines. Let's take the address exploder module
        that was discussed a while ago. First, visualise following familiar
        programming language pseudo code:

        (ret-val1, ret-val2 ...) = Function( arg1, arg2, arg3 ...)

        *Function* may return multiple arguments and multiple arguments can
        be passed to it. Clear so far. Let's show how this applies to
        procmail modules:

            RC_FUNCTION  = $PMSRC/pm-xxx.rc # name the subroutine/module
            RC_FUNCTION2 = ...

            INPUT       = "value"           # Set the arg1 for module
            INCLUDERC   = $RC_FUNCTION      # Call Function( $arg1 )

            :0                              # Examine function ret val
            * ERROR ?? yes
            ...

        This should be pretty clear too. You just have to look into the
        subroutine/module which you intend to use, to find out what
        arguments it wants which you _need_ _to_ set (INPUT) before calling
        it. The documentation also tells you what values are returned, e.g.
        one of them was ERROR.

        If it were recipe/module, the call would be almost the same, but
        instead of returning values, the recipe/module most likely does
        something to your message or writes something to the data files
        etc. A *Recipe/module* is much higher level, because it may
        call multiple subroutine/modules. The distinction between
        subroutine and recipe module type is not crystal clear, but I hope
        the above will clarify a bit the Procmail module/subroutine/recipe
        concept.

       Header file modules

        These are like #include .h files in C, they define common
        variables, but do not contain actual code.

        o   pm-javar.rc -- Defines standard variables: SPC WSPC NSPC SPCL and
            perl styled \s \d \D \w \W and \a \A (alphabetic characters only)
        o   headers.rc -- From Alan's procmail-lib. Define standard regexp
            and macros: address, from, to, cc, list_precedence

       General modules

        o   *pm-jafrom.rc* -- Derive FROM field without calling `formail'
            unnecessarily. If all else fails, use formail.
        o   *get-from.rc* -- From Alan's procmail-lib. get the "best" From
            address. Sets FROM and FRIENDLY, the latter being the "friendly"
            user name sans address.
        o   *pm-jaaddr.rc* -- Subroutine to extract various email components
            from INPUT. Like address=foo@some.com, net=com, account=foo...
        o   *pm-jastore.rc* -- Subroutine for general mailbox delivery.
            Define MBOX as the folder where to drop
            message and this subroutine will store it appropriately.
            Supports single mboxes, ".gz" mbox files, directory files and
            MH folders with rcvstore.

       Date and time handling

        For these, you get the date string from somewhere, then feed
        it to some of these subroutines:

        o   *pm-jatime.rc* -- a low-level subroutine. Parse time "hh:mm:ss"
            from variable INPUT
        o   *pm-jadate1.rc* -- a low-level subroutine. Parse date
            "Tue, 31 Dec 1997 19:32:57" from variable INPUT
        o   *pm-jadate2.rc* -- a low-level subroutine. Parse ISO standard date
            "1997-11-01 19:32:57" from variable INPUT
        o   *pm-jadate3.rc* -- a low-level subroutine. Parse date
            Tue Nov 25 19:32:57 from variable INPUT
        o   *pm-jadate4.rc* -- Call shell command "date" once to construct RFC
            "Tue, 31 Dec 1997 19:32:57" and parse the YY MM HH and other
            values. You usually use this subroutine if you can't get the date
            anywhere else.

       Date and time handling

        You use these recipes to get the date directly from the message:

        o   *pm-jadate.rc* -- higher-level recipe. Read date from message's
            headers: From_ Received, or call shell `date' if none succeeds.
        o   *date.rc* -- higher-level recipe.
            From Alan's procmail-lib: parse date or from headers
            Resent-Date:, Date, and From

       Forwarding and account modules

        o   *pm-japop3.rc* -- Pop3 movemail implemented with procmail. You can
            send a "pop3" request to move your messages from account X to
            account Y. Each message is send separately. This recipe listens
            to "pop3" requests.
        o   *pm-jafwd.rc* -- control forwarding remotely. You can change the
            forward address with a "control message" or turn
            forwarding on/off with a "control message"
        o   *pm-japing.rc* -- Send short reply when subject contains the word
            "ping" to show that the account is up and email address is
            valid.
        o   *correct-addr.rc* -- From alan's procmail lib. To help forward mail
            from an OLD address to a NEW address, and do some mailing list
            mail management. This recipe file is intended to make it easy
            for users to forward their mail from their old address to a new
            address, and, at the same time, educate their correspondents
            about it by CC'ing them with the mail.

       Vacation modules

        o   *pm-javac.rc* -- A framework for your vacation replies. This
            recipe will handle the vacation cache and compose an initial
            reply; which you only need to fill in. (Like putting vacation
            message to the body)
        o   *ackmail.rc* -- From Alan's procmail lib. procmail rc to
            acknowledge mail (with either a  vacation message, or an
            acknowledgement)

       Message-id based modules

        o   *pm-jadup.rc* -- Handle duplicate messages by Message-Id.
            Store duplicate message in separate folder.
        o   *dupcheck.rc* -- From Alan's procmail-lib. If the current mail has
            a "Message-Id:" header, run the mail through "formail -D",
            causing duplicate messages to be dropped. Can use MD5 hash in
            cache.

       Cron modules

        o   *pm-jacron.rc* -- A framework for your daily cron tasks. This
            recipe contains all the needed checks to ensure that your
            includerc is called whenever a day changes. (Day change is
            subject to messages you receive). Your own cron includerc is
            run once a day.

       Backup modules

        o   *pm-jabup.rc* -- Save messages to backup directory and keep only N
            messages per day. Idea by John Gianni, packaged by Jari. Note:
            The implementation will always call shell for each message you
            receive; so using this module is not recommended if you get
            many messages per day. Instead, use the cron module to clean
            the messages' backup directory only once a day, and not everytime
            a message arrives.

       Confirmation modules

        o   *pm-jacookie.rc* -- Handle cookie (unique id) confirmations.
            Also known as Procmail authentication service (PAS).
            This simple procmail module will accept messages only from
            users who have returned a "cookie" key. You can use this to
            to protect your mailing list from false "subscribe" messages
            or from getting mail from unknown people, typically spammers
            who won't send the cookie back to you to "validate" themselves.
            Uses subroutine pm-jacookie1.rc, which generates the unique
            cookie; CRC 32 by default.
        o   See also Michelle's confirmation module for SmartList

       File Servers

        o   *pm-jasrv.rc* -- A Mime Procmail file server (MPFS) It contains
            all the instructions and supports several MIME encoding types:
            text/plain and gzip. The keyword SEND is configurable. You
            can set up as many file servers as you need to different
            directories by changing the SEND keyword. MPFS supports
            password for file access.
        o   *commands.rc* -- From Alan's procmail-lib, check for commands
            in the subject line. Handles commands (send|get)
            [help|info|procmail info|procmail lib|procmailrc] and a few
            others.
        o   *send-file.rc* is a very simplistic piece of procmail code
            to send file (non-MIME support) requested in subject line.
            http://www.universe.digex.net/~mbr/unix/send-file.html

       Mime modules

        o   *pm-jamime.rc* -- Subroutine to read MIME headers and put the
            mime version, boundary string, content-type information to
            variables.
        o   *pm-jamime-decode.rc* -- recipe to decode quoted-printable
            or base64 encoding in the body.
        o   *pm-jamime-kill.rc* -- Recipe for attachment killing: wipes out the
            extra mime cruft leaving only the plain text. Applications for
            killing: ms-tnef attachment (MS Explorer 7k),
            html attachments (netscape, MS Express) vcard (Netscape),
            PCX attachment (Lotus Notes).
        o   *pm-jamime-save.rc* -- Recipe for saving simple file attachment.
            When you receive _ONE_ file attachment in a message, this
            recipe can save it in a separate directory. The content is
            also decoded (base64,qp) while saving.

       Filtering message body or headers

        o   *pm-jadaemon.rc* -- Handle DAEMON messages by changing subject to
            reflect a) the error reason b) to whom the message was originally
            sent c) original subject sent and what was the subject. Store the
            DAEMON messages to separate folder.
        o   *pm-jasubject.rc* -- Standardize Subject "Re[32]: FW: Sv: message"
            or any other derivate to de facto "Re: message"
        o   *pm-janetmind.rc* -- Reformat http://minder.netmind.com/ messages,
            The default 4k message is shortened to a few important lines.

       Miscellaneus modules

        o   *pm-jaempty.rc* -- check if message body is empty (nothing
            relevant). Define variable BODY_EMPTY to "yes" or "no" if
            message is empty.
        o   *pm-janslookup.rc* -- Run nslookup on given address. If you
            compose return address with "formail -rt -x To:" you can
            verify if domain is registered before sending reply. Uses cache
            for already looked up domains.
        o   *guess-mua.rc* -- Guess the Mail User Agent and set MUA:
            MH,PINE,MAIL

       Mailing list modules

        o   *Microlist* a small mailing list module by david hunt 
            ...This version contains vars set for my environment and needs,
            and requires resetting of those vars before use. Its exact
            function and use will remain a mystery until I get a readme
            file written for it. If anyone wants to use it, I suggest you
            write to me first. If anyone has any suggestions or criticisms
            (no matter how harsh) please write
            http://www.west.net/~dh/homedir/microlist/microlist4.3
        o   *pm-jalist.rc* -- Subroutine to extract mailing list name from
            message. Do you need to add a new recipe to your .procmailrc
            every time you subscribe to new mailing list? If you do,
            take a look at this module, which examines the message and
            defines variable `LIST' to hold the mailing list name. You
            can use it directly to save the messages adaptively to
            correct folders. No more hand work and manual storing
            of mailing list messages.

    4.8 Where to get Procmail code and modules

       "Alan's procmail modules"
        Send subject "send procmail library" to Alan Stebbens
         http://reality.sgi.com/aks/

       "pm-code, Jari's Procmail modules"
        http://www.procmail.org/jari/ --> See pm-code.zip or *shar* file.

       "Elijah's"
        http://www.qz.to/~eli/src/procmail/rc.master.html

       "Concordia scripts"
        http://alcor.concordia.ca/topics/email/auto/procmail/
        ...We provide sample sets of recipes to get you started. The great
        thing about the concordia scripts is the fact that they are
        designed to run from a central location and be called from a
        .procmailrc installed in the user's ~/home directory.
        

       "Meng on procmail"
        http://icg.resnet.upenn.edu/procmail/
        http://res2.resnet.upenn.edu/procmail/
        ...goes into exhaustive detail about how I manage my mailing lists

        "David's" David Hunt 
        ...My .procmailrc and .forward files can be viewed at
        http://www.west.net/~dh/homedir/pmdir/

    4.9 Procmail code to filter UBE

        _Sysadms_ _remember_ : Spam filtering is much more efficiently done
        in the MTA, especially if you are just looking at From and To lines.
        For example, you can setup in Exim a rule that blocks \d.*@aol\.com
        (that is any aol.com local part that begins with a digit). AOL
        guarantees that _none_ of their addresses begin with a digit. Exim
        rejects such bogus addresses at the SMTP level before the message
        is received.

       "Daniel's smap filter"
        1997-09-13 Daniel Smith  sent excellent spam filter
        called `spamc.rc'. It used some nice heuristics and filters from
        various people, including [david] and [philip].
        Later Dan made substantial changes to it and the new version is
        available from ftp://ftp.bristol.nl/pub/users/DanS/spamcheck

       "pm-jaube.rc Jari's ube filter (compiled from others)"
        After Daniel Smith posted his spam recipes to procmail mailing
        list, Jari investigated them and compiled other recipes to a
        general purpose UBE module that needs no special setup and can be
        installed via simple INCLUDERC. No additional ube-list files are
        used, all UBE detection happens using procmail rules. The module
        is included in kit `pm-code.zip'.

       "Catherine A. Hampton's Spambouncer"
        http://www.best.com/~ariel/nospam/
        ...The attached set of procmail recipes/filters, which I call
        The Spam Bouncer, are for users who are sick of spam (unsolicited
        junk email) and want to filter it out of their mail as easily
        as possible. These recipes can be used as shared recipes for a
        whole system, or by an individual for their own mailbox only.

       "Protect yourself from spam: A practical guide to procmail"
        http://www.sun.com/sunworldonline/swol-12-1997/swol-12-spam.html
        ...take you, step by step, through everything you need to know in
        order to enlist the aid of a Unix host in filtering unwanted e-mail
        traffic. This page is excellent to get you started with procmail
        and filtering with simple recipes and how to store messages to
        folders. Recommended for newcomers to Procmail.

       "Junkfilter" by Gregory Sutter 
        http://www.pobox.com/~gsutter/junkfilter/
        ...Junkfilter is a user-configurable procmail-based filter system
        for electronic mail. Recipes include checks for forged headers,
        key words, common spam domains, relay servers and many others.

       "Download procmail spam filters"
        http://www.telebyte.com/stopspamr
        This is excellent site and contains many other spam stop pointers.

       "SpamDunk"
        http://www.interlog.com/~waltdnes
        http://www.interlog.com/~waltdnes/beta/techie.htm
        ...This webpage shows a commented example of a working .procmailrc
        file that works for me. I have tried to make things as generic as
        possible, but there are no guarantees that it will work for anyone
        else.

5.0 Dry run testing

    5.1 What is dry run testing

        It means that you call your procmail test script directly with sample
        test mail

            % procmail $HOME/pm/pm-test.rc < $HOME/tmp/test-mail.txt

        The script pm-test.rc has the procmail recipe you're testing or
        improving. The test-mail.txt is any valid email message containing
        the headers and body. You can make one with any text editor, e.g.
        `vi', `pico' or `emacs' in your Unix system. Here's a
        simple test mail skeleton:

            From: me@here.com
            To: me@here.com (self test)
            X-info: I'm just testing

            BODY OF MESSAGE SEPARATED BY EMPTY LINE
            txt txt txt txt txt txt txt txt txt txt

        Remember that you can define environment variables as well in
        the dry run call. Here's an example where procmail just executes
        the script and does nothing fancy.

            % procmail VERBOSE=on DEFAULT=/dev/null \
                ~/pm/pm-test.rc < ~/txt/test-mail.txt

        Suppose the script prints something to logfiles, but you'd instead
        like to get it all dumped to screen. No problem, first find out
        your tty value by calling `tty' at shell prompt and pass
        that on the command line. Here the default LOGFILE is directed
        to take care of redirecting "LOG=" commands and statement
        "MYTEST_LOG=${MYTEST_LOG:-$HOME/pm/pm-test.log}"

            #  `tty' tells what to fill in /dev/..

            % procmail VERBOSE=on DEFAULT=/dev/null                         \
                LOGFILE=/dev/pts/0 MYTEST_LOG=/dev/pts/0                    \
                ~/pm/pm-test.rc < ~/txt/test-mail.txt

    5.2 Why the From field is not okay after dry run

          It now says "From foo@bar Mon Sep 8 14:38:06 1997"

        [philip] Don't worry about this. It's a side-effect of running the
        message through formail after having generated any auto-reply --
        the auto-reply generated by "formail -rt" doesn't have a "From "
        header (it's pointless for outgoing messages), so the second
        formail adds one, not knowing that it'll just be ignored by
        sendmail later (well, sendmail will extract the date from it, but
        that's ignorable). You only see it because you're saving to a
        folder instead of the mailing it.

    5.3 Getting default value of a procmail variable

        [david] There's always this way to learn a variable's
        initial value (note the strong quotes), which Stephen uses to get
        procmail's value for $SENDMAIL in the scripts that build SmartList:

            procmail LOG='$PATH' DEFAULT=/dev/null /dev/null < /dev/null

        Since LOGFILE hasn't been defined, $PATH will be printed to the
        screen. One caution: if there are any variables in the definition
        of $PATH (such as $HOME), they'll be expanded in the output.

6.0 Things to remember

    6.1 Get the newest procmail

        Lot of troubles surface only because you have an old procmail version.
        Be sure to have the latest which is 3.13.1 since 1999-04-05. Here is
        a command to check your procmail version number:

            % procmail -v

        Knock your sysadm or ISP until he installs this version; don't give
        up, if you're serious about using procmail.

    6.2 Csh's tilde is not supported

        Real csh or Emacs freaks have grown accustomed to using tilde (~)
        everywhere, but must drop that habit now. Procmail doesn't support it;
        just use `$HOME'. When you write procmail recipes, think *sh* not
        *csh*. This mindset will automatically get your brain tuned to the
        right programming habits.

    6.3 Be sure to write the recipe starting right

        The recipe starts with `:0' or just with `:' but the latter one is
        somewhat dangerous and easy to miss. Beware writing it `0:' as it
        happens easily. The Procmail code checker, Lint, also requires that
        you use the `:0' recipe start convention.

        [philip] Always put a zero after the colon that begins the recipe. In
        the first versions of procmail, you would put the number of
        conditions, with a default of 1. That was annoying, and the computer
        can do the counting easier, so Stephen made it so that a count of 0
        indicates that the conditions are all the lines beginning with a `*'.
        The default is one, unless the `a', `A' , `e', or `E' flags is given,
        in which case the default is zero. *ALWAYS* *START* a *RECIPE* *WITH*
        `:0'.

    6.4 Always set SHELL

        [faq] If your login shell is a C shell (csh or tcsh), avoid
        havoc: as a precaution, always put following at the top of
        your .procmailrc.

            SHELL = /bin/sh

       If system has no /bin/sh and you're forced to use csh/tcsh

        [] Csh and tcsh execute the .cshrc
        first, THEN if, and only if it is the login shell (not a sub shell)
        it executes the .login, which should contain basic important system
        setting like `stty' commands. Likewise, bash and ksh users are
        taught to define and export PATH in .profile, so our per-shell
        startup files would not have clobbered the PATH set in .procmailrc
        the way your .cshrc did.

        [philip] ...I have been told by other sysadmins that there are
        systems on which csh was hacked to source the .login before the
        .cshrc. For various reasons I suspect these to be systems based on
        older versions of BSD (say, 2.3 BSD).

        As for tcsh, the order in which the .login and .cshrc is sourced is
        a compile-time option which defaults to the .cshrc (or .tcshrc)
        before the .login. There may be some wackos out there who change
        the default in memory of the system(s) that they were raised on. I
        suggest electroshock as the proper treatment.

          ...done sys admin on Crays, Convexes, Suns, SGIs, Decs, PC
          running BSDI, Linux and Free BSD, and I have never run into a
          system where the .cshrc is sourced AFTER the .login. If someone
          goes to the trouble to change the order, I would love to know a
          valid reason for it.

       Procmail won't work well with SHELL set to csh derivate

        [1998-08-17 PM-L  Volker Kuhlmann]
        ...The blame lies with procmail and its documentation. Obviously,
        procmail is programmed with the assumption that the login shell is
        a sh derivative. This assumption is a) not very nice, and b) not
        stated in the otherwise very good documentation. Of course a user
        can set SHELL to tcsh. If then procmail is too stupid to hack it,
        it ought to say so clearly, and the above-mentioned questions of
        people using tcsh will disappear from this list. One could also be
        nice and point out pitfall (3) mentioned above in the procmail
        docs. It is customary to have terminal configuration in .login. If
        it is shifted to .cshrc it should be properly surrounded by if ..
        endif. Perhaps it is not customary to configure the terminal in
        .bashrc (where else then? - only a rhetorical question), but that
        is no reason to blame it on tcsh.

        My .cshrc only setenvs the environment when it is a login shell
        (shell level 1). Obviously procmail runs a login shell. As I said
        earlier, there are good reasons for setting a full PATH
        independently whether the shell is interactive or not. So, when
        procmail executes programs with SHELL=tcsh, PATH is set to the tcsh
        defaults. That may or may not be desirable, depending on the
        individual case. No problem with that and avoidable (run tcsh with
        -f). Nice if it was in the procmail docs.

        But then, the PATH getting clobbered is not the point here (just a
        side-effect I didn't realise until 2 people pointed it out).

    6.5 Check and set PATH

        [jari] It is very likely that the default PATH environment variable
        that your .procmailrc sees it not enough. To play safe, so that all
        the needed binaries can be found when escaping to shell in
        .procmailrc, set the `PATH' variable as a very first statement.
        Here is one example that I use for HP-9 HP-10 and in SUN-OS.
        You can add paths that don't exist, that way you can use the
        same .procmail on multiple servers (On HP and SUN as I do)

            PATH        = $HOME/bin:\
            /usr/contrib/bin:\
            /bin:/usr/bin:/usr/lib:/usr/ucb:/usr/sbin:\
            /usr/local/bin:/opt/local/bin:\
            /vol/bin:/vol/lib:/vol/local/bin:${PATH}

        [Richard] It is dangerous to have many directories in the PATH,
        especially if you do not control the content of any of them. A
        sysadmin could put a newer, incompatible version of a program you
        rely upon in one of them and you cause difficult-to-diagnose
        problems. It may make more sense to link the binaries you need into
        your own ~/bin directory and include just that in your PATH.

        [jari] In principle I agree with Richard's advice, but in practice
        the newer version of the program seldom breaks the procmail code
        you have written. It depends on your "threat level": be more
        cautious and use Rik's advice; alternatively trust the system and
        adapt to (rare) changes. Your call.

    6.6 Keep the log on all the time

        It's best that you put these variables at the very start of
        your .procmailrc. When you start using procmail, you also want to know
        all the time what's happening there and why your recipes
        didn't work as expected. The answer to almost all your questions can
        be found in the log file. As the log file will grow to be quite big,
        remember to set up a cron job to keep it moderate size.

            LOGFILE     = $PMSRC/pm.log
            LOGABSTRACT = "all"
            VERBOSE     = "on"

    6.7 Never add a trailing slash for directories

        [philip] Drop the trailing slash: it'll choke if you ever end up on
        Apollo's DomainOS where double slashes are network references. If
        the directory has a trailing slash, it will choke
        on most OSes (they treat it like "/.").

            DIR         = /full/path/to/www/directory/    # Wait...
            FILE        = $ARCHIVEDIR/file                # Ouch !

    6.8 Remember what term DELIVERED means

        [alan] When procmail delivers a piece of mail, whether to a
        file or a pipe-command, if the write succeeds, then the mail is
        considered to have been delivered, and processing stops with that
        recipe file. Here is the relevant text from man page:

          ...There are two kinds of recipes: delivering and non-delivering
          recipes. If a delivering recipe is found to match, procmail
          considers the mail (you guessed it) delivered and will cease
          processing the rcfile after having successfully executed the
          action line of the recipe. If a non-delivering recipe is found to
          match, processing of the rcfile will continue after the action
          line of this recipe has been executed.

    6.9 Beware putting comment in wrong place

        You like commenting a lot, sticking them everywhere possible?
        Yes, I do that too, and got into trouble because one is not that
        free to comment code in procmail. Pay attention to the following
        example

            :0          # comment, nice tune...
            * condition # OUCH, Ouch, ouch. This comment must not be here!!
                #         Hm, Old procmail versions don't understand this
                #         Are you sure you want to put comments inside
                #         Condition line?
            * condition
            {               # comment ok
                            # comment ok
                :0          # comment ok
                /dev/null   # comment ok
            }               # comment ok

        So, the place to watch is the *condition* line. Some later procmail
        versions promised to correct this misfeature, but it never came
        true. No procmail exists yet that allows putting comments
        on the same line with a condition clause.

    6.10 Brace placement

        Be careful with your braces and remember that old procmail
        versions aren't as forgiving as newest version. Below you see
        classical "Test OK condition first, and if that fails then do
        something else". See the side comments.

            :0
            * condition
                                # No space allowed here!
            {}                  # Wrong, at least _one_ empty space
            :0 E
            {do_something }     # Again mistake, must have surrounding spaces

    6.11 Local lockfile usage

        Lockfiles are only needed when procmail is doing something that
        should be serialized, i.e., when only one process at a time should
        be doing it.

        This generally means that any time you write to a file, you should
        have a locallock, preferably based on the name of the file being
        written to. Forwarding actions ('!'), and 99% of all filters don't
        need lockfiles. However, if a filter action writes to a file while
        filtering, then you may need a lock. Procmail always does kernel
        locking when it writes mail to files via simple file actions. So
        even if you forgot the lock colon, procmail tries to play safe if
        kernel locking has been compiled in.

        Beware misplacing the lock colon(:)

             :0: a      # Ouch! Wrong unless you want a lockfile named a
             :0 a:      # Okay.

        Note that in delivering recipes where you manually write the
        content, you must use local lockfile with `>' token, because
        procmail can't determine lock by itself. It can only determine the
        lockfile from the `>>' token. [stephen] However, putting a
        lockfile on a recipe like this is, of course, utterly useless. So
        you might as well omit the locking entirely.

            #   Save last body of message to file mail.body

            :0 b:  mail.body$LOCKEXT
            | cat > mail.body

        o   If the command line in the procmail rcfile contains ">>",
            a name for the local lockfile will be implicit, and the second
            colon alone is enough.
        o   If the command doesn't write to a file, or doesn't write to the
            same file as anything else (including a matching letter that makes
            procmail run the same command) that might run at the same time,
            the local lockfile is unnecessary.

        [philip] Watch this too. A nesting block that does not launch
        a clone cannot take a local lockfile on the recipe that starts the
        braces. A nesting block that does launch a clone can. (see
        the error)

            :0: file$LOCKEXT
            {
                #  error: "procmail: Extraneous locallockfile ignored"
                #  - This lock file will be ignored
                #  - If the recipes inside the braces try to use file.lck
                #    as  a lockfile, then you'll have a deadlock situation.

                :0 :
                /tmp/tmp.mbx
            }

        Let me also explain why the `w' is so important. Notice, that the
        two here are equivalent. The `W' here is implicit. _NOTE_: this is
        only true on the recipe that opens a nested block. On a recipe with
        a program, forward, or delivery action, `W'' is different from `w'
        is different from missing both.

            :0 c: file$LOCKEXT      :0 Wc: file$LOCKEXT
            { ... }                 { ... }

        To quote the comment in source code, "try and protect the user from
        his blissful ignorance". The parent will always wait for the cloned
        child to exit when a lockfile is involved. The only question is
        whether or not it should be logged. If you want failure of the
        cloned child to be logged, then you should use the `w' flag, ala:

            :0 wc: file$LOCKEXT
            { ... }

        A local lockfile can be used to lock a clone; the parent procmail
        will remove it when the clone exits (thus it serves as a global
        lockfile for the clone). If the braced block does not launch a
        clone, asking for a local lockfile generates an error.

    6.12 Global lockfile

        [david] If you want to block everything while the recipe runs, even
        during the _conditions_, use global lock. For example in this
        construct the `formail' which updates the message-id cache file
        must be protected with a global lockfile.

            MID_CACHE_LEN   = 8192
            MID_CACHE_FILE  = $PMSRC/msgid.cache
            MID_CACHE_LOCK  = $PMSRC/msgid.cache$LOCKEXT

            LOCKFILE        = $MID_CACHE_LOCK

            :0
            * ^Message-ID:
            * ? $FORMAIL -D $MID_CACHE_LEN $MID_CACHE_FILE
            {
                    LOG = "dupecheck: discarded $MESSAGEID from $FROM $NL"

                    :0                  # no lockfile !
                    $DUPLICATE_MBOX
            }

            LOCKFILE                    # kill variable

        You cannot use local lockfile as below:

            :0 : $MID_CACHE_FILE$LOCKEXT
            *   ^Message-ID:
            * ? $FORMAIL -D $MID_CACHE_LEN $MID_CACHE_FILE

        because the local lockfile named on the flag line will be created
        only if the conditions have matched and the action is attempted.

        One more note: watch carefully, that there is _no_ `:' lock when
        delivering to `DUPLICATE_MBOX' because the outer global lockfile
        already prevents all other procmail instances from executing this
        part of the recipe.

    6.13 Gee, where do I put all those ! * $ ??

        Ahem. I can't tell you exactly what to do or how to write your own
        procmail recipes, but I can tell how I'm writing them. Here is my
        condition line token order:

            * $ ! ? BH VAR ?? test

        That won't say much unless I give you something to compare with.
        Here is one perfectly valid rule, but not my style

            :0
            *$ ^Subject:.*$VAR
            *! ^From:.*some
            *B ! ?? match-the-string-in-body
            *$? $IS_EXIST $FILE
            *VARIABLE ?? set

        I prefer lining up things in the condition lines. The first column is
        reserved for dollar sign, the second for *not* operator and so on.
        The important thing is that I can see at a glance if I have set the
        variable expansion dollar in the line (leftmost).

            :0
            *$       ^Subject:.*$VAR
            *  !      ^From:.*some
            *  ! B ?? match-the-string-in-body
            *$ ?      $IS_EXIST $FILE
            *         VARIABLE ?? set

    6.14 Sending automatic reply, use X-loop header

        Do not send automatic reply without checking "! ^FROM_DAEMON"
        condition and always include `X-Loop' header and check its existence
        to prevent mail loops

            :0
            *    conditions-for-auto-reply
            *$ ! ^$MY_XLOOP
            *  ! ^FROM_DAEMON
            | $FORMAIL -A "$MY-XLOOP" ...other-headers...

    6.15 Avoid extra shell layer (check command for SHELLMETAS)

        [dan] It is very important to study your shell command calls and try to
        save the overload of the extra layer of shell. It may be extra work
        once when you write your rcfile but it saves effort on each piece of
        arriving email. When procmail sees a character from `SHELLMETAS', it
        runs

            # Default SHELLMETAS: &|<>~;?*[
            # Default $SHELLFLAGS: -c

            % $SHELL $SHELLFLAGS "command -opts args"

        instead of

            % command -opts args

        That is because procmail's ability to invoke other programs does not
        include filename globbing ([, *, ?), backgrounding (&), piping
        (|), succession (;), nor conditional succession (&&, ||). If it
        sees any of those characters (before expanding variables), it hands the
        job over to a shell.

        Sometimes those characters appear in arguments to a command without
        having their shell meta meaning and procmail really could invoke the
        command directly without the shell. You can see the distinction in a
        verbose logfile: if procmail runs the command itself, it logs

            Executing "command,-opts,args"

        with a comma between each positional parameter, but if it calls a
        shell, the original spacing from the rcfile appears unchanged in
        the logfile:

            Executing "command -opts args"

        So, if you know you won't be needing shell expansion, wrap your
        shell calls with this:

            savedMetas  = $SHELLMETAS
            SHELLMETAS    # Kill variable

            ..command that does not need shell expansion features..

            SHELLMETAS  = $savedMetas

    6.16 Think what shell commands you use

        For every message, procmail launches the processes you have put into
        your .procmailrc. If you haven't paid attention to optimization
        before, now it's serious time to take a magnifying glass and check
        every recipe and the processes in them. When you write you private
        shell scripts, the performance hit is not so important, but for
        mail delivery, the matter is totally different. First, let's see
        some programs and sizes: The following is from HP-UX 10, where the
        binaries seem to include debug and symbol table code.

            131072 Aug 21  1996 /usr/bin/awk
            196608 Oct  1  1996 /usr/bin/sort
            245760 Jun 10  1996 /usr/bin/grep
            262144 Jun 10  1996 /usr/bin/sed
            303552 Dec  7  1995 /usr/local/bin/gawk
            544768 Jun 10  1996 /usr/contrib/bin/perl       [perl 4.36]
            822232 Aug 25 13:58 /opt/local/bin/perl5.00401

                    text    data     bss
            awk:    72727 + 51316 +  15317   = 139360
            sort:  173225 + 18496 + 183076   = 374797
            sed:   237248 + 16992 +  56252   = 310492
            grep:  221591 + 16176 +  53816   = 291583
            perl4: 502220 + 36044 +  65632   = 603896
            perl5: 633812 + 69612 +   2385   = 705809
            gawk:  160018 +  5264 +   7168   = 172450

        The binary siszes above are not the typical cases: these are from
        another system

                 4 Sep 28 14:25 /usr/local/bin/awk -> gawk
             32768 Nov 16  1996 /usr/bin/grep
             49152 Nov 16  1996 /usr/bin/sed
            114688 Oct 20  1996 /usr/local/contrib/gnu/bin/grep
            155648 Nov 16  1996 /usr/bin/awk
            155648 Nov 16  1996 /usr/bin/nawk
            221184 Nov 16  1996 /usr/bin/gawk
            311296 Jan 27  1997 /usr/local/bin/gawk
            958464 Nov  2 16:34 /usr/local/contrib/bin/perl
            1196032 Sep 14  1996 /usr/local/bin/perl

        Stan Ryckman  wants you to know that:

          Comparing byte sizes on disk means nothing here... these
          things may or may not have been stripped. Any symbol tables included
          in the byte counts you see above won't affect process start-up time.

          The `size' command will give a better handle on what will be needed
          in starting a process. The three segments may each have their own
          overhead, though, and the relative contributions of those segments
          to startup time may well be system-dependent.

        Hm. Can we draw some conclusion? Not anything definitive, but at
        least something:

        o   While sed and grep may be bigger than awk in some systems, this
            is an exception. They are usually much smaller and fast to use.
        o   But complex commands that would require many processes to be
            chained together, like `grep -v | grep | sed' combination could
            be usually accomplished with one awk call. Ask somewhere how to
            do it with awk if you don't know the language, it's quite alike
            perl
        o   Don't use anything else but standard awk, _gawk_ and _nawk_
            are bigger and may not be found on all systems.
        o   Avoid perl at all costs; it's many times (6) bigger than awk.
            Perl is slow-to start up, due to intermediate compilation
            process at startup.
        o   Remember that if procmail is running in a dedicated mail host, it
            probably doesn't even have any goodies installed, just the boring
            standard versions; which may not be even the same as what you see
            on current host. e.g. My mail host is running HPUX-9, while my
            login is on HPUX-10. They have very different installations.

        Here are some more programs. Don't even think of extracting fields with
        `grep' or `awk', like "grep Subject", because `formail' is
        much smaller and more optimized for tasks like that.

            37007 Sep  5 15:53 /usr/local/bin/formail   # 3.11pre7
            28672 Jun 10  1996 /usr/bin/tr
            20480 Jun 10  1996 /usr/bin/tail
            20480 Jun 10  1996 /usr/bin/cat
            20480 Sep 26  1996 /usr/bin/expr
            16384 Jun 10  1996 /usr/bin/head
            16384 Jun 10  1996 /usr/bin/cut
            16384 Jun 10  1996 /usr/bin/date
            16384 Jun 10  1996 /usr/bin/uniq
            16384 Jun 10  1996 /usr/bin/wc
            12288 Jun 10  1996 /usr/bin/echo

    6.17 Using absolute paths when calling a shell program

        Shell programmers know that if you use absolute path when you call
        the executable, shell doesn't have to search through long list of
        directories in $PATH. This may speed up shell scripts remarkably.
        The correct way to use such an optimization is to define variables to
        those programs.

        Hm, should you use such optimization in your procmail code? That's
        two folded question and I....would say yes and no. How many
        shell calls do you have? Do you use grep or formail a lot? Then you
        could optimize these calls. To be portable, define variables for
        executables:

            #  perhaps defined in separate INCLUDERC
            #
            #   INCLUDERC = $PMSRC/pm-mydefaults.rc

            FORMAIL     = /usr/local/bin/formail
            GREP        = /bin/grep
            DATE        = /bin/date

            :0 fhw
            | $FORMAIL -rt

        And when you port your .procmailrc to different environment that
        has different paths, you could use this recipe in addition to one
        just mentioned above:

            FORMAIL     = ...as above

            :0
            * HOST ?? second-host
            {
                #   In this host the paths are different. Reset.

                $FORMAIL    = "formail"
                $GREP       = "grep"
                $DATE       = "date"
            }

    6.18 Disabling a recipe temporarily

        If you have a recipe that you would like to disable for a while,
        there is an easy way. Just add the "false" condition line before
        any other conditions. The "!" also nicely visually flags that
        "this recipe is NOT used".

            #  This recipe stops at "!" and doesn't get past it.

            :0
            * !
            * condition
            * condition
            {
                ...
            }

    6.19 Keep message backup, no matter what

        It's good to have a safety measure in your .procmailrc.
        Although you are an expert and have checked your recipes 10 times,
        there is still a chance that something breaks. One morning, when you
        browse your *BIFF* reminder log; you notice "Hm, there is that
        interesting message but it was not filed, where is it?". And when
        you go to study the procmail logs (you do keep the log going all
        the time) and it hits you: "Gosh; a mistake in my script! Message was
        fed to malicious pipe and I had that `i' flag there... *sniff*".
        And you greatly regret you didn't back up the message in the first
        place.

        So, before your procmail does anything to your message, put the
        message into some folder which is regularly expired. For example I
        use Emacs Gnus to handle the expiring. One could also use a cron job
        instead. Then you can relax knowing your email is safe.

            SPOOL      = $HOME/Mail/spool

            #   Backup storage
            #   - This could be directory too. In that case you could use
            #     cron job to expire old messages at regular intervals
            #   - For once a day expiration, see procmail module list
            #     and pm-jacron.rc

            BUP_SPOOL  = $SPOOL/junk.bup.spool

            :0 c:
            $BUP_SPOOL

        Naturally you can filter out mailing list messages from the backup,
        because losing one or two (hundred) of them may not be that serious.
        Maybe you could use two backup spools, one for mailing lists and the
        other for your non-list messages.

            :0 c:
            * ! mailing-list1|mailing-list2
            $BUP_SPOOL

        If you have the date variables set up as described below, you
        could also create a backup folder per day:

            $BUP_SPOOL    = $SPOOL/$YYYY$MM$DD

        This makes it very easy to delete backups that are older than
        a given number of days, either manually or through a cron job.

    6.20 Order of the procmail recipes

        When you start writing a lot of procmail recipes, you soon find out
        that it matters a great deal in which order your put your recipes. When
        each group of recipes starts growing too big, it's good practice to
        move each group to a separate rc file. Here is one recommended order:

            -   backup important messages
            -   cron-subroutine
            -   handle duplicate messages
            -   handle DAEMON MESSAGES
            -   handle plus addressed message  (RFC plus or sendmail plus addresses)
            -   handle server requests (file server, ping responder...)

            -   drop MAILING LIST messages

            -   send possible vacation  replies only after all above
            -   apply kill file
            -   detect mime and format or modify the message body
            -   save private messages

            -   and last: FILTER UBE.

        The backup, cron and duplicate handling go naturally to the beginning
        of your .procmailrc. Next comes a grey area where Daemon, plus handling
        (#REF #using_rfc_comment_trick_for; Note plus;); and server messages can
        be put.

        Mailing lists should be handled as early as possible, but after the
        server messages, because you want your services handled first.

        Do not send vacation replies before you have handled mailing lists
        to prevent annoying vacation replies to lists.

        After that you are left with "known" private messages and those of
        unknown origin. A kill file (to block based on sender) for rapid
        spammers, who send you message or several per day need to be
        checked before checking other messages.

        Last but not least: Put your UBE checkers to the end to avoid mishits
        of valid mail. DO NOT SEND AUTOMATIC COMPLAINT BACK. Drop the
        UBE to a folder, manually select the messages that need actions
        and send message to postmasters in the Received chain explaining that
        their mail relay has been hijacked.

7.0 Procmail flags

    7.1 The order of the flags

        Order does not matter of course, but here is one stylistic suggestion.
        The idea here is that the most important flags are put to the left,
        like giving priority 1 for `aAeE', which affect the recipe
        immediately. Priority 2 has been given to flag `f', which tells if
        a recipe filters something. Also (h)eader and (b)ody should
        immediately follow `f', this is considered priority 3. In the
        middle there are other flags, and last flag is `c', which ends the
        recipe, or allows it to continue. In addition according to [david]:
        "...I'm quite sure that putting anything other than the opening
        colon and the number to the left of `AaEe' will cause an
        error."

            :0 aAeE HBD fhb wWir c: LOCKFILE
               |    |   |   |    |
               |    |   |   |    (c)ontinue or (c)lone flag last.
               |    |   |   (w)ait and other flags
               |    |   (f)ilter flag and to filter what: (h)ead or (b)ody
               |    (H)eader and (B)ody match, possibly case sensitive (D)
               The `process' flags first. (A)nd or (E)lse recipe

        You can write the flags side by side

            :0Afhw:$MYLOCK$LOCKEXT

        Or, as I prefer, leave flags in their own slot for more
        distinctive separation. Note that $LOCKEXT must be next to $MYLOCK,
        because it contains string ".lock".

            :0 A HB fhw: $MYLOCK$LOCKEXT

    7.2 Flag w and recipe with |

        [alan] If the filter program exits with a 0 status (0 == okay), then
        procmail will replace the original input body with the output of the
        filter program. If the filter program exits with anything but zero,
        procmail will report an "error" to the log, and "recover" the input
        (not filter it)

        [david] I am very sure that that's the case _only_ if you have the
        `w' or `W' flag on the filtering recipe. Without `w' or `W',
        procmail won't care about a bad exit status from the filter and will
        replace the filtered portion with whatever standard output the
        filter produced. It may still report an error to the log but it
        won't recover the previous text. This, for example, will destroy the
        body of a message, even without `i':

            :0 fb
            | false

        With this, however, procmail will recover the original body:

            :0 fbW      # same results even if we add `i'
            | false

        [stephen] No, not on all occasions. Procmail will not care about the
        exitcode here. However, if procmail detects a write error, it *will*
        recover (because of the missing `i' flag). Procmail will only detect
        a write error in such a case if the mail is long enough and does not
        fit in the pipe buffer that's in the kernel (typically 10KB).

    7.3 Flag w, lockfile and recipe with |

        [manual] In order to make sure the lockfile is not removed until the
        pipe has finished, you have to specify option `w' otherwise the
        lockfile would be removed as soon as the pipe has accepted the
        mail. So if you see anything that looks like ">" or ">>" in your
        recipe, then that should immediately ring your bells. immediately
        check that you have included the `w' flag _and_ the lockfile `:'.

            :0 hwc: headc$LOCKEXT
            * !^FROM_MAILER
            | uncompress headc.Z; cat >> headc; compress headc

    7.4 Flag f and w together

          The w tells Procmail to hang around and wait for the script to
          finish. [Wouldn't you think this ought to be implied by the f
          already?]

        [david] Of course the `f' flag is enough to make procmail wait for
        the filter to finish, but the `w' means something more: to wait to
        learn the exit code of the filtering command. If sed fails with a
        syntax error and gives no output, without `W' or `w' procmail would
        happily accept the null output as the results of the filter and
        go on reading recipes for the now body-less message. On the other
        hand, with `W' or `w' sed will respond to a non-zero exit code by
        recovering the unfiltered text.

    7.5 Flags h and b

        [david] `hb' is the default; you need to use `h' only when you
        don't want `b' or vice versa. You can think of it this way: `h'
        means "lose the body" and `b' means "lose the header," but the two
        together cancel each other out.

        [philip] `hb' (feeding whole message) is the default for actions.
        You need to specify `h' without `b' if you want the action applied
        only to the head. `H' is the default for conditions. You need to
        specify `HB' or `BH' if you want to test a condition against the
        entire message.

    7.6 Flag h and sinking to /dev/null

        When you drop something to /dev/null, use the h flag so that
        procmail does not unnecessarily try to feed whole message there.

            :0 h
            * condition
            /dev/null

        [philip] Procmail knows that it shouldn't create a locallock on
        /dev/null and that it shouldn't kernel lock /dev/null, and it knows
        to write it "raw" (no "From " escaping or appended newline).  This
        means that procmail simply opens /dev/null, does its write with
        one system call, and closes it.

        I'm not sure if adding the `h' flag makes a real difference on
        modern UNIX kernels. I suppose it depends on how optimized the
        write() data is and in particular, whether a user-space to
        kernel-space copy is _required_, or whether it's delayed. If it's
        delayed then the code for handling /dev/null would presumably not
        do it, and the size of the write wouldn't actually matter.

    7.7 Flag i and pipe flag f

         Flag `i' is useless in mailbox deliveries.

        [faq] The following will work some of the time, when the message is
        short enough, but that's a coincidence. With a longer message,
        though, Unix starts paying attention to what is happening, because
        it will have to buffer some of the data, and then when the buffered
        data is never read, an error occurs. The error is passed back to
        Procmail, and Procmail tries to be nice and give you back your
        original message as it was before this malicious program truncated
        it. Never mind that in this case you wanted to truncate the
        data. Anyway, the fix is easy: Just add an `:i' flag to the recipe
        ( `:0fbwi' instead of `:0fbw') to make Procmail ignore the error.

            :0 fbw
            * condition
            | malicious-pipe

        [dan] here's why the `i' flag is needed (courtesy of Stephan): You
        told procmail to filter the entire mail (header and body), so it
        does and it attempts to write out header and body to the filter.
        Then procmail notices that not the entire body is being consumed.
        Procmail, being rather paranoid when it comes to delivery of mail
        assumes something went wrong and considers this a failure of the
        filter.

            :0 fbwi
            | head -2

    7.8 Flag r

        [philip] Procmail automatically turns on the `r' (raw mode) flag for
        deliveries to /dev/null, so there's no need to do it yourself.

            :0 r        # you can leave out the `r'
            * condition
            /dev/null

        [david] You can use the `r' flag (for raw mode) on every recipe
        where you do not want a From_ line added. I'm assuming that there
        isn't one already there; the `r' flag keeps procmail from making
        sure that there are a From_ line at the top and a blank line at the
        bottom, but it will not make procmail remove them if they are
        already present. Also, be careful to use the `-f' option on all
        calls to formail so that formail won't add a From_ line.

        Someone who didn't need From_ lines -- I forget who -- found it
        annoying to put `r' onto every recipe and altered the source to
        prevent procmail from adding From_ lines at all, ever. I think a
        better idea would be a procmailrc Boolean to enable or disable them
        for all recipes without affecting other users. (Then perhaps we'd
        need a reverse `r' flag to undo raw mode for one recipe at a time?)

    7.9 Flag c's background

          ...Interesting. My vision of `c' is to think of CONTINUE
          with message processing afterwards even if conditions matched.


        [david] Precisely: when you have braces, thinking "continue"
        instead of "copy" or "clone" can get you into trouble.

        Early versions of procmail, before braces and before cloning,
        called the `c' flag "continue" in their documentation; I think it
        is still called that in the source.

        When Stephen introduced braces (but not cloning at this point), it
        was of course implicit that an action line of "{" was
        non-delivering, and a `c' was extraneous. People put c's there
        because they wanted procmail to continue to the recipes inside the
        braces on a match, and procmail brushed it off with an "extraneous
        c-flag" warning. No harm done.

        When Stephen introduced cloning, though, I was rather upset that he
        was giving double duty to `c' instead of introducing something new
        like `C' for it, especially because people who absolutely wanted no
        clone but intended the recipes inside the braces to run in the same
        invocation of procmail as everything else were mistakenly putting
        c's on their braces to make sure procmail would "continue". People
        would (and did) get double deliveries.

        Roman Czyborra, though, said that if you consider `c' to stand for
        "copy", that covers both uses of `c': provide a copy to a simple
        recipe or, if there are braces, to a clone procmail that will
        handle the recipes inside the braces. Stephen agreed and changed
        the documentation accordingly.

        Longtime users of procmail and people who read old docs may still
        think of it as "continue", but since the introduction of clones,
        that is not a good way to look at it. "Copy" is much safer.

    7.10 Flag c before nested block forks a child

        [alan] The combination of a nested block and the `c' flag causes
        procmail to fork a child process for the nested block, while the
        parent skips over it and continues on. The child process doesn't
        necessarily stop unless a *delivering* recipe (without the `c' flag)
        action succeeds.

    7.11 Flag c and understanding possible forking penalty

          ... I run shell commands that need not to be serialized, so
          instead of doing the standard way:

            :0 hic                  # nbr.1 / standard way
            | command

          I assume I can avoid the extra fork caused by (c)lone flag
          altogether by using these. Any difference between these two?

            :0                      # nbr.2 / alternative
            * ? command
            { }                     # ...No-op, Procmail syntax requires this

            dummy = `command`       # nbr.3 / alternative

        [philip] There is a misunderstanding here. Let me clarify:

          Procmail only forks a full-blown clone on a recipe with the 'c'
          flag whose action is a nested block.

        If it's a simple mailbox deliver, pipe, or forward action then
        procmail does not fork a 'clone' (for pipe and forward actions
        procmail does have to fork, but only so it can execute the
        action). `nbr.1' and `nbr.2' take the same number of forks to
        execute. They also take the same effective number of writes
        (in case you're concerned about that). The latter also
        requires that procmail wait for the command to finish.

        `nbr.3' is worse than the above two, as procmail has to not
        only wait for the command to complete but also save the output
        into the named variable.

    7.12 Flags before nested block

        Given the following recipe, let's examine the flag part

            :0 $FLAGS
            {
                do-something
            }

        [david] `HB' `AaEe' and `D' affect the conditions and
        thus are meaningful when the action is to open a brace. `HB' and
        `D' would be meaningless, of course, on any unconditional recipe, but
        they should not cause error messages.

        Generally, flags that affect actions are invalid there, and `bhfi'
        and `r' always are, but the others are partial exceptions: if you
        are using `c' to launch a clone, then `w' `W' and a local lockfile can
        be meaningful. If there is no `c', then `w' `W' and a local lockfile
        are invalid at the opening of a braced block.

    7.13 Flags aAeE tutorial

        [david] `AaEe' are mutually exclusive and no more than one should
        ever appear on a single recipe. [philip] Actually, this is not
        true. e does not work with `E' or `a' (and procmail gives a warning
        if you try), and `A' is redundant if a is given, but at least some
        of the other combination make sense and work.

        o   *A* = try this recipe if the conditions succeeded on the most
            recent recipe at that nesting level that did not itself have an
            A nor an a
        o   *a* = same as `A', but moreover the action must have succeeded
            on the most recently tried recipe at that nesting level
        o   *e* = Almost like `A', try this recipe if the conditions matched
            but the action failed on the most recently tried (not skipped)
            recipe at this nesting level. universe, `e' is the opposite of `a'.
            `e' only looks backwards past `E' recipes that were skipped
            because of their `E'. It doesn't care whether a previous recipe
            had an `A' or `a' flag.
        o   *E* = try this recipe if the conditions have failed on the most
            recent recipe at that nesting level that did not have an `E' and
            on since then every recipe at that level that did have an `E';
            essentially opposite of `A'

        These mnemonics might help:

        o   *A:* if you did the recipe at the start of the chain, try this one
            (A)lso
        o   *a:* if the last action at that nesting level was (a)ccomplished)
        o   *e:* if the last action at that nesting level (e)rred
        o   *E:* (E)lse because the conditions down the chain so far have not
            matched. Or "try this recipe unless the last tried recipe matched".

            #   [philip] demonstrates `e'

            :0 :            # match, but action fails
            /etc/hosts/foo


                :0 A        # no match
                * -1^0
                /dev/null

            :0 e # this is skipped because the last tried recipe didn't match
            {
                ...whatever
            }

        How they interact with one another when used consecutively has not
        been fully tested to my knowledge. Consider this:

            :0
            * conditions
            non-delivering-action1

                :0 a
                action2

            :0 e
            action3

        Is action3 done if action2 failed or if action1 failed (or perhaps
        in both situations)? [philip] Action 3 is only done if action2 failed.

        If the answer is action2, does this work to get action3 done if
        action1 failed? I think it does, but does it also run action3 if
        the conditions didn't match on the first recipe? [philip] Yes, and
        yes.

            :0             #   [david]
            * conditions
            non-delivering action1

                :0a
                action2

            :0E
            action3

        [philip] If that's not what you want, combine some flags:

            :0
            * conditions
            non-delivering action1

                :0 Ae
                action3

            :0 a
            action2

        If the conditions match, action1 will be executed. action3 will
        then execute if action1 failed, otherwise action2 will be executed
        [if action1 succeeded].

        [david] I know what this structure does because I use it:

            :0
            * conditions
            non-delivering action1
                :0A
                action2

            :0E
            non-delivering action3
                :0A
                action 4

        If the conditions match, action1 and action2 are performed and
        action4 is not (of course action3 is not either), even if action2
        is non-delivering; if they fail, action3 and action4 are performed.
        The `A' on the fourth recipe refers back to the third and no farther.
        But I don't know about this:

            :0
            * conditions
            non-delivering action1
                :0A
                * more conditions
                action2

            :0E
            non-delivering action3
                :0A
                action 4

        Now, suppose the conditions on the first recipe match but those on
        the second recipe do not match. Would the third recipe (and thus
        the fourth one) be attempted? I would expect so. [philip] Yes. The
        last tried recipe didn't match, therefore the `E' flag will be
        triggered.

        If that isn't what you want, you can prevent it this way:

            :0
            * conditions
            {
                :0
                non-delivering-action1

                :0
                * more-conditions
                action2
            }

            :0 E # ignores mismatch inside braces, looks only at same level
            non-delivering action3

            :0 A
            action4

        If that is what you want, you can be positive this way:

            # if action2 is non-delivering or vulnerable to error that
            # would cause fall-through

            DID2         # Kill variable

            :0
            * conditions
            non-delivering-action1

                :0 A
                action3

            :0
            * ! DID2 ?? (.)
            non-delivering-action3

                :0 A
                action4

            # if action2 is delivering and sure to succeed
            :0
            * conditions
            non-delivering-action1

                :0 A
                * more-conditions
                action2

            :0
            non-delivering-action3

                :0 A
                action4

        [philip] or those who are interested, I'll note that there are only
        3 combinations of the `a', `A', `e', and `E' flags that aren't
        either illegal or redundant. They are `Ae', `aE', and `AE'. I've
        shown a use for `Ae' up above. Here's an example of `AE':

            :0
            * condition1
            non-delivering action1

                :0 A
                * condition2
                non-delivering action2

            :0 AE
            action3

        action3 will only be executed if condition1 matched but condition2
        didn't match. Without the A flag, action3 would be executed if
        either of them failed. This can also be done with a instead of A
        with analogous results.

        Procmail's "flow-control" flags may not be particularly easy to
        describe in straight terms (and this can all be made more
        complicated by throwing in a more varied mix of delivering vs
        non-delivering recipes), but I've found that it usually does what I
        expect it to do, and when it doesn't or I'm in doubt or I want to
        be particularly clear, I can always fallback to doing it explicitly
        via nesting blocks. Pick your poison...

8.0 Matching and regexps (regular expressions)

    8.1 Philosophy of abstraction in regexps

        Here are two ways to view or write regexps. Make up your own mind.

          People who are in favor of writing pure native regexps in the
          recipes:

            [    ]<[    ]*("([^"\]|\\.)*"|[-!#-'*+/-9=?A-Z^-~]+)...  # "

        o   I'm not planning on "maintaining" that code, as the syntax for
            XXX will not ever change <>>
        o   I some how doubt that anyone else will change that regexp more than
            trivially
        o   If none of your other regexps use the categorical variables, and
            you're not changing the regexp, then what's the point?
            The variablized version will be slower, and will clutter the
            environment with subprocesses.

        Where someone that immediately wants to abstract things says
        (This is from philip's great Message-Id matching recipe)

            dq = '"'                                # (literal) double-quote
            bw = "\\"                               # (literal) backwhack
            atom       = "[-!#-'*+/-9=?A-Z^-~]+"
            word       = "($atom|$dq([^$dq\]|$bw.)*$dq)'
            local_part = "$word($s\.$s$word)*"

            $s<$s$local_part...                     # ignore comment here

          ....abstraction: It makes code clearer when you break it
          to manageable parts, which possibly surfaces reusable parts. It
          also makes thing look simpler, and enables even novices
          to understand what's going on there. After we're not
          connected to the net anymore, others could possibly understand
          it too.

          So, naturally we can't agree with any of the previously mentioned
          arguments presented for keeping regexp "in pure native format".

        o   Although you won't maintain it, it's an example for others. What
            you post first, people will save it to their mailboxes and
            circulate elsewhere in the net: "Hey, I've saved this, try it"
        o   You can write cryptic regexps or break them into parts where
            the whole looks much simpler. Consider novice's welfare :-)
            This has nothing to do with the "It never changes in my lifetime".
        o   The speed penalty imposed by additional variables is not
            something we can measure in practice. CPU won't even hiccup.
            An extra `formail' call in your recipes is 10x as expensive as
            100 variables. (I don't know how to measure that, but launching
            a shell and creating a process is a much more expensive task).
        o   Cluttering the env process? C'm on. That won't matter either.
            No outside process uses lowercase environment names, or then it
            must be real special program. So called "cluttering" of
            environment space is also no-issue. CPU won't even get a hiccup
            for that.

    8.2 Matches are not case-sensitive

        Okay, okay; if you read the manual you knew that already. But
        sometimes someone with years of experience with Unix may take it for
        granted that procmail would be case-sensitive as the rest of the
        unix tools are. Use the `D' flag to turn on case-sensitivity.

    8.3 Procmail uses multiline matches

        Procmail uses multiline matches by default. This means that ^ and $
        match a newline, even in the middle of a regexp. Now you know this,
        you can easily interpret e.g. `$[^>]' as: `a newline followed by a
        line not starting with a `>'.

        If you put a '$' after the '\/' match token then procmail will
        include the matched newline if there's one there. Solution? Don't
        put a dollar sign there unless you really want a newline, use period
        that matches all but newline:

            :0 B
            * ^Search-string: \/.+

    8.4 Headers are folded before matching

        If you have a header that continues on separate lines, you don't have
        to worry about the linefeeds. Procmail silently folds the header onto
        one line, before matching it

            Received: from unknown (HELO Desktop01) (208.11.179.72) by
                palm.bythehand.net with SMTP; 4 Dec 1997 23:29:09 -0000

            :0                          # note, match on continuation line
            * ^Received:.*bythehand\.

    8.5 Improving Space-Tab syndrome

        Procmail doesn't know about standard escape codes like `\t' and `\n'
        or [\0x00-\0x133]:

            #  Not what you think       # You have to write: space + tab
            [ \t]                       [   ]

        But using the space+tab is not very readable and it's a very error
        prone construct. I suggest using the following to improve the
        readability:

            WSPC   = "    "         # whitespace = space + tab
            SPC    = "[$WSPC]"      # regexp whitespace, the short name
                                    # SPC was chosen because you use this
                                    # a lot in condition lines.
            NSPC  = "[^$WSPC]"      # negation of whitespace

            #   match anything except space and tab

            *$ var ?? $NSPC

            #   match anything ecxept space and tab and newline

            *$ ! var ?? ($SPC|$)

        But you cannot use newline inside brackets.

            WSPCL  = "   "'
            '

            #   Won't work although WSPCL definition is correct.

            *$ var ?? [$WSPCL]

        Instead use variable syntax:

            SPCNL = "($SPC|$)"      # space + tab + newline

        If you absolutely need a range of characters, see if you have `echo'
        command in your system to define variables like this:

            NUL_CHAR        = `echo \\00`
            DEL_CHAR        = `echo \\0177`
            REGEXP_NON_7BIT = "[^$NUL_CHAR-$DEL_CHAR]"

    8.6 Handling exclamation character

        [philip] you do need the first backslash, to keep procmail from
        considering the backslash as a request to invert the sense of the
        match. For example, these two conditions are equivalent:

            * ! 200^1 foo
            *   200^1 ! foo

        Therefore, a leading '!' must either be backslashed, enclosed in
        either parens or brackets (I suspect that parens would be more
        efficient), or prefaced with an empty pair of parens. I would
        recommend writing the condition with one of these:

            * 200^1 \!!!!
            * 200^1 ()!!!!
            * 200^1 (!!!!)

    8.7 Rules for generating a character class

        In a "character class" (things between "[" and "]"), metacharacters
        don't need to be escaped. Well, a backslash is an exception.
        e.g. [$[^\\] would match any one of the literal characters dollar,
        opening bracket, caret, and backslash.

        o   To match "])" use [])]
        o   To match "[(" use [[)]
        o   To include a literal ^          must not be first
        o   To include a literal -          must be first, last or \-
        o   To include a literal \          you must use \\
        o   To include a literal ]          must be first
        o   To include a literal [ ( ) or $ just use it anywhere

        [elijah] If you are inverting a character class "first" means just
        after the(^). So the character class that contains everything but ]
        ^ and - must look like this:

            [^]^-]

        [david] What if I want literal $ inside bracket? A $ inside
        brackets, unless it begins a variable name and the "$" modifier is
        on, always means a literal dollar sign. It cannot mean a newline if
        it appears inside brackets. A good way to keep it exempt from "$"
        interpretation is to put it last inside the brackets (unless one
        also need to include a literal hyphen and one can't put the hyphen
        first; then you'll need to escape the dollar sign with a backslash
        and put the hyphen last -- well, you could alternatively escape the
        hyphen, I guess), because procmail knows that "$]" cannot possibly
        be a reference to a variable.

        General guideline:

        o   ($) always matches a newline, with or without "$" interpretation;
        o   [$] always matches a dollar sign, with or w/o "$" interpretation;

    8.8 Matching space at the end of condition

        [david] If you need to have tab or space at the end of condition line
        you can use these:

            * rest of string .*
            * rest of string[ ]
            * (rest of string )
            * rest of string ()
            * rest of string( )         # I prefer this one

        [philip] From my looking at the source, the last two should be
        equal in efficiency, and except for a trace difference in regcomp
        time, should match at the same speed as a solitary trailing blank.
        The character class version [ ] will be slower.

        Of course, I suspect that neither you nor your sysadmin will ever
        notice the difference in speed, and given that 99% of all systems are
        I/O bound and not CPU bound, the system is incredibly unlikely to
        notice either. I can't complain though, as I also go to various
        extremes to seek out every last bit of possible performance. Ah well.
        The first one would be slower yet, though perhaps no slower than the
        bracket form.

    8.9 Beware leading backslash

          I am trying to come up with a procmail recipe that among other
          things should have the condition 'body does not contain a
          particular word'. Here is what I tried:

            * ! B ?? \

        [david] You have fallen into the leading backslash problem, If the
        first character of a regexp is a backslash, procmail takes it as "end
        of leading whitespace" and strips it. What you coded means "a less-than
        sign, then the word, then any non-word character." (It also prevents
        the less-than sign from being taken as a size operator.) Unless the
        non-word character immediately to the left of the word was a less-than
        sign, that regexp would fail (and thus the condition would pass). Try
        this:

            * ! B ?? ()\

        This would work too:

            * ! B ?? \\

        but in a casual reading it would look like "literal backslash,
        less-than sign, the word, word boundary character," so we on the list
        generally recommend the empty parentheses.

        Do note that the difference in meaning of \< and \> in procmail (where
        they must match a non-word character) from their meaning in perl and
        egrep (where they match the zero-width transition into and out of a
        word respectively) does not come into play here. Because procmail's \<
        and \> can match newlines (both real and putative), it rarely is a
        factor. It's a problem only when a single character has to serve both
        as the ending boundary of one word an also the opening boundary of
        another. Well, it's also a problem when you have one as the last
        character to the right of \/, but that's easily solved.

    8.10 Correct use of TO Macro

        o   `TO' is not a normal regular expression; it is a special
            procmail expression that is designed to catch any destination
            specification. For details, see the miscellaneous section of
            the `procmailrc(5)' man pages.
        o   Prefer `TO_' instead of `TO' if you have new procmail. `TO_' is
            better because TO used to be too loose
        o   Please remember to write `^TO', with the anchor in it.
        o   Do not put a space between the caret (^) and the word `TO' in
            `^TO'.
        o   Do not put a space between the `^TO' and the text that you are
            matching on; it must be `^TOtext' If this bothers you, you can
            use `TO()text' instead to get better separation of text.
        o   Both letters in `TO' must be capitalized.

    8.11 Procmail's regexp engine

        [philip] procmail's regexp engine has no special optimization
        for anchoring against the beginning of the line. Most program that
        have such an optimization have it because they need the line
        distinction for other reasons (for example, grep by default prints
        the entire line containing a match).  Procmail has no such other
        reason, so it treats newline like any other plain character in the
        regexp. There should be no speed difference as long as procmail
        can say: "the first character I see must be a 'foo'".  Note that
        case insensitivity is handled by making everything lowercase, so a
        letter being first doesn't bring in the spectre of character-classes
        or anything like that.

          .> recipe may have just changed the size of the head, procmail
          .> cannot keep a byte-count pointer nor a line-count pointer to
          .> where the body begins but must scan through the head to find the
          .> blank line at the neck before it begins a body search.

        Procmail does this when it reads in the head, not when it goes to
        search the body, so that cost can't be avoided. Let me repeat; that
        searching the body is no slower than searching the header, if we
        forget the minimum impact of the size of these two.

    8.12 Procmail and egrep differences

        [By david]

        o   ^ and $ are non-zero-width and anchor to real or putative
            newlines (rather than to the zero-width start and end of a line);
        o   An initial ^^ or a final ^^ anchors to the opening or closing
            putative newline respectively;
        o   ^ and $ in the middle of a procmailrc regexp match to an embedded
            newline (and must be escaped to match to a caret or a dollar sign);
        o   \< and \> are non-zero-width and match to a character that
            wouldn't be in a word (or to a real or putative newline) [rather
            than to the zero-width transition into or out of a word]; it
            always matches one non-word character. It will fail when there is
            no whitespace after the colon. This is rather pathological but
            still perfectly compliant with RFC822. For this reason,
            you should use (.*\<)? instead of just .*\< after the colon that
            terminates a header field name:

                ^Subject:.*\        # Wrong
                ^Subject:(.*\<)?humor\>     # Right, notice ?

        o   *, ?, and + in the absence of \/ are stingy rather than greedy,
            and that generally won't matter, but in the presence of \/ they
            are stingy to the left of \/ and greedy to the right of \/,
            while in most applications the leftmost wildcard on a line is
            the greediest and greed decreases from left to right.

    8.13 Understanding procmail's minimal matching (stingy vs. greedy)

          ...I want to have a procmail recipe that will save certain mail to
          folders where the folder name (always a number) is specified in
          the subject.

            :0 :
            * ^Subject: *\/[0-9]*
            $HOME/Mail/$MATCH

        [philip]...and this won't quite work. For a subject with a space
        after the tab, the '*' on the left hand side will be matched
        minimally (zero times), and then the stuff on the right hand side
        will be matched maximally, but starting at the space still, which
        will match nothing. This is a case were procmail's minimal matching
        can cause massive confusion and frustration. The solution is
        usually the following:

            FORCE THE RIGHT HAND SIDE TO MATCH AT LEAST ONE CHARACTER

        By Changing the recipe to:

            :0 :
            * ^Subject: *\/[0-9]+
            $HOME/folders/$MATCH

        it'll work, because then the left hand side will have to match all
        the way up to the first digit (but not the digit itself). If you
        follow the rule in caps then you'll almost always be able to ignore
        procmail's weirdness in this area.

        [david] And examine how procmail matches "Subject: Keywords 9999"

            * ^Subject:.*Keywords.*\/[0-9]*

            procmail: Match on "^Subject:.*Keywords.*\/[0-9]*"
            procmail: Matched ""

        The right side was as greedy as it could be; the problem is that we
        seem to expect greed on the left as well. MATCH is set to null, in
        contrary to our expectation. It is not a bug but rather a frequently
        misunderstood effect of the way extraction is advertised to operate.

          Remember that only the right side is greedy; the left side is
          stingy, and left-side stinginess takes precedence over right-side
          greed.

        Extraction is implemented this way: the entire expression, left and
        right, is pinned to the shortest possible match; then the division
        mark is placed and the right side is repinned to the longest
        possible match starting at the division. The tricky part is to
        remember that the division is marked during the stingy stage.

        If the expression is

            ^Subject:.*Keywords.*\/[0-9]*

        and the text is

            Subject:Keywords9999

        then the shortest possible match to the entirety is

            Subject:Keywords

        because ".*" and "[0-9]*" both match to null. Then the division
        mark is placed on the space after "Keywords" and procmail looks for
        the longest possible match to [0-9]* starting with that space.
        That, again, is null, so MATCH is set to null.

        We see that it works as expected if regexp is changed to this:

            ^Subject:.*Keywords.*\/[0-9]+

        That is a whole other ball of wax. Now the shortest match to the
        entirety is

            Subject:Keywords9

        and the division mark is placed at the 9. Then procmail refigures
        the longest match to the right side starting at the division mark
        and sets MATCH=9999. However here

            ^Subject:.*Keywords\/.*[0-9]*

        the second ".*" would have reached not just up to the digits but
        through them to the end of the line. MATCH would contain the rest of
        all of it matched to ".*" plus null match  "[0-9]*".

        [for curious reader]

        Given line

            Subject: Keywords 9999

        the second, which differs only by inserting the extraction marker,
        would not match and would not set $MATCH:

            ^Subject: Keywords *9999        # matches ok
            ^Subject: Keywords *\/9999      # won't !

        because the left side would be matched to "Subject:
        Keywords" and the immediately following text, " 9999", did not match
        the right side. It would actually make the condition fail and keep
        the recipe from executing. It took a lot of circuitous coding to
        allow for not knowing in advance exactly how many spaces there would
        be before the digits.

          Call it counterintuitive, but it's not a bug. General advice:
          always make sure that the right side cannot match null or that the
          last element of the left side cannot match null. Or in other words:
          force the right-hand side of the \/ to match at least one character.

    8.14 Explaining \/ and ()\/

        `MATCH' strips all leading blank lines in 3.11pre7

        [david] \/ with nothing to the left of it means "one foreslash". To
        start a condition with the extraction operator, use ()\/ or \\/;
        the latter looks counter intuitively like "literal backslash and
        literal foreslash" (as it would mean if it appeared farther along
        in the regexp), so most of us prefer the former.

            *$ var ?? $s+\/$d+      # ok, \/ in the middle
            *$ var ?? \/$d+         # Wrong, when \/ is at the beginning
            *$ var ?? ()\/$d+       # No ok, () at the beginning

    8.15 Explaining  ^^ and ^

        [philip] Procmail doesn't think *lines* when it matches; but it
        concatenates all lines together and then runs the regexp
        engine. This may be a bit surprising, but consider the following where
        we want to discard any message that is likely a html advertisement

            #   Body consists entirely of html code
            #   something which'll match any message which has ""
            #   in the body

            :0 B:
            *$ $s*
            html.mbox

        The condition test is applied to the entire body. If you want to
        limit it to match only against the beginning of the body, you have
        to say so using the ^^ token, as you discovered. A simple line
        anchor (^ or $) just says that there must be a newline (or the
        beginning or end of the area being searched) at that particular
        point in the text being matched. notice the leading anchors below.

            #   trap spam where the *very* first line of the body started with
            #   

            :0 B:
            *$ ^^$s*
            html.mbox

          What, exactly, does "Anchor the expression at the very start of
          the search area..." i.e. the ^^ ?

        [dan] Technically, an opening ^^ anchors to the putative
        newline that procmail sees before the first character of the search
        area (and a closing ^^ anchors to the putative newline that
        procmail sees after the end of the search area).  When the search
        area is B, that is a point equivalent to the second of the two
        adjacent newlines that enclose the empty line that marks the end of
        the head.

        The reason I'm bringing that up is this: if there are multiple
        empty or blank lines between the head and the body, ^^ will mark
        the start of the second of those lines, not the start of the first
        line of the body that contains some text.

        So if you want to test whether  is the first printing text
        in the body, even if it is not necessarily flush left on the very
        first line, you might need a condition like the following, where
        there is space/pipe/tab/pipe/dollar.

            *$ B ?? ^^$SPCNL*

    8.16 ANDing traditionally

        Erm, you knew this already if you read the man pages. Stacking
        condition lines one after another does the AND operation, where
        all of the conditions must be present:

            * condition1
            * condition2

    8.17 ORing traditionally

        Here is simple OR case. There are some cases where it's impossible
        to OR conditions with this style. [philip] knows more about those
        cases.

            *  condition1|condition2

        Likewise, two exit code tests can often be ORed like this

            * ? command1 || command2

        But there are many situations where two tests cannot be ORed by
        combining them into one condition:

        o   a regexp search of one area ORed with a regexp search of a
            different area
        o   a positive regexp search [i.e., for a match to its pattern] ORed
            with a negative regexp search [i.e., for the absence of any
            match to its pattern]
        o   an exit code condition ORed with a regexp search condition
        o   an exit code condition seeking success ORed with an exit code
            condition seeking failure
        o   a size test ORed with anything else (even another size test)

          How can I make OR conditions that all use the SAME action? I want
          to be able to test for a number of variants on certain requests,
          all in one block.

        [hal] Yes, this can be easily done

            CASE = ""

            :0
            * case 1 tests
            {
                CASE = 1
            }
                :0 E
                * case 2 tests
                {
                    CASE = 2
                }

            :0
            * ! CASE ?? ^^^^
            {
                # real work, perhaps with explicit tests on CASE
            }


         Case study: Finding text from header and body

        [david] In addition to the standard ways of coding OR, here's a
        special one for searching the subject and the body for a given word
        in either:

            * HB ?? ^^(.+$)*(Subject:(.*[^a-z0-9])?|$(.*\<)*)remove\>

        If the string doesn't have to be preceded by a word border, it gets
        a little simpler:

            * HB ?? ^^(.+$)*(Subject:.*|$(.|$))*string

    8.18 ORing and score recipe

        Once any of the conditions match, the score gets a positive value and
        the recipe succeeds. Idea by Erik Selke 

        [era comments] ...allegedly the scoring system is going to cost you
        more than plain old regex matching. Floating-point math and all that,
        even if you use extremely simple scoring. Thus, it would probably be
        slightly more efficient to do it the De Morgan way.

            * 1^0 condition1
            * 1^0 condition2

        We can now write the previous case stydy (HB ORing traditionally)
        with scores. I was tempted to write it like this, when [david]
        told me the following.

            * 1^0 H ?? match-it
            * 1^0 B ?? match-it

        [david] That will work, but it isn't the best way to do ORing,
        because if a match is found to the first condition procmail still
        takes the trouble to test the second one. Better, use the supremum
        score on each condition:

            $SUPREME = 9876543210

            *$ $SUPREME^0 first_condition_to_be_ORed
            *$ $SUPREME^0 second_condition_to_be_ORed
            * ... etc. ...
            *$ $SUPREME^0 last_condition_to_be_ORed

        Upon reaching the supreme score, procmail will skip all remaining
        weighted conditions on the recipe, deeming them matched. Since all
        conditions on this recipe are weighted, once procmail finds one
        matched condition it will skip the rest and execute the action.

    8.19 ORing by using De Morgan rules

        [Tim Pickett ] I thought I'd point out that
        there are a few ways to do a logical OR of conditions. Someone posted
        a solution here that involved using procmail's scoring system, but I
        figured you could do it without scoring by taking advantage of De
        Morgan's rule:

            a or b      is same as   not(not a and not b)

        or mathematically:

            a || b <=> !( !a && !b )

        Here's a way to do ORing

            :0
            * ! condition1
            * ! condition2
            { }             # official procmail no-op
            :0 E
            action_on_condition1_or_condition2

9.0 Variables

    9.1 Setting and unsetting variables

        You have already set variables with the "=" syntax. Variable names
        are case sensitive: `var' is different from `VAR'

            VAR = /var/tmp  # directory
            VAR = "this"    # literal
            VAR = 1
            VAR = $FOO      # another.
            VAR = "$VAR at" # combined with previous value

        Unsetting a variable is done like this

            VAR             # kill variable.
            VAR=            # same but with old style
            VAR = ""        # Variable is said to be "null" now

        And you can put multiple assignments on the same line

            VAR=1  VAR=2  VAR=3

        Examine the following, which are all equivalent. The backticks will not
        require a shell in the absence of any `SHELLMETAS' so neither of
        these will spawn a shell

                #   _case1_: We Don't care if file exists this time...

                VAR = `cat file`

                #   _case2_: The use of {} is considered "modern"

                :0
                * condition
                {
                    VAR = `cat file`
                }

                #   _case3_: oldish, and procmail specific and errors have
                #   been reported if you use this construct.
                #   Note: There must be no space in "VAR=|"

                :0
                * condition
                VAR=| cat file

    9.2 Variable initialisation and sh syntax

        Procmail borrows some sh syntax for variable initialisation.
        Note that sh's ${var:=default} and ${var=defaultvalue}
        syntaxes are not available in a procmail rcfile.

        o   VAR1 = ${VAR2:-value}
            sets VAR1 to VAR2 if VAR2 is set and non-null, and sets VAR1 to
            default "value" otherwise
        o   VAR1 = ${VAR2-value}
            sets VAR1 to VAR2 if VAR2 is set, and sets VAR1 to default
            otherwise
        o   VAR1 = ${VAR2:+value}
            sets VAR1 to "value" if VAR2 is set and non-null, and sets VAR1
            to VAR2 otherwise.
        o   VAR1 =${VAR2+value}
            Sets VAR1 to "value" if VAR2 is set and sets VAR1 to VAR2
            otherwise.

        And here are the classic usage examples

            VAR = ${VAR:-"yes"}     # set VAR to default value "yes"
            VAR = ${VAR+"yes"}      # If VAR contains value, set "yes"

        Ever wondered if this calls `date` in all cases?

            VAR = ${VAR:-`date`}

        No, procmail is smart enough to skip calling `date' if VAR already
        had value. It doesn't evaluate the whole line. Below you see what
        each initialising operator does. Study it carefully

            VAR = ""                # Define variable
            VAR = ${VAR:-"value1"}  # VAR = "value1"
            VAR = ""
            VAR = ${VAR-"value2"}   # VAR = ""

            VAR = ""
            VAR = ${VAR:+"value3"}  # VAR = ""
            VAR = ""
            VAR = ${VAR+"value4"}   # VAR = "value4"

            # Note these:
            VAR = "val"
            VAR = ${VAR:+"value3"}  # VAR = "value3"
            VAR = "val"
            VAR = ${VAR+"value4"}   # VAR = "value4"


            VAR                     # kill the variable
            VAR = ${VAR:-"value1"}  # VAR = "value1"
            VAR
            VAR = ${VAR-"value2"}   # VAR = "value2"

            VAR
            VAR = ${VAR:+"value3"}  # nothing is assigned
            VAR
            VAR = ${VAR+"value4"}   # nothing is assigned

        And if you want to choose from several initial values,
        you might use the recipe below
        instead of the standard var = ${var:-"value"}.

            :0
            * VAR ?? ^^^^
            {
                #   no value (or was empty), set default value here based on
                #   some guesses

                VAR = "base-default"

                :0
                * condition
                {
                    VAR = "another-default"
                }

                ...more conditions..
            }

        You could also use equivalent, but less readable condition line in
        previous recipe:

            *$ ${VAR:+!}

        It works, because if variable contains a value the line expands to

             * !

        Where "!" is the procmail "false" operation. One more way to do the
        same would be, that we require at leastone character to be present.
        You could use also regexp (.), which would require at least one
        character to be present, but you might not like matching pure spaces.

            * ! VAR ?? [a-z]

    9.3 Testing variables

        If possible, perform positive tests, rather than negative, like below:

            * ! TEST_FLAG ?? yes

        Alternative with a positive test:

            *  TEST_FLAG ?? no
        To my opinion, this is more
        readable. You're free to disagree with me at this point, but all in
        all, it's nicer to look at code that has as few ! flags as
        possible, especially in variable tests.

        [philip] The following fails if the variable is unset or null.

            * variable ?? (.)

        That was why I'd be better off to use

            *$ variable ?? $NSPC

        Or

            * variable ?? (.|$)

        to require that *variable* contain at least one character. But
        neither is a way to check whether a variable is set or not, because
        each treats a null variable the same as an unset one. This is the
        best way I know to check whether a variable is set or not:

            *$ ! ${VAR+!}

        [] Here is yet another way to test if variable
        is set  and if it isn't, sets it to a default value.

            :0
            *$ ! VAR^0
            {
                VAR = "value"
            }

    9.4 What does $\VAR mean?

        [era and david] Procmail 3.11, $\VAR will escape regex metacharacters.
        It should produce a suitably backslash-escaped expression for
        Procmail's own use. In addition $\VAR will always begin with leading
        empty parentheses.

        You can't pass the $\VAR construct to shell programs, because there
        is that leading parenthesis. Here's a recipe to standardize the regexp.
        You can pass SAFE_REGEXP to an external programs like `sed'.

            PROCMAIL_REGEXP = "$\VAR"

            :0
            * PROCMAIL_REGEXP ?? ^^\(\)\/.*
            {
                SAFE_REGEXP = "$MATCH"
            }

        [era] Note that this is slightly inexact; Procmail will
        backslash-escape according to Procmail's needs, not sed's. For
        example, Procmail doesn't think braces are magic (although that would
        be nice to have in Procmail as well) whereas many modern variants of
        sed do.

    9.5 Common pitfalls when using variables

        Procmail is picky and forgives nothing. Here are some of the favourite
        mistakes one can make:

            $EMAIL  = "foo@site.com"      # Done Perl lately? Remove that $

            # Erm, this is ok, but many procmail recipe writers want to
            # take extra precautions and include the regexps in parentheses.
            # So, maybe (yabba|dabba|doo) would be more safe

            REGEXP  = "yabba|dabba|doo"

            *  Subject:.*$REGEXP  # Hey, you need the "*$ Subject..."

            *$  $REGEXP ?? hello  # surely you meant '* REGEXP ?? hello'

    9.6 Quoting: Using single or double quotes

        Pay attention to this:

            VAR = "you"
            NEW = 'hey "$VAR"'  # won't extrapolate $VAR; you get literal
            NEW = "hey '$VAR'"  # extrapolates to: hey 'you'

        You can even combine separate words together

            VAR = "1 ""and"" 2" # same as "1 and 2"

        Don't let these many quotes disturb you, just count the beginning
        and ending quotes. Superfluous here, but you may need some similar
        construct somewhere else.

            VAR = '1 '"'"'and'"'"' 2'  # same as: 1 'and' 2

        [david] Beware forgetting quotes, like when you'd do

            SENDMAILFLAGS = -oQ/var/mqueue.incoming -odq

        Procmail translates `!' into | "$SENDMAIL" "$SENDMAILFLAGS" as the
        procmailrc(5) man page warns us. By the rules of sh quoting, that
        means that shell sees only the first switch

            % sendmail -oQ/var/mqueue.incoming

        My suggestion: since you need a soft space inside `$SENDMAILFLAGS',
        use the quotes when you define `$SENDMAILFLAGS' but do this instead
        of using the `!' operator for forwarding:

            SENDMAILFLAGS = "-oQ/var/mqueue.incoming -odq"

        [Walter Haidinger ] Here's yet another
        approach: deliver messages from procmail directly to mailboxes in
        all those users' homes. No sendmail involved, _much_ lower loads.

            :0:
            * 
            /var/spool/mail/someuser

        [philip] Assuming that "someuser" is an actual user in the
        password file (I haven't been following this thread, some maybe
        that isn't true here), then the following is probably better:

          Walter Haidinger comments on this recipe: I'm happy to announce that
          this works *really* well. No harm is done to the system-load
          anymore. What a relief!

            :0 w
            * conditions
            |procmail -d someuser

        That lets procmail's very tricky "screenmailbox()" routine take
        care of bogus mailboxes in a secure fashion.

          Is that as safe as forwarding? Does another sendmail delivering
          to /var/spool/mail/someuser use the same locking mechanism and notice
          that mailbox is already locked? I don't want to risk a corrupt
          mailbox.

        [philip] Sendmail only delivers directly to files through
        aliases that say things like:
                whatever: /some/local/file
        Under normal circumstances, sendmail calls the local mailer to actually
        store mail in a file, and since that's procmail (right?), there
        shouldn't be a problem. Also, sendmail 8 does kernel-level locking
        when it delivers directly.

    9.7 Quoting: Passing values to an external program

        Remember to include the double quotes when you send variables'
        values to the shell programs. Below you see a mistake,
        because the content of the SUBJECT is not quoted and
        thus not available from perl variable $ARGV[1].

            :0
            * condition
            | perl-script $SUBJECT      # mistake; use "$SUBJECT"

        There is also another way. If your script can access environment
        variables (almost all programs can), then you do not need to pass
        the variables on the command line. Above, the SUBJECT is already
        in the environment and in perl you can get it.

            $SUBJECT = $ENV{SUBJECT};   # or "use Env;" and you see $SUBJECT

        Next, do you know what is the difference between these two recipes?

            :0
            | "command arg1 arg2 arg3"

            :0
            | command "arg1" "arg2" "arg3"

        You guessed it. The first one quotes the entire command and does not do
        the right thing, the latter is correct and depending on the content of
        argN variables. Anyway, play safe and always add quotes.

        Sometimes you need trickier quoting to to get single quotes around
        the `arg'. Pay attention to this, because this may be the reason
        why your grep command doesn't seem to succeed as you expect.

            #  If $GREP "$arg" doesn't seem to work

            * ? $GREP "'"$arg"'" $DATABASE

    9.8 Passing values from an external program

        External programs cannot set procmail variables directly. Programs
        must write the values to external files and then read the values
        from these files. Capturing only one value is easy:

            var = `command`      # capture STDOUT

        But if a program modifies the body and exports some status
        information it is trickier. We assume here that the script is
        controlled by you and that you have added the switch
        --export-status option which causes the program to print
        information to a separate file.

            LOCKFILE    = $HOME/.run$LOCKEXT  # protect external file writing
            valueFile   = $HOME/tmp/values

            #   modify body, and export status values to external file: one
            #   value in every line
            #
            #       VALUE1
            #       VALUE2
            #       VALUE3

            :0 fb
            | $NICE script.pl --export-status $valueFile

            values = `cat $valueFile`

            # Derive values from each line

            :0                              # line 1
            *$ values ?? ^^\/[^$NL]+
            {
                var1 = $MATCH
            }

            :0                              # line 2
            *$ values ?? ^^.*$\/[^$NL]+
            {
                var2 = $MATCH
            }

            :0                              # line 3
            *$ values ?? ^^.*$.*$\/[^$NL]+
            {
                var3 = $MATCH
            }

            LOCKFILE    # Release lock

        [richard] Alternatively write valueFile from your rc or external
        program with lines like

            PARAM1="value for param 1"
            PARAM2="value for param 2"
            PARAM3="value for param 3"

        and read it with

            INCLUDERC $valueFile

        Now there is no need to worry about synchronizing the read with the
        lines, or about adding new parameters, since each is labeled in
        valueFile.

    9.9 Incrementing a variable by a value N

        [dan, phil and Richard] Here's a recipe for incrementing a variable
        by a value N. If $VAR is not a number, we get an error. Note that
        if $VAR + $N is not greater than 0, this recipe will not change the
        value of VAR if the assignment happens inside braces. You must
        place the assignment after the closing curly brace.

            :0
            *$ $VAR ^0
            *$ $N   ^0
            { }             # procmail no-op
            VAR = $=

    9.10 Comparing values

        It's too expensive to call the shell's `test' function to do
        [-lt|-eq|-gt] because you can do the same with procmail. The
        do-something below is run if SCORE <= MAXIMUM. The recipe simply
        subtracts SCORE from MAXIMUM and determines if the result is
        positive.

            :0
            *$ -$SCORE   ^0
            *$  $MAXIMUM ^0
            {
                .. do-something
            }

        [idea by era] it's getting slightly cumbersome if it's between MIN
        and MAX:


            :0
            *$   $SCORE ^0
            *$  -$MIN   ^0
            {
                dummy               # no-op, just for the LOG

                :0
                *$ -$SCORE  ^0
                *$  $MAX    ^0
                {
                    suitable
                }
            }

        Eg. When values are MIN=1, MAX=5, SCORE=4

            procmail: Assigning "SCORE=4"
            procmail: Score:       4       4 ""
            procmail: Score:      -1       3 ""
            procmail: Assigning "dummy"
            procmail: Score:      -4      -4 ""
            procmail: Score:       5       1 ""
            procmail: Assigning "suitable"

    9.11 Strings: How many characters are there in a given string?

            # 1998-06-23 PM-L [walter]

            :0
            *  1^1 VAR ?? .
            { }
            LENGTH = $

    9.12 Strings: How to strip trailing newline.

        Suppose you have used regexp, which left newline($) in the `MATCH'.
        If you wonder why the recipe works, remind yourself that regexp
        operator "." never matches a newline.

            :0
            * VAR ?? ^^\/.+
            {
                VAR = $MATCH
            }

    9.13 Strings: deriving the last N characters of a string.

            #   1998-06-23 PM-L [walter] Note the use of
            #   the $ sign below to anchor to end-of-string...
            #
            #   For last 2 characters use * VAR ?? ()\/..$
            #   For last 5 characters use * VAR ?? ()\/.....$

            :0                      # Last character
            * VAR ?? ()\/.$
            {
                TAIL = $MATCH
            }

    9.14 Strings: Getting partial matches from a string.

        [dan] Getting a match to the right is quite easy with procmail's
        `match' operator.

            VAR = "1234567890"

            :0
            * VAR ?? ()\/3.*
            {
                result = $MATCH                         # now 34567890
            }

        but deleting 2 characters from the end is nearly impossible without
        forking an outside process. The cheapest might be `expr' because it
        doesn't need a shell to pipe `echo' to it (as `sed' would and I
        believe `perl' would):

            #   by resetting the shellmetas, this will only call
            #   `expr'. If we wouldn't have fiddled with shellmetas,
            #   this would have called two processes: sh + expr

            saved       =   $SHELLMETAS
            SHELLMETAS
            result      =   `expr "$VAR" : '\(.*\)..'`  # now 12345678
            SHELLMETAS  =   $saved

        ksh or bash could do it as well:

            #   semicolon to force invoking a shell, actually
            #   first question mark will force a shell already.

            saved       = $SHELL
            SHELL       = /bins/sh
            result      = `echo ${VAR%??} ;`
            SHELL       = $saved

        Now, if you know that the last two characters will be "90", that's
        different. Of course, this totally screws up if the third-to-last
        character is a 9.

            :0
            * VAR   ?? ()\/.*[^0]
            * MATCH ?? ()\/.*[^9]
            {
                result = $MATCH                         # now 12345678
            }

        [jari] Comments: If a shell must be used, then `awk' is a good tool for
        simple string manipulation. Its startup time is faster that perl's
        whose overhead is due to internal compilation. `awk' also consumes
        less recourses overall than `perl'. Following will only work if VAR
        is a string of continuous block of characters. (ARGV[1] can be used)

            saved       =   $SHELLMETAS
            SHELLMETAS

            VAR = ` awk 'BEGIN{ v = ARGV[1];                                \
                    print substr(v,1,length(v)-2); exit }'                  \
                    "$VAR"                                                  \
                  `

            SHELLMETAS = $saved

        This version requires _some_ file, any file, so that we get awk
        started. In the previous code all the work was done in the BEGIN
        block and no file was ever opened.

            saved       =   $SHELLMETAS
            SHELLMETAS

            VAR = ` awk '{print substr(v,1,length(v)-2); exit }'            \
                    v="$VAR" /etc/passwd                                    \
                  `

            SHELLMETAS = $saved

        [dan] comments awk: `expr' is sure to be a smaller binary than awk
        for procmail to fork, and it needs much less command-line code to
        do this job. Note also that one still has to diddle with SHELLMETAS
        to avoid a shell, because the awk code contains brackets; thus it
        doesn't replace all.

        There is also a way to remove words from the end of string by
        procmail means if the strings are separated by same separator. Let's
        use the word this-mailing-list-request which we would like to shorten
        to this-mailing-list. [david] presented the recipe 1998-06-16 in PM-L.


            VAR = "this-mailing-list"

            #   1) if there is match at the end ending to these words
            #   2) Get everything up till last match and store it to MATCH
            #   3) Read MATCH, but exclude last dash "-"

            :0
            * VAR   ?? -(owner|request|help)^^
            * VAR   ?? ^^\/.*-
            * MATCH ?? ^^\/.*[^-]
            {
                VAR = $MATCH
            }

    9.15 Strings: Procmail string manipulation example

        [1998-06-23 PM-L walter] ... Now we get to apply these formulas
        to strip the last character off a string. It gets a bit ugly for
        special cases. I've deliberately chosen a worst-case scenario.


            VAR         = "Testing 012301230111"
            RC_APPEND   = $PMSRC/pm-myappend.rc

            :0
            * VAR ?? ()\/.$
            {
                TAIL = $MATCH           # last character of VAR "1"

                # Get the longest match that does not end in the TAIL character

                :0
                *$ VAR ?? ()\/.*[^$TAIL]
                {
                    HEAD = $MATCH      # now "Testing 012301230"

                    #   if the last two or more characters in VAR are
                    #   identical, they all get chopped, oops

                    :0
                    * -1^0
                    *  1^1 VAR  ?? (.)
                    * -1^1 HEAD ?? (.)
                    {
                        dummy     = "tooshort"
                        INCLUDERC = $RC_APPEND
                    }
                }
            }

            result = $HEAD              # "Testing 01230123011"

            # ........................................ pm-myappend.rc
            #   LENGTH(HEAD) plus 1 SHOULD equal LENGTH(VAR). That is
            #   not the case when the last 2 (or more) ending
            #   characters are identical. in that case, call appendrc
            #   recursively to stick back an appropriate number of
            #   TAIL characters.

            :0
            * -1^0
            *  1^1 VAR  ?? (.)
            * -1^1 HEAD ?? (.)
            {
                HEAD      = "$HEAD$TAIL"
                INCLUDERC = $RC_APPEND
            }

    9.16 How to raise a flag if the message was filed

            FILED = !       # ! is procmail "false"

            :0 c:           # We process the message more
            * condition
            foo

                :0 a
                {
                    FILED   # Kill variable
                }

            ...

            :0              # Stop if previous cases filed the message
            *$ $FILED
            {
                HOST = "_done_"
            }

        Or alternatively: procmail automatically sets `LASTFOLDER' if
        it delivers message to mailbox.

            LASTFOLDER      # kill variable

            :0 c:
            * condition
            foo

            :0 c:
            * condition
            bar

            ... et cetera ...

            :0
            * ! LASTFOLDER ?? ^^^^      # Or   ${LASTFOLDER+!}!
            {
                HOST = "_done_"         # Force procmail to stop
            }

    9.17 Dollar sign in condition lines.

        #todo, check this recipe

            This doesn't seem to work for me...

            * ^TO()$\foo@bar.com

        [david] An unescaped dollar sign later in the line represents a
        newline, so what you have there is searching for the following:

        .   An expression that matches the expansion of the ^TO token (which
            is anchored to the start of a line by its definition), followed
            by
        .   A newline, followed at the start of the next line by
        .   "foo@bar" [the backslash escapes the f, which didn't need
            escaping], followed by
        .   any character that is not a newline (the period is unescaped),
            and finally
        .   "com".

        Try this instead:

            *$ ^TO()$\foo@bar\.com

        #todo: the dollar seems exactly the same in the above two
        #todo Examples: are you sure that this is correct?

        In fact, to avoid matches to things like foo@bar.community.edu,
        you might want to do it this way:

            *$ ^TO()$\foo@bar\.com\>

    9.18 Finding mysterious foo variable

          I have my fellow worker's procmail code and he uses a variable FOO
          that I can't find in his code anywhere. It's not a shell variable
          either, because it's literal. Where does it come from?

        Your procmail runs /etc/procmailrc when it starts, please check
        that. It may define some common variables already for all users.

    9.19 Storing code to variable

        One way to run complex code in a procmail recipe is first to store
        it in a variable. Idea by [era]. You could do this in a separate shell
        script too. The following example reads URLs from the body of
        a message: the URLs have been put to separate lines and some special
        Subject is used to trigger the dumping of the html pages:

            #   Code by [era]
            #
            COMMAND='while read url; do
                case "$url" in
                  *://*)
                    lynx -traversal -realm -crawl -number_links "$url" |
                    $SENDMAIL $LOGNAME
                    ;;
                esac
            done'


            #  Notice the trailing semicolon after `eval' !
            :0 bw
            * ^Subject: xxxxx
            | eval "$COMMAND" ;

        If you want to run the code inside the nested block, then look
        carefully, there are double quotes around the command in backticks.
        If you leave double quotes out, then each word in SH_CMD would be
        interpreted separately:

            $SH_CMD = '$echo "$VAR" >> $HOME/test.tmp'

            :0
            * condition
            {
                #   condition satisfied; run the given shell command
                #   and do something more.

                dummy = `"$SH_CMD"`

                ..rest of the code..
            }

        A similar construct works for message echo-ing too:

            MESSAGE='Thank you so much for your message.
            Unfortunately, the volume of mail I receive .... (blah blah blah).
            If your matter is urgent, try calling +358-50-524-0965.
            '

            :0 hw
            * ! ^X-Loop: moo$
            | ($FORMAIL -rt -A "$MY_XLOOP"; echo "$MESSAGE") | $SENDMAIL

    9.20 Getting headers into a variable.

        [david] Here are several ways to get the entire header into a variable:

            HEADER = `$FORMAIL -X ""` # The space after the X is vital.
            HEADER = `sed /^$/q` # also writable as   HEADER=`sed /./!q`

            :0 h
            HEADER=|cat -

        will save the entire header into one variable. It has to be smaller
        than $BUFSIZE, though. This way might work as well, and will require no
        outside processes if it does:

            :0
            * ^^\/(.+$)*$
            {
                HEADER = $MATCH
            }

    9.21 Converting value to lowercase

        If you know that a word belongs to set of choices, you can do
        this inside procmail

            LIST = ":word1:word2:word3:word4"   # Colon to separate words
            WORD = "WORD1"

            :0
            *$ LIST ?? :\/$WORD
            {
                WORD = $MATCH
            }

        But if you don't know the word or string beforehand, then this is
        the generalized way: [idea by era and david]

            :0 D
            * WORD ?? [A-Z]
            {
                WORD = `echo "$MATCH" | tr A-Z a-z`
            }

10.0 Suggestions and miscellaneous

    10.1 Speeding up procmail

        o   Use absolute paths to take the burden of searching binary along path
            from shell: Use $FORMAIL variable abstraction.

                $FORMAIL = "/usr/local/bin/formail"

                :0 fhw
                | $FORMAIL -I "X-My-Header: value"

        o   Multiple `echo' commands that spread many lines can be converted
            to single echo command if \n escape is supported. You usually
            see these in autoresponders

                echo "........."; \
                echo "........."; \
                echo ".........";

                -->

                echo ".........\n" \
                     ".........\n" \
                     ".........\n";

        o   You can avoid multiple and possible expensive FROM_DAEMON tests
            by caching the result at the top of your .procmailrc. You can
            now use variable $from_daemon like the big brother FROM_DAEMON.
            The same idea can be applied to FROM_MAILER regexp. If you have
            *pm-javar.rc*, it already defines variables `$from_daemon' and
            `from_mailer' exactly like here:

                from_daemon = "!"

                :0
                * ^FROM_DAEMON
                {
                    from_daemon = "!!"  # double !! means "OK"
                }

                :0
                *$ ! $from_daemon
                {
                    ..do-it..
                }


        o   Count the backticks and you know how many shell calls procmail
            has to launch. See if you can minimize them and use some procmail
            code instead.
        o   ^TO and other macros are expensive, see if you can use simple
            Header:.*\ instead. Well, it's not clear if this
            gives you much speed advantage.
        o   Don't call "$FORMAIL -xHeader:" every time you need a header
            value, consider if it suffices to use `match' operator \/.
        o   You can minimize the calls to only one `formail' if you add many
            headers along the way: See formail usage tips in this document
        o   Searching body is expensive, simply because it contains more text.
            There isn't much to do about this, because you use `B' anyway
            when you need it.
        o   See if you can move some tasks to your .cron file. procmailrc is
            not meant for those purposes. Instead of calculation daily
            values every time in procmail, let cron do that at 04:00 or
            21:00. Don't run cron at midnight if you can, because everybody
            else is running their crons at the same time. If "logical" date
            change time can be used (when you arrive to work, when you
            leave the work), use it in cron jobs.
        o   [philip] Setting `LINEBUF' permanently to a big value slows
            procmail down.
        o   Remove all calls to `perl' and use programs that are nicer to
            the system (If you just call command line perl, there is
            probably an equivalent alternative with `awk' `tr' `sed' `cut')
        o   Examine each shell command and see if you do need `SHELLMETAS.'
            If you can set `SHELLMETAS' to empty, this saves calling "sh" for
            each invocation of the external command.

    10.2 See the procmail installation's examples

        Did you remember to look at the examples that come with procmail? If
        not, it's time to give them a chance to educate you. Here is one
        possible directory you could take a look. Ask from your sysadm if you
        can't find the directory where to look into.

            % ls /usr/local/lib/procmail-3.11pre7/examples/

        Or if you're really anxious to get on your own, try this. The directory
        /opt/local is for HP-UX 10 machines and the *forward* contains example
        how to define your `.forward' for procmail.

            % find /opt/local/ -name "forward" -print

        If the find succeeded and found the file, then you know where the
        procmail files installation directory is.

    10.3 Printing statistics of your incoming mail

        If you keep the procmail log crunching, it will record to which
        folder the messages was filed. There is program `mailstat' which
        can process the procmail.log file and print nice summary out of it.
        If you generate the summary at midnight and clear the log, you
        get pretty nice per day/per folder traffic analysis.

            # -m merges all error messages into a single line

            % mailstat -km procmail.log

    10.4 Storing UBE mailboxes outside of quota

          I want to store spam outside disk space. Problem: if I tell
          procmail to deliver to, say, /tmp/spam.box, it does so just fine
          (according to the log). Unfortunately, it delivers to /tmp on the
          mail host which I cannot access. spam.box doesn't appear in the
          /tmp directory of the shell machine when procmail is invoked for
          incoming mail.

        [philip] Under the most likely configuration of sendmail in
        this situation, it is impossible to have procmail invoked by
        sendmail on the shell machine: sendmail is probably set to just
        forward all mail to the designated mail delivery machine.

        There are other options: you could temporarily store the mail in
        your account, then have a cronjob on the shell machine that
        reprocesses the message. That would probably be more efficient than
        having each message trigger an rsh to the shell machine. If you
        actually get enough spam that it's pushing against your quota, then
        the rsh is too expensive -- use a cronjob that invokes something
        like:

            cd your-maildir     &&
            lockfile spam.lock  &&
            test -s spam        &&
            {
                cat spam >> /tmp/spam.box && rm -f spam spam.lock || \
                rm -f spam.lock;
            }

        WARNING: the above assumes the following:

        o   everything in your-maildir/spam is spam and belongs in
            /tmp/spam.box
        o   no further filtering of the messages is necessary: they just need
            to be moved (it actually treats everything in the
            your-maildir/spam as a single message and uses procmail as a
            reliable copy command, thus the `DEFAULT' assignment as the use
            of /dev/null as a empty procmailrc)
        o   /tmp/spam.box is a not a directory

        If the latter two of those conditions isn't true OR IF THEY MIGHT
        CHANGE then you should use `formail' `-s' to break the message apart
        and invoke procmail on each one separately.

        [era] Many sites cross-mount directories for various reasons. /tmp
        is always local but /var/tmp might be cross-mounted between the
        login host and the mail host; another one to try is /scratch -- and
        if all else fails, ask your admin to set up an NFS share for this
        purpose.

    10.5 Using first 5-30 lines from the message

        [era] The regex to grab few lines (or all of them, if there are
        less than fifty) is not going to be very pretty, but it saves launching
        an extra process.

            :0 B
            * $ ^^$SPCNL*\/$NSPC.*$(.*$)?(.*$)? ... etc, the rest of the lines
            {
                toplines =  $MATCH
            }

        The skipping of whitespace at the beginning of the message is of
        course not necessary. You should probably set `LINEBUF' reasonably
        high if you grab many lines, say 30: 80*30 = 2400 bytes; probably
        setting it to 8192 or 16384 is a good idea, depending how much you
        want to match. The above gets ugly quickly, so

            #  But if N=30, sed ${N}q if you don't have head

            :0 Bi
            {
                toplines = `head -$N`
            }

            :0 a
            * toplines ?? pattern
            {
                ...do-it
            }

    10.6 Using cat or echo in scripts?

          I have seen a lot of examples that use 'echo', i.e.,

            :0
            * condition
            | echo "first line of message"  \
                   "second ..."             \
                   "et cetera"

          I started out with spam.rc from "ariel" which got me into the
          habit of

            :0
            * condition
            | cat file_containing_message

          although I note that spam.rc did have one recipe using the echo
          method. What are the reasons for choosing each method over the
          other?

        Here is a comparison table. Choose the one you think is best for you

        o   Echos don't have dependency on an external file:
            everything is contained in the .procmailrc file. Echos keep
            all the relevant stuff in one file. Cat's make you
            maintain multiple files. That's the main
            reason I lean toward echo's; you may have accounts on
            several machines. It is easier to be able to copy just one
            generic .procmailrc between them without having to copy a bunch
            of messages also. Mostly, though, there's no real difference
            between the two methods.
        o   Echo is easier to use with variables.
        o   Echo starts many processes, cat only starts one, but this is
            not always true: In most current Bourne shell implementations,
            echo is a builtin. This holds true with tcsh too.
        o   The main problem I see with the use of cat is "what happens when
            you forget the file or destroy it ?". I suggest to, at least,
            test that the file is readable before catting it.
        o   [richard] An argument against echo is that it is not well
            standardized, and different versions may exist on the same
            machine. Some recognize -n, some don't; some recognize embedded
            metacharacters, some don't.This is an argument in favor of
            `print'. Print, however, is not a built-in on all systems. The
            comment on built-ins is pertinent to situations when a shell is
            spawned. When procmail handles the call directly, it will
            always look for a stand-alone executable. I guess echo may be
            better, as long as we are aware of any differences in behavior
            between built-in and stand-alone versions.

    10.7 How to run an extra shell command as a side effect?

          [jari] I was once wondering what would be the wisest way to send
          messages to my daily "biff" log file about the events that
          happened during my .procmailrc execution. This is how [david]
          commented on my ideas

            # case 1: print to BiffLog

            dummy = `echo "message: $FROM $SUBJECT" >> $biff `

        [david] Problems you get no locking on the destination file, and
        unless you put it inside braces you have to run it on every message
        unconditionally. (Also procmail tries to feed the whole message to
        a command that won't read it, but the remedies for that don't help
        very much.)

            # case 2: We consume delivering recipe and therefor have to use
            #        `c' flag.

            :0 whic:
            | echo "message: $FROM $SUBJECT" >> $biff

        Here it locks the destination file and you can add conditions to
        it, so it's probably the best. If the head or the body is less than
        one bufferful, you can limit the unnecessarily written data with `h'
        or `b', but I think that in most OSes a partial buffer and a full
        one are the same amount of effort.

            # case 3: We use side effect of "?" here. Cool, but this
            # doesn't do $biff file locking thus message order may
            # not be what you expect.

            :0
            *  condition
            *  ? echo message: $FROM $SUBJECT >> $biff
            { }         # procmail no-op

        We have conditions possible, but there is no locking on the
        destination file. I'd go with method #2 or a variation thereof:

            :0 hic:                 #   we don't necessarily need `w'
            * condition
            | echo message: $FROM $SUBJECT >> $biff


            :0 hi:                  #   Or you could use this
            * condition
            dummy=| echo message: $FROM $SUBJECT >> $biff

          [jari] Now, when [david] has explained how various ways differ
          from each other, I present the recipe where I used the case 3.
          When I was dropping a message to a folder, I wanted to send a
          message to my biff log too. The idea is that the drop-conditions
          have already matched and then we run extra command by using side
          effect of "?" token. As far as the recipe is concerned, the "?"
          is a no-op. The pedantic way would have been to add the LOCKFILE
          around to the recipe, but imagine 50 similar recipes like
          this...and you understand why the LOCKFILE was left out. It's
          only necessary if you worry about sequential writing to the biff
          file.

            :0 :
            * drop-condition
            * ? echo message: $FROM $SUBJECT >> $biff
            $MBOX

    10.8 Forcing "ok" return status from shell script

          ...the "?" trick only allows running some additional shell
          commands (`true' command always succeeds) while conditions
          above have already determined that drop will take place. And you
          can always make condition to succeed if a misbehaving shell script
          always returns a failure exit code.

            * ? misbehaving-shell-script || true

        [david] If the script *always* returns a failure code, just do this:

          * ! ? misbehaving-shell-script

        The more complex case is a script that can return either success or
        failure but you don't care which; if the drop conditions passed,
        you want to run the action line. `echo' can also fail if the
        process lacks permission or opportunity to write to stdout. A more
        reliable choice is true(1); its purpose in life is to do nothing
        but exit with status 0.

        The command `:' is a shell builtin which always returns true
        status. Not exactly more readable than true(1) "|| :" will save the
        invocation of true (unless true is built into $SHELL), but procmail
        will still run a shell. On the other hand, as long as the command
        itself has no characters from `SHELLMETAS' a weight of 1^1 and no
        "|| anything" will avoid the shell process as well.

        However, there is yet a better way to make sure that a failure by the
        script doesn't make procmail abort the recipe:

            :0 flags
            * other conditions
            * 1^1 ? shell-script
            action

        Regardless of the exit status of the script, the condition will score
        1 and not interfere with procmail's decision about the action line of
        the recipe. Weighted exit code conditions behave like this (see the
        procmailsc(5) man page):

            * w^x ? command

        scores w on success or x on failure.

            * w^x ! ? command

        scores the same as this:

            * w^x  pattern_that_appears_in_the_search_area_$?_times

    10.9 Make your own .procmailrc available to others

        There is never too much to learn about procmail and the best source
        is the rc files that people have done. Remember to comment
        your .procmailrc file well before you put it available. Below
        is a recipe
        for sending your .procmailrc upon request. If you want to send
        anything more that one or two files (many times you want to put
        other files available too), then please do not use this code but a
        general file server module.
        (Note: #REF #procmail_module_list ;Procmail module list;)

            :0
            * ! ^Subject:.Re:
            *   ^Subject:.*send.*procmailrc
            * ! ^FROM_DAEMON
            {
                :0 fhw:
                | $FORMAIL -rt                                              \
                           -A "Precedence: junk"                            \
                           -I "Subject: Requested .procmailrc";             \
                           -I "$MY_XLOOP"

                :0 a hwic
                | ( cat - $HOME/.procmailrc ) | $SENDMAIL

                :0              # trash the "Send procmailrc" request
                /dev/null
            }

    10.10 Using dates efficiently

          _Note_: See module list, where you will find `date' and `time'
          parsing modules. You can also parse the date from the first
          `Received' or `From_' header if it is the same each time in your
          system. That would be orders of magnitude faster and decreases
          your system load if you receive lot of mail.

        Calling `date' in your procmail script _many_ times is not a good
        idea. Use the `MATCH' as much as possible to be efficient in
        procmail, like below where we call `date' only once. If you are not
        in the same time zone as your server, and you want an accurate
        report of the date, you might amend the invocation to the following:

            date = `TZ="KDT9:30KST10:00,64/5:00,303/20:00";date "+%Y %m %d"`

        The basic recipe is here

            # By [richard] add %H:%M%S if you want these as well

            :0
            * date ?? ^^()\/....
            {
                YYYY = $MATCH
            }

            :0
            * date ?? ^^..\/..
            {
                YY = $MATCH
            }

            :0
            * date ?? ^^.....\/..
            {
                MM = $MATCH
            }

            :0
            * date ?? ()\/..^^
            {
                DD = $MATCH
            }

            TODAY   = "$YYYY-$MM-$DD" # ISO std date: like 1997-12-01

    10.11 Keep simple header log

        Here is a simple strategy: Record all what comes in and record all what
        happened to that message. See how brief info is constantly recorded to
        *BIFF* folder. You can now check the *BIFF* log every day to
        see if the messages were sunk to right folders: Remember to add *BIFF*
        rule to every recipe, so that the sink message [sunk-somewhere] is
        recorded after incoming message headers.

        I use this one-liner log in my Emacs window which is updated by
        `live-mode' process all the time (See the Emacs tools section
        later). It gives a nice overview of email messages the I'm receiving:
        it's my biff(1) equivalent in Emacs.

            # this requires that HH and MM have been setup before,
            # see pm-jadate.rc

            NOW     = "$HH:$MM"            # the time only
            TODAY   = "$YY-$MM-$DD $NOW"   # ISO 8601: date and time

            $NULL   = $SPOOL/junk.null.spool    # /dev/null is dangerous
            BIFF    = $PMSRC/pm-biff.log

            # or if you prefer a log per day (easy for cleanup):
            # BIFF   = $PMSRC/pm-biff.log.$YYYY$MM$DD

            # .............................................. headers ...

            # DON'T USE THESE: they call shell
            #
            # FROM    = `$FORMAIL -zxFrom:`
            # SUBJECT = `$FORMAIL -zxSubject:`

            :0                    # Use procmail match feature
            * ^From:\/.*
            {
                FROM = "$MATCH"
            }

            :0                    # Use procmail match feature
            * ^Subject:\/.*
            {
                SUBJECT = "$MATCH"
            }

            # ............................................. incoming ...
            #  record log of incoming mail

            # or if you use a biff file per day, you could have:
            # echo "$NOW $FROM $SUBJ" >> $BIFF

            :0 hwic:
            |  echo "$TODAY $FROM $SUBJ" >> $BIFF

            # ......................................... null recipe ...
            # spam-like addresses - let friends@planetall.com fall through

            :0 :
            * From:.*(remove|delete|free|friend@)
            * ? echo "  [null-AddrReject]" >> $BIFF
            $NULL

    10.12 Gzipping messages

        [Sean B. Straw ] On the recipe
        delivery line where you'd normally be tossing it into a folder do
        this instead:

            :0 c:
            |gzip -9fc >> $MAILDIR/mail.mbox.gz

        This will compress each message as it comes in (and since most are
        TEXT, it does a fine job - MIME, OTOH is one of the best ways to
        mailbomb someone since it doesn't compress well - but the indirect
        bombing via mailing lists doesn't do this), reducing the disk space
        required, usually dramatically. Done in conjunction with something
        like the following at the end of your .procmailrc, you could have a
        header file you could quickly rummage through looking for valid
        messages to add to a procmail recipe, then run:

            gzip -d -c mail.mbox.gz | formail -s procmail -m recipe.rc

        (note that if the recipe delivers into the mail.mbox.gz file on any
        condition, then you should look to MOVE the file before running
        this process, and use the moved version. In fact, this would be a
        good idea anyway, as newly delivered mail may appear in the end of
        the gzip file while you're doing this - and since your ultimate
        goal is to be able to eliminate junk, you'll want to know that
        after you've processed a gzipped mail file, you can delete it
        without accidentally whacking new mail).

            :0
            * LASTFOLDER ?? ^^^^
            {
                # Save the message in case we need to retrieve it.

                :0 c:
                |gzip -9fc >> $MAILDIR/mail.mbox.gz

                # copy headers for easy browsing - including being able to
                # identify lists you're being subscribed to.

                :0 h:
                header.log
            }

    10.13 Emergency stop for your .procmailrc

        [jari] If I have a bad luck while I am testing a new recipe, it may
        run in a loop and and it may send me continuously email messages. I
        then have to quickly recall .procmailrc and start disabling my
        individual "control" recipe files. Yet I figure, in situations like
        this where every second is important, there must be a better way.
        [alan] This is quite easy already; put this at the top of your
        .procmailrc:

            #   instead of leading dotfile, you may prefer
            #   stopFile = $HOME/procmailrc.stop which shows up in default ls.
            #   In the other hand you can do ls ~/.procmail* to see both...

            stopFile = $HOME/.procmailrc.stop

            :0
            *$ $IS_EXIST $stopFile
            {
                EXITCODE = $EX_TEMPFAIL # Means: retry later; requeue
                HOST     = "_stopped_by_external_request_"
            }

        Then, when testing your procmailrc and disaster happens, you can
        simply do following to disable your procmailrc filtering.

            % touch $HOME/.procmailrc.stop

        [richard] This is also a candidate recipe for including in
        an INCLUDERC. Combining the two ideas, we have a file
        procmailrc.stop which contains the recipe and is included near the
        top of .procmailrc, When you don't want it, mv it to procmailrc.go.
        Procmail complains about missing INCLUDERCs, but it does not
        complain about them if they exist and are empty. Another reason to
        not use dotted file names, but to use cp instead of mv.

11.0 Scoring

    11.1 Using scores by an example

        First make all the needed matches and let the `SCORE' value to be
        set. Examine the score after the final value has been calculated.
        The condition lines say:

        o   Start with some threshold: -250.
        o   Read the subject into `MATCH'
        o   Add 50 for each match of !. Notice the "^1": if it read
            "^0", only one 50 would be added for "!!!!", now that counts
            as 4 x 50 = 200. See procmailsc(1) for "^N" syntax.
        o   Any dollar sign is likely spam.
        o   find uninteresting subject words
        o   And a negative count for replies.
        o   Usually spam doesn't seem to have Re: in subject field.
            (but don't rely on this, spammers have started to use "re:")
        o   letters such as !!! frequently found in the body are usually
            indication of spam. Add 100 for each match.

            # Idea by 26 Sep 97 Stephane Bortzmeyer 

            :0
            *     -250 ^0
            * ^Subject:\/.+$
            *       50 ^1    MATCH ?? [!]
            *       50 ^1    MATCH ?? [$]
            *      100 ^1    MATCH ?? ()\<(free|sex|opportunity|money|great)\>
            *     -250 ^0   ^Subject: *(Fwd|Fw|re):
            * B ?? 100 ^0    ()!!!
            { }             # official procmail no-op

            SCORE = $=      # Score has been calculated

            :0 fhw
            | $FORMAIL -i "X-Spam-Score: scored $SCORE"


            :0:             # If score had positive value, sink message
            *$ $SCORE^0
            junk.spam.mbox

        Given the following subject:

            "Great opportunity for free sex; no money required!!!!"

        procmail scores it this way: ! was found 4 times (200/weight 50),
        "free|sex..." regexp matched 4 times (400/weight 100).

                     condition score    Total sum so far
                                ----    ----------------
            procmail: Score:    -250    -250 ""
            procmail: Score:     200     -50 "[!]"
            procmail: Score:       0     -50 "[$]"
            procmail: Score:     400     350 "^Subject:.*\"
            procmail: Score:       0     350 "^Subject: *(Fwd|Fw|re):"
            procmail: Score:       0     350 ! ""
            procmail: Assigning "SCORE=350"

        [david] Some notes on possible regexps and their differences:

            * 100^1 ^Subject:.*\<(free|sex|opportunity|money|great)\>

        That condition says to score 100 for every subject line that
        contains any of those five words ... not to score 100 for every one
        of those words in the subject, but 100 for every subject line that
        contains any of those words. So it will never score more than 100
        unless there are multiple subject lines. You see, it offers five
        alternative regexps:

            ^Subject:.*\
            ^Subject:.*\
            ^Subject:.*\
            ^Subject:.*\
            ^Subject:.*\

        Offhand, I think regexp below would score 400: 100 for
        "Subject.*free" and 100 for "sex" etc. Of course, the score might
        be higher if other lines in the header included the strings "sex",
        "opportunity", "money", or "great", but appearances of
        "free" outside the subject wouldn't be counted.

            * 100^1 ^Subject:.*\

            [translates to]

            ^Subject:.*\

        And this one would score 400 too. How? `MATCH' would contain whole
        subject and there would be non-overlapping matches to " great ", "
        opportunity ", and " free ". If we got rid of either or both of the
        word-border marks, it would score 500.

            Subject: Great opportunity for free sex; no money required!!!!
            * 100^1 MATCH ?? ()\<(free|sex|money|opportunity|great)\>

    11.2 Brief Score tutorial

        #todo: test

        [elijah] If you're serious about using scores, please spend
        a minute reading this short example.

            VERBOSE = "yes"

            :0
            *  1^1 foo
            * -2^2 bar
            { }
            a = $=

            :0
            *  1^1 foo
            * -2^2 bar
            {
                :0 f
                | echo Whee: fun ; cat -
            }
            b = $=

            :0
            *  1^1 foo
            * -2^2 bar
            {
                whee = "fun"
            }
            c = $=

            :0 h
            /dev/null


        Then if you would send a message

            From foo Fooof
            To: bar
            Subject foobar

            body-something-here

        The log file will tell you what happened.

            procmail: [20175] Fri Sep 26 10:25:23 1997
            procmail: Score:       3       3 "foo"
            procmail: Score:      -6      -3 "bar"
            procmail: Assigning "a=-3"
            procmail: Score:       3       3 "foo"
            procmail: Score:      -6      -3 "bar"
            procmail: Assigning "b=0"
            procmail: Score:       3       3 "foo"
            procmail: Score:      -6      -3 "bar"
            procmail: Assigning "c=-3"
            procmail: Assigning "LASTFOLDER=/dev/null"
            procmail: Opening "/dev/null"
            From foo Fooof
              Folder: /dev/null 46

    11.3 Score's scope

        If you have a delivering recipe and the score is positive, the
        action lines are executed. If the score is less or equal to 0, then
        the `$=' information is lost, but also at the next recipe
        definition, even if the recipe is never executed. Study following
        example:

            :0
            * 10^0
            {
                dummy = "Score for condition xxxx was: $= $NL"

                :0
                {
                    dummy = "Next recipe, Score no longer available: $= $NL"
                }
            }

            #   Wont' work.  $= is getting set back to 0 outside of
            #   the delivering recipe.

            dummy = "Score outside of all recipes: $= $NL"

        Here is interesting anomaly which [richard] discovered. It is
        presented here only as a curiosity. DO NOT USE IT IN YOUR RECIPES.
        (this not "clean programming", but a hack)


        [david] If you want to save the score for later use (even if it is
        zero or negative):

            :0
            * 10^0
            { }                     # procmail no-op

            SCORE = $=

            :0 A
            action_if_positive

        If other recipes that clobber the references for the `A' flag
        intervene, this will work:

            :0
            * 10^0
            { }                     # procmail no-op

            SCORE = $=

            ... more stuff ...

            :0
            *$ $SCORE^0
            action_if_positive

    11.4 Counting length of a string

        Supposing `VAR' contains some text, we can count the characters
        by using dot to match every character and increasing score for
        every match.

            :0
            * 1^1 VAR ?? .
            { }

            LENGTH = $=

    11.5 Counting lines in a message (Adding Lines: header)

        [1995-10-03 PM-L Idea by David Karr ] [david]
        later corrected 1998-01-02: For one thing, the second condition
        always counts one too many (the final newline plus the closing
        putative newline create the extra match); second, after making that
        correction, an empty body would score zero and leave the variable
        undefined.

            :0
            * 1^1 .
            * 1^1 ^.*$
            * -1^0
            { }
            lines = $=

            :0 fhw
            * ! ^Lines:
            | $FORMAIL -a "Lines: $lines"

        The reason we used it at all was that size conditions worked only on
        the entire text regardless of H or B or HB flags at the top of the
        recipe. Nowadays we can do this and get the accurate figure in one
        condition:

            # leave `B ??' out to measure the entire message
            :0
            * 1^1 B ?? > 1
            { }
            size = $=

        If you want to be silly about it (as some of us very often do),

            :0
            * -1^1 B ?? > -1
            { }
            size = $=

        gives the same result, and as long as the search area is non-empty,
        so do these, which are even sillier:

            :0
            * 1^-1 B ?? < 1
            { }
            size = $=

            :0
            * -1^-1 B ?? < -1
            { }
            size = $=

        [Karr] This recipe counts bytes in the message, you could use this
        Content-length replacement, prefer using the next recipe. The first
        score counts every character, and the second score sums up every
        line (that is: newlines are added).

            :0 HB                       # use B to measure body only
            *    1^1 .
            *    1^1 ^.*$
            {
                textsize = $=

                :0 fhw
                * ! ^Content-length:
                | $FORMAIL -a "Content-length: $textsize"
            }

    11.6 Determining if body is longer than header

            :0
            *  1^1 B ?? > 1
            * -1^1 H ?? > 1
            {
                ..body was longer
            }

    11.7 Matching last Received header

        [david] Here is way to use scores to hit the bottommost `Received'
        header.

            :0
            * $ 1^1 ^Received:.*by$s+\/.*
            action

    11.8 How to add Content-Length header

          We use procmail for local delivery, and would like to get it
          to generate the content-length header, if one doesn't exist. SUN-OS
          mailtool at least gets confused and merges messages together if
          there is no message body.

        [stephen] All you need to do is: a) Make sure that procmail is started
        without the -Y flag. b) Either, in your sendmail.cf, insert:

            H?l?Content-Length: 0000000000

        Or (slightly less efficient), insert the following recipe in your
        /etc/procmailrc file and Procmail will take care of any necessary
        magic.

            :0 hfw
            * !^Content-Length:
            | /usr/bin/formail -a "Content-Length: 0000000000"

    11.9 Testing message size or number of lines

        Size conditions ignore `H' and `B' on the flag line and always work on
        `HB' unless another search area is specified on the condition's own
        line. To test *only* the body,

            :0                      # Note: this is in BYTES
            * $ B ?? < $NBR
            {
                ...whatever when fewer bytes
            }

        This syntax would obey a B flag on the flag line:

            :0 B                    # Note: this is in LINES
            * -1^1 .
            * -1^1 ^.*$
            *$ $NBR^0
            {
                ...whatever when fewer lines
            }

    11.10 Counting commas with recursive includerc

        [jari] Foreword: David and Phil really are experts with procmail,
        and let this section serve as an example to "what on Earth is
        recursive procmailrc and how it is used?". I would not personally
        use recursive includerc, simply because I would not trade clarity:
        I find this easier to understand and maintain. `split' just
        explodes input according to comma and the `print' return how many
        elements were exploded to array `a'. The performance hit is not
        bigger than forked `procmail' binaries in recursive version.

            :0
            * ^CC:\/.*
            {
                field       = $MATCH

                saved       = $SHELLMETAS
                SHELLMETAS
                commaCount  = `echo $field | awk '{print split($0,a,",")}' `
                SHELLMETAS  = $saved
            }

        See the recursive RC implementation at
        

        [richard] Here is recipe that needs no recursion. MAX_RECIP
        is set to 9, but you may prefer some other value. This counts each
        comma. It allowed in addresses.Some folks sum Resent-xx *or*
        non-Resent-xx headers. I sum all.

            :0
            *             1^1 ^(resent|apparently-)?(to|b?cc):\/.*
            *             1^1 MATCH ??,
            * $ -$MAX_RECIP^0
            {
                :0
                * $         $=^0
                * $ $MAX_RECIP^0
                {
                    RESULT = "Count of commas is $="
                }
            }

12.0 Formail usage

    12.1 Fetching fields with formail -x

        If you're new to procmail your first though to read a header
        content from the message would might be call:

            SUBJECT = `$FORMAIL -xSubject:`

        That's not good. DON'T Do THAT. You just created expensive shell
        subprocess where procmail calls `formail' and feeds full message to
        it. We can do the same with minimum efforts:

            :0
            * ^Subject:\/.*
            {
                SUBJECT = $MATCH
            }

        No shell subprocess called. This is 10x faster and consumes fewer
        resources, while it may need more typing. Use it and your your sysadm
        is happy with your well behaving procmail recipes that don't load
        the CPU unnecessarily. There are cases where you need to launch
        `formail', eg let's derive the address of the sender and store it
        to variable:

            FROM = `$FORMAIL -rtzxTo:`

        But you can still make this more efficient. Here is one example where
        you actually want to use "old" `=|' style variable assignment,
        make sure there are _no_ extra spaces:

            :0 hw
            FROM=|$FORMAIL -rtzxTo:

        That way only the _header_ gets fed into formail, whereas the
        previous backtick fed the _whole_ message. Also, you can then check
        the return code of formail with `a' or `A' recipe after this one.

    12.2 Always use formail's -rt switch

        [faq] `-r' breaks RFC822, so always use `-rt' if you don't know
        what this means. Perhaps you should always use it anyway.

        [david] There is formail -r[t] rank bar graph in the source code of
        3.11pre4. It might be easier to follow as a top-to-bottom listing
        (and again, Tom Zeltwanger appears to be using one of the older
        versions where From_ was mistakenly overpromoted). These are the
        rankings in version 3.11pre4:

            formail -r:                     formail -rt:

            Resent-Reply-To:                Resent-Reply-To:
            Resent-Sender:                  Resent-From:
            Resent-From:                    Resent-Sender:
            Return-Receipt-To:              Reply-To:
            Errors-To:                      From:
            Reply-To:                       Sender:
            Sender:                         Return-Receipt-To:
            From_                           Errors-To:
            Return-Path:                    Return-Path:
            Path:                           From_
            From:                           Path:

        [Stephane Bortzmeyer ] Always use `-rt' and
        never `-r'. Because such precedence (Sender over From) is an
        important violation of RFC 822. There is one canonical order,
        described in the RFC and nothing else should be used, like fuzzy
        ranking or, worse, reordering. This is a serious problem with
        formail.

        The proper order is:

            Reply-To, else From, else Sender, else 

          And, how would you deal with resent mail?? Ie: Resent-Reply-To,
          Resent-From, and Resent-Sender?

        It treats `Resent-X' as X (" Whenever the string `Resent-' begins a
        field name, the field has the same semantics as a field whose name
        does not have the prefix. "). So you have to choose an order between
        them, the RFC does not specify it.

        [david] I think that the idea is that `-r' is intended to determine
        the origination address, not the place to reply; `-rt' is for
        determining the place to send replies. For addressing a response,
        yes, `-rt' will invert the header in a way more in line with the
        rules; for figuring out the origination point,

            formail -r -zxTo:

        might be better than

            formail -rt -zxTo:

        And here's an additional problem: `formail' `-rD' always uses the
        `-r' precedences; you can't make it use the `-rt' precedences
        and the `-D' cache checking function at the same time.

         4.4.4.  AUTOMATIC USE OF FROM / SENDER / REPLY-TO (RFC 822 excerpt)

        For systems which automatically  generate  address  lists  for
        replies to messages, the following recommendations are made:

        o   The `Sender' field mailbox should be sent notices of
            any  problems in transport or delivery of the original
            messages. If there is no  `Sender'  field, then  the
            `From' field mailbox should be used.
        o   The `Sender' field mailbox should NEVER be used
            automatically, in a recipient's reply message.
        o   If the `Reply-To' field exists, then the reply should
            go to the addresses indicated in that field and not to
            the address(es) indicated in the `From' field.
        o   If there is a "From" field, but no `Reply-To' field,
            the  reply should be sent to the address(es) indicated
            in the `From' field.

        Sometimes, a recipient may actually wish to communicate with the
        person that initiated the message transfer. In such cases, it is
        reasonable to use the `Sender' address.

        This recommendation is intended only for automated use of
        originator-fields and is not intended to suggest that replies may
        not also be sent to other recipients of messages. It is up to the
        respective mail-handling programs to decide what additional
        facilities will be provided.

    12.3 Using -rt and rewriting the From address

        Sendmail adds the `From' header which points to your account. But in
        some cases you may wish to rewrite the `From'.

        o   You respond to spammer and you want to hide in some extents your
            address. ( The headers will still be there, but at least
            hitting `r' in most MUA's pick up the `From' )
        o   You want to rewrite `From' to show your virtual address
            me@forever-lasting-address.com instead.
        o   You are in some other account currently, but you want to send
            message to some Net service (eg. Mailing list) that expects to
            see the same address you first time used in subscription.

        You could also use `Reply-To' to signify where you want further
        responses to go, but that doesn't hide your true `From' address. And
        there are still MUAs that don't obey `Reply-to'. Whatever reason
        you have to rewrite From header, here is the command.

            :0 fhw
            | $FORMAIL -rt -I "From: me@forever-lasting-address.com"

    12.4 Formail -rt and Resent-From header

        Here is something that made me scratch my head a lot. Let's examine
        scenario first which explains how the email travels.

            account --> virtual-address --> Local-address

        In this chain I was sending message from my University account to
        my official work address, the *virtual-address* delivers
        the mail to right local domain. There is only one problem with this
        picture. When I generated response from *Local-address* with
        `formail' `-rt', the generated address pointed back to
        *virtual-address, which pointed back to *Local-address* of
        course. A loopback was ready, I never got route travelling to
        original address *account*

        What was happening here was that the mailserver that handled the
        virtual-address, didn't *forward* the message, but instead
        *resent* the message. In this process a set of new headers were
        generated:

            Resent-From: 
            X-From-Line: 
            Received: from 
            Resent-Message-Id: <199710151903.WAA28670@virtual-address>
            Resent-Date: 
            Resent-To: 
            Received: ...
            Message-Id: <199710151904.WAA05050@account-domain>
            From: 

        And now when the formail -rt command was used, it picked up the
        `Resent-From' added destination where the message should be returned.
        Surprising, but according to procmail, 100% correct. `Resent-From'
        has higher priority than `From'.

        The Resent-* headers are considered *informative*, and should never
        be used when automatically generating a response. The problem here
        is the middleman, it should not _resend_ a message, but rather
        _forward_ it. So I put this into my .procmail to handle the broken
        middleman in our site.

            #   Remove that misleading Resent-From if it was added by our
            #   "middleman"

            :0 fhw
            * Resent-From: 
            | $FORMAIL -IResent-From:

        [edward] adds to this that: As you know, `formail' `-rt' is
        for composing a response to the address from which an e-mail was
        sent. Let's say you are on vacation and have set up a procmail
        recipe to autorespond to all e-mail you receive. Furthermore, let's
        say Joe sends me an e-mail and I re-send it to you. If you wanted
        to respond to the sender of the e-mail that you received, would you
        e-mail me or Joe? You better e-mail me because I was the one who
        sent it to you. Joe may not even know you. Imagine if you did send
        your response to Joe. It would probably cause him considerable
        confusion as to why you are sending him e-mail informing him that
        you are vacation.

        formail `-rt' uses a heuristic algorithm to determine who it should
        respond to, based on the presence of various headers and their
        contents. If you look at the formail.c source code, you'll see a
        graphical representation of this algorithm. It will also explain
        difference between the results of `-r' and  `-rt'.

        `Resent-Reply-To' has the highest relative importance/reliability of
        all header fields. Next is Resent-From and Resent-Sender, followed
        by Reply-To, From, Sender, et al.

    12.5 Quoting the message

        Use formail -rtk

    12.6 Without quoting the message

        Use formail -rkb or formail -rkt -p '' or formail -rkt -b

    12.7 How to include headers and body to the reply message

        [david] ...It does require that the entire head fit into sed's hold
        space, but it almost always will; exceptions are cases where the
        sender messed around and added a bunch of uninformative (and
        usually self-congratulatory) additional headers or when the message
        got caught in a loop for a while but finally escaped before being
        bounced for too many hops.

            :0 fhw r
            | sed -e H -e '$ G'

            :0 fhw
            | $FORMAIL -rt; ... now generate reply ...

    12.8 Adding text to the beginning of message

        We don't actually filter anything here. It's just a trick to
        reprint headers and add some text after them: text appears
        at the beginning of body.

            :0 fhw
            | cat - ; echo "This text comes after the headers."

    12.9 Adding text to the end of message

            :0 fb
            | cat -; echo "added text after body"

    12.10 Adding text before quoted message

        If you are generating an auto-reply message where you want to place
        the notification to the beginning of body followed by the quoted
        original message, here is recipe for it. Substitute *condition* to
        trigger the reply condition.

            :0
            * condition
            {
                :0 fhb
                | $FORMAIL -rtk -p '>'   \
                  -I "From: me@here.com" \
                  -I "$MY_XLOOP"

                :0 fhw
                | cat -; echo "added message at the start of body"
            }

   12.10 How to truncate headers (save filing space)

        [Idea by Rodger Anderson ] As a last
        recipe, if you're tight of space, you could remove extraneous
        headers. But make sure you want to that, because headers may
        contain useful information about URLs and other things like mail
        server addresses. Some people keep signature information in
        separate X-header (say: `X-My-Info') instead of at bottom of
        message so that it won't bother people and disturb reply quoting.

            #   Strip header to bare minimum
            #   If this is MIME multipart, then skip recipe

            :0 fhw
            * ! multipart
            |   $FORMAIL -k                                                 \
                -X Date:                                                    \
                -X Subject:                                                 \
                -X Message-Id:                                              \
                -X From                                                     \
                -X To:                                                      \
                -X Cc:                                                      \
                -X Reply-To:                                                \
                -X Mime-Version:                                            \
                -X Content-type:

            :0 :
            mail.default.mbox

        [david] comments the final recipe

        o   You should keep the `Reply-To' header if there is one. If the
            sender wanted replies directed to a different address than that
            in the `From' header, you are losing that information and, when
            you respond, writing to the wrong place.
        o   You ought to keep `To' and `Cc' so that you can tell when you read
            your mail who else was sent it. If your mail user agent has a
            groupreply or replyall function, keeping `To' and `Cc' will allow
            that feature to continue working. This way you are cheating
            yourself out of it.
        o   '-X From' is enough to keep both the `From_' line and the `From'
            header. You don't need to specify -X From: again after it.
            (To keep `From_' without `From:' you need to say -X "From " or
            something similar, with a quoted space.)
        o   All mail is going to have a line (usually two) beginning
            'From'.

        Another slightly different approach is to kill the headers that
        take the most of the space. If you're not interested in tracking
        down the original sender of possible UBE message, then you can
        remove the `Received' headers. You may want to fill out the
        condition line to simplify only your work or campus messages,
        and let other messages retain their full headers.

            :0  fhw
            *   possible-condition-to-handle-only-certain-messages
            |   $FORMAIL -I Received:

    12.11 Adding extra headers from file

        [stephen] Notice that the obvious solution won't do here:

            :0 fhw
            * condition
            | $FORMAIL -rt | cat - $HOME/newHeaders

        The problem here is that there will be a newline in the middle, which
        causes the header to be shortened (procmail determines the new
        header/body boundary after having processed each filter). Use the
        following instead.

            :0 fhw
            * condition
            | $FORMAIL -rt -X "" ; cat $HOME/pm-newHeaders.txt ; echo

        [david] If $HOME/newHeaders ends in a blank line, you don't
        need the "; echo".  Under some circumstances procmail puts back the
        blank separating line if it gets lost, but I'm not sure exactly what
        those are, and you have a SHELLMETAS character in there already (the
        first semicolon), so a shell is forked anyway.

        But this is my favourite way (it assumes that formail -r will never
        generate a continuation line for From:); if you use it, make sure
        that the newHeaders file does NOT contain a trailing blank line:

            :0 fhw
            * whatever
            | $FORMAIL -rtn

              :0 A fhw
              | sed "/^From:/r $HOME/newHeaders"

    12.12 Splitting digest

        [Idea by David Hunt] One interesting idea to handle digests
        automatically as single messages if that we call procmail
        recursively. First Call formail to split the mail when headerfields
        are contained in the body, calling procmail again as the
        output-program of formail. insertion of X-Loop makes it possible to
        reuse .procmailrc for the separate messages.

            #   If it looks like more than one mail, send to formail for
            #   splitting, then send back to procmail for sorting again.

            :0 B
            *  ^From [-_+.@a-z0-9]+  (Sun|Mon|Tue|Wed|Thu|Fri|Sat)
            *  ^From:
            *  ^TO
            *$ ! H ?? ^$MY_XLOOP
            | $FORMAIL -A "$MY_XLOOP" -m4s procmail

    12.13 Mailbox: Splitting to individual files

        [david] To split some old email archives into individual
        files while stripping unimportant header fields, use following. The
        keys are to use procmail's `-p' option, to strong-quote `$FILENO' in
        the setting of `DEFAULT', and to use /dev/null or a known empty file
        as the rcfile.

            % setenv FILENO 0000
            % formail -kXDate: -XFrom: -XTo: -XSubject: -XIn-Reply-To:      \
                -XX-Mailer +1ds                                             \
                procmail -p DEFAULT=`pwd`/'$FILENO.txt'                     \
                /dev/null < inputfile

    12.14 Mailbox: Extracting all From addresses from mailbox

        The -ns causes formail to split the mailbox and feed each mail
        separately to next process.

            % cat mailbox | formail -ns formail -xFrom: | sort -u

    12.15 Mailbox: Applying procmail recipe on whole mailbox

            % cat mailbox | formail -ns procmail pm-experiments.rc

    12.16 Mailbox: run series of commands for each mail (split mailbox)

          ...Maybe the heat has melted my brain, but I can't seem to get
          formail to perform a series of commands on each mail that it has
          split from a folder. Here's an example of a simple debugging
          attempt: I've tried parentheses, putting the commands into a
          shell function, and other flailings too numerous to remember, all
          to naught.

            % formail -s addr=`formail -XFrom: | formail -r | formail -zx To`;\
                echo "$addr" >>output

        It appears that formail doesn't use the shell when executing the
        command specified when splitting. No SHELLMETAS here. Given that, the
        secret is to fire up the shell explicitly yourself to do the piping:

            % formail -s sh -c 'formail -XFrom: | formail -rzxTo:' >> output

        Note that you only need two formails in the pipe, not three, as the -r
        flag works correctly when combined with other flags.

          ...To me, a large mailbox would consists of about 10,000 messages
          per month (that's about what I get). That would mean that my
          mailbox would contain 60,000 messages in 6 months. I sure as heck
          wouldn't want to skim through it all or even try to load it up in
          an MUA.

        [1998-08-27 Bennett Todd ] I also deal with monster
        volumes of email. I've switched over entirely to Maildir in all my
        email handling; the only place I still see mboxes is in the save
        folders of my netnews reading (using slrn) and whenever I want to
        process them I either convert them into Maildir (e.g. for archival)
        or simply split them into multiple messages. Splitting into
        multiple messages turns out to be preposterously easy; using GNU
        csplit:

          [richard] The csplit invocation shown here will catch
          occurences of ^From embedded in the message body if your MUA
          hasn't escaped them with a >. Some MUAs use content-length
          headers and don't escape ^From. Procmail supports this. Be cautious
          if you choose to use this simple split.

            csplit -n4 - '/^From /' '{*}'

        That will create an empty xx0000 which I delete, and leave the
        messages in files named xx0001, xx0002, etc. If you have more than
        9999 messages in a folder then go -n6, or -n9, or whatever. Once
        they're split it's really easy to use shell tools to bundle
        messages into batches, file them into categories, etc.

        If you are archiving all email traffic forever (which I do) then
        another dandy tool to add to the mix is glimpse
        http://glimpse.cs.arizona.edu/ it takes a while to build the index,
        but that's a fine job to run out of cron at night. Once the index
        is built it's a pleasingly quick way to root through big archives
        of messages.

    12.17 Option -D and cache

        [Bob Weissman ] and [stephen] These files are
        self-limiting. The number after the `-D' is the size in bytes above
        which the older entries will be removed. E.g., my .procmailrc has

            :0 Wh:  .msgid.cache$LOCKEXT
            |$FORMAIL -Y -D 12288 .msgid.cache

        And the file never exceeds 12288 bytes by very much. Though
        formail indeed exceeds this size by as much as the length of one
        `message-ID', the file size should never grow significantly beyond
        that, even if used indefinitely. The file is in binary format, each
        entry terminated by single null byte, and an occasional (significant
        placeholder) double null

        [philip] The format of the cache is initially as follows:

            entry\0entry\0entry\0\0

        When the file size grows to equal-to or greater-than the size
        specified on the command line, formail starts over at the
        beginning, using a double-null to mark where it stopped. However,
        entries after the double-null, except for the partially overwritten
        one, are still valid and checked, so that the file is then in the
        format:

            entry\0entry\0entry\0\0partial-entry\0entry\0entry\0\0

        New entries will be written after the first double-null, so that it
        implements a circular cache. Check out lines 319-322 of formail.c

    12.18 Option -D and message-id in the body

          Some of my messages contain the original Message-ID in the body
          of the letter and not the Header. Is there an option for Formail to
          over come this problem?

        [david] This is strictly untested; I don't know where in the
        body the Message-ID's appear, but if they're at the top of the body,
        this might help:

            :0 hW        # Message-Id: in the head,
            *$ ^Message-Id:.*$NSPC
            | $FORMAIL -D $cache_size $cache_name

            :0 E BbW    # If not but there's one the body, try body.
            *$ ^Message-Id:.*$NSPC
            | $FORMAIL -D $cache_size $cache_name

        You might want to copy a `Message-Id' from the body to the head in
        any case (if there's none already in the head) just to have it in
        the right place, so we could do that first and then `formail' `-D'
        will work normally. This form will run formail twice if the
        `Message-Id' header is in the body instead of the head, but it will
        look for `Message-Id' on *any* line of the body, not just at the
        top:

            :0 fhw
            *$ ! H   ?? ^Message-Id:.*$NSPC
            *$   B   ?? ^\/Message-Id:.*$NSPC
            | $FORMAIL -A "$MATCH"

            :0 hW
            | $FORMAIL -D $cache_size $cache_name

    12.19 Reducing formail calls (conditionally adding fields)

        #todo: url

        Suppose you want add fields to the message when some condition is met:

            :0              # compose initial reply
            | $FORMAIL -rt

            :0
            * condition1
            | $FORMAIL -A "X-Header1: value1"

            :0
            * condition2
            | $FORMAIL -A "X-Header2: value2"

        Hm, we have three processes called here, can we minimize the calls?
        Yes, this is idea from [philip] and [david]. Notice that there is
        only ONE process needed.

            :0
            * condition1
            {
                hdr1 = "-AX-Header1:value"
            }

            :0
            * condition2
            {
                hdr2 = "-AX-Header2: value"
            }

            :0 fhw
            | $FORMAIL -rt  ${hdr1+"$hdr1"} ${hdr2+"$hdr2"}

        And if you want to stack all headers to only one variable, it is
        a bit of extra work. Below we use short variable names only because
        of the line space: the calls fit on one line.

        o   field  = all (f)ields stacked to one string.
        o   nl     = continuation newline terminator of previous field

        The recipe says: if *field* has previous value, set `nl' to newline
        separator, later concat previous contents of *field* with possible
        newline and new header field.

            field       # kill variable
            :0
            {
                nl
                nl     = ${field+"$NL"}
                field  = "$field${nl}X-Header1: value"
            }

            :0
            {
                nl
                nl     = ${field+"$NL"}
                field  = "$field${nl}X-Header2: value"
            }


            :0 fhw                          # If we have something in *field*
            * ! field ?? ^^^^
            | $FORMAIL ${field+-A"$f"}

        The above recipe was the most general one, each recipe determined
        by itself if the *f* existed previously or not. But if you know
        that *f* is already set, you can write simpler recipe:

            :0          # We know f has value before our module
            {
                field = "$field${NL}X-Header1: value"
            }

    12.20 Formail -A -a options

        You can't use option -A with -a or -I if the header name is the same.
        Like below where you try to keep only the last definition of X-1,
        but the first -A isn't seen when -a is applied.

            formail -A "X-1: 1" -a "X-1: 2"
            -->
            X-1: 1
            X-1: 2

        Whereas; separate pipes give you the desired results.

            formail -A "X-1: 1" | formail -a "X-1: 2"
            -->
            X-1: 1

            formail -A "X-1: 1" | formail -I "X-1: 2"
            -->
            X-1: 2

    12.21 Formail -e -s options

        [david] I had a file of alternating `From' and `Date' lines and
        wanted to convert it into an mbox.

            formail -dem2 -s < input > mailbox

        should have done it, right? Nope; `formail' `-s' took it all as one
        message, even with -m1. When I edited in blank lines, the command
        worked. My first reaction was that the -e option wasn't working as
        advertised and that the blank lines were necessary after all.

        Then I realized the real problem: there was no interruption in the
        succession of valid header lines in the input for anything that
        could look like a body. I could have put something other than blank
        lines between each pair of headerfields and then -e would have done
        its job, but as long as every additional line looked like a valid
        RFC822 headerfield, even if its name was the same as one that had
        appeared earlier, `formail' `-s' assumed that it was still the same
        message's head.

13.0 Saving mailing list messages

    13.1 Using subroutine pm-jalist.rc to detect mailing lists

        Because I didn't have sendmail plus addressing capabilities
        (explained in next section) I wrote module *pm-jalist.rc*. It
        is included in the pm-code.zip

        The subroutine tries to detect and derive the mailing list name
        directly from the message. Many Mailing daemons: ezlm, smarlist,
        listserv, majordomo use standardized headers from where the list name
        can be picked. After this subroutine has been applied to message,
        the variable `LIST' contains the mailing list name. You no longer
        have to manually insert separate recipes for each new mailing list
        you subscribe to, because this subroutine adaptively finds new new
        mailing lists.

        Once the mailing list name has been grabbed, you can easily "map"
        or convert the name to any suitable folder name before saving it:

            LIST            LIST name    Description of mailing list
            (as grabbed)    you want
            --------------------------------------------------------------
            jde             java.jde    Java Development Env
            java            java.prog   Java programming
            FLAMENCO        flamenco    Flamenco music
            tango-l         tango       Argentine Tango dancing
            tm-en-help      tm-en       Emacs TM mime package mailing list
            w3-beta         w3          Emacs WWW mailing list


        You set then conver grabbed `LIST' to new folder name with
        conversion table:

            JA_LIST_CONVERSION = "\
            jde       java.jde,\
            java      java.prog,\
            FLAMENCO  flamenco,\
            "

        And to detect all mailing lists, you only need one recipe, like
        below:

            INCLUDERC = $PMSRC/pm-jalist.rc

            :0 :                          # if list name was grabbed
            * ! LIST ?? ^^^^
            $LIST_SPOOL_DIR/list.$LIST

    13.2 Using plus addressing foo+bar@address.com

        If you have a recent enough (8.8.8+) `sendmail', please ask your
        sysadm to activate the plus addressing. Procmail gets `bar' in `$1'
        automatically.

        http://www.qz.to/~eli/faqs/addressing.html
        http://www.faqs.org/faqs/mail/addressing/

        [Bennett Todd ] The PLUS feature has also been
        Implemented in *qmail* and *Postfix* (nee VMailer). By default
        qmail uses "-" rather than "+", but it can be configured to use
        different rules; Postfix doesn't come with either enabled, but its
        example main.cf has a commented-out line to enable "+"-based
        support.

        [Roy S. Rapoport ] Plus addressing is
        implemented using sendmail (well, I'm sure the other MTAs can also
        do it, but my experience is with sendmail). The last few releases
        of sendmail (8.8.6, 8.8.7, 8.8.8) all seem to automatically default
        to allowing it. Basically, for any address of the form `foo+baz',
        sendmail ignores the `+baz' part and just delivers it to foo.

        If you want the easiest method to handle mailing list mails, then
        subscribe to list by using dedicated plus address:

            login+list.procmail@example.com
            login+list.debian@example.com
            login+list.linux@example.com

        When you receive message from any of these mailing lists to your
        `login' account, the `list.procmail' is already in variable `$1' and
        the recipe to sink all mailing lists to their individual folders is
        very simple:

            #   Note: The $1 contains value only _IF_ procmail
            #   is invoked with option -m or -a (with an argument).
            #   Be sure procmail is invoked with that oprion either as from
            #   LDA or ~/.forward.
            #
            #   $1 is pseudo variable and it can't be used in condition line,
            #   so we copy the value to ARG.

            ARG = $1

            :0 :
            * ARG ?? list
            $ARG

        [david] Here is what I have configured to sendmail.cf to support
        plus addressing:

            Mprocmail, P=/usr/bin/procmail, F=DFMmShu,                      \
                            S=11/31, R=21/31,                               \
                            T=DNS/RFC822/X-Unix,                            \
                            A=procmail -m $h $f $u

        Well, this is definition of the procmail mailer, not the local
        mailer. Furthermore, there's more to plus-addressing support than
        the definition of the local mailer. Ruleset 0 or 5 needs to be set
        up to move everything after the + into the 'host' variable ($h).
        Unless you have a strong understanding of sendmail rulesets and
        rewriting rules, you should not attempt to add plus-addressing to
        your sendmail.cf, but instead just install the latest version of
        sendmail and use the m4 sendmail.cf generation tools with a .mc
        file that contains:

            FEATURE(local_procmail, `/usr/local/bin/procmail')

        plus whatever else your site requires.

            ...Ok, I corrected it. Well, here's what that looks like. I did
            look into the part about Ruleset 5 while trying it on
            originally. But all I could do was make sure that the
            plus-addressing section was there.

            Mlocal, P=/usr/bin/procmail, \
                            F=lsDFMAw5:/|@qSPfhn9, S=10/30,
            R/40,
                            T=DNS/RFC822/X-Unix,
                            A=procmail -Y -a $h -d $u
            Mprog, P=/bin/sh, F=lsDFMoqeu9, S=10/30, R/40, D=$z:/,
                            T=X-Unix,
                            A=sh -c $u

    13.3 Using RFC comment trick for additional information

        Recall from [rfc1036] that the preferred Usenet email address
        formats are following

              From: login@example.com
              From: login@example.com (First Surname)
              From: First Surname 

        I invented this idea after reading Eli's excellent faq about email
        addressing. Please read it (especially section 19.) before you
        continue in order to understand what I'm going to present.

        I have an account which does not support plus addressing and I was
        kinda jealous to everyone that could use this neat sendmail
        addressing scheme. The plus addressing helps so much better to deal
        with mailing list messages.

        But as it turns out, we can simulate in some extent plus addressing
        with pure RFC compliant address. We exploit RFC comment syntax,
        where comment is any text inside parentheses. According to Eli's
        paper, comments should be preserved during transit. They may not
        appear in the exact place where originally put, but that shouldn't
        be a problem. So, we send out message with following `From' or
        `Reply-To' line:

            first.surname@domain (First Surname+list.procmail)

        Now, when someone replies to you, the MUA usually copies that
        address as is and you can read in the receiving end the PLUS
        information and drop the mail to appropriate folder: `mail.procmail'.

       [About subscribing to mailing lists with RFC comment-plus address]

        It's very unfortunate that when you subscribe to lists, the comment
        is not preserved when you're added to the list database. Only the
        address part is preserved. I even put the comment inside angles to
        fool program to pick up everything between angles.

            

        But I had no luck. They have too good RFC parsers, which throw away
        and clean comments like this. Eg. procmail based mailing lists, the
        famous `Smartlist', use `formail' to derive the return address and
        `formail' does not preserve comments. The above gets truncated to

            first.surname@example.com

        Also many mailing lists send out messages as `Bcc', so your address
        is not even available in headers anywhere, neither is this nice RFC
        comment. Ah well, but this RFC comment trick works very well in
        private communication, virtually all MUAs copy whole contents of a
        `From' or `Reply-To' header to `To' header, preserving comments and
        you get the benefit of plus addressing. Here is procmail code
        to demonstrate reading the PLUS information from RFC comment-plus
        field:

            RC_EMAIL = $PMSRC/pm-jaaddr.rc      # Address explode module

            :0
            *$ To:\/.*
            {
                INPUT       = $MATCH
                INCLUDERC   = $RC_EMAIL         # Explore grabbed To address

                #  If COMMENT_PLUS was defined, module found "+"
                #  address which contained, say, "mail.procmail".
                #  Save it to folder.

                :0 :
                * $COMMENT_PLUS ?? [a-z]
                $COMMENT_PLUS
            }

        Pretty simple. And you can put anything inside RFC comment and do
        whatever you want with these plus addresses. _NOTE_: there are no
        guarantees that the RFC comment is preserved everytime. Well, the
        standard RFC822 says is must be passed untouched, but I'd say it is
        90% of the cases where mail is delivered from one server to
        another, it is kept.

        Example: if you discuss in usenet groups, you could use address

            first.surname@example.com (First Surname+usenet.default)
            first.surname@example.com (First Surname+usenet.games)
            first.surname@example.com (First Surname+usenet.emacs)
            first.surname@example.com (First Surname+usenet.linux)

    13.4 Simple mailing list handling

        [Peter S Galbraith ] I have used
        this in the past (by simply looking at the spool file and seeing the
        `From_' line of the message):

            :0 :
            * ^From debian
            list.debian.mbox

            :0 :
            * ^From procmail
            list.procmail.mbox

        Now, I collect specific high-volume mailing lists (like Debian) into
        their own spool files like above, and let other recipes catch all
        other mailing lists (like procmail and fvwm) into a single spool
        file with later rules:


            :0 :                                    # Majordomo lists
            * ^Sender: owner-\/[-a-zA-Z0-9_.]*
            list.$MATCH.mbox


            :0 :
            * ^X-Mailing-List: <\/[-a-zA-Z0-9_.]*   # SmartList lists
            list.$MATCH.mbox

        So Debian mailing list mail goes to Debian, procmail and fvwm mail
        go to maillists and mail addressed to me yet CC'ed to a list go to
        my main spool file.

    13.5 Archiving according to TO

        Traditional way to detect and save mailing list messages is:

            :0 :
            * ^TO()procmail
            list.procmail

            [and so on...]

        The following code will save the message to folders list.foo, list.bar,
        list.procmail when the name is in the TO address.

            #   generalised version
            #   By dattier@wwa.com (David W. Tamkin)
            #   cases desired for foldernames

            LISTS = "(foo|bar|procmail)"

            :0:
            * $ ^TO_()\/$LISTS
            * $ LISTS ?? ()\/$\MATCH
            list.$MATCH

    13.6 Using Return-Path to detect mailing lists

        [philip] For most mailing lists, a more accurate way to determine
        whether it came from the list is to examine the Return-Path:, From_
        or Resent-From: header. This catches messages from the list,
        regardless of whether they were To: the list, Cc: the list, or even
        Bcc: the list, something which doesn't show in the message at all.

        For instance, I refile message from the procmail mailing list using
        the following recipe:

            :0
            * ^Return-Path: + is not a bad idea, though because it's not a zero-width
        assertion but rather an actual character class, you have to strip
        it from the match

            LISTS  = "(foo-list|bar-list)"

            #   1) to get the match
            #   2) rematch sans the trailing \>
            #   3) Note: preserves capitalisation of the string

            :0
            * $ ^TO_()\/$LISTS\>
            * $ MATCH ?? \/$LISTS
            * $ LISTS ?? ()\/$\MATCH
            {
                M = $MATCH
                
            }

        [Era] gives this sample example to describe what happens above:

            VAR =  "MOO"
            what = "(moo|bar|baz)"

            :0                              # Search what from VAR
            * $ VAR ?? ()\/$what
            {
                #  Now; what is was that really matched, there were several
                #  choices: moo,bar,bar
                #  Beware: $MATCH must not contain regexp characters

                :0
                * $ what ?? ()\/$MATCH
                { }                         # no-op

                # Fine, New MATCH contains moo
            }

14.0 Procmail, MIME and HTML

    14.1 Mime Bibliography

       "List of annoying things that various MIME implementations do."
        ...The result is a sort of style guide for implementors of things that
        generate MIME. Feel free to send comments or contributions.
        http://www.cs.utk.edu/~moore/mime-style.html

    14.2 Mime notes

        

        [1998-07-28 PM-L Brett Glass ] MIME filename
        buffer overflow bug described at

            http://www.sjmercury.com/business/microsoft/docs/security0728.htm

        This bug is particularly insidious because it can be exploited via
        by spamming software and could impact millions of users in a very
        short time.

        Use procmail to plug the hole at the mail server, by truncating the
        excessively long file names in the MIME headers: eliminate the
        extra-long filenames, truncating them back to (say) 64 characters
        max? All that's required is to recognize header below and make sure
        that  is chopped to a reasonable size.

            Content-Disposition: attachment; filename=""

        [era] I believe that the problem isn't really that the
        filename is over the allowed length for some platform (Macintoshes
        allow something like 27 characters if memory serves) but a bug in
        how some particular email clients allocate memory for the file name
        string (but I am really only speculating here).

          ...So far Eudora, Netscape Mail, Outlook Express, and mutt (Unix)
          have all been found to have buffer overflow problems.
          (mutt-0.93.2i and up are fixed. A patch for 0.93.1 is available.)

    14.3 Software to deal with mime or html

        See also nearest Perl CPAN module, http://www.perl.org, site and
        CPAN/modules/by-module/MIME/

        There's also Unix program `munpack' to explode a MIME message
        to separate files. Use ftpsearch to find it.

        [MIME aware mail agents in Unix]

        See `mutt' that could handle HTML mail. (Pointer to Mutt
        mentioned below)

        All Emacs Mail agents can handle MIME if you install some of the
        mime handling packages: TM, SEMI, rmime.el. See
        http://www.bmrc.berkeley.edu/~trey/emacs/mime.html

    14.4 Mime content type application/ms-tnef

          ...A member of one of my mailing lists appears to be using
          Microsoft Mail. His messages to the list are usually accompanied my
          an encoded attachment like this one:
          "c:\eudora\users\steven@idma.com\attach\WINMAIL11.DAT" The message
          headers include the following clause: Content-Type: multipart/mixed;
          boundary="openmail-part-058c9f3d-00000001" This is driving people
          crazy. What is causing this and is there any way to make it stop?

        Most likely the sender is using Exchange (or Windows Messaging or
        Outlook97) and sent the messages in Rich Text Format. It puts the RTF
        message in an attachment called WINMAIL.DAT (application/ms-tnef). But
        this attachment is useless unless the recipient is also using
        Exchange.

        The sender can turn off the RTF option for messages to you. For more
        information, see: XCLN: Sending Messages In Rich-Text Format
        http://support.microsoft.com/support/kb/articles/q136/2/04.asp

    14.5 Trapping html mime messages

        [era] Here's a simple filter to throw out unwanted html that is sent
        by using mime. [jari] This recipe detects if the message is
        classified as mime *text/html* and junks it to separate folder. It
        does not change the message content. If you want to actually
        remove html or other attachments from the message, see
        `pm-jamime-kill.rc' in the module list.

            :0:
            *$ ^Content-Type:$s*multipart/(mixed|alternative);\
               $SPCNL*boundary="?\/[^;"]+
            * $ B ?? ^--$\MATCH\$([-a-z]+:.*)*Content-type:$s*text/html
            junk.html.mbox

        Some more examples can be found from section: 'Explaning  ^^ and ^'

    14.6 Complaining about html messages

        [Marek Jedlinski ]. This how I
        respond to html messages. In my `noHTML.txt' I politely explain
        why I don't appreciate receiving HTML email, and ask to resend the
        message as plaintext. What happens in the majority of cases is that
        the sender resends the same message again ("oh, it bounced, let's
        try again") and I assume they don't actually read my explanation
        since they just happily resend the HTML cr*p. It bounces again at
        which point they give up... Tough luck, I say ;)

        BTW, the above recipe is placed *after* mailing list mail gets
        sorted. When someone sends HTML mail to a mailing list I read, I
        just flame them in person

            TXT_NO_HTML = $HOME/noHTML.txt

            :0 BH
            *  ! ^FROM_DAEMON
            *$ ! ^$XLOOP
            *    ^Content.Type.+multipart.alternative
            *    ^Content.Type.+text.html
            {
                    LOG = "$NL --TRASH: multi-part HTML $NL"

                    :0
                    | ($FORMAIL                                             \
                          -rtk                                              \
                          -A "X-Mailer: Procmail Autoreply"                 \
                          -A "$XLOOP" ;                                     \
                        cat $TXT_NO_HTML                                    \
                        ) | $SENDMAIL
            }

    14.7 Converting HTML body to plain text

          _Note:_ Older lynx has security holes:
          http://ciac.llnl.gov/ciac/bulletins/h-82.shtml
          http://lynx.browser.org/

        The most popular solution to convert html body into plain text is to
        use `lynx'. Another more straightforward method is to use a perl one
        liner: it's quicker, easier to use with procmail but it doesn't pretend
        to know about HTML DTD. The recipe below should be taken with grains of
        salt: seeing HTML tag is no guarantee that the body "only" has html. A
        cautious recipe writer also watches for MIME multipart messages. (See
        `pm-jamime.rc' to draw some mime characteristics from message)

        This recipe has been written so that you can add more alternative
        html conversion scripts. You may even want to select the appropriate
        conversion for a message: eg. perl for unimportant ones.

        _Note_: This is oversimplified method of checking if body contains
        html. It would be probably a good idea to check mime headers which
        indicate html encoding here as well.

            :0 B
            * ()
            * ()
            {
                conversion = "lynx"     # or select this conditionally

                :0
                * conversion ?? lynx
                {
                    # In new lynx version you can read from stdin. If
                    # /dev/stdin doesn't exits try /dev/fd/0
                    #
                    # lynx -dump -force_html -nolist -restrictions=all \
                    #   /dev/stdin
                    #
                    #  Without a global lock on this, you have a chance
                    #  that two procmail instances will try to write to
                    #  msg.dump

                    file = "$HOME/tmp/msg.dump"

                    LOCKFILE = $file$LOCKEXT

                    :0 fbw
                    | cat > $file && lynx -dump $file

                    LOCKFILE

                }

                :0 E fbw
                | perl -0777 -pe 's/<[^>]*>//g'

            }

    14.8 Getting rid of unwanted mime attachments (html, vcard)

        Microsoft and Netscape MUAs are conquering the PC world and it's
        likely that you will receive messages from people that use this
        software. The unfortunate thing is that you receive the message in
        mime format:

            HEADERS
            --mime-boundary
            plain text
            --mime-boundary
            Some idiotic html (or other type) copy of the text
            --mime-boundary

        When you would like to see a traditional message in the format:

            HEADERS
            plain text

        Good news. There's a procmail module that addresses this problem. The
        module can kill any mime attachment and the predefined sets include
        typical cases:

        o   Microsoft Explorer has a bad habit of including 7k
            application/ms-tnef attachment to the end of message.
        o   Lotus Notes sends similar extra attachment.
        o   Microsoft Express sends a copy of message in html format in the
            attachment.
        o   Netscape's Mozilla sends a copy of message in html. See
            example. It Also sends annoying `vcards'.

        The module is called `pm-jamime-kill.rc' and included in Jari's
        `pm-code.zip'.
        (Note: #REF #procmail_module_list ;Procmail module list;)

    14.9 Sending contents of a html page in plain text to someone

        [timothy] Send an email with the subject: "GetPage:
        some.url.here/". And it comes back. Kurt Thams 
        also pointed out that lynx allows file:// protocol and since
        procmail is running as you, this would be a security risk.

            GetFile: ~user/.login

        We make the script safe here by forcing "http://$MATCH" and not
        simply using "$MATCH"

            :0
            *$   ^Subject:$s+GetPage:()\/.*
            *$ ! ^$MY_XLOOP
            |   ($FORMAIL                                                   \
                    -rt                                                     \
                    -I "Precedence: junk"                                   \
                    -I "Subject: Requested page: $MATCH"                    \
                    -I "$MY_XLOOP" ;                                        \
                 lynx -dump "http://$MATCH"                                 \
                )| $SENDMAIL

        [era] If all you need is to create a suitable MIME package, there
        are various MIME command-line utilities such as metasend (which is
        for interactive use, and so doesn't work very well with Procmail)
        and mpack you can try. If your needs are simple, you could even
        read up a bit on the MIME spec and generate the necessary headers
        and separators yourself (echo Content-Type: multipart/mixed etc etc
        etc). Conversely, if your needs are complex, get the Perl MIME
        package from CPAN and cook up your own tool. The MIME FAQ
        (especially part 6) is a good place to look for info.
        http://www.faqs.org/faqs/by-newsgroup/comp/comp.mail.mime.html

        [jari] See procmail module list at the beginning of this
        document for procmail based MIME file servers.
        (Note: #REF #procmail_module_list ;Procmail module list;)

15.0 Simple recipe examples

    15.1 Saving: MH folders -- numbered messages

        Hm. This is explained in the procmail man pages, but not very
        well. There are just one or two occasions where the man page tells
        how to create individual files instead of catenating messages
        to a folder. Notice the `/.' at the end of folder name

            :0
            * condition
            dir-folder/.

        [manual] When delivering to directories (or to MH folders) you
        don't need to use lockfiles to prevent several concurrently run-
        ning procmail programs from messing up.


          On a save to a directory, how does procmail determine what to put
          after $MSGPREFIX to complete the name of the file?

        [philip] It's the inode number of the file encoded in
        base-64 with the set of characters A-Za-z0-9-_, in reverse order.
        So, for example, the inode numbered 59699 would be encoded as
        follows:

            59699 = 51 + 64 * ( 36 + 64 * 14 )
            A=0, B=1, ..., N=13, O=14, ..., a=26, ..., k=36, ..., z=51,
            0=52, ...
            --> zkO

    15.2 Saving: to monthly folders

            # Use any date method mentioned previously to define variables
            # YYYY YY MM DD. Archive digests monthly

            :0 c:
            * ^From:.*\/mailing-list-digest@some.net
            {
                # Get the "mailing-list-digest" string, do not use following
                #
                #       MBOX = `echo $MATCH | sed -e 's/@.*//' `
                #
                # Because we really don't need those extra shell processes.
                # Procmail can derive the word 10x more efficiently

                :0
                * MATCH ?? ()\/[^@]+
                {
                    MBOX = $MATCH
                }

                :0 :
                $YYYY-$MM-$MBOX
            }

    15.3 Modifying: Filtering basics

        Pay attention to the `cat' command position in each recipe.

            :0 fbw
            | echo "This is a line of text _before_ the body"; \
              cat -

            :0 fbw
            | cat - ; \
              echo "This is a line of text _after_ the body"

            :0 fbw               # prepend text before the body
            | cat msg.txt -

            :0 fbw               # append text at the end of body
            | cat - msg.txt

            :0 fbwi              # replace the body with text from file
            | cat msg.txt

    15.4 Modifying: Squeezing empty lines around message body

        [david] Anything that replaces the body is going to require an outside
        process, even if it's only /bin/echo. In order to trim empty lines
        from the beginning of message and from the end of message, you can
        do this, if the entire body fits into `LINEBUF'

            :0 B fbw
            * ^^$*\/.(.|$)*.$
            | echo "$MATCH" # trailing extra newline intended

        If your version of cat is BSD-ish,

            # SysV's cat has a different meaning for -s and cannot do this

            :0 B fbw
            * $$$
            | cat -s

        otherwise, it can be done with a very simple sed filter:

            :0 B fbw
            * ^^($)|$$$
            | sed /./,/^$/!d

        Note that cat -s has slightly different results from the others: if
        there are any empty lines at the top of the body, cat -s will keep
        one. The echo and sed suggestion will remove all empty lines from
        the top and, like cat -s, keep one at the bottom.

    15.5 Modifying: shuffling headers always to same order

        [phil] To sort the headers in the message into predictable order,
        you can use following recipe. The spaces have been eliminated
        between the `-I' and its argument in the above. The shell may or
        may not allow unquoted spaces in the second part of the
        ${variable:+blah}. For example, under solaris 2.6, /bin/sh barfs on
        ${FROM:+-I "From: $FROM"}, while /bin/ksh handles it just fine. I
        think the POSIX shell standard requires that it be allowed, but,
        well, will your _next_ system be POSIX compliant?


            :0
            * ()\/^From: +\/.*
            { FROM = $MATCH }

            :0
            * ()\/^Reply-To: +\/.*
            { RT = $MATCH }

            :0
            * ()\/^X-Mailer: +\/.*
            { XM = $MATCH }

            :0
            * ()\/^Message-Id: +\/.*
            { MID = $MATCH }

            :0
            * ()\/^Date: +\/.*
            { DATE = $MATCH }

            :0
            * ()\/^To: +\/.*
            { TT = $MATCH }

            :0
            * ()\/^CC: +\/.*
            { CC = $MATCH }

            :0
            * ()\/^Subject: +\/.*
            { SUBJ = $MATCH }

            :0 fh w
            | $FORMAIL                                                      \
                ${XM:+-I"X-Mailer: $XM"}                                    \
                ${TT:+-I"To: $TT"}                                          \
                ${FROM:+-I"From: $FROM"}                                    \
                ${RT:+-I"Reply-to: $RT"}                                    \
                ${CC:+-I"Cc: $CC"}                                          \
                ${MID:+-I"Message-Id: $MID"}                                \
                ${DATE:+-I"Date: $DATE"}                                    \
                ${SUBJ:+-I"Subject: $SUBJ"}

    15.6 Service: Auto answerer to empty messages

        [elijah] Here is piece of code that responds to empty
        messages.

            :0 B
            * ! ...
            | (echo "From: me@here.com" ;                                   \
              $FORMAIL -r -A"Precedence: junk"                              \
              -A"X-Loop: me@here.com" ;                                     \
              echo "Your blank message was received.\n"                     \
                   "Did you mean to say something?\n"                       \
                   "\n"                                                     \
                   "-- \n"                                                  \
                   "My Signature\n"                                         \
                   "this has been an automated response\n"                  \
              ) | $SENDMAIL

    15.7 Service: File server -- send fileas as attachments upon request

        This section is here only for indexing purposes. The File servers
        are described in the #REF #procmail_module_list; Procmail module list;

    15.8 Service: Ping responder

        Sometimes I'm on the road and I don't seem to get access to the
        site where my messages are. The telnet connection fails and
        standard unix "ping" plays dead for me. "What's happening in that
        site?" I wonder. Here is a recipe that I have added to all of my
        accounts. It sends an immediate reply if at least the mailhost is
        up and gives some status information.

            :0
            * ^Subject: ping$
            {
                :0 fh
                | $FORMAIL -rt

                #   Remember, Don't send back anything that would be vital to
                #   attacker. It doesn't matter if the `uptime` or other
                #   scripts fail, the reply is sent anyway.

                :0 c    # Record this ping request
                |   ( cat -;                                                \
                      echo `uptime`;                                        \
                      echo "$HOST User count: " `who | wc -l`;              \
                    ) | $SENDMAIL

                :0 :                    # or sink to $DEFAULT
                $PING_SPOOL
            }

    15.9 Service: simple vacation with procmail

        Don't forget to look into procmailex(5) man pages which also has
        vacation example. The ones presented below may not work for you.
        Here is a very simple vacation recipe. Whenever the file ~/.vac
        exists, the vacation program is called. Be sure that you have the
        ~/.vacation.msg file ready too. Remember that `vacation' does not
        _save_ you messages; so we need `c' flag here.

            #  Some prefer the non-dotted file which shows up in ls listing

            vacationFlagFile = $HOME/.vac

            :0 wc
            *$ ? $IS_EXIST $vacationFlagFile
            |  vacation $LOGNAME

        Some people like to raise a flag in .procmailrc instead of creating a
        file. If you like the variable approach better, here is the equivalent
        implementation of the above

            VACATION = "yes"    # Comment this when not in vacation

            :0 wc
            * VACATION ?? yes
            | vacation $LOGNAME

        [philip] and [era] Since vacation only sends replies -- it
        never sends the original # messages, one way to do two things with
        your .forward file. Substitute "abc" with your login name.

            |/usr/ucb/vacation","exec /usr/local/bin/procmail -f- ||exit 75 #abc

    15.10 Service: vacation code example

        [By Eric Black ] Here is the procmail part

            OFFSITE = "my_guest_login@wherever.I.am"

            #  Forward urgent mail to me at my offsite address; afterward,
            #  continue processing it as normal The procmail pattern match
            #  may be case-insensitive, in which case this rule could be
            #  simplified...

            :0 c
            * ^Subject: .*urgent
            | $SENDMAIL $OFFSITE


            #  Use "vacation" to tell other people I'm not here To enable,
            #  un-comment the next two lines; to disable, comment them out
            #
            #  The -a Identifies another name that can legitimately
            #         appear in the To: line of the mail header instead
            #         of your login name

            :0 wc
            | vacation -a ericb eric

        And here the ~/.vacation.msg file

            Subject: I'm out of town for a while
            From: eric (via the vacation program)

            I'm out of town until .  Your mail regarding
                   "$SUBJECT"
            will be read when I return, or possibly at some unknown
            time before then if I get a chance to check for email.

            If your message must be seen by me before I return,
            you can send it with the word "URGENT" in the subject header.
            Such mail will be automatically forwarded to me so that
            I see it sooner.
            --Eric

    15.11 Service: Auto-forwarding

        [timothy] I have my .procmailrc setup to forward email to another
        (email only) account. When I am not going to be at the account, I
        want to turn forwarding off

            #   look for the file to tell us whether or not to forward mail
            #   if the file exists, forward the mail
            #   or not

            ELSWHERE = "me@elsewhere.com"
            FILE     = "$HOME/.forwardmail"

            :0 c
            *$ ? $IS_EXIST $FILE
            ! $ELSWHERE

            #   if a message arrives from the other account
            #   with the Subject 'forward-off' then remove the
            #   file, effectively turning off forwarding

            :0 hwic
            *$ ^From:.*$ELSWHERE
            *  ^Subject: forward-off
            | $NICE mv -f $FILE $FILE.off

            #   if a message arrives from the other account
            #   with the Subject 'forward-on' then remove the
            #   file, effectively turning off forwarding on

            :0 hwic
            *$ ^From:.*$ELSWHERE
            * ^Subject: forward-on
            | $NICE mv -f $FILE.off $FILE

    15.12 Service: forward only specific messages

        Here is piece of code that triggers forwarding according to
        addresses. If you have lot of these kind of forwarding,
        you should use simple `awk' database which you would grep.

            #   By Jim Hribnak 
            #   info@domain1.com goes to joe@domain1.com
            #   info@domain2.com foes to fred@domain2.com

            :0
            * ^TO_()info@domain1.com\>
            {
                FORWARDTO = "$FORWARDTO joe@domain1.com"
            }

            :0
            * ^TO_()info@domain2.com\>
            {
                FORWARDTO = "$FORWARDTO fred@domain2.com"
            }

            :0 fhw
            *    FORWARDTO ?? @
            * ! ^$MY_XLOOP
            | $FORMAIL -A "$MY_XLOOP"

              :0 a
              ! $FORWARDTO

    15.13 Service: Making digests

            # By 
            # Add this message to the digest accumulator

            :0 c:
            | $FORMAIL -k -X From: -X Message-Id -X Date -X Subject >> $DIGEST

            #Check size of digest, and send it off if it's big enough
            :0
            * $       -$DIGSIZE     ^0
            * $ `wc -l <$DIGEST`    ^0
            | $NICE send-digest $DIGEST

    15.14 Kill: killing advertisement headers and footers

          A mailing list that I subscribe recently began adding a block of
          "boiler plate" text to the beginning and end of every message
          that goes through the list (groan). The text is always the same,
          and is always at the beginning and end of the message.

        [david] sed could do both at once, but the problem is that
        sed never knows when it is N lines from the end if N>0; it knows
        the last line when it reads it, but when it is looking at the
        next-to-last line it doesn't know that there is only more one line
        to come. It does, however, know how many lines of input it has
        already read.

        So I have three suggestions: if you know that the header is X lines
        long [let's say 5 for this example] and that the first line of the
        footer contains some string or pattern that will not occur in the
        significant part of the post,

            :0 fbwi
            * conditions
            | sed -ne 1,5d -e '/pattern/q' -e p

        If you recognize the end by the last line that you want to keep
        instead of the first line that you want to delete, omit the n
        option and the p instruction:

            | sed -e 1,5d -e '/pattern/q'

        Finally, if the only reliable way to spot the footer is by reaching
        so many lines from the end (because any search pattern might occur
        in the real text as well), we can score as you've been doing to get
        the number of the last significant line. Let's say the footer is
        three lines long; because ^.*$ always counts one line too many
        (long story), we subtract four instead of three:

            :0 fbwi
            * conditions
            * 1^1 B ?? ^.*$
            * -4^0
            | sed -e 1,5d -e "$="q

    15.15 Kill: simple killfile recipe with procmail

        Kill files are widely used with newsreaders to delete uninteresting
        posts when you enter a newsgroup. A kill file usually contains one
        single entry per line to match the message content and this can be
        easily done with procmail. Remember however that for every message
        procmail forks a process, so before you apply the killfile rules to
        the messages, be sure your recipes are in this order: the killfile
        rules are applied only to *unknown* messages

            SINK MAILING-LISTS
            SINK ANNOUNCEMENTS
            SINK WORK MESSAGES
            OTHER DELIVERIES
            apply killfile rules and UBE recipes to the rest

        Recipe will drop the message (i.e. consider it 'delivered') if one
        of its headers matches a pattern in killfile.

            :0 hW:  $HOME/.killfile$LOCKEXT
            | egrep -i -f $HOME/.killfile

        The reason why there is explicit lockfile is that you must be able to
        update the killfile while your procmail is running. An example edit
        script is presented below.

            #!/bin/sh
            # program: killfile.sh
            #
            file=$HOME/.killfile
            lock=$file.lock
            cp $file $file.tmp
            emacs -q $file          # or use whatever you prefer: vi, pico
            lockfile $lock
            mv $file.tmp $file
            rm -f $lock

    15.16 Kill: duplicate messages

        [Lars Kellogg-Stedman ] Put this as a first entry in
        your .procmailrc and you won't see any duplicates as long as the 8K
        cache doesn't get full. The duplicates folder is cleaned out
        weekly via a cron job. While it may be tempting to simply sink
        duplicates to /dev/null, I have come across broken mail clients the
        stick the same value in the `Message-id' header of all outgoing
        mail.

            :0
            * ^Subject:\/.*
            {
                SUBJECT = $MATCH
            }

            MID_CACHE_LEN   = 8192
            MID_CACHE_FILE  = $PMSRC/msgid.cache
            MID_CACHE_LOCK  = $PMSRC/msgid.cache$LOCKEXT

            LOCKFILE = $MID_CACHE_LOCK

            # IF  the message has a message-id header
            # AND formail -D is successful (exit status=0)
            # THEN
            #   log a message to the procmail log
            #   sink the message

            :0
            *  ^Message-Id:
            * ? $FORMAIL -D $MID_CACHE_LEN $MID_CACHE_FILE
            {
                LOG="dupecheck: discarded message, $SUBJECT $NL"

                :0              # Store duplicates, notice no lock!
                duplicate.mbox
            }

            LOCKFILE            # Release lock by killing variable

        And here is a bit simpler recipe, a slightly modified version from
        the [manual]. Procmail notices formail's success, considers the
        message delivered and does not stop processing the rcfile due to
        `c' flag, which let's a message to fall into safety copy inbox.

            :0 hWc: $PMSRC/pm-msgid.cache$LOCKEXT
            *  ^Message-id:
            | $FORMAIL -D 8192 $PMSRC/pm-msgid.cache

              :0 a:
              duplicate.mbox

        There was a pretty heavy thread around September 1997 about
        duplicate detection, where some promising stuff was posted. One
        item you should definitely have in your collection is Eli's `hashd'
        

        Matt Saroff also started a thread about duplicates:
         where several of the replies are also helpful.

    15.17 Kill: spam filter with simple recipes

        [Ed McGuire ] Seeing several junk mail filters
        posted recently, varying from the simple to the complex, I thought
        I would also share my own. I junk whatever comes from my ISP but is
        not addressed to my domain or to one of the mailing lists I
        subscribe to.

            #   1.  mail to my domain
            #   2.  NOT addressed to me directly
            #   3.  NOT coming from mailing lists I'm subscribed to.

            0:
            * ^(received):.*psi\.com
            * ! ^((apparently-)?to|cc):.*(i2|intellection)\.com
            * ! ^(to|cc):.*(pdp-?8-lovers|procmail|sunshine|info-pdp11)
            junk.ube.mbox

        [Gordon Matzigkeit ] I have just discovered an
        effective rule for separating SPAM from the rest of my e-mail.
        Just substitute your username for `gord' in the line below

            # Anything which is not addressed to me is probably SPAM.
            :0:
            * !^TO().*\
            junk.ube.mbox

        This only works because I handle all mailing list addresses above
        that point in my .procmailrc (i.e. all traffic that arrives from
        mailing lists that I am subscribed to goes into other folders).
        Most SPAMmers seem to do it nowadays by sending mail via mailing
        lists, rather than creating huge `To' lists of users

        Many times sysadm install a list of know addresses that
        send spam and then they check the incoming mail against the "black
        list". Keep in mind that that some `fgrep' implementations have a
        problem with the -w word switch. Note that the above recipe scans
        the FULL HEADER, so use it with some caution, i.e., be careful what
        you add to your list of spam domains.

            # by [philip]; egrep would do here too, if it is posix
            # compliant, it may have -f switch that makes it behave
            # like fgrep.
            #
            # Note: option -F would make [ef]grep to search fixed string
            #       instead of regexps.
            #

            BLOCK_FILE  = $HOME/Mail/DeniedNames.lst
            UBE_MBOX    = $HOME/Mail/junk-ube.mbox

            # To filter out the Subject lines, so that emails sent
            # with the subject "Have you received a message from
            # blah-blah@spam" don't get filtered.
            # [era] suggested we use formail
            #
            # Edsel Adap  agrees there is a
            # likely bug in Solaris 2.5.1 "/usr/bin/fgrep -i" and
            # suggested the use of /usr/xpg4/bin/fgrep instead.
            #
            #  Sun Microsystems Developer Support
            # Files in /usr/xpg4 are available via the SUNWxcu4 package,
            # which is part of the user, developer, all, or Xall Solaris
            # clusters.
            #
            # Solaris 2.4 doesn't have /usr/xpg4/bin/fgrep :-(, you
            # must use  `tr A-Z a-z' before piping the message to fgrep.

            :0 hw:
            *$ ? $FORMAIL -ISubject: |fgrep -i -f $BLOCK_FILE
            $UBE_MBOX

        The file DeniedNames.lst is simply a list of addresses

            82338201@compuserve.com
            Dwnliner@ix.netcom.com
            Emerald@earthstar.com
            FreeWay@dm1.com

    15.18 Kill: (un)subscribe messages

          I'm getting tired of those pesky (un)subscribe messages that
          certain "other" mailing lists seem to pass through to the list at
          large instead of capturing them at the list server, like SmartList
          does.

        [Adam Shostack ] The following do help,
        although they're often too broad. (I use a .safe rule to cover those
        cases) The < 1000 is a useful hueristic. It's rare that unsubscribe
        messages are long.

            :0 :
            * (Delete|u*n*Sub(s| )*| add | leave | help )
            * < 1000
            junk.misc.mbox

        [Rodger Anderson ] I've been
        working on a recipe to filter out those pesky s*bscribe and
        uns*bscribe messages from mailing lists, and I'm posting what I have
        so far. As an aside, it also filters out very short messages, which
        I've found are usually some sort message meant for list owner/request
        address.

        I give heavy weight to Subjects starting with (un)?s*bscribe, with
        also pretty heavy weight to Subjects containing either of those
        words. I then give heavy weight to the body of messages starting
        with those words, and a lighter weight to lines starting with them.
        Then multiple occurrences get some weight too, up to a point. Then I
        count the words in the message against all that.

            :0 B
            *  1^0
            *  30^0 H ?? ^Subject: +(un)?subscribe\>
            *  20^0 H ?? ^Subject:.*\<(un)?subscribe\>
            *$ 20^0   ^^$SPCNL*(un)?subscribe\>
            *$ 10^0    ^$SPC*(un)?subscribe\>
            *  8^.4   \\<(un)?subscribe\>
            * -.4^1  \\<$a+\>
            junk.misc.mbox

        [Adam Shostack ] How about looking for sub &
        unsub, as well as a perennial misspelling 'unsuscribe me'?  I also
        find filtering on add, leave and help to be useful. This may well be
        the only word on the line. I think it has to do with broken list
        management packages.

            | :0B
            | *  1^0
            | * 30^0 H ?? ^Subject: +(un)?subscribe\>

            * 20^0 H ?? ^Subject: +(un)?sub?(scribe)?\>

            (The B is often missing, as is the word fragment 'scribe')

            | * 20^0 H ?? ^Subject:.*\<(un)?subscribe\>

            * 20^0 H ?? ^Subject: +(add|leave|help)$

              # fewer points if more words

            * 15^0 H ?? ^Subject: +(add|leave|help)

        [david 1998-10-20] You want to match on messages where the
        first non-blank thing in the body is "unsubscribe" at the end of a
        line, where there are five lines or fewer in the body?

            :0 B
            *$ ^^$SPCNL*unsubscribe$
            * 7^0
            * -1^1 ^.*$
            junk.misc.mbox

        ^.*$ always counts one line too many, so a five-line body will be
        counted as six; that's why we need a prejudice of 7. But if the
        first non-blank text in the body is "unsubscribe" alone on a line,
        is a line count really necessary? True posts that include the word
        will have it in the middle of a sentence, such as the preceding
        one. What you'll find by specifying a line limit is that
        unsubscribe requests with long signatures or attachments at the
        bottom of a previous message will get through.

    15.19 Time: Once a day cron-like job

        [Bill Moseley ] If you want to do something only
        once a day, they you have to store the date somewhere and check
        against that stored date.

            YYMMDD_FILE = $HOME/.yymmdd
            YYMMDD      = $YY-$MM-$DD

            #   Contains single line of procmail code
            #   YYMMDD_PREV = ..

            INCLUDERC $YYMMDD_FILE

            #   If different date, then enter this block
            #   The echo updates stamp in file.

            :0
            *$ ! YYMMDD ?? ^^$YYMMDD_PREV^^
            *  ? echo "YYMMDD_PREV = $YYMMDD" > $YYMMDD_FILE
            {
                ...do the cron jobs..
            }

    15.20 Time: Running a recipe at a given time

          If I put a program to my recipes, it will be executed every time
          message arrives. That's a problem, and I'm not allowed to use cron
          in this account. I'm looking for some sort of condition to check
          the current time and if its outside of the hours 11pm and 7am then
          execute the action.

        [david] How do your From_ lines look? If they're the traditional
        kind that sendmail and smail add, they include the local time on your
        system at receipt. So include a check that the hour is between 07
        and 22 inclusive, like this:

            :0 c
            *  ^From .*some-address.* (0[789]|1.|2[012]):[0-5][0-9]:
            |  command

        I included the minutes and the colon that separates the minutes from
        the seconds so that the expression for testing the 07-22 range can
        match only on the hour.

    15.21 Time: Triggering email and using cron

        [david] Put something like the following entries in your personal
        crontab for your userid (and not knowing if you particular cron
        "cd's" to your home directory first):

            0 23 * * *        touch $HOME/.mail.relay.on
            0 7 * * * rm -f $HOME/.mail.relay.on

        And if your cron doesn't know the HOME variable (that'd be an
        exception)

            0 23 * * *  /bin/csh -c 'touch ~LOGNAME/.mail.relay.on'
            0 7 * * *   /bin/csh -c 'rm -f ~LOGNAME/.mail.relay.on'

        Then, in your .procmailrc do:

            :0 c
            *  ^From.*some-address
            *$  $IS_FILE $HOME/.mail.relay.on
            | command

        the script will run_my_program only if both the subject matches and
        the file test succeeds. The file test will succeed only between 11pm
        and 7am.

        In all honesty, if system gives usable From_ lines, I like following
        suggestion better. I use it all the time to turn blocks of procmail
        code on and off at given times or dates, and it works likes a charm.
        It uses many fewer processes and is less likely to get the status
        wrong if for any reason one of the cron jobs fails to run or doesn't
        do its job.

        This pages only at day time

            :0 c
            * ^From .*some-address.* (0[789]|1.|2[012]):[0-5][0-9]:
            | command

        This pages at night

            :0 c
            * ^From .*some-address.* (0[0-6]|23):[0-5][0-9]:
            | command

    15.22 Decoding: Uudecode

        [philip] here is piece of code to do uudecode match when certain
        condition is matched. The magic string here is "begin ...file",
        the *body* is then fed to `my_uudecode_program' whatever it does
        to it.

            :0 b
            *      ^From:.*someone@somewhere\.com
            *      ^Subject: Subject
            * B ?? ^begin 644 file.tar.gz
            | my_uudecode_program

    15.23 Decoding: MIME

            #   by Peter Galbraith 
            #   MIME filtering of accented characters and split lines.
            #
            :0
            * ^Content-Type: *text/plain
            {
              :0 fbw
              * ^Content-Transfer-Encoding: *quoted-printable
              | mimencode -u -q

                :0 A fhw
                | $FORMAIL -I "Content-Transfer-Encoding: 8bit"

              :0 fbw
              * ^Content-Transfer-Encoding: *base64
              | mimencode -u -b

                :0 A fhw
                | $FORMAIL -I "Content-Transfer-Encoding: 8bit"
            }


            #   1995-10-18 Tim Pickett 
            #
            #       Decode MIME quoted-printable Content-Transfer-Encoding
            #
            #   Conditions
            #
            #       Mail has a MIME-Version header with a number in it.
            #       Header saying "Content-Transfer-Encoding: quoted-printable"
            #       exists

            :0
            *$ ^MIME-Version:$s*$d*(\.$d*)
            *$ ^Content-Transfer-Encoding:$s*quoted-printable
            {
              :0 fhw     # Remove header
              | $FORMAIL -I"Content-Transfer-Encoding:"

              :0 fbw             # Decode the body.
              | mmencode -u -q
            }

    15.24 How to send commands in the message's body

            :0 b
            * ^Subject: ARCHIVE
            | sed -e '/$s*[^a-zA-Z]/,$ d' | sh

    15.25 Matching two words on a line, but not one

        How does one write a recipe that will do this: Put mail in mailbox
        which has a line with two string (one and two) like:

                one     two

          but save mail in error-folder if the line as only the first
          string like: one (string two is missing)


        [philip] I presume these lines would be located in the body of the
        message, and that by "space between one and two" you mean
        "whitespace between one and two".  If those assumptions are wrong
        then you'll need to tweak the following recipes:

            # The 'B' tells procmail to look in the body instead of the header.
            # The second colon tells procmail to lock the mailbox with a
            # locallockfile -- if mailbox is a directory then you don't need
            # it. The brackets in the condition contain a space and a tab.
            #
            :0 B:
            *$ one$s*two
            default.mbox

            :0 B:
            * one
            error.mbox

        Now, the above will match even if "one" or "two" is part of another
        word (at the end in the case of "one" and at the beginning in the
        case of "two").  If you don't want that then you'll need to change
        the recipes to read:

            :0 B:
            *$ ()\
            default.mbox

            :0 B:
            * ()\
            error.mbox

    15.26 How to define personal XX macros?

        By macro, I'm referring to the procmail's FROM_DAEMON, TO and TO_
        that you can use in matches. Here is one way to make one's own macro

        [alan] Define HEADERS to include those headers you care about. Pick
        one of the definitions below (and remove or comment out the
        others). Here are three ways to define user `to_' macro

        .   use only To:
        .   use either To: or Cc:
        .   To:, Cc:, or Apparently-To:

            to_ = '^To:(.*\<)?'
            to_ = '^(To|Cc):(.*\<)?'
            to_ = '^((Apparently-)?To|Cc):(.*\<)?'

        And you use it like this

            :0 :
            *$ $to_()foo@bar.com
            address-matched.mbx

        [jari] and here are some more examples

            cc_      = "(^((Original-)?(Resent-)?(Cc|Bcc)):(.*[^a-zA-Z])?)"
            from_    = "(^(Apparently-|Resent-)*\
            (From|Reply-To|Sender):(.*\<)?|\
            ^From $NSPC+)"}

    15.27 How to change subject by body match

        Suppose you to change the email's subject when there is a match in
        the body. The desired outcome would be this:

            From: foo@this.is
            Subject: Fault: NNNN in program block YYY    << changed

            Fault: NNNN in program block YYY

        Here is the answer

            :0 fhw
            *       ^Subject: NOK case report
            *$ B ?? ^$s*\/Fault: [0-9a-f]+ in program block.*
            | $FORMAIL -I "Subject: $MATCH"

    15.28 How to change Subject according to some other header

        Suppose you want to change the subject when mail comes to some
        particular address; or when some other header field. Here is one
        way to do it, we suppose that mail comes to various internal mail
        addresses. See the HEADERS macro in previous section.

            # By [alan]
            # Examine headers, create a subject tag if we recognize a list

            TAG = ""

            :0
            * $ ${HEADERS}info@foo.com
            {
                TAG = "info"
            }

            :0E
            * $ ${HEADERS}check@foo.com
            {
                TAG = "check"
            }

            # ...and so on...
            # now, if TAG is set, insert it into the subject

            MATCH       # kill this

            :0 fhw
            * !  TAG ?? ^^^^
            *   ^Subject: *\/[^ ].*
            | $FORMAIL -I "Subject: $TAG - ${MATCH:-}"

        *Or* you could use the command line arguments, add following
        line to your `.forward'. (alias file syntax)

            foo: "|/usr/local/bin/procmail -m /usr/local/etc/pm-tagit.rc foo"

        Then in `tagit.rc' you would instead say:

            ARG = $1

            :0
            * ARG ?? ^^foo^^
            {
                TAG = "foo@go"
            }

            :0
            * ARG ?? ^^somethingelse^^
            {
                TAG = "somethingelse@go"
            }

        This method will work even if someone Bcc:s a message to
        foo@some.com.

    15.29 How to call program with parameters

          ...now, suppose I want to call `program' with parameter $FOUND,
          and get the result back in RESULT, how do I do it ?

        The stdout of myprogram will be captured at stored in the variable
        RESULT. Also consider what should happen if there are spaces or tabs
        in the value of $FOUND. Perhaps it should be better off enclosed with
        quoted.

            #   Make sure FOUND is not empty before passed to program

            :0
            * ! FOUND ?? ^^^^
            {
                RESULT = `program "$FOUND"`
            }

16.0 Miscellaneous recipes

    16.1 Matching valid Message-Id header

        [philip] wrote full RFC compliant matcher. Follow the link

        

            dq = '"'                                # (literal) double-quote
            bw = "\\"                               # (literal) backwhack
            ws         = "[         ]*"                     # whitespace
            atom       = "[-!#-'*+/-9=?A-Z^-~]+"
            word       = "($atom|$dq([^$dq\]|$bw.)*$dq)'
            local_part = "$word($ws\.$ws$word)*"
            domain     = "(\[$ws([^][\]|$bw.)*$ws\]|$atom($ws\.$ws$atom)*)"

            :0
            * ! $ ^Message-Id:$ws<$ws$local_part$ws@$ws$domain$ws>
            thats-non-valid-message-id

    16.2 Sending two files in a message

        If you plan to send multiple files in a message, be sure that every
        file has extra blank line at the end so that they can be *cat*d
        together. Instead of doing

            (cat THIS; echo " "; cat THAT ) | $SENDMAIL

        You do

            (cat THIS THAT ) | $SENDMAIL

        But sometimes you don't have control over the files, then you can
        do this to make sure there is blank line. Notice, only two
        processes used compared to first choice.

            (echo '' | cat THIS - THAT ) | $SENDMAIL


        [David] And an sed expert would do it this way

            (sed -e '$ !b' -e '/./G' -e "r THIS" THAT ) | $SENDMAIL

        o   $: the last line
        o   !: everywhere except the range (in this case, everywhere except
            the last line)
        o   b: branch to a label. No label: branch to the end
            (and, since -n is not in effect, print the pattern space)

        Now remember that everywhere except the last line, we've
        skipped ahead, so the rest of the code will be executed only for
        the last line of the input.

        o   /./: on lines that contain a character (but we get here only for
            the last line, so on the last line if it contains a character)

        o   G: append a newline and the contents of the hold space to the
            pattern space (the hold space is empty, so basically, if the
            last line was already empty, do nothing, but if the last line
            was not empty, append a newline and thus add a blank line after
            it).
        o   r file: After finishing with this run through the sed
            instructions, read the named file and copy it to the output.

        This side of sed comes out only after sed has had a few drinks...

    16.3 Excessive quoting of message

          [25 Nov 1997 buck@Compact.COM] I administer a LISTSERV mailing
          list and our host has asked us to reduce excess quoting of
          previously posted material. ...Subject: asking if this was
          excessive quoting. With the weights below, this extra copy will
          activate at 66% quoted lines of all body lines.

        [era] I would definitely tolerate 75% quotes. And in the
        end, you will of course always have to face the kinds of people who
        would rather change their quoting style to evade such constraints
        than quote less. An idealized quote parser should perhaps realize
        that a non-blank prefix that recurs on a lot of lines is probably a
        customized quote string.

        This will preserve the correspondent's original subject (with a Re:
        added if it didn't already have one) and thus the template text
        should indicate the nature of the problem.

        I'm not sure what would be appropriate to generate behaviour more
        like I suggest below, any takers? Perhaps no score at all for empty
        lines, neutralize .signatures (hope sender obeys "-- " convention)
        and add 10^0.5 for each quoted line and dish out -15^0.3 for
        nonquoted? (I haven't really explored this -- could be completely
        up the creek.) [Also, perhaps long runs of quoted material should
        be penalized harder than quoted snippet -- reply text -- quoted
        snippet -- reply text alternations?]

            COPY_ADDRESS = "listAdm@foo.com"

            :0
            * ^Sender: 
            {
                # - quoted lines
                # - non-blank, non-quoted lines
                # - completely blank lines

                :0B
                *$  10^1 ^$s*>
                *$ -15^1 ^$s*[^>$WSPC]
                *$ -15^1 ^$s*$
                {
                    # You don't need to repeat the original condition here
                    # You also don't really need to extract SENDER
                    # Generate a reply with appropriate headers and the
                    # body quoted

                    :0 fhw
                    | $FORMAIL -rtk -A "Bcc: $COPY_ADDRESS"

                    # Now "replace" the body with template text + body (In
                    # other words, add the template before the quoted body)

                    :0 fbw
                    | cat $HOME/template.txt -

                    # Now send it off to recipients mentioned in generated
                    # header

                    ! -t
                }

                # Wasn't excessively quoted; save it
                :0 :
                $SOME_MBOX

    16.4 Sending message to pager in chunks

          I have a 200 character limit on my pager. But I have wordy contacts
          who go over that limit. What I would like to do is have a recipe
          split up messages addressed to my pager into 200 character (max)
          messages.

        

        [era] This stuff about forwarding to pagers is a recurring
        topic on this list. I've tried to find a good summary of all the
        issues but there always seems to be some tiny twist to what people
        would like to have implemented. As a general comment for future
        generations, the Procmail part is usually trivial and the problem
        reduces to writing a good program (shell script or otherwise) for
        formatting the text precisely the way you want it, and spitting it out
        in suitable chunks.

        Here's something to split up the body of the message into smaller
        chunks and do a shell script on each chunk. The -s option to fold says
        to only wrap lines on whitespace if possible

            #   Create a duplicate of the message to forward to the pager.
            #   This will be reformatted and have most headers stripped off.

            :0 c
            {
                # Construct header with only From: and Subject: retained

                HEADER = `$FORMAIL -XFrom: -XSubject:`

                #   Reformat body as 200-character lines and send each
                #   as a separate message with the preconstructed minimal
                #   header

                :0 bw
                |   tr '\012' ' ' | fold -s -w 200 | while read line; do
                    echo -e "$HEADER\n\n$line" | \
                    $SENDMAIL pageraddress@wherever.com ; done
            }

        If your version of echo doesn't understand \n to mean newline
        (and/or the -e option to enable this escape processing), you need to
        tweak this. (You might need to anyway -- this is mostly untested. In
        my limited testing, I found the messages would arrive in more or less
        random order. Inserting pauses in the script should help to some
        extent, but could lead to other problems and is not an ideal solution
        anyhow.)

        I don't know if the header trimming is required; some pager gateways
        appear to count the headers as part of the message, while others
        don't. Again, for future generations, details like this are relevant
        to include when you ask about how to do this.

    16.5 Playing particular sound when message arrives

        [Peter S Galbraith ] Here is the command
        in shell to produce the sound:

            % cat anyfile | /usr/X11R6/bin/auplay /usr/lib/exmh/drip.au

        However, it won't work directly in the recipe

            procmail: Executing "/usr/X11R6/bin/auplay /usr/lib/exmh/drip.au"
            Can't connect to audio server

        Strange. The command works from the shell if I `su' to user `mail'.
        Anyway, I got it to work by fully specifying the audio server (which
        is my workstation, where I receive mail)

            AU      = /usr/X11R6/bin/auplay
            TUNE    = /usr/lib/exmh/drip.au

            :0 hwic
            * ^From:.*foo@bar.com
            | cat > /dev/null; $AU -audio tcp/mixing:8000 $TUNE

    16.6 Combining multiple Original-Cc and Original-To headers

          How can I use procmail/formail to combine the information in these
          headers into their *CORRESPONDING* header MINUS the Original-*
          Note that I can have multiple Original-Cc: headers and I want all
          the recipients combined into one Cc: header.

            #   1998-01 by [david]
            #   initialize as unset

            ORIG_TO ORIG_CC

            #   The -c option to formail takes care of headers continued onto
            #   indented lines; the pipe to tr takes care of multiple
            #   Original-To: headers by linking their contents with commas.
            :0
            * ^Original-To:.*[^   ]
            {
                ORIG_TO = `$FORMAIL -zcxOriginal-To: | tr \\12 ,`
            }

            #   Drop trailing comma from tr:
            :0 A
            * ORIG_TO ?? ,^^
            * ORIG_TO ?? ^^\/.*[^,]
            {
                ORIG_TO = $MATCH
            }

            #   Likewise for Original-Cc: lines:

            :0
            * ^Original-Cc:.*[^   ]
            {
                ORIG_CC = `$FORMAIL -zcxOriginal-Cc: | tr \\12 ,`
            }

            :0 A
            * ORIG_CC ?? ,^^
            * ORIG_CC ?? ^^\/.*[^,]
            {
                ORIG_CC = $MATCH
            }

            #   Now, let's install the changes if needed:
            #   with -A instead of -I or -i it should
            #   not clobber existing To: or Cc: information.
            #   -A : Append a custom headerfield onto the header in any case.

            :0
            * ORIG_TO ?? ^^^^
            * ORIG_CC ?? ^^^^
            { }
            :0 E fhw
            | $FORMAIL                                                      \
              ${ORIG_TO:+-A "To: $ORIG_TO"}                                 \
              ${ORIG_CC:+-A "Cc: $ORIG_CC"}

    16.7 Forwarding sensitive messages in encrypted format

            #   by [alan]
            #   See if addressed *directly* to me, and ..
            #   ..has not already been forwarded

            KEY             = "TheMagic"
            FORWARD_EMAIL   = "foo@bar.com"

            :0
            * $   ^To:.*$LOGNAME(@|[^0-9a-z]|$)
            * $ ! ^$MY_XLOOP
            {
                # now let's encrypt the body using mimencode

                :0 fbw
                |   echo "MIME-Version: 1.0" ;                              \
                    echo "Content-Type: application/crypt" ;                \
                    echo "Content-transfer-encoding: base64" ;              \
                    echo "" ;                                               \
                    crypt $KEY | mimencode -b

                #   Now let's prepare the headers for forwarding the mail,
                #   and mark it so we don't loop

                :0 fhw
                | $FORMAIL   -I"Resent-To: $FORWARD_EMAIL" -I"$MY_XLOOP"

                :0
                ! $FORWARD_EMAIL

            }

17.0 Procmail and PGP

    17.1 Decrypt pgp messages automatically

        _Warning_: if you use remailers or anonymous services, you must use
        different passwords and different user id's to decrypt incoming
        messages. If you just receive messages encrypted with one key, then
        you this may be useful to you. However, it is generally considered a
        huge security risk to keep your password carved into your .procmailrc.

            :0 fbw
            * B ?? PGP ENCRYPTED MESSAGE
            | pgp -z "your pass phrase" -f +batch 2>&1

    17.2 Getkeys from keyserver

            # by Adam Shostack  1996-02
            #
            # This first ruleset protects me from mailbombs from an automated
            # service that I often send incorrect commands to, generating 5mb
            # of reply. It also sorts based on success of the command.
            #
            # swissnet.ai.mit.edu is fast keyserver

            :0
            * From bal@swissnet.ai.mit.edu
            {
               :0 h
                * >10000
                /dev/null

                :0 h
                *^Subject:.*no keys match
                /dev/null

               :0 E
               | pgp +batchmode -fka
            }

    17.3 Auto grab incoming pgp keys

            #  [Opher Kahn ] This first ruleset protects
            #  me from mailbombs from an automated service that I often send
            #  incorrect commands to, generating 5mb of reply. It also sorts
            #  based on success of the command.
            #
            #  swissnet.ai.mit.edu is PGP key server

            :0
            * From bal@swissnet.ai.mit.edu
            {
               :0 h
               * >10000
               /dev/null

               :0 h
               *^Subject:.*no keys match
               /dev/null

               :0 E
               | pgp +batchmode -fka
            }

            #  auto key retrieval
            #
            #  I have an elm alias, pgp, points to a keyserver The logfile
            #  gets unset briefly to keep the elm lines out of my logfile.

            :0 W
            * B   ?? -----BEGIN PGP
            * H ! ?? ^FROM_DAEMON
            {
                KEYID = `/usr3/adam/bin/sender_unknown`
            }

            LOGFILE=

            #   #todo: We should get rid of the 'elm' dependency here.
            #   #todo: correct this sometime... [jari]
            #

            :0 ahc
            * ! ^X-Loop: Adams autokey retrieval
            | $FORMAIL -a"X-Loop: Adams autokey retrieval" | elm -s"mget $KEYID" pgp


            #!/bin/sh
            #
            #   Script: sender_unknown
            #
            #   unknown returns a keyid, exits 1 if the key is known. $output
            #  is to get the exit status. Otherwise, this would be a one
            #  liner.

            OUTPUT=`pgp -f +VERBOSE=0 +batchmode  -o /dev/null`
            echo $OUTPUT | egrep -s 'not found in file'
            EV=$?
            if [ $EV -eq 0 ]; then
                    echo $OUTPUT | awk '{print $6}'
            fi
            exit $EV

            # end of sender_unknown

18.0 Includerc usage

    18.1 Using: multiple rc files

          ...Do INCLUDERC statements function as a kind of "call" which
          returns control to the "original" rc file if processing falls off
          the end of the included rc file? Or if processing falls off the
          end, does mail then get delivered to $DEFAULT and processing
          stop? Suppose I have these commands

            INCLUDERC = $PMSRC/pm-a.rc
            INCLUDERC = $PMSRC/pm-b.rc
            INCLUDERC = $PMSRC/pm-c.rc

        Yes, the control is returned to the original file where the
        includerc was called from. And No, mail does not get delivered in
        the $DEFAULT because the includerc just ends: processing continues
        until there is no more statements in the top level.

        Includerc is nothing more that a sliced top level recipe.

    18.2 Using: You can call rc file conditionally

        One interesting way to prevent false hits when filtering UBE is to
        try to see if the message comes from some valid destination
        first. If it comes, then it shouldn't be run through UBE filter,
        because it may filter valid messages out. No ube filter is
        completely bullet proof.

        Here is an example where the UBE detection is put into use only when
        the message comes from somewhere that I don't know beforehand (or I
        have just forgot to tweak my .procmailrc)

            ME      = "(me@here.is)"
            LISTS   = "(procmail|list-a|list-b)"

            :0                      # Idea by Bill Moseley
            *$ ! ^TO_()$ME
            *$ ! $LISTS
            {
                # Could be UBE or I might be on a unknown distribution list.
                INCLUDERC = $PMSRC/pm-ubecheck.rc
            }

        [dan] That would work; common practice, however, is to put recipes
        for filing mail from lists (and, per Bill's preferences, anything
        mentioning procmail in the head gets treated the same as mail from
        this list) first; then the only remaining condition to consider
        there would be unexpected blind carbons: * ! ^TO_moseley. This
        method is good if you get much more spam than legitimate mail
        (including mail from list subscriptions as legitimate) and you want
        procmail to deal with spam right away. I belong to several very
        active mailing lists, so I actually receive more pieces of
        legitimate mail than pieces of spam.

        One way to get the best of both worlds is this:

            * $ ! ()\/(^TO_$LOGNAME|procmail|list-(ABC|123|XYZ))

        because then, if the regexp matches (and thus the negated condition
        fails and you don't detour into $PMSRC/checkspam.rc), MATCH is
        already set to the name of the mailing list, and you can do further
        tests by just examining MATCH (or a variable you copy it into)
        instead of a repeating a complete head search.  [I prefer to use
        the variable $LOGNAME rather than hard-coding my name because then
        others can use the code, and I can use it unchanged on sites where
        my logname is different, and if my logname is changed my
        .procmailrc will keep up with it.]  For example (I've separated the
        conditions into two lines so that, per Bill's preferences, a
        mention of procmail in the head will get the message into the
        Procmail List folder, even if a match to $^TO_$LOGNAME is also
        present and appears sooner):

            :0
            * ! ()\/(procmail|list-(ABC|123|XYZ))
            * $ ! ^TO_$LOGNAME
            {
                INCLUDERC=$PMSRC/pm-ubecheck.rc
            }

            #   The next recipe has an `E' flag, so it will be examined
            #   only if the preceding one didn't match; thus if $MATCH was
            #   set inside pm-ubecheck.rc, it won't hurt anything here, and a
            #   value for $MATCH set in pm-ubecheck.rc
            #   won't be mistaken for a list name:

            :0 E: # MATCH is non-null only if it matched a list name
            * MATCH ?? (.)
            $MATCH

            #   Remaining recipes will be read only for two types of mail:
            #   those that met $^TO_$LOGNAME but not any expected list
            #   name, and those that went through pm-ubecheck.rc but came out
            #   undelivered.

    18.3 Autoloading an rc file

        Now when you know that includerc can be called conditionally, let's
        discuss about "autoloading of module". For example I use following
        statement in nearly all my modules to import predefined variables

            :0
            * !  WSPC ?? ( )
            {
                INCLUDERC = $PMSRC/pm-javar.rc
            }

        It says that "If variable WSPC does not contain space, then load
        module". If the module has already been loaded by some other rc
        file, the WSPC would exist. If it does not exist yet, then we load
        the module. This is classical example of conditionally loading
        functions or variables into current module:

            Check if feature is present, No? Then load module module.

        Justin Lloyd  suggest a general way of caching
        the included rc files. We use top-level script that records every
        module that was included. The module is loaded only if it it not
        yet included:

            #   pm-xximport.rc

            :0
            * ! INCLUDE_CACHE ?? ()\<$RC\>
            {
               #    Module was not there yet, add it to the list
               INCLUDE_CACHE  = "$INCLUDE_CACHE$RCFILE$NL"
               INCLUDERC      = $RC
            }

        This different approach then the previous one. Instead of checking
        features, the presense of module is checked. Two sides of the coin
        which can be used for the same thing. You can pick either solution
        but to my opinion

        o   Adding extra top level INCLUDE_CACHE is extra work. Procmail
            must open a separate top-level rc file every time with call

            RC="pm-xxscript.rc"   INCLUDERC=pm-xximport.rc

        o   If feature already existed, you would still have to open the
            pm-xximport.rc file for every call to find it out. Eg. Here you
            pm-xximport.rc is called 3 times no matter if 1,2,3 were
            already present

            RC="pm-xxscript1.rc"   INCLUDERC=pm-xximport.rc
            RC="pm-xxscript2.rc"   INCLUDERC=pm-xximport.rc
            RC="pm-xxscript3.rc"   INCLUDERC=pm-xximport.rc

        o   With simple feature test, procmail can evaluate the condition
            in place without the need of opening separate file.

            if no feature present..
                then load

            if no feature present..
                then load

        Note however, that both suggestions accomplish the same thing; the
        implementation is only different. If the typical count of including
        RC files per module were big enough, I'd use justin's way. Currently
        the typical count is only 1-2 (VAR,DATE).

    18.4 Making: naming of the rc file

        When you write an rc file, think whether or not it could be
        generalized so that others could use it. I have adopted style where
        all procmail files start with prefix `pm', so that I can stack
        other files as well to the same directory. If I simply named them
        as `rc.*', look what happens:

            % ls rc*        # fine, print rc files

        but I wold like to print all procmail relates files and backup
        them with one command, so this would print all procmail relates
        files

            % ls pm-*

            --> pm-mytest.rc
                pm-jaube.rc
                pm-tips.txt
                pm-art.txt
                pm-incoming.log
                pm-list.mbox        # the mailing list

        I usually use a name like `pm-xxSCRIPT-NAME.rc' for a rc file where
        `xx' is my initials from firstname and surname, like (J)ari
        (A)alto. These scripts are product versions, that can be
        distributed. I also have private scripts that handle my mailing
        lists, work messages and so on and they have prefix `my'.

            pm-jascript.rc
            pm-myscript.rc

        and when I download someone else's script I would like to see it
        named so that it's unique to the person who did it.

            pm-jdscript.rc      # John Doe's script.

    18.5 Making: Using namespace when saving procmail variables

        If you're going to write rc file that works like any other
        programming language subroutine, you must separate it from the
        world and make it well behaving. A subroutine is traditionally a
        black box: you call it with arguments and it responds with
        returned values. You don't need to know what happens in there. And
        you expect that the subroutine hasn't changed the existing
        environment, like procmail variables `DEFAULT' `LOGFILE' etc. when
        it ends.

        So the process diagram of a good RC subroutine is:

                                pm-xxscript1.rc
            call            --> +------------+
            arguments           | black      | --> it may call
                                | box        |     other subroutines
                                |            | <-- pm-xxscript2.rc
            output values   <-- +------------+

        Procmail does not have local variables, so you must put the
        variables to global namespace. Let's see an example where
        subroutine uses `MAILDIR' for `chdir' purposes.

            MAILDIR_xxscript1   = $MAILDIR              # save
            ...
            MAILDIR             = new location
            ...
            ...at the end of subroutine
            MAILDIR             = $MAILDIR_xxscript1    # restore

        Here the original value is saved when subroutine started and the
        original value was restored when subroutine exited. The global
        namespace (xxscript1) used was unique and is guaranteed not to
        clash with anyone else's. If the `pm-xxscript2.rc' would have also
        used `MAILDIR' the saved value would have been in

            PROCMAILVAR_xxscript2

        and the two wouldn't mix up with each others `MAILDIR'. The general
        name for saved variable is therefore:

            PROCMAILVAR_scriptname

        This follows the simple "onion" or "stack" model, where variable's
        value is saved before changing it and restored on exit point.

            save-x-1
            set--x-1

                save-x-2
                set--x-2
                ..
                restore-x-2

            restore-x-1

    18.6 Making: Public and private variables in rc file

        As you learned above, the variables should be put to RC file's
        namespace. The user interface variables (public) should be all caps
        and private variable should start with lowercase letter. Whether
        you use "theVarStyle" or "the_var_style" is up to you.

            [script pm-xxscript.rc]

            # ........................... public

            XX_SCRIPT_FLAG = ${XX_SCRIPT_FLAG:-"default"}
            XX_SCRIPT_VAR  = ${XX_SCRIPT_VAR:-"default"}

            # ........................... private

            charset = "a-z1-2"
            regexp  = "something-that-matches"

        Whether you need to stick prefix `xx_script' to the private variables
        depends on whether you call another includerc which may happen to use
        same names as you:


            [pm-xxscript.rc]
            charset = ...           # watch this
            ...
            INCLUDERC = ..          # call another subroutine

                charset = ..        # holy cow, it used same variable

            ..back in the pm-script.rc

            :0
            * $charset              # BOOM, not what you think.

        In this case it would be wise a) not to define `charset' at the top
        of the file but to move the definition to just before the recipe
        where it is used or b) make the name unique, with
        `pmScriptCharset'.

    18.7 The rules of thumb for constructing general purpose rc file

        o   Write good documentation at the beginning of file: how to set up
            the includerc and explain what it does. If you don't include
            docs, people may skip your extraordinary useful script. Also,
            remember that the script lives in the Net and passes through many
            hands long after you have been disconnected.
        o   Keep the layout like this: the user interface variables must all
            be in capital letters. Familiarize yourself with what(1) tags
            too. Notice the first and last lines: if you keep the format
            like this, then any universal tool can rip your code from any
            file (or email), because it's delimited by "pm-xxScript.rc -- "
            and "end of pm-xxScript.rc". See unix what(1) for first line's
            syntax.

                # @(#) pm-xxScript.rc -- procmail script for ...
                # DOCS

                USER VARIABLES

                private variables

                CODE

                # end of pm-xxScript.rc

        o   Always include version number or last modification date somewhere.
            Prefer some version control tool, like RCS, VCS, ClearCase,
            whatever you have at hand.
        o   Use a variable name like `dummy' in appropriate places to tell
            what's happening in the code. Remember that the *VERBOSE*
            setting isn't much help if you can't tell by looking at the
            *LOG* where on earth the code is executing.

                dummy = "start of pm-xxScript.rc"
                ...
                dummy = "Now testing if we have control message XXX"
                :0
                * condition
                {
                    dummy = "Now testing if the command is YYY"
                    :0
                    * condition
                    ...
                }
                ...
                dummy = "end of pm-xxScript.rc"

        o   If you need the value of some common headers, don't just call
            formail like this because the value may already be available
            prior your includerc. For example the user may already have needed
            the *Subject* value and stored it in a variable

                [in pm-xxScript.rc]

                XX_SCRIPT_SUBJECT = `$FORMAIL -xSubject:'

                [User may have already read the content to SUBJECT]

                SUBJECT   = `$FORMAIL -xSubject:'
                INCLUDERC = $PMSRC/pm-xxScript.rc

            Your pm-xxScript.rc launches an unnecessary formail call. Instead,
            use the existing SUBJECT.

                [user]
                :0
                * ^Subject:\/.*
                {
                    SUBJECT = $MATCH
                }

                ...

                XX_SCRIPT_SUBJECT   = $SUBJECT            # Note this!
                INCLUDERC           = $PMSRC/pm-xxScript.rc

                [ in the pm-xxScript.rc variable definitions  ]

                #   User should initialise the variable
                #   XX_SCRIPT_SUBJECT if he already has read the
                #   subject.

                :0
                * XX_SCRIPT_SUBJECT ?? ^^^^
                * ^subject:\/.*
                {
                   SUBJECT = $MATCH
                }
                ...the rest of the code

        o   _Add_ header `X-Loop' and test against it if you are sending an
            automated reply. The `X-loop' prevents responding to already
            responded message.

                :0
                *    condition
                *  ! ^FROM_DAEMON
                *$ ! ^$MY_XLOOP
                {
                    # Ok, now we're clear to send an automated reply
                }

    18.8 An includerc skeleton

        Here is my includerc file skeleton that i use in all my modules. The
        funny looking ".$" are for the text2html Perl filter. The
        documentation section can be ripped and turned into html very easily
        is you just keep the standard 4 tab column positions and start the
        description with "File id" and end it with "Change Log". The command
        to make the html is:

            % ripdoc.pl pm-xxscript.rc | t2html.pl > pm-xxscript.html

        These two perl files are available from my ftp directory.

            # @(#) pm-xxscript.rc -- one line description string here
            # @(#) $Id: pm-tips.txt,v 1.76 1999/11/10 13:55:12 Jari Aalto Exp $
            #
            #   File id
            #
            #       .Copyright (C)  1997-98 Foo Bar
            #       .$Contactid:     $
            #       .$Created:      YYYY-MM $
            #       .$keywords:     procmail [subroutine|recipe] whatItDoes $
            #
            #       This code is free software in terms of GNU Gen. pub. Lic. v2 or later
            #       You can get newest version by sending email to maintainer with
            #       subject "send "
            #
            #   Description
            #
            #       This subroutine Parses  from variable INPUT
            #
            #   Required settings
            #
            #       PMSRC must point to source directory of procmail code.
            #       This subroutine will include
            #
            #       o   pm-xxScriptA.rc
            #       o   pm-xxScriptB.rc
            #
            #   Call arguments (variables to set before calling)
            #
            #       o   INPUT, the string from where to parse...
            #       o   VAR1, description, default is ...
            #       o   VAR2, description, default is ...
            #
            #   Returned values
            #
            #       ERROR will have value "yes" if couldn't parse INPUT
            #       OUTPUT will have result after successful parse
            #
            #   Example usage
            #
            #           :0
            #           * condition\/.*
            #           {
            #               INPUT     = $MATCH
            #               INCLUDERC = $PMSRC/pm-xxscript.rc
            #               #  OUTPUT has the result
            #           }
            #
            #   Change Log: (none)

            # ..................................................... &init ...

            dummy       = "init: pm-xxscript.rc start"

            #  Read the standard variable definitions if they are not
            #  yet defined: that's "if WSPC variable does not contains space,
            #  as it should, then global variables haven't been read yet"

            :0
            * !  WSPC ?? ( )
            {
                INCLUDERC = $PMSRC/pm-javar.rc
            }

            # .................................................... &input ...
            # - User configurable variables with reasonable defaults
            # - But parameters like "INPUT" that must be set beforehand
            #   are not mentioned here.

            VAR1    = $VAR1{VAR1:-"default1"}
            VAR2    = $VAR2{VAR2:-"default2"}

            # .................................................... &do-it ...

            dummy       = "subroutine: pm-xxscript.rc parses now that and that"

            

            dummy       = "subroutine: pm-xxscript.rc end."

            # end of pm-xxscript.rc

19.0 Mailing list server

    19.1 Mailing list server pointers

       "Meng on procmail"
        http://res2.resnet.upenn.edu/procmail

    19.2 Simple Mailing list server

            # by Lars Hecking 
            #

            MAJORDOM = "majordomo-(users|docs|workers)"

            :0 w
            * $ ^(Sender|To|Cc):.*\/$MAJORDOM
            * $  MAJORDOM ?? ()\/$\MATCH
            | $APPNMAIL $LISTS/$MATCH

        Here is another, by Brock Rozen  with ideas
        from [dan]

            # get the date in RFC822 format for insertion into some messages;
            # the "Resent-Date:" field is copied from the "Date:" field on
            # some systems. RFC1123 says "All mail software SHOULD use 4-digit
            # years in dates..."

            LIST_NAME = "myList"
            LIST_ADDR = "$LSIT_NAME "
            LIST_DATE = `date '+%a, %d %h %Y %H:%M:%S %Z'`
            LIST_ERR  = "$EMAIL"        # my admin address

            #   Sendmail ignores "To:" in the presence of "Resent-To:"
            #

            :0 fhw
             *$ !^X-List: $LIST_NAME
             *$ ^TO()$LIST_NAME
             |  $FORMAIL
                    -A "X-List: $LIST_NAME"                                 \
                    -I "Resent-To: $LIST_ADDR "                             \
                    -i "Resent-Date: $LIST_DATE"                            \
                    -I "Errors-To: $LIST_ERR"                               \
                    -A "Precedence: bulk"                                   \
                    -A "X-Loop: $COMSAT"

            :0 a
            ! -oi `cat /var/tmp/src/power-users.list`

20.0 Common troubles

    20.1 Procmail modes: normal, delivery, and mailfilter.

          ... a) what recipes procmail goes through if there's no
          /etc/procmailrc on the system b) how it decides whether an
          address/local-part is valid or not c) how procmail selects the
          mailbox to drop the email

        [philip] *Delivery* mode is invoked using the -d flag. All
        arguments are the -d are usernames. It is usually used by the MTA
        to deliver mail to users, and indeed, procmail will return failure
        if it is given an invalid username. In delivery mode, procmail
        reads /etc/procmailrc before the user's .procmailrc.

        _Note_: Procmail will work in delivery mode only if it is
        setuid root, if it is invoked with the ruid of the recipient named
        in -d, or, under certain OSes where the build routines have
        determined that it is safe, if the euid is that of the recipient
        and the egid is the recipient's login group.

        *Mailfilter* mode is invoked using the -m flag. It accepts
        only one rcfile as an argument -- other arguments are either
        variable assignments or arguments that are made availible to the
        rcfile itself as $1, $2, etc. If the specified rcfile is located
        under /etc/procmailrcs/ then procmail will take on the uid of the
        owner of that file. Otherwise, it will run as the user who invoked
        it. /etc/procmailrc, that procmail -d reads, is ignored. In
        mailfilter mode, procmail unsets `ORGMAIL' and `DEFAULT' to
        suppress normal delivery -- reaching the end of the rcfile results
        in the mail bouncing. If the rcfile sets either of them then
        procmail will attempt delivery to that mailbox if it falls off the
        end of the rcfile; however, the mailbox will have to be writable by
        the uid/user that procmail is running as.

        _Note_: Only one rcfile can be named on the command line, but names
        of other rcfiles can be passed in the positional parameters to be
        used later in INCLUDERC assignments.

        *Normal* mode is invoked by not using the `-m' or `-d' flags. It
        accepts any number of rcfiles and variable asssignments as
        arguments. Procmail runs as the invoking user in this mode.
        /etc/procmailrc is ignored.

        So, to answer your questions: if procmail reaches the end of the
        specified rcfile, it bounces the mail (/etc/procmailrc is ignored).
        Everything is up to the rcfile -- how to determine whether the
        address is valid and where to put the message if it is.

    20.2 Procmail as sendmail Mlocal mail filtering device

          ...I'm a new sys admin at my company, and I've been trying to set up
          Procmail as the mail filtering device (still using mail as the
          Mlocal) I've tried setting up the sendmail.cf to use Procmail as
          a filter (we want to use the current mailer as the local mailer)
          with one local procmail rc file. Procmail seems to work just fine
          if set up as the local mailer, but I'm still having problems
          setting it as the filter.

        [John M Vinopal  answers sendmail.cf]

            R$+ < @ $=a . > $*
                $#procmail $@ /etc/mail/procmailrc $: $1 < @ procmail > $3
            R$+ <@ procmail > $*                      $1 < @ resort.com .> $2

        so this sends anything of the form foo@resort.com through procmail
        and rewrites it as foo@procmail. the procmail script reinjects it
        and it bypasses the call to procmail and then is rewritten back to
        foo@resort.com.

            /etc/mail/procmailrc:
            :0
            ! -oi -f "$@"

    20.3 Procmail doesn't pass 8bit characters

        You've mistaken. Procmail does not do that to your mail.
        Frank Gadegast  tells you:

        o   procmail wasnt the problem, it was sendmail
        o   I uncommented this line in sendmail.cf
            and now I get all nice German Umlauts.

                # strip message body to 7 bits on input?
                # O SevenBitInput

        The problem was that some mails run through the local mailer
        procmail and arrived all right (local mail), all mail from external
        (that dropped into my most used mailbox where I use a
        procmail-filter), did not arrive all right. This made me think it
        procmail, but these mails came from external and it was sendmail to
        blame.

    20.4 My ISP isn't very interested in installing procmail

          ...I recently requested my ISP to install procmail, and they
          responded by saying no. Their main reason was they did not wish to
          incur the traffic from any/ all of their subscribers setting up
          mailing lists.

        [Jon Lewis ] Wouldn't you need
        write access to either /etc/aliases or /etc/procmailrc to setup
        mailing lists? Tell the ISP that procmail will greatly improve mail
        delivery and enable all users to filter out junkmail without ever
        seeing it. If they still refuse, find a better ISP.

    20.5 My ISP has systemwide procmailrc; is this a good idea?

        [eli] I, for one, do not like my ISPs to put stuff in
        /etc/procmailrc. There is precious little I will gain from that and
        plenty of opportunity for them to make mistakes I would not have. At
        one ISP I know people got upset at some sendmail level filtering of
        email. One of those upset is a habitual complain-to-spammer-ISP
        person. He did not want problems seeming to go away if they were
        really there. Another guy just didn't trust the filtering.

        Writing a shell script that will give the user a .procmailrc which
        includercs a system wide shared procmailrc is the best way to do it.
        This forces the filtering to be "opt-in".

    20.6 Procmail changes mailbox and directory permissions

        By Ed McGuire . Before procmail was used:

            > -rw-rw----   1 foo      mail  1127 Sep 11 07:33 foo

        After:

            > -rw-------   1 foo      mail  1517 Sep 11 07:34 foo

        when the UMASK environment variable is more restrictive than the
        mode of the mailbox, procmail changes the mode of the mailbox. The
        default value of UMASK is 077.  If you want to preserve the group
        access to your mailbox, I think you can set UMASK to 007 in the
        rcfile:

            UMASK = 007

          Further note: the above `UMASK' suggestion in .procmailrc does not
          work. See comment by Gjermund Sørseth 

        However the permissions on `DEFAULT' are handled before procmail
        even opens the .procmailrc, so changing the umask there will have
        no effect on the mailspool.

        [Scott J. Kramer ] it's documented in the MISCELLANEOUS
        of the procmail(1) man page:

          If /var/mail/$LOGNAME already is a valid mailbox, but has got too
          loose permissions on it, procmail will correct this. To prevent
          procmail from doing this make sure the u+x bit is set.

        Otherwise, you might notice a syslog message like:

          procmail: Enforcing stricter permissions on "/var/mail/sjk"

        when it chmod's the file to 600. As you've discovered, this is
        inconsistent with the SYSV (Solaris 2 anyway) default mailbox
        protection of 660, gid=6 (mail). I think that's an OS-dependent bug,
        with the `chmod u+x ...' as the workaround.

    20.7 Changing mbox permission during compilation to 660

          ...it appears that mail that procmail delivers back into the
          spool it is writing out with owner.group user.mail and rights
          600. To me this is reasonable. Mail delivered to the spool by
          /bin/mail is written out owner user, group mail 660.

          When procmail delivers mail 600 later attempts at delivery with
          procmail removed from the .forward file fail: /bin/mail doesn't
          have permissions (or refuses to uses its permissions).

          Since we have fickle and unruly users who will be moving their
          .forwards in and out of place this is a problem.

          Is the correct solution to force procmail to write 660? If so, how
          is this done? I assume in the section of config.h just below the
          warning about only messing with a section if you think you know
          what you are doing. I don't like feel like I know well enough what
          I'm doing to walk into that territory without some guidance.

        [alan] I used to be the manager of the system support in the
        College of Engineering, at the University of California, Santa
        Barbara.

        We supported about 1500 users from two HP 9000 G30's, using one of
        them as the centralized mailer. Mail was available via NFS exported
        /usr/spool/mail to over 200 workstations, of all kinds: SGI, HP,
        Sun, etc.

        We replaced /bin/mail with procmail as the local mailer (Mlocal)
        because procmail correctly avoided NFS-locking problems, and it
        supported user-configurable mail filtering, without compromising
        system security.

        In over two years subsequent to the change, we had no loss of mail
        due to procmail being used as the local mailer. If you wish further
        comment from the current system managers, send email to
        "postmaster@eci.ucsb.edu".

        To answer your specific questions:

        * you can configure the permissions directly, by changing one of the
        following defines in config.h:

            /* bit set on mailboxes when mail arrived */
            #define UPDATE_MASK     S_IXOTH
            /* if found set */
            #define OVERRIDE_MASK   (S_IXUSR|S_ISUID|S_ISGID|S_ISVTX)
            /* the permissions on the mailbox will be left untouched */
            #define INIT_UMASK      (S_IRWXG|S_IRWXO)       /* == 077 */
            #define GROUPW_UMASK    (INIT_UMASK&~S_IRWXG)   /* == 007 */

        We did not find it necessary, however:

        o   We did disable all locking except dot-locking, since the kernel
            locks were the source of the NFS-locking problems. There have
            continued to be occasional locking problems, but these are
            "victim"-induced problems caused by using non-supported and
            discouraged mailers, such as "mailtool" from older Suns. These
            locking problems have nothing to do with mail delivery, but from
            the mail client using kernel-advisory locks, and then orphaning
            them or, leaving them locked all day long.
        o   An alternative to having users use .forward files, is to create a
            file of users who would like to use procmail as their local
            delivery agent, and use this file to initialize a class
            variable.

        Write a special rule in sendmail.cf which delivers mail using
        Mprocmail instead of Mlocal when the destination user is in the
        special procmail user class.

        This allows users who want procmail-direct delivery in spite of
        management worrying.

        I set this up to test procmail delivery on our system before
        changing Mlocal to use procmail. We placed some "volunteer" users in
        the procmail class file, and they never had any problems (I was one
        of them).

    20.8 The .forward file must be real file

          ...I tried to make a softlink to ~/.forward, but then my
          procmail wouldn't run. When I made a real ~/.forward file, then it
          worked again. My question is -- why would procmail treat a link to a
          file any differently than the actual file itself?

            ln -s ~/.procmail/forward ~/.forward

        [Werner Reisberger ] That's not a problem with
        procmail, this is an MTA issue. Due to security reasons sendmail will
        not deliver mail to files whicharesymlinks.

        [david] procmail has restrictions on what permissions it will tolerate
        on an rcfile. For example (I'm just guessing here) it can tell whether
        it can read the target file but it cannot tell who might be able to
        write to it. This prevents a major security hole

        You can make hard link to the file, since A hard link is completely
        indistinguishable from the original file. But note: a file hard-linked
        to two or more names is very distinguishable from a file with only one
        (hard) link, and procmail, for example, will not deliver to a plain
        folder that has two or more hard links.

        You can also put the real file at ~/.forward and let
        ~/.procmail/forward be a symlink to

        [< mikk0022@maroon.tc.umn.edu>] I suppose, the reasoning behind
        procmail's folder policy is that procmail locks the file by name, not
        inode. Hence it cannot guarantee mutual exclusion for access to a file
        which has multiple names.

        My understanding of the .forward policy is that a symlink need not
        share the permissions of its target. Therefore somebody's .forward
        symlink could have proper permissions, while its target could be
        writable by others. This would allow anybody with the write
        permissions to execute any program (potentially) from the user's
        .forward file.

        Two hard links share the same permission, so this argument doesn't
        hold.

    20.9 Using .forward if procmail already is LDA

          [Elie Rosenblum ] If you have a .forward, it is
          used by sendmail to replace a call to the LDA for the user in
          question. So if you have a .forward that doesn't call procmail,
          procmail is ...

        [david] Elie sent the answer to me with a carbon to the list, but
        since reading my personal copy my inbox got trashed. As of this
        writing the list copy hasn't reached me, but the rest of that
        sentence (as I recall from reading it before it got hosed) was to
        the effect that procmail is then never invoked at all on your
        incoming mail; a .forward takes precedence over the LDA. That
        scenario never occurred to me. Thank you for explaining.

          [Philip] Scratch the bit about /etc/procmailrcs/$LOGNAME. You're
          mixing up procmail -d with procmail -m.

        Ah, got it ... after rereading the man page. The part about
        /etc/procmailrcs really can apply only when procmail is setuid
        root, so again it's something I've no experience with and never
        quite followed or retained. So no file in /etc/procmailrcs is ever
        used implicitly, but /etc/procmailrc can be.

          [Philip] $HOME/.forward is handled by sendmail. If you have a
          .forward, then sendmail rewrites attempts to deliver to you into
          attempts to deliver to the addresses listed in the .forward file.

        Or in other words, the .forward takes precedence over the LDA.
        Thank you both.

    20.10 Mail should be put in the mailqueue if write fails

          ...We want to deliver directly to a user's home
          directory. But this can of course be temporarily full. Then the
          mail should _not_ bounce, but instead be put back in the
          mailqueue and tried again until either it succeeds or sendmail
          bounces it after 5 days (as usual). The README file says this is
          my choice (to bounce or not), but I cannot find any place where
          I can set this. What is the correct place to set this behaviour

        [1998-06-24 PM-L phil] The -t flag causes procmail to
        return EX_TEMPFAIL where it normally would have returned
        EX_CANTCREAT. If you've made procmail the local delivery agent then
        you should add -t to the A= define, _before_ the -d flag.

    20.11 Qmail: how to make it work with procmail

        [1998-11-10 PM-L John Conover ] All you
        do is install fastforward and dot-forward, (they are optional, and
        are not required.) Then cp /var/qmail/boot/proc or
        /var/qmail/boot/proc+df, to /var/qmail/rc.

        [1998-11-10 PM-L Greg Boes ] From the qmail
        FAQ (4.4 How do I use procmail with qmail?) Put

            | preline procmail

        into ~/.qmail. You'll have to use a full path for procmail unless
        procmail in in the system's startup PATH. Note that procmail will
        try to deliver to /var/spool/mail/$USER by default; to change this,
        see INSTALL.mbox.

        if you are trying to get procmail to deliver to maildir format as
        well, take a look at rhi for patches:

            http://www.qcc.sk.ca/~bguenter/distrib/procmail-maildir/

    20.12 Qmail: Procmail looks file from /var/spool/mail only

          ...Procmail seems to want to do something in /var/spool/mail. But
          since I use qmail, I don't have a /var/spool/mail. Is there a way
          to have procmail not to create temp stuff there?

        [philip] Get procmail 3.11pre7 and uncomment and and correct for your
        local setup the MAILSPOOLHOME="/.mail" define in src/authenticate.c.
        Compile and install. t's relative to the user's home directory.
        Thus the name `MAILSPOOLHOME'.

        [Ekkehard Knopp ] On the www.qmail.org page is a
        patch that lets procmail 3.11pre7 work with Maildir's, (qmail's NFS
        safe delivery format), and not must Mailbox's.

        Very useful. Really slows down delivery though. On my test box,
        just adding procmail to the delivery where all it did was deliver
        to the default mailbox, and no other rules whacked my speed test
        from something like 600,000 messages/day to about 180,000.

        Killer. I suspect Procmail's locking of the Maildir 8 ways from
        Sunday is probably partially to blame.

    20.14 AFS: How to use Procmail when HOME is in AFS cell

          ...I've viewed some of the archived posts concerning AFS and
          procmail, but each seems to have a different perspective on the
          subject. Besides the fact that AFS isn't the greatest product in
          the world, does everyone agree that it is not possible to use
          procmail when your $HOME lies in an AFS cell? Mail sent locally
          seems to work with procmail, but mail from users w/o a token or
          AFS id just gets delivered to /var/spool/mail/someone.

        [Christopher Lindsey  1998-03-09 PM-L] AFS
        is awesome! You just have to treat it nicely. :) The only viable
        solution that we've been able to come up with involves patching the
        procmail-3.11pre7 sources to "fake" user home directories out of
        another directory.

        For example, my home directory in AFS is

            /afs/ncsa.uiuc.edu/.u1/lindsey/

        It is kept as such on the mail server in /etc/passwd as well.
        However, we have some space set up via NFS in /var/forward with
        space for each individual user (so /var/forward/lindsey in my
        case).

        The procmail patch intercepts requests for the user's home
        directory and replaces it with the "fake" directory (the
        /var/forward one). So for all practical purposes, procmail things
        that my home directory is /var/forward/lindsey, and everything
        works fine.

    20.15 Help, some idiot sent my address to 30 mailing lists

        You can make a procmail recipe to junk incoming mail from the
        lists until you get the unsubscribe messages delivered to cancel
        your participation. You should complain to the list's maintainer
        that such things was even possible: The mailing list should have
        sent you a confirmation message with unique "participate ID number"
        that you need to send back in order for the subscription to take in effect.

            KILL_FILE = $PMSRC/.kill-immediately

            :0
            *$ ? $IS_READABLE $KILL_FILE
            {
                KILL = `cat $KILL_FILE`
            }

            # 1) Make sure KILL has value
            # 2) if match is found from header.
            # 3) /dev/null does not need lockfile

            :0
            *  KILL ?? [a-z]
            *$ $KILL
            /dev/null

        [sean] ...In the long haul, your best bet with dealing with this
        problem is to stamp out the offender - bring this harassment to
        the attention of their ISP and get their account closed. Repeat as
        necessary. Most of the mailing lists should have some record of the
        submission request. Even if forged, the abuser probably has their
        IP address in the headers somewhere (and if the person is actively
        subscribing your friend to so many lists and actually WORKING at
        covering their tracks, apparently you've REALLY crossed them). Most
        people who stoop to these immature harassment tactics aren't
        bright enough to fully cover their tracks.

        Another alternative to having to manually deal with unsubs on
        certain lists is once you've identified filterable characteristics
        of the lists, BOUNCE them. Most semi-intelligent listserv
        implementations will unsub you if they get repeated bounces. Yea,
        not nice to the listserv maintainer - but then, if perhaps they'd
        implement a subscription verification system, it wouldn't have been
        a problem to begin with.

            :0
            * condition
            {
                # may expose your .forward - but if you're bouncing lists,
                # it probably doesn't matter much.
                EXITCODE = 67

                # save header for examination.
                :0 h:
                bounce.log
            }

        You've got a sticky situation. You can't simply ditch all
        unrecognized mail - you need to be able to review potential refuse
        first, and take action on anything which doesn't belong (because
        you certainly don't want to continue getting the non-wanted lists
        till the end of eternity - you should want to unsubscribe from them
        to simplify your mail).

    20.16 Help, Procmail beeps and prints to my console

          ...when messages get filtered through procmail I get a beep and
          then first 10 lines or so are also sent to the console. I get a
          lot of messages so the beeps, and stuff on my screen is getting
          very annoying.

        [sean] One or the other should do the trick (or both even): Go to
        your login file (what it is named depends on the shell you're
        using), and add:

            biff -n

        Or/also, in your .procmailrc add:

            COMSAT = "no"

        [manual] has information on the COMSAT variable. It also states
        (contrary to reasoning I gave in above) that COMSAT defaults to
        'no' if you specify an rc file on the commandline (otherwise, it is
        on by default).

        Doing this latter one should keep procmail from generating
        COMSAT/BIFF notifications, but would still leave your shell capable
        of receiving them, say, if you only processed certain mail in
        procmail manually or somesuch. Personally, I turn biff off AND set
        the COMSAT off. I read my mail when I read my mail, and I check it
        often enough (with a POP client at that).

    20.17 Help, procmail dumps mail to console

          ...I have installed sendmail and procmail on my linux machine
          (latest version of slackware) it works ok, but procmail if run
          with -d $u dumps all mail after receiving immediately on the
          console with ---- more ---- I don't like it, a beep is ok, but I
          do not want all the garbage on my screen. Is there a way to tell
          procmail that I just want the mail in my mailbox
          (/var/spool/mail/$u) ? Thanks for the help!

        [Xavier Beaudouin ] Check your /etc/inetd.conf for a
        in.comstat, add a '#' at the beginning of the line, save the file
        and killall -HUP inetd. This should stop this ;-)

    20.18 Help, corrupted From_ line in mailbox

         [Jeffrey S. Gilton 
         1998-02-11 in procmail mailing list " Solved the FFrom problem"]

        Thanks to everyone who responded to my questions about a problem where
        the From line was getting corrupted. Here I tell what was the real
        problem.

        To recap, when our Caldera OpenLinux 1.1 system received multiple
        email messages very quickly, some messages would get multiple F's on
        the from line and then subsequent messages would be missing the F's.

        Most responses said that it sounded like a file locking problem.
        Suggested solutions were to get the latest version of procmail or
        recompile our version so that it would look at the file locking
        mechanisms.

        The funny thing was that three systems with new installs didn't
        exhibit the problem.

        The file locking recommendation eventually led to the real problem. On
        a good system I would run our spam script (we spammed ourselves to
        trigger the problem) and everything would work. Using top I would see
        multiple instances of procmail running. Looking at the directory where
        the spool files were, I would see a spool_file.lock file get created
        and then go away.

        Finally, I did the exact same thing on the system that wouldn't work.
        There I would see the multiple instances of procmail running but no
        lock file being created. I said to myself "Now that I know what is
        happening, the question is why."

        It turned out to be a permission problem on the spool directory. On
        the system that worked, the permissions were rwxrwxr-x with the owner
        being root and the group being mail. On the system that didn't work,
        the permissions were rwxr-xr-x with the owner and group being root.
        This meant that procmail, which is run as mail couldn't write the
        directory file. We changed the broken system to rwxrwxr-x with owner
        root and group mail. The problem disappeared.

        As I said, the suggestions about lock files were key. It guided our
        investigation until we found the real problem. I thank everyone who
        responded.

        I've seen other posting about corruption of the From line. Perhaps you
        have the same problem.

        [Christopher B. Smith ] I had the exact same
        problem with my upgraded OpenLinux system. For the record, if you are
        running the imapd that comes with it, you should really set your
        permissions for the directory is as follows:

            chmod 1777 /var/mail/spool

        I got that feedback from the guy who wrote imapd, and it works very
        well.

    20.19 Directing user's mail to HOME instead of /var/spool/

          ...I have a need to direct all a single user's mail to a mailbox in
          his home directory, to $HOME/mailbox,

            # One possible solution, not perfect

            UHOME       = /tmp_mnt/users
            UHOME_LIST  = "(login1|login2|login3)"

            *$ ^TO\/$UHOME_LIST@
            *   MATCH ?? ()\/[^@]+
            $UHOME/$MATCH

        [era] Perhaps preferably use ^TO_ if you have Procmail 3.11pre7
        or newer. This is the classical case of using Procmail where you
        really need the envelope recipient information. The headers are
        +not+ enough to determine who a message is for. If Procmail is your
        MDA, you can have this, but I'd still think something involving
        Sendmail would be more appropriate. For one thing, what if this
        user would suddenly really want to use Procmail? You can set
        `DEFAULT' and `ORGMAIL' for this one user in `/etc/procmailrc' to come
        around that, but the bottom line, as so many times before, is that
        Procmail might not be the right tool for this.

    20.20 NFS mounting /var/mail is a good way to get bad performance

        


            > /var/mail stays at a Solaris 2.5 machine. Cucipop is working
            > at the same machine. It's fine there. But, I want to have
            > more than one machine with cucipop and when I put cucipop at
            > another machines, NFS clients, it is delaying more 30 or 40
            > seconds to close the session.

        [1998-06-23 PM-L Brad Knowles ] NFS mounting
        /var/mail is a good way to get bad performance, especially when
        you're doing any NFS writes. Even if you're not doing any NFS
        writes, just having to deal with local file locking and trying to
        translate that into NFS file locking is a nightmare (in general,
        file locking is one of the single biggest problems left with NFS).

            > Procmail is working good on NFS, it finishes quickly. But when
            > cucipop is put on a NFS client, procmails starts to delay too.

        Procmail probably isn't writing to NFS, or if it is, it's probably
        not using the same locking mechanism as cucipop. Unfortunately,
        each vendor and each program have their own ideas on how to best do
        that.

          [philip] cucipop was written by the author of procmail. Ideally,
          when you compile cucipop you edit its config.h to use the locking
          techniques that procmail's autoconf process determined for your
          system(s). However, even if you didn't do that, cucipop uses the
          same dotlocking algorithm as procmail.

        Also, keep in mind that any POP3 server will have to copy the
        mailbox in order to work on it, and many of them copy the mailbox
        to /var/mail/.username (you got it -- creating lots of NFS writes).
        When they're done, they copy the mailbox back to /var/mail/username
        (after they copy any new mail messages that have come in to the end
        of /var/mail/.username and locked then truncated the original
        /var/mail/username file).

          [philip] cucipop doesn't use a temporary file: it keeps it all in
          memory. On deletes it updates the mailspool in place which should
          never _lose_ data, though if the server crashes in the middle of
          this you can end up with one or more bogus messages.

        This is a *real* nightmare when you start talking about users who
        select "Leave mail on server" and have multi-megabyte mailboxes.

          [philip] Assuming you have enough memory, cucipop should be pretty
          fast.

        I think maybe now you're starting to understand why POP3 really
        doesn't scale well at all in multi-machine environments (unless
        you've cooked up a custom mail store that uses a real database
        back-end, like Oracle Parallel Server), with /bin/mail (or
        procmail) as a writable interface to this message store and POP3
        and/or IMAP as a readable (and writable) interface to this same
        message store. Then you can let the database vendors deal with the
        hard data replication and distribution problems.

        Otherwise, it's a pain-in-the-ass.

            > Is there another good pop server?

        Have you tried QPopper from Qualcomm? It's the single best POP3
        server I've ever run across, although I wouldn't put even it in an
        NFS write environment.

        BTW, I used to be the Mail Systems Administrator for GNN (Global
        Network Navigator), the web site/National ISP co-operative between
        O'Reilly & Assoc. and AOL. At our peak, we had hundreds of
        thousands of registered users, of which up to five to six thousand
        were logged in at any one time, with their MUA set to check their
        mail every minute.

        We had a single primary Mail/POP3 server machine (Dec Alpha 2100 w/
        four 250Mhz processors, 4GB RAM, 28GB hardware mirrored/striped
        mail spool), and one warm spare (same CPU/RAM configuration,
        physically hooked up to the same disks, but through DECsafe ASE not
        mounting them unless the primary died).

    20.21 I can't see the sendmail's response in LOGFILE

          ...As the man page says, this should've written to my LOGFILE. It
          didn't. But it DID activate the pipe in the recipe. So what's up
          here?

            :0 hc
            *$ ? $IS_EXIST $HOME/.vacation
            | LOG=| ($FORMAIL -r; echo $IM_NOT_HERE) | $SENDMAIL -t

        [david] The man page says that a variable capture recipe assigns the
        standard output of the command to the variable. Since you are repiping
        the output of formail and echo to sendmail, sendmail sucks up the
        standard output of formail and sendmail. Sendmail itself does not
        write to standard output, so the stdout of ( $FORMAIL -r ; echo
        $IM_NOT_HERE ) | $SENDMAIL -t is nothing.

        Thus you're assigning a null string to $LOG, and when procmail writes
        $LOG to the logfile you can't see a difference.

    20.22 Compiling procmail and choosing locking scheme

        General advice: Everything except dot locking is usually broken.

        [stephen, <199607292139.XAA12433@hera.cuci.nl>]. Remove fcntl() and
        lockf(), only allow flock() (or omit it completely) Kernel locks
        don't work. But that's all some programs use. Across a networked
        filesystem, lockf() doesn't work, fcntl() and flock() should, but
        they don't either because the lockd is buggy. Mailtool uses fcntl()
        but does it wrong, so that's another problem. The only thing that
        works on all platforms, all networks, all the time are .lock files.

        Makefile refers to:

            # Uncomment (and change) if you think you know
            #LOCKINGTEST=100
            #        it better than the autoconf lockingtests.
            #        This will cause the lockingtests to be hotwired.
            #        100     to enable fcntl()
            #        010     to enable lockf()
            #        001     to enable flock()
            #        Or them together to get the desired combination.

        config.h refers to:

            /*#define NO_fcntl_LOCK uncomment any of these three if you */
            /*#define NO_lockf_LOCK definitely do not want procmail to make */
            /*#define NO_flock_LOCK use of those kernel-locking methods */

    20.23 Forwarding lot of mail causes heavy load

          ...There are several forward (e.g. ! walter@localhost) recipes
          For every forwarded mail, a distinct sendmail process is created.
          This leads to a heavy (IMHO unbearable) system load. How can I
          stop procmail from running a sendmail process for every mail
          forwarded?

        SUMMARY: Look at qmail, it's better than sendmail.

        [era 1998-08-15 PM-L] (Blows dust off old underutilized Bat
        Book/ORA sendmail book) Yeah, setting QueueFactor (q) and QueueLA
        (x) to suitable values should do what you want. You need to have
        load-balancing support compiled in, though; according to the Bat
        Book, sendmail -d3.1 tells whether you have it or not. (Mine just
        says getla:0 which I would imagine means I have the support but the
        load average was below the cutoff level.

          AFAIK using load averaging would have the first messages
          delivered and the rest queued. However, also not being a sendmail
          guru, I do not know how to empty a sendmail queue for incoming
          mail only. Moreover, even if I knew how to do this, it would have
          to be done after procmail finishes.

        [Liviu Daia ] Instruct sendmail to queue
        messages when called from procmail:

            SENDMAILFLAGS="-oi -od d"

        then disable the normal sendmail daemon from your system init
        scripts, and run it in flush queue mode only, that is, replace

            /usr/sbin/sendmail -bd -q 15m

        in your init scripts with

            /usr/sbin/sendmail -q 15m

        ("15m" is how often the queue will be run (15 minutes).
        Change it to whatever is appropriate
        for your purposes). Also make sure to disable forking in your
        sendmail.cf.

        The downside of this approach is that it will also delay the
        delivery of local messages.
        Different approach: pipe messages to sendmail instead of using
        '!' and use the wait flag. Something along the lines of:

            :0 w
            * conditions
            | $SENDMAIL $SENDMAILFLAGS 

        Well, I'm actually not sure you can use the 'w' flag without 'f'
        (the manual doesn't say it, and I'm not too familiar with procmail
        internals), so if that doesn't work you might also try Sendmail
        will rewrite the `From_' header (which you can probably safely
        ignore), and it will (optionally) add a `From:' if one doesn't
        exist, but it won't touch an _existing_ `From:'. Well, actually it
        will encode or decode any 8-bit characters in the `From:' according
        to the options in sendmail.cf, but it won't change the _meaning_ of
        the "From:". In fact, that's exactly what procmail does too in the
        '!' recipes.

            :0 fw
            * conditions
            | $SENDMAIL $SENDMAILFLAGS 

            # dummy recipe to stop procmail from delivering an empty message
            :0
            a /dev/null

    20.24 What happens to mail if MDA Procmail fails

          ...When procmail is the local mailing agent distributing
          e-mail to a user's $HOME and the target machine is 'down', where
          does the e-mail go? I was given the impression that the mail
          would be collected on the 'mailhub' in /usr/mail/BOGUS.xxx
          (Solaris system). It is not happening and we have the potential
          of losing mail.

        [philip] I assume that by "target machine" you mean the NFS
        server for the given user's account. Procmail's attempt to read
        ~/.procmailrc will timeout, then when it tries to write to $DEFAULT
        (which you say is in their home directory) it'll time out (again)
        and return EX_CANTCREAT to sendmail. Sendmail will then presumably
        bounce the message.

        Now, if sendmail is looking for .forward files in user home
        directories, then procmail will never be called, as sendmail will
        try to open the .forward file and consider it a transient error
        when it times out, causing the message to be queued for a later
        delivery attempt.

        (Note: invoking procmail with the -t flag causes it to return
        EX_TEMPFAIL instead of EX_CANTCREAT. This would cause the message
        to be requeued. However, this is not generally recommended.)

    20.25 Procmail reads entire 90Mb message into memory

          ...last week my workstation ground to a halt when procmail received a
          90Mb Email message (ran out of memory). The point is, such
          message sizes are fine by me, as long as the system can handle
          it. Is there any way I could make procmail only read the headers
          of that message before scanning /etc/procmailrc/ ~/.procmailrc and
          acting on it? That way it wouldn't need to read the entire
          message into memory.

          ...Recently, I modified the sendmail.cf file to pipe messages
          through procmail before sending them to deliver, so that I can
          use system-wide procmail recipes for spam filtering. However,
          yesterday we had a client send a 22 megabyte e-mail message (on
          purpose, no less) and the system just came to its knees trying to
          deliver it to the user's mailbox.

        [philip] Btw, All the versions of /bin/mail (or mail.local) that
        I've seen the source for either read the entire message into memory
        first or use a temp file. Depending on where temp files are
        located, a 90MB temp file may be just as bad as holding it in
        memory.

        And, No, there isn't. Hacking it in would not be non-trivial, mainly
        because the current code runs with the assumption that the entire
        message is there, and determining when it actually needs to see the
        entire body (to do demand loading) would not be easy. Remember
        that a condition on the size of the message, ala

            :0
            * > 10000000
            /dev/null

        would require the body to be read...  It really is just better to
        simply have sendmail enforce the limit. You should be doing it
        there anyway to cut down on the totally trivial denial-of-service
        attacks and because it's more efficient.

          ...I am running procmail ver 3.11pre7 and I keep getting "out of memory
          as i tried to allocate 8xxxxxx bytes.". I have over 100 meg
          available swap space so i have a difficult time understanding
          this. Is this a known error?

        Procmail's memory allocation technique appears to non-optimal for
        some OS/libc combos, namely implementation of the libc system
        function realloc() (FreeBSD has been reported). It's conceivable
        that the configuration process could be enhanced to detect this
        system limitation to use a strategy more efficient on them. Don't
        hold your breath.

        [ed] There is a patch available that should fix the problem
        for you. See the messages at
        .

    20.26 Help, procmail uses occasionally huge chunk of memory

          ...we've noticed that occasionally, procmail uses a huge chunk of
          memory. It's always the same 17MB as reported by the top command.
          Can anyone enlighten me as to why sometimes procmail creates such
          a huge footprint and other times doesn't, for the same user with
          an unchanged .procmailrc file?

        [ed] Is your operating system a BSD variant such as FreeBSD or
        OpenBSD? If so, then the problem is due to a poor implementation of
        the Standard C Library system function realloc() on those
        platforms. A patch that works around this is available. See the
        messages at

        

        Specifically, the patch is located at

        .

        It's an artifact of procmail's memory management. It reads
        an entire message into memory before working on it. Fear the system
        with procmail as the local delivery agent, where people are
        slagging 100M CAD files around. :-)

    20.27 Procmail signalled out of memory in my verbose log

          ...I notice in my procmail verbose log the following
          'transaction':

            procmail: [10239] Sat Jan  9 08:49:02 1999
            procmail: Out of memory
            buffer 0: "formail"
            buffer 1: " formail -A "X-Check: List""
              Folder: **Bounced** 5744
            procmail: Notified comsat: "bhoule@:**Bounced**"

          If I act quick enough when this happens, I can look in
          spool/mqueue and find a message with a gazillion addresses in the
          To: line. So it seems that formail is having trouble adding my
          X-Check header to an already large set of headers.

        [philip] No, it's procmail that's unable to allocate enough memory.
        The buffer dumps indicate that procmail was unable to get enough
        memory somewhere between parsing the action line and reaching the
        next recipe -- buffer 0 would not contain the string "formail" if
        procmail had gotten to another recipe or variable assignment.
        What's weird is that the message is so small (only 5744 bytes
        according to procmail). Do you only see this error on this recipe,
        or at random places in your .procmailrc? If the later, then I would
        guess that your mailserver is running out of memory for some other
        reason and that procmail happens to be an innocent bystander. If
        the former, then, well, I'm not sure.

          The message is never delivered to me. Is there anything I can do
          so that procmail/formail will act as if it was never there so the
          incoming dumps into my inbox rather than returning an error to
          the mailer? This "**Bounced**" business is not a very helpful
          action.

        Giving procmail the -t flag will cause fatal internal errors that
        are normally returned as permanent errors to be returned as
        temporary failures instead. Otherwise there's no way to control
        that. (Setting EXITCODE won't work because procmail needs to malloc
        memory to handle TRAP and EXITCODE, and it'll refuse to try that
        when it was malloc that caused the exit.)

    20.28 Variables DEFAULT and ORGMAIL

          ...According to the man pages, `DEFAULT' is defined as `ORGMAIL'
          ...so if I redefine ORGMAIL, then `DEFAULT' should change as
          well, which doesn't help me. Any help on this would be
          appreciated

        [david] `DEFAULT' is initially defined as equal to `ORGMAIL'. Once
        procmail has started reading /etc/procmailrc (if it is the MDA) or
        your .procmailrc, you can change the value of either without affecting
        the other.

        In fact, you can even set DEFAULT on the command line when you invoke
        procmail (I'm not sure about doing that with `ORGMAIL', though), and
        that value will override its normal initial value equal to `ORGMAIL'.

        What if it is possible that dropping to DEFAULT fails due to disk full?
        Then you would better have another drop place in another file system.
        Peek at bdf(1) or df(1) to find out the different mounted file systems.

            # Place this to the end of your .procmailrc and define
            # DEFAULT_SECONDARY

            :0 :
            $DEFAULT

            :0 E
            $DEFAULT_SECONDARY

        If you deliver explicitly to $DEFAULT, procmail treats it like any
        other save-to-folder recipe, and if the write fails, it continues
        reading recipes.

          ...If I had set the "deliver" destination as `ORGMAIL' rather than
          `DEFAULT,' would it have made any difference?

        Nope. If you write a recipe for it, procmail just expands the variable
        and doesn't give a heck if it happens to be the same destination as
        `DEFAULT' or `ORGMAIL'. `DEFAULT' is special to procmail only when it
        uses it on its own after falling off the end of the rcfile; `ORGMAIL'
        is special only at startup (without -m) and when procmail falls off
        the end of the rcfile and finds that it cannot save the message to
        `DEFAULT'.

          In general, if procmail falls off the end of the rcfile,
          fails to save to `DEFAULT', and then fails to save to `ORGMAIL',
          does it revert to the compiled-in value of `ORGMAIL' ?

        [philip] Procmail has no fallback beyond the current value of
        `ORGMAIL'. If delivery to both `DEFAULT' and `ORGMAIL' fail, then
        procmail gives up and exits with error code 73 (EX_CANTCREAT) or 75
        (EX_TEMPFAIL), depending on whether the -t flag was given. Setting
        `EXITCODE' would probably override those. The message is logged
        as "*Bounced*".

    20.29 When DEFAULT cannot be mailed to

        If procmail gets to the end of the rcfile without delivery (or without
        being directed to another rcfile by an INCLUDERC or HOST assignment),
        it assumes these:

            :0:
            $DEFAULT

            :0 e:
            $ORGMAIL

        That is, it tries to deliver to $DEFAULT and if it can't, it tries
        $ORGMAIL. If that fails too ("deep, deep trouble" as Stephen says in
        the man page), it exits without delivery and reports failure to the
        MTA, which, depending on other factors, will either requeue the letter
        and try delivering later or will bounce it to the sender.

    20.30 Variable DROPPRIVS

          ...I have procmail invoked from a mailtable for a virtual domain.
          Presently that runs as root, inherited from sendmail. I'd like to
          have it run less privileged. I tried chown'ing the rc file to the
          user I want used and setting "DROPPRIVS=yes". That didn't do it.
          So I added "LOGNAME=user" and "USER=$LOGNAME" before the
          DROPPRIVS assignment and that didn't work.


        [philip] DROPPRIVS only has an effect inside the /etc/procmailrc used
        when procmail is running in delivery mode (-d), not when it's
        running in mailfilter mode (-m). USER and LOGNAME have no effect on
        the working of DROPPRIVS, as procmail is just going to change to
        the uid/gid of the user specified on the command line after the -d.
        Your mailtable entry should be specifying the procmail mailer,
        which runs procmail in mailfilter mode.

        If the following are true:

        o   procmail is running in mailfilter mode
        o   no assignments were given on the command line
        o   the -p flag was not specified
        o   the rcfile specified is located under /etc/procmailrcs/ without
            backwards references ("/../"s)
        o   the rcfile is not a directory (duh!)

        then procmail will assume the uid and gid of the owner of the
        rcfile. If the rcfile is actually a symlink, the procmail will
        assume the uid and gid of the link itself, not the underlying file.
        If your OS allows anyone to give away ownership of files with
        chown, the procmail adds the following restriction to those above:

            /etc/procmailrcs must be owned by root and mode 700.

    20.31 Variable HOME

        [david] Since procmail doesn't understand tilde, you have to use
        variable HOME instead.

            CONTENT   = `cat ~/file.txt`        # Won't work
            CONTENT   = `cat $HOME/file.txt`    # ok

        But accessing other user's home is another story. You could change
        the SHELL temporarily to get procmail understand the reference,
        like this:

            SHELL   = /bin/csh
            CONTENT = `cat ~user/file.txt`
            SHELL   = /bin/sh                   # restore original setting

        Because the tilde is in $SHELLMETAS, when procmail sees a tilde,
        it will invoke a shell. It's better to skip the extra process of a
        shell and use the $HOME variable: put a symlink somewhere under
        your own home directory that points to the other user's file so
        that you can use the $HOME variable in your .procmailrc and avoid
        the shell invocation.

        However, there are dangers on this too, because sysadm may move
        home directories and your symlinks may be out of date. If you
        expect such changes and broken links, then you could cache the
        needed home directories at time you need them:

            HOME_PHIL   = `ksh -c "echo ~phil"`
            HOME_ED     = `ksh -c "echo ~ed"`

    20.32 Variable HOST

        [philip] If a assignment to the "HOST" variable occurs where the
        assigned value doesn't equal the hostname of the machine on which
        procmail is running, procmail will stop reading the procmailrc, and
        if there are other procmailrcs specified on the command line, it
        will start reading them.

        [david] It goes back to the early days of procmail, before Stephen
        thought of INCLUDERC or the "var ?? condition" syntax. When people
        had to use different code based on which local host machine was
        processing a particular message, the method was to list a number of
        rcfiles on procmail's command line. The first one would start out
        with general code for all messages and all hosts and then have a

            HOST = some.specific.machine

        assignment, followed by code for mail delivered on that machine.
        If the first nine characters of "some.specific.machine" matched the
        real value of $HOST, procmail would stay in that rcfile; on a
        mismatch, it would jump to the second rcfile named on the command
        line.

        The second rcfile would probably be for another particular machine,
        so (unless it first had some universal code for all machines except
        the first one, or unless there were only two machines where
        procmail might run) right at the top it would have

            HOST = this.specific.machine

        Again, a match for the first nine characters would keep procmail
        reading this rcfile, but a mismatch would make it jump to the next
        rcfile.

        And so it went. An incorrect HOST assignment (note that "HOST"
        alone attempts to unset the variable, so it is *always* an
        incorrect assignment) in the last rcfile on the command line made
        procmail drop the message and exit. Since we almost never name
        more than one rcfile on the command line now, attempting to unset
        HOST in .procmailrc will have that effect.

        I would guess that the only use of this original setup still around
        is in SmartList, where flist invokes procmail with a number of
        rcfiles on the command line and uses things like
        HOST=go.to.the.next.rcfile.now to move from one to the next. Also,
        procmail's -m facility (which didn't exist back then) is
        incompatible with using HOST to jump among rcfiles, because it
        requires naming exactly one rcfile on the command line.

        Nowadays we can do something like this to use different rcfiles on
        different hosts:

            :0
            * HOST ?? ^^\/[^.]+
            {
                INCLUDERC = $HOME/.$MATCH.rc
            }

    20.33 Variable LINEBUF

          ...[manual] Length of the internal line buffers, cannot be set
          smaller than 128. All lines read from the rcfile should not
          exceed $LINEBUF characters before and after expansion. If not
          specified, it defaults to 2048. This limit, of course, does not
          apply to the mail itself...

        _Note:_ Beware of simply setting LINEBUF to a huge value: such an
        assignment causes procmail to immeadiately allocate twice that much
        memory (procmail has two buffer internally of size $LINEBUF).

        [philip] Those 160 lines of condition are almost certainly
        overflowing LINEBUF. You should either a) use one of the
        innumerable recipes sent to the list demonstrating the use of
        fgrep; b) break it into multiple recipes; or c) increase LINEBUF.
        If you modify this list of domains regularly, then you should
        strongly consider (a), as (b) and (c) just put off it happening
        again.

        `LINEBUF' only applies to lines from procmailrcs. You generally only
        have to worry about `LINEBUF' when you have a variable expansion or
        command expansion (backquotes) that doesn't have an obvious and
        reasonable bound on its size. procmail will avoid overrunning its
        `LINEBUF' length buffer when doing command expansions by ignoring the
        extra output, so you're safe there, as long as data truncation is
        fine. Variable expansion isn't checked like that, so you can cause
        procmail to coredump by doing something like:

            :0
            * ^Subject: \/.*
            |some-program $MATCH

        then feeding procmail a message with a huge Subject: header field:
        since no shell meta characters appear in the action, the action
        line will be expanded and exec()ed by procmail directly instead of
        by the shell. On the other hand, the following is fine:

            :0
            * ^Subject: \/.*
            |some-program $MATCH ; ;

        The semicolon forces a shell invocation, and the shell should be
        safe. If your /bin/sh can buffer overrun on variable expansion, then
        you're in more trouble than you know.

        Action lines aren't the only place to watch your variable
        expansions. Variable assignments and condition lines that have a
        leading dollar sign also undergo expansion. For example, this isn't
        safe:

            SUBJECT = `$FORMAIL -x Subject:`
            NEWSUBJ = "Subject: $SUBJECT"

        procmail won't buffer overrun in the first line, but a really long
        subject could cause the second to do so. The following should be
        safe:

            NEWSUBJ = "Subject: `$FORMAIL -x Subject:`"

        but even then only if you're sure the shell is doing the expansion
        of NEWSUBJ.

        Note that matching against the value of a variable (using the "var
        ??" condition special) is safe no matter what the size of the
        contents of the variable. The problem is when you interpolate the
        variable into something else.

          Is there any easy way to know default LINEBUF value for
          specific procmail? I'm sure there's a much easier way, but this
          will work:

            #   Mitsuru Furukawa
            #
            $OUT    = $HOME/tmp/linebuf.lst

            :0 wc:  $OUT$LOCKEXT
            *$ ! ?  $IS_EXIST $OUT
            | echo "$LINEBUF" > $OUT

        [philip] If you examine the procmailrc manpage, you'll note that it
        lists fourteen variables (among them DEFAULT but not LINEBUF) whose
        values are reset in the environment by procmail, plus some
        additional ones like IFS, ENV, PWD, and PATH which come out of the
        top of config.h. Following this is a list of all of procmail's
        magic variables, including those fourteen. The idea is that while
        procmail has thirty magic variables, only fourteen of them are put
        into the environment by procmail.


        The others may have default values, but they're 'input only': if
        what you're doing depends on one of the others having a certain
        value, then you should just go ahead and set it to that value. I
        know of only two ways to find out what value procmail is using by
        default: a) check the manpage (the manpages should show the correct
        default for the machine), or b) fire up your favourite debugger and
        hope that no one stripped the procmail binary.

        There will be no error message when Procmail dumps core, even
        though the reason is apparently precisely that `LINEBUF' is being
        exceeded too much.

          Is there a limit on the length of a single line

        [david] Yes, both before and after variable expansion and
        command substitution, it must be shorter than `LINEBUF' characters.
        The exceptions are (1) comments and (2) commands that are run by a
        shell rather than directly by procmail. The entire condition must
        be under `LINEBUF' characters

        Unfortunately, LINEBUF seems to be a write-only variable; you can
        change its value but you can't find out its current setting.

    20.34 Variable LOG and LOGFILE

        If you want to print something to the `LOGFILE', you could do it
        like this

            LOG = "  This message goes to LOGFILE"
            LOG = " $NL$NL  And this has linefeeds around $NL$NL"

        Or like this, which proves to have some nice feature in respect to
        `VERBOSE' setting:

            dummy = "  This message goes to LOGFILE"
            dummy = " $NL$NL  And this has linefeeds around $NL$NL"

        You see, if you set `VERBOSE="off"' Then the `dummy' lines are not
        printed and recorded to the `LOGFILE'. `LOG' messages are aways
        printed, and that's not very nice if you're trying to suppress
        messages while you call some subroutine:

            saved   = $VERBOSE
            VERBOSE = "off"

            #   Hope this subroutine does not use LOG
            #   Eg. $PMSRC/pm-jaaddr.rc

            INCLUDERC = $RC_ADDR

            VERBOSE = $saved            # restore original value

    20.35 Variable TRAP

        Here is one example how to write to the logfile, Be sure that you
        have preset all the variables, this just demonstrated the usage of
        `TRAP'. Pay attention to right use of single and double quotes if
        you pass the values to the shell. Like in this example where the
        `/dev/' is removed from the `FOLDER' variable's value.

            TRAP = 'echo "
            FROM    $FROM
            TO/CC   $TO / $CC
            SUBJECT $SUBJECT
            FOLDER  $LASTFOLDER
            " | sed -e "s#FOLDER   /dev/#FOLDER   #g"'

        And if your MUA expects the file to be touched before it sees new
        incoming mail, here is recipe by [david]:

            TRAP = 'touch -m $HOME/Mail/$LASTFOLDER' # with strong quotes

        Place it early in your rcfile; then each recipe that saves to a
        directory can look simply like this, and the trap will take care of
        the touching:

            :0 flags # no local lockfile needed for save to directory
            * conditions
            directoryname/.

        [david] Procmail terminates when it exits ... after final delivery
        of the message. It doesn't terminate (nor execute TRAP) after
        delivering a copy to a `c' recipe [however, a clone does execute
        TRAP when it terminates, unless you unset TRAP for it]. It doesn't
        execute the trap after a variable assignment, a variable capture
        recipe, a filtering recipe, nor any other non-delivering action.

        On the other hand, it *does* execute the trap if you do a quick
        bail-out by unsetting or missetting $HOST.


         [Recipe to record Subject lines on exit]

        <> [david] ...this will
        list all subject lines in the logfile upon exit if there are two
        or more. The earliest would appear twice: once in the trap output
        and once in the logabstract.

            :0
            * ^Subject:.*$(.+$)*Subject:
            {
                #  If there is already `TRAP' set, combine the
                #  old trap recipe with this

                TRAP = "${TRAP:+$TRAP ; }$FORMAIL -XSubject:"
            }

    20.36 Variable UMASK

        There is a better way to find out which folders contain new
        mails if you are using procmail to filter the mails. (This was a
        hack by one of my friends) procmail allows you to set UMASK on the
        folders. So before doing anything, set UMASK to 076, which means the
        perms will be -rwx-----x to any folder which receives mails. now
        using `find' -perm -001, you can print the folders which have new
        mails. the shell script which does this will also have to chmod o-x
        on all these folders.

          ...How does this work? AFAIK umask only applies to new files created
          and not to appending to existing files which is what procmail
          essentially does, right?

        [era] Procmail does interpret UMASK this way, so this works,
        but I don't think it's a particularly good solution. It's actually
        hinted at in the documentation for UMASK in procmailrc(5). `find' is
        a rather heavy program to start up every time you want to look for
        mail. (Haven't done any timings, though.)

        o   I just grep -c '^From ' on my mail folders to see how many
            messages there are in them. (This is only an approximation, in
            the case where one or more messages contain unescaped From_
            lines.)
        o   For a really pedestrian solution, keep all your spool files in
            their own directory (I think this is a good idea for other
            reasons as well) and do an ls -lrt on that directory, possibly
            piped into a sed script to trim off files with timestamps older
            than, say, 24 hours.
        o   If your mail reader will reset permissions on spool files when
            it gets mail from them, the UMASK trick is a good base for a
            mail checking script, but I would then only ls -l the spool
            files and look for files with an x01 permission.

    20.37 UMASK and permissions

          My mail folder says -rw-r--r-x, Is there a bug in Procmail's
          umask handling? (see last x bit)

        [philip] That's a feature, not a bug! To quote the procmailrc(5)
        manpage:

        _UMASK_: The name says it all (if it doesn't, then forget about
        this one :-). Anything assigned to UMASK is taken as an octal
        number. If not specified, the umask defaults to 077. If the umask
        permits o+x, all the mailboxes procmail delivers to directly will
        receive an o+x mode change. This can be used to check if new mail
        arrived.

          Anyhow, normally, under Unix, the create system call will set
          default permissions of 666 and the umask can only be used to mask
          off the bits you don't want (and not to e.g. add x bits).
          Shouldn't Procmail work this way, too, just to be consistent with
          the rest of the system?

        creat() will set the permissions to whatever you want it to,
        modulus the umask. If the umask is zero, you can set the
        permissions to 7777, though that would be kind of stupid (and
        actually, most versions of UNIX won't let you set both the sticky
        bit and an executable bit unless you're root, for historical
        reasons). Most programs that call creat() or open(..,O_CREAT,...)
        give a mode argument of 0666, as they generally don't write out
        executables. Procmail just happens to call open() with a mode
        argument of 0667, to be modified by your umask.

    20.38 Performance difference between backtick and "|" recipe

        Procmail sends the whole message to stdin whenever
        it sees backticks used. And if you use recipe, you can add
        the `h' flag to feed only the header to the program, and not the
        whole message. Let's ask academic question: Which one
        of the choices below is efficient?

            # Side effect: Do something with shell
            dummy = `echo hi there > some-file.txt`

            :0 hwic
            | echo "hi there" > some-file.txt

        Procmail sends whole message to first line and only headers
        to second recipe. Answer: It doesn't matter. Either way procmail
        will make one write system call which will return 0 [bytes written]
        and off it goes. You should use the first one, because the latter
        affects the `A' and `E' flags later, first one is more clear overall.

        While someone suggested following, it was rejected because it
        hurts performance more [stephen]. The cat process is useless and
        directing to dev null does not buy anything.

            :0 hwic
            | cat - /dev/null; echo "hi there" > some-file.txt

    20.39 Procmail's temporary file names while writing file out

          ...Any ideas what might make those .nfs* files? They contain
          messages which seem to have been successfully processed by
          procmail in the later parts of the .procmailrc . However, I doubt
          they'd ever get cleaned up if I didn't discover them.

            /disk3/home/foobar/Mail 119) ls -la backup
            total 22
            drwx------  2 stanr         512 Nov 11 21:00 .
            drwx------  3 stanr        2560 Nov 11 21:11 ..
            -rw-------  1 stanr        3063 Nov  4 03:31 .nfsA0c724.4
            -rw-------  1 stanr        1780 Nov  3 23:00 .nfsA47da4.4
            -rw-------  1 stanr         849 Nov  3 23:22 .nfsA481f4.4
            -rw-------  1 stanr        2293 Nov 11 11:28 .nfsA737d4.4
            -rw-------  1 stanr        2598 Nov 11 20:39 msg.HCJB
            -rw-------  1 stanr        3127 Nov 11 21:00 msg.ICJB
            -rw-------  1 stanr        1884 Nov 11 20:45 msg.KCJB
            /disk3/home/stanr/Mail 120)

        [david] procmail uses temporary name while it is trying to
        write a file out, which it renames if things go well. I noticed
        that they all came from a 4h 31 span overnight; perhaps there was
        some systems work being done on your machine that screwed things
        up?

            :0 ic
            | cd backup && rm -f dummy `ls -t msg.* .nfs* | sed -e 1,3d`


        [aaron] When a file that is being used by a program on an NFS
        client gets unlinked the NFS server renames it to something like
        that. It should then actually get unlinked when the file is closed,
        but it looks like the NFS server never got the close message for
        those.

        [Keith Pyle ] It is a result of using NFS, but
        the fault lies with the operating system on the NFS client. Keep in
        mind that NFS is stateless from the perspective of the NFS server.
        It keeps no information on how any file is being used. So, if a
        client tells the server to delete the file, the server deletes the
        file. This is not normally a problem, but many programs use a
        "trick" of Unix where the program opens a file, unlinks (deletes)
        it, and then continues to use the file. For all local files, the
        Unix kernel will not actually delete the file until all processes
        which have the file open exit. This works very well for temporary
        files.

        If a client tells an NFS server to delete a file, it will delete
        the file immediately because of the stateless nature of NFS. The
        server has no way of knowing if any client still has the file open.
        To avoid this problem, if a client unlinks an open file on an NFS
        filesystem, the file is renamed to .nfs* where * is a unique value.
        The NFS client system is supposed to delete the .nfs* file when the
        process exits. However, there are some versions of Unix which do
        not do this well (e.g., AIX). If one of these OS's is used, it is
        common to find .nfs* files in various places. Therefore, it is a
        good idea for system administrators to periodically purge any .nfs*
        files over a certain age to eliminate the unsightly buildup in the
        filesystems.

    20.40 Parameter $@

        [david] Of version 3.11pre7 procmail does not grok "$*", nor does
        it grok "$@" outside a pipe or forward action. The only way to get
        the positional parameters all quoted together into "$*" is
        something like this:

        This doesn't work after all

            ARGS = `echo "$@"`

        Procmail substitutes null for "$@" there.  *This* works, though:

            :0 ir
            ARGS=|echo "$@"

        After that you use "$ARGS" instead of "$*".

        If you try to set ARGS with ARGS="$@", procmail doesn't substitute
        for "$@" and makes $ARGS null. If you try ARGS="$*" you get the
        literal text '$*'.

        [philip] Of course, $ARGS differs greatly from $@ in that $ARGS will
        either be split on whitespace (if unquoted) or one argument (if
        double-quoted).  $@ has the cool property that if double quoted
        it'll still be split into multiple arguments on the original
        argument boundaries. Since full-blown email addresses often have
        spaces, this distinction should not be casually dismissed. Note
        that while you might not type in such an addresses, your MUA's
        reply builder may.

    20.41 Procmail variables are null terminated (detecting null string)

        You can't catch null in the message. Eg if you try like this

            NUL=`/usr/5bin/echo "\000"`

            :0 HB
            * $ $\NUL
            {
                LOG = "Caught NUL"
            }

        [philip] It won't work as expected. The problem is that environment
        variables (and therefore procmail variables) are null-terminated,
        and therefore cannot contain a null. The above line creates an
        empty variable. The solution is to use an inverted character class:

            NUL = `/usr/5bin/echo '[^\001-\377]'`

        Note that procmail handles 8-bit characters except for null in
        procmailrcs, so you can use a literal control-A and octal-377 in
        your .procmailrc and save an echo and shell invocation right there.

    20.42 FROM_DAEMON TO and TO_ and case-sensitiveness

        [david] ^TO is case-insensitive by default. Stephen once told me
        something to the effect that tokens like ^TO, ^TO_, ^FROM_DAEMON,
        and ^FROM_MAILER are *always* case-insensitive, even if the recipe
        has the `D' flag, but I'm not positive that that was what he was
        saying, and we never pursued it. Certainly they are insensitive to
        case if there is no `D'.

        [philip] If a regexp contains the ^FROM_DAEMON token, then that entire
        regexp is treated as case-insensitive. Other conditions in the
        recipe are not affected by this. The other tokens have no effect on
        the case-sensitivity. (This is with procmail 3.11pre4)

    20.43 TO_ macro deciphered

          ...What is the essential difference between TO and TO_ ?

        [phil 1996-03-21] The difference is that ^TOalias1@site may match
        something like bobs-alias1@site while ^TO_ won't.

        [elijah 1997-09-16] Let's rewrite that in perl /x format. See
        below. The definition of the word boundary in block (E). See below.
        The ^TO_ expansion was added in v3.11pre4. You'll probably have to
        just ^TO (no '_'), which should work almost as well.



            /                       # [begin regexp]
             (                      # [Block (A)]
              ^                     # Anchor to start of line
               (                    # [Block (B)]
                 (Original-)?       # Optionally proceed (C) with "Original-"
                  (Resent-)?        # Optionally proceed (C) with "Resent-"
                          (         # [Block (C)]
                           To       # "To"
                          |Cc       # or "Cc"
                          |Bcc      # or "Bcc" {very rare in practice}
                          )         # [end (C)]
                 | (                # [Block (D)]
                    X-Envelope      # Proceed line 17 with "X-Envelope"
                   |Apparently      # or "Apparently"
                       (-Resent)?   #    with optional "-Resent" appended
                   )                # [end (D)]
                      -To           # "-To" [line 14]
                )                   # [end (B)]
                   :                # ":"
                   (                # [Block (E)]
                    .*              # any text
                      # any single char other than letters, numbers,
                      [^-a-zA-Z0-9_.]
                                    # hyphen (-), underscore (_), or period (.)
                   )                # [end (E)]
                    ?               # Block (E) is optional
             )                      # [end (A)]
            /x                      # [end regexp]

    20.44 TO_ macro and RFC 822

          ...According to RFC822 the From address can contains almost anything
          and the valid email address can be extracted from the line as
          long as it is enclosed between <...>. Like .

        [by Vikas Agnihotri ] Block (E){see
        TO_ macro explanation} is there to slurp up that part. The
         is not needed, and a case such as:

            From: "jester@fun.house" 

        Will confuse a test for "^TO_jester@". Yes, I have seen people do
        that stuff, apparently not even maliciously. And although valid
        following is also valid

            From: someone@somewhere.com 

        [Elijah continues] it will also confuse the regexp. I don't like
        the ^TO and ^TO_ macros for most things and typically use stuff
        like this:

            ^(Resent-)?(To|CC):.*[< ]{address}([ >]|$)

        It still can be confused, but the things that will cause problems
        are fairly rare in practice. You might prefer something like this:

            ^(Resent-)?(To|CC):([^(]+([(].*[)])?)*[, <]{address}([, >]|$)

        Which can correctly deal with

            To: (hatter@tea.party) {address}
            To: (fake {address}) bill.the.lizard@the.jury.box
            To: Alice , "W. Rabbit (late)"
                    , Gentle Reader <{address}>
            To: jabberwocky@vorpal.swords.r.us, duchess@the.croquet.game,
                    chesire@no.where, {address}, dinah@meow.org

        It will still fail for

            To: (fake <{address}>) mockturtle@tortoise.edu

        If someone is malicious enough to send you such mail.

    20.45 FROM_DAEMON deciphered

        Here is the exploded FROM_DAEMON regexp as of 3.11pre7

            (^(Precedence:.*(junk|bulk|list)
              |To: Multiple recipients of
              |(
                ((Resent-)?(From|Sender)|X-Envelope-From):|>?From )
                ([^>]*[^(.%@a-z0-9])?
                (
                    Post(ma?(st(e?r)?|n)|office)
                    |(send)?Mail(er)?
                    |daemon
                    |m(mdf|ajordomo)
                    |n?uucp
                    |LIST(SERV|proc)
                    |NETSERV
                    |o(wner|ps)
                    |r(e(quest|sponse)|oot)
                    |b(ounce|bs\.smtp)
                    |echo
                    |mirror
                    |s(erv(ices?|er)
                    |mtp(error)?|ystem)
                    |A(
                       dmin(istrator)?
                       |MMGR
                       |utoanswer
                      )
                )
                (  ([^).!:a-z0-9][-_a-z0-9]*)?
                   [%@>         ][^<)]*(\(.*\).*)?
                )?
                $
                ([^>]|$)
              )
            )


        [era] explains the last regexps as follows:

            (([^).!:a-z0-9]   End of e-mail address token
              [-_a-z0-9]      Another alpha token
              )?              ... or maybe not;
             [%@>\t ]         Address separator -- either  or
                                
or a bare address with whitespace around it [^<)]* Skip as long as we don't run into another bracketed address or end of comment (presumably to prevent this from matching inside parenthesized comments in the first place) (\(.*\).*)? Skip optional parenthesized comments and anything after them if found )? ... or maybe not; maybe we just see an ... $ ... end of line instead ([^>]|$) Uh, I should know what this is supposed to do, but I can't quite remember what it's for. I think it had something to do with continued header lines ... Anyone? Does ^FROM_MAILER match on the Return-Path: line? [david 1998-04-29] Apparently not, but it does match on the UNIX From_ line, which usually contains the same address as the Return-Path: header. Does anyone have an idea how I can use this macro but tell it to ignore the Return-Path line in the header? There's probably some way within procmail without the extra fork of formail, but this is easy to think of and easy to write: :0h HEAD_WITHOUT_FROM_=| formail -IReturn-Path: -I'From ' :0 * HEAD_WITHOUT_FROM_ ?? ^FROM_MAILER action If you want to consider only the From: header, try this: :0 * ^\/From:.* * MATCH ?? ^FROM_MAILER action 21.0 Technical matters 21.1 List of exit codes The right place to look is /usr/include/sysexits.h, but the codes should be pretty much standard. These ones are from HP-UX 10 and the code that you will be using mostly is EX_NOUSER or EX_NOPERM. It tells to the sender of UBE to "piss off and delete me from your list; I'm not here" EX_OK 0 successful termination EX__BASE 64 base value for error messages EX_USAGE 64 command line usage error EX_DATAERR 65 data format error EX_NOINPUT 66 cannot open input EX_NOUSER 67 addressee unknown EX_NOHOST 68 host name unknown EX_UNAVAILABLE 69 service unavailable EX_SOFTWARE 70 internal software error EX_OSERR 71 system error (e.g., can't fork) EX_OSFILE 72 critical OS file missing EX_CANTCREAT 73 can't create (user) output file EX_IOERR 74 input/output error EX_TEMPFAIL 75 temp failure; user is invited to retry EX_PROTOCOL 76 remote error in protocol EX_NOPERM 77 permission denied I thought that by using the EXITCODE, I would be assured that the email would be *rejected* but in fact Sendmail 8.8.7 attempts to deliver the "user unknown" to netcom.com, which is obviously wrong? [sean] Sendmail accepts the message, then passes it on to Procmail, either as the local delivery agent, or via a .forward file (depending on your system's configuration). Procmail says "gee, gotta lie about not being here" and rejects the message, when is sent back into the spool, and delivered according to who it appeared to come from. Had SENDMAIL determined the user didn't exist (password file / aliases / virtusertable.txt), then it would have rejected the message right when the remote was doing SMTP RCPT. But the user WAS valid, and so it accepted it. Another scenario is when you have a mail secondary, and your primary (where the user account and procmail are) is down. Some system goes to deliver mail to you, and resolves to your secondary -- which simply holds mail for your primary -- it hasn't a clue which user is valid and which isn't. Well, the (E)HELO (the system sending your primary the message) takes place during the SMTP session, the message is coming from your secondary - not from the original sender. At THAT point, if the user didn't exist, I believe sendmail would be issuing an unknown user error to the secondary, which in turn should mail that message back to who it thinks is the sender (I can't check my Bat book from where I'm at - any sendmail pros are welcome to elaborate). is there any way at all to get around this (force the rejection at delivery time)? Better yet, is there some sort of check to make sure that the Received domain reasonably matches the From: domain? You'd need to have a ruleset in your SMTP Daemon (generally Sendmail) to check domains (which WILL fail on many valid messages, BTW) and reject it WHILE the SMTP delivery session (actually, the negotiation) is in progress. By the time Procmail has the message, you've completely accepted the message, and any rejection you might hope to do is bouncing the mail - to the apparent sender. Such is the problem with forged mail. I wouldn't suggest this tactic for fighting spam anyway - so much of it is forged, and any bounce you send out simply uses up system resources on your machine and those on the system that was spoofed. Spammers don't REMOVE addresses from their lists (they want the lists to look as big as possible when they go to sell it to someone else) -- some have even taken to GENERATING addresses at domains and sending messages to them with the assumption that somebody will probably have an account by that name ("bill@ joe@ dave@ ..."). Use procmail to trashbin (or otherwise file) all the junk and then manually take action on those which get through. 21.2 List of precedence codes The priorities most sendmails recognize are following. The lower the priority, the later the message gets dealt with. A smart vacation program will ignore anything with a list, bulk, or junk priority. --Adam Shostack 0 first-class 30 list 60 bulk 100 junk 100 special-delivery [dan] You should use `bulk' when you distribute files via File Server. The value in the Precedence: header says absolutely NOTHING about the contents of the message itself, it merely suggests a priority level to the mail system. From pp. 668 of the O'Reilly's *sendmail* book, bulk typically has a value of -200 while `junk' -100; thus a message with `junk' will get *higher* priority than that of `bulk' (although this can be changed in the `sendmail.cf' file). Other than on heavily loaded machines, this value won't matter anyway, since all mail will be quickly processed. [Stephen] ...Mail sent by a person is usually considered to be more important than autoreplies generated by some daemon. One way to express the lower priority of autoreplies is by adding a "Precedence: junk" field. This allows mail transport agents to make educated decisions about which mail to forward first (in case the mailqueue gets clogged). Another point is: other autoreply services, like `vacation'. They try to make an effort not to accidentally reply to a message generated by another daemon (e.g. yours). One way they detect this is by looking at the Precedence field. If it contains `junk', they know, this is not something we should respond to. 21.3 Sendmail and -t sendmail -t tag reads To, Cc, Bcc, etc, for the recipient of the auto response? :0h * condition * !^X-Loop: foo@site\.com | ($FORMAIL -rA "X-Loop: foo@example.com" ) | sendmail -oi -t [david] That's not a problem, because formail -r will not generate any Cc: or Bcc: headers unless you tell formail to add them. The only line where sendmail -t will look for recipients will be the To: line. 21.4 RFC822 Reply-To and formail problem with multiple recipients [david] formail -r extracts only one return address, even when the Resent-Reply-To: or Reply-To: header contains more than one (and Stephen has told me he plans to leave it that way). o Looking for the best address to reply to is a completely different algorithm than looking for the best group of addresses to reply to. Finding a group of addresses involves actually determining that you even are searching for a group and not only for one address. Then finding out the best address for each. It's already a tricky business doing this just for one address. o It makes thousands of autoreply recipes vulnerable to mail-storm attacks. Formail tries its best to control the damage even if operated by someone who doesn't know what he is doing. If it were to reply to multiple addresses at times, this damage control is severely undermined. [dan]I understand these concerns; however RFC822 specifically allows for multiple recipients in a Reply-To: header. Given that, it seems that there should be a straight-forward way to deal with this in formail; even worse is that "formail" silently ignores multiple Reply-To: addresses. For (a), wouldn't the Reply-To: (or Resent-Reply-To:) header supersede all other addresses and thus greatly simplify the searching? For (b), how about only using multiple (Resent-)Reply-To: addresses if formail's "-t" option is also specified? Or if you are really worried about mail-storms and existing recipes, a new formail option. 21.5 Procmail and IMAP server [ed] See also ftp://ftp.cac.washington.edu/mail/imap.vs.pop ...This paper is an elaboration on a short note entitled "Comparing Two Approaches to Remote Mailbox Access: IMAP vs. POP", which was written in 1993 and recently updated. The purpose of this paper is to provide more extensive background on message access paradigms and protocols, and then to specifically compare the Internet's Post Office Protocol (POP) and the Internet Message Access Protocol (IMAP) in the context of "online" operation. ...I log in to a set of NFS-ed servers (or more precisely AFS-ed), and my mail comes into another server (not a part of this set) which is running IMAP. So sendmail never delivers mail into /var/mail/$LOGNAME on my login machines, and instead delivers to the IMAP server. Since sendmail never reads my .forward file in the home directory, I figure procmail never gets invoked. You need a program which will fetch your e-mail from the IMAP server and then feed it to procmail. One such program that can do this is fetchmail. Check out http://locke.ccil.org/~esr/fetchmail/. The bad news is that once you do this, you probably won't be able to use an IMAP client to read your e-mail anymore. But that might be good news if you prefer an MUA that reads mbox files but doesn't grok IMAP. 21.6 Machine which processes mail ...The just-installed procmail does not work and I am assuming that sendmail is trying to run procmail on another machine. Is there anyway I could find out the appropriate ARCHITECTURE for that machine [era] The following should tell you the name of the machine which processes mail for the machine you're asking about. You can then try to log in to that machine if you have shell access there, which is something you need to have in order to compile Procmail on it. nslookup -q=mx machine # alternatively use host(1) command If you don't have nslookup (doh) or don't understand what it says, try adding this to your .forward "|uname -a >/full/path/to/home/.uname.out" i.e. this should be there in +addition+ to what else you do. Otherwise this will lose your mail thoroughly, since it reads the mail but doesn't save it anywhere. You might want to save a copy of all incoming mail to a safety mailbox, too, just in case. Like so: /full/path/to/home/safetymailbox |"uname -a >/full/path/to/home/.uname.out" |"IFS=' '&& exec /usr/local/bin/procmail -Yf- || exit 75" If you try this, it is very important that the file safetymailbox exists and is writable. (`man' `5' `forward' if you have that -- I don't seem to have this manual page on systems with newish versions of sendmail, is that correct?) Try the `uname' command (and/or read the manual) to see what you should expect to find in the file .uname.out 21.7 Compiling procmail and MAILSPOOLHOME ...I am compiling 3.11pre7 on a new system and have a couple of questions. I edited the makefile to be the home directory "/home/a/abc" for example. I defined MAILSPOOLHOME as "/mail". The incoming mail is actually stored in "/usr/mail/abc". When I pipe test messages through procmail (using "procmail] ...The trouble with rapid delivery MTA's and SmartList(is not!) is multigram and the bounced email processing feature and large lists. If you run small lists it's not so bad. But with 30k+ subscriber monthly announcement lists, I have to redirect bouncing email someplace else. The faster the mail is delivered the faster the bounces come in. The larger the list, the more multigram bogs down and the more procmail processes accumulate and then you get file table overflows and all that stuff from the load being sky high and hundreds of processes hanging around. Also because alot of mail servers don't behave the way we would like them to and return to us proprietary bounced email messages that SmartList can't understand, the bounce feature is only able to successfully remove a portion of the offensive email addresses, requiring the listowner or some one to manually go through it all anyway. SmartList is not covered by the GNU GPL -- the copyright notice in the docs do say that changes/modifications can be made if they are 'marked'. Personally, I think the bounced email removal system needs to be entirely redone if SmartList(is not!) is to be able to look forward to the future enough to merit further development. "Other MLM software" o majordomo uses Perl o Mailman uses Python http://www.list.org/ o SmartList uses scripts/procmail o Listar Mailing List Package (free): http://www.nausicaa.net/~listar/ o L-Soft international, Inc. (non-free): http://www.lsoft.com/ o The Petidomo Mailing List Processor and Manager (non-free): http://www.petidomo.com/ o Mailagent (free): http://www.foretune.co.jp/people/shigeya/mailagent/ o BeroList (free): http://www.in-trier.de/~bero/BeroList/ 22.3 SmartList code (mailing list implementation with procmail) "Smartlist 'faq'" http://www.mindwell.com/smartlist/ "Mark's Smartlist add-ons" Mark David mcCreary ftp://ftp.mail-list.com/ ...front-end for Smartlist mailing lists, and allows people to mail to list-on@domain.com and list-off@domain.com, which then creates and sends a properly formatted subscribe/unsubscribe message to the list-request address. It also handles change of address, switches to/from digest lists, moderated subscriptions, and a few other things. My experience is that if you are planning on running lots of lists, then eliminating questions/problems from subscribers is of paramount importance, and those procmail recipes may be worth the time to learn/tweak. --Mark "Michelle's (SmartList add-ons) Confirmation cookie" ftp://ftp.fatfree.com/ confirm-1.1.tar.gz ftp://ftp.rahul.net/pub2/artemis/ confirm-1.1.tar.gz To add subscription confirmation to smartlist "The mail-list.com front-end for Smartlist Mailing Lists" ftp://ftp.mail-list.com mark david mcCreary to start the subscription process. "Auto respond to 'help: is there faq?' " http://www2.inow.com/~conover/ rel.tar.zip "Moderated lists -- Perl script" http://www.mjolner.com/~lbr/moderate/ 22.4 Installation trouble: getparams Does anyone out there know what the error means when it occurs when installing Smartlist? Procmail is already installed on the system (by the sysops) make: *** No rule to make target `getparams' [Hal Wine] Yes, it means that you haven't built procmail yet. Build procmail first, then execute Smartlist's install.sh script. You need to get and untar the procmail sources in your own directory, then get and untar the corresponding Smartlist sources in the same directory tree. Then build (but don't install) procmail, then install Smartlist using the install.sh script. Smartlist uses and builds files in the Procmail source tree, so that has to be done first [sysops] don't have the time to mess with getting Smartlist running. Obviously,when I attempt to install Smartlist, it's not finding Procmail. What do I have to do to get the install program to find Procmail? If the sysops aren't going to install Smartlist, read all the sections in Manual about non-root use of Smartlist (it works fine). You should make sure that smartlist, when invoked, uses the matching version of procmail. This means either use the version of Smartlist that matches the sysop installed version of procmail, or set up your PATH such that you use the version you built. If you use your own version, make sure it uses the same locking strategies as the "official" version. 22.5 Accepting mail only from users in whitelist(s) 1998-10-08 PM-L Dave Robbins. ML = /usr/local/lib/aliases ACCEPTLIST = "$ML/mylist.accept $ML/everyone $ML/others" FROM = `$FORMAIL -rtzxTo:` :0 * ? echo "$FROM" | $EGREP -i -f $ACCEPTLIST * ? test -r $ACCEPTLIST -a -s $ACCEPTLIST { :0 HB ! `cat $ML/mylist` } :0 * ! ^FROM_DAEMON * ! ^FROM_MAILER *$ ! $MY_XLOOP_LIST | ($FORMAIL -rtk \ -A "$MY_XLOOP_LIST" \ -A "Precedence: junk"; \ echo "Your post to mylist@magic.geol.ucsb.edu was not successful\n" \ "because the mailing list is restricted to submissions\n" \ "from only certain individuals and groups. Sorry.\n" \ ) | $SENDMAIL -oi -t 23.0 Additional procmail or MUA software 23.1 Comstat to handle multiple mailboxes ftp://ftp.belwue.de/pub/unix/xcomsat.tar.gz 23.2 Elm and pgp support (Mutt is the successor to elm.) Mutt's primary site is ftp://ftp.guug.de/pub/mutt/ with various mirrors outside the US to avoid the crypto distribution problem. If you want elm, "Michael Elkins' ftp directory" ftp://ftp.cs.hmc.edu/pub/me/ http://www.cs.hmc.edu/~me/mutt/ http://www.cs.hmc.edu/~me/elm/me.html [Liviu Daia mentions that] ...Provided that you configure it correctly, it will use lynx to convert HTML attachments to plain text automatically, and display them in its pager. You can reply in plain text to those attachments, and you can also do the same thing with any kind of attachment for which you give it a way to convert to plain text. It's definitely not aimed at the beginner level like Pine, but it's far more powerful too. Also GPL-ed. 23.3 MH sites "New MH" ftp://ftp.math.gatech.edu/pub/nmh/nmh.tar.gz http://www.math.gatech.edu/nmh/ 24.0 Additional procmail software for Emacs 24.1 What is Emacs ...first thing I learned on a Unix machine was that `vi' is a text editor and `Emacs' is a way of life. --David W. Tamkin _Emacs_ refers to a programming platform (it's not only a text editor, or a programming editor, but it does almost everything you tell it to do except make your coffee) which can be found almost in any Unix platform. Nowadays Emacs is also available for the PC platform too. There are two flavours to choose from: Emacs, maintained by the FSF (Free Software Foundation), and XEmacs, sometimes called "Emacs the next generation", because it has a better graphical user interface (gui) and internally advanced OO design (it can highlight on tty, whereas Emacs can't). XEmacs is being maintained by group of programming wizards. See #URL-BASE/elisp.html Emacs add-in packages are lisp and the lisp file extension is *.el*. Inside each package one finds instructions how to use and how to install the package into Emacs. 24.2 Emacs and procmail mode and Lint Procmail mode for Emacs (which can also lint procmail recipes) is available. People familiar with C-coding know lint, which is a rigorous code syntax checker. You can read about this Emacs mode from http://poboxes.com/jari.aalto/tiny-tools.tar.gz 24.3 Emacs and lining up backslashes Some time ago I wrote makefile to my Emacs tgz kit and as a side effect I got frustrated with the use of backslashes within the make rules. This backslash problem is universal in almost every programming language, (eg. C/C++ macros) including procmail, where you sometimes use `echo' a lot, :0 h * condition | ( cat -; \ echo "And the body text\n" \ "follows here with\n" \ "these echoes"; \ ) | $SENDMAIL Ouch. That looks bad. Any line up tool anywhere? Yes, get my Emacs tiny-tools.tar.gz and look at the file `tinymy.el' which defines function `timy-backslash-fix-paragraph'. Here is piece of lisp code that you stick to your .emacs to make the key `Control-\' to run the backslash fix (global-set-key "\C-\\" 'my-backslash-default-column) (defun my-backslash-default-column (&optional arg) "Col 76." (interactive "*P") (autoload 'timy-backslash-fix-paragraph "tinymy" t t) (timy-backslash-fix-paragraph (or arg 76) 'verb) ) After that, you just put your cursor inside paragraph and hit `Control-\' to get the following line up effect. The column position is best to set near right margin, but not further than a regular page's maximum column 80. :0 h * condition | ( cat -; \ echo "And the body text\n"; \ "follows here with\n"; \ "these echoes"; \ ) | $SENDMAIL Guys, Emacs is available for every platform, even for Windows95 and WindowsNT. So, go ahead and install one if you haven't already. Setting up your personalised Emacs may require steep learning curve, but it's well worth the effort :-) 24.4 Emacs and browsing mailbox files If you use Gnus as your MUA, then you already can browse mailboxes. If you just want to read some arbitrary mailbox without firing up Gnus, then you can use package `tinymbx.el' It defines a special mailbox reading minor mode that is activated when you visit mailbox file. You can copy, file, delete messages or mail the author of the current message. There is no separate summary buffer as in RMAIL, but you move from message to another with PgUp and PgDown keys. #URL-BASE/tiny-tools.tar.gz 24.5 Emacs and live-mode.el http://www.zanshin.com/~bobg/ from Bob Glickstein 1997 ...`live-mode' is a minor mode that works like the "tail -f" Unix command. If the file grows (or changes in any other way) on the disk, then the buffer copy is periodically updated to show the new file contents. This makes `live-mode' ideal for viewing such things as log files. --Bob You definitely want this if you browse procmail log files. This package updates the logfile buffers whenever they change on disk. You can think it like *biff* if you record incoming file to short `$BIFF' log. 24.6 Emacs and font-lock.el `Font-lock' comes standard in Emacs releases. You can colorize your .procmailrc if you use `font-lock'. Here is some lisp code; put it in your .emacs and reload it with `M-x' `load-file'. When you load file that matches `procmailrc' or `procmail.log' the font-lock attributes for the file get set. Change the regexp if your procmail filenames are different. (add-hook 'find-file-hooks 'my-find-file-hooks) (defun 'my-find-file-hooks () (require 'cl) ;; colors are available to Emacs only under X window (when (and window-system (fboundp 'font-lock-mode) ;; make sure this is present ) (cond ((string-match "procmailrc" buffer-file-name) (setq font-lock-keywords (list '("#.*" . font-lock-comment-face) '("^[\t ]*:.*" . font-lock-type-face) '("[A-Za-z_]+=.*" . font-lock-keyword-face) '("^\\*.*" . font-lock-doc-string-face) )) ;; Turn the fontifying mode on if it's not on already (unless font-lock-mode (font-lock-mode 1)) ) ((string-match "procmail.log" buffer-file-name) ;; The strings "" in the procmail log makes font-lock crazy, ;; We kill the String class from the buffer with ;; these statements. ;; (let ((table (make-syntax-table))) (modify-syntax-entry ?\" "_" table) ;; Change " (set-syntax-table table)) (setq font-lock-keywords (list (cons "Opening " 'font-lock-type-face) (cons ".* error .*" 'font-lock-keyword-face) (cons "Folder:" 'font-lock-type-face) )) (unless font-lock-mode (font-lock-mode 1)) )) )) ;; End code 25.0 Procmail, Emacs and Gnus 25.1 Gnus pointers "Gnus" http://www.ifi.uio.no/~larsi "Gnus manual: procmail" http://www.ifi.uio.no/~larsi/www.gnus.org/manual/gnus_6.html#IDX1501 "Gnus Hypertext search archive" http://www.miranova.com/gnus-list/ 25.2 Why use procmail with Gnus Gnus has very powerful mail split methods and one normal reaction against the need of procmail is: "Hey, Gnus does my mail splitting, I don't need procmail". The difference between Gnus and procmail splitting is quite easily explained: you want procmail to preprocess the mail before gnus ever sees it and then postprocess the mail with Gnus (read, move mail from the inbox to another) _Case1_: Gnus and regular mailbox, no procmail. Gnus reads directly one huge mailbox where all incoming messages are. When the user starts Gnus, it slurps in the whole mailbox and starts splitting the mail according to the its split rules. mail -> $MAIL --> fire up Gnus --> split1.mbx split2.mbx .... _Case2_: procmail and Gnus. The email is always delivered to procmail first. Procmail is free to put the mail anywhere or just let it drop to the user's default inbox, usually pointed by environment variable $MAIL. mail -> procmail --> Post processing with Gnus [the ~/Mail/spool] --> split1.mbx --> split2.mbx [The default procmail rule drops to inbox] --> $MAIL You can let gnus to process the messages further: like moving messages from one inbox to another. Summary o If you use procmail, the incoming messages are immediately categorized. The incoming email is put in the folder of your choice. The mailboxes are there waiting for you all the time. You can use `less' or `more' to view them in a hurry. o If you don't use procmail and let Gnus to do all the splitting, you always see one huge inbox, $MAIL. It will not be split until you fire up Emacs and Gnus. If you're in a hurry, you may not have time to start Emacs & Gnus, before reading the important messages. Your only option is to read all messages in $MAIL and try to find the ones that consider eg. you work. So, let procmail drop messages to their inboxes and Gnus to possibly "fine process" these inboxes. 25.3 Setting up gnus for procmail - Basics Procmail and Gnus communicate with each other very nicely when you use the mail backends like: `nnml', `nnmh' and `nnfolder'. See Emacs info Gnus::Node: `Select' `Methods' for more. Here are step by step instructions for reading the mail with `nnml' mail backend. We suppose that you have the following definition in your .procmailrc so that the incoming mail is delivered to the right directory. The important point here is that the name of the gnus `nnml' group is identical; except the `.spool' suffix, to the spool file where procmail writes. So if you write to `list.procmail.spool', the group name in gnus is named `nnml:list.procmail' # .procmailrc excerpt PMSRC = $HOME/pm MAILDIR = $HOME/Mail SPOOL = $MAILDIR/spool RC_LIST = $PMSRC/pm-jalist.rc # The file name must be list.xxxxx.spool in order to # `nnml' to work in Gnus.Define procmail mailing list PROCMAIL_SPOOL = $SPOOL/list.procmail.spool # GNUS must have unique message headers, generate one # if it isn't there. By Joe Hildebrand :0 fhw | $FORMAIL -a Message-Id: -a "Subject: (None)" # detect mailing lists and store messages to spool directory INCLUDERC = $RC_LIST :0 : * ! LIST ?? ^^^^ $SPOOL/list.$LIST.spool . Copy the Lisp code below to your ~/.gnus . Start Gnus with `M-x' `gnus-no-server' (M-x means ESC followed by x). You will see *Group* buffer to appear. . Make the new group with `G' `m' `list.procmail' RET `nnml' RET. You can read the group as usual and query new mail with `g' command. (setq gnus-secondary-select-methods '((nnml "")) ;; See also nnmail-procmail-suffix which is .spool by ;; default ;; nnmail-use-procmail t nnmail-spool-file 'procmail nnmail-procmail-directory "~/Mail/spool/" nnmail-delete-incoming t ) And then I have procmail always deliver to ~/Mail/spool/. If you add more inboxes, create them inside gnus *Group* buffer with `G' `m'. 25.4 Gnus for procmail - More gnus Okay, let's continue our journey in Emacs. What you read previously was the minimum you needed to get your Gnus to read procmail delivered files. However, if you're new to Gnus, here are some more tips and basic instructions. The best advice I can give is that you go to each buffer: In group, you press `G' `C-h' and in Summary `C-h' `m' and print the commands to printer that you see listed. In Group buffer o When you press `g' to get new mail to these groups, the group _disappears_ if there is no mail. If you want the group to be permanently visible, then set (setq gnus-permanently-visible-groups "^nnml\\|^nnfolder") In emergency, press `L' to list all groups. o If you made a mistake and wrote `list.procmaill' with an extra `l' accidentally in the group name, use `G' `r' to rename group. o Raise or lower the priority of your procmail mail groups with `S' `l'. Values 1 or 2 or 3 are good. Consider reserving 1 for your primary mail and 2 and 3 for mailing lists. o When you exit a group and have read some articles, they won't show up next time you go there. But by giving prefix argument before entering the group with `SPC', Gnus will list all read articles. You give the command like `C-u' `SPC', where `C-u' is the prefix argument. Settings o You want gnus to tell you everything it does (setq gnus-verbose 10) ;; 0..10 o You expire articles (get permanently rid of them) with the 'E' command in the *Summary* buffer. The default expiry time is 7 days. You can define the expiry time in days with (setq nnmail-expiry-wait 7) o If you read mailing lists, you want automatic expiry when you have read the article. Use the following to set up groups that use this automatic expiration. (setq gnus-auto-expirable-newsgroups (concat "procmail" "\\|other-list" "\\|and-some-other-list" )) o `B' `e' in the *Summary* buffer expires current expirable articles. o If you want to kill an article; permanently remove it from disk, use `B' `delete'. o If you want to mark an article as *persistent* (never expires), use `*' o You don't want these mail groups cached because mail is already in "cache" format. The cache is needed only when you read newsgroups and want to store messages locally. (setq gnus-uncacheable-groups "^nn\\(virtual\\|m[hlk]\\|db\\)") 25.5 Emacs and Gnus -- Fiddling with spool files Well, to tell you the truth, managing Gnus is scary at first: You can make a lot of mistakes along the way or otherwise change your mind about group names and so on. It's a tricky task to move mail from one directory to another if you decide to rename the spool file name where procmail is putting the filtered mail. Let's take an example: Say you decide to change the spool file name list.procmail.spool to mail.procmail.spool, because you come to think that all your mail groups should have the same prefix "mail." in your Gnus group buffer. You already changed procmail to output to that file, so now you have two files sitting in your spool directory. ~/Mail/spool/list.procmail.spool ~/Mail/spool/mail.procmail.spool # make sure this exists o Let Gnus read the old file as usual. Press `g' read new mail to `list.procmail'. list.procmail.spool will now be empty and merged to `nnml' backend file nnml:list.procmail. o Make a new group with `G' `m' nnmail `mail.procmail' in *Group* buffer. o Go to the old `list.procmail' group and select all articles with `M' `P' `a'. Move the messages with `B' `m' to `mail.procmail'. You will see `G' marks appear to the beginning of moved articles. o Exit the *Summary* buffer and hit `g' to see that the messages hat were transferred to your new `mail.procmail' o Kill the old group