SAC:Mailing Lists

Mailing lists are administered with Mailman on the virtual host lists.osgeo.org. Physically this is the "Mail" VM running on osgeo4. Frank Warmerdam (warmerdam) and Alan Boudreault (aboudreault) do most of the mailman work.

Notes for List Administrators

Requesting a new list

If your project or activity would like to have a mailing list at lists.osgeo.org please take the following steps:

• First, confirm that there isn't already an appropriate list for this activity, and consider whether OSGeo is the right home for the proposed list.
• File a ticket at http://trac.osgeo.org/osgeo (use your OSGeo Userid to login) and make sure the component is set to "SAC". This is the request to the System Administration Committee to create the list.
• Please include the exact name you would like for the list (ie. foo-announce), and the email address of the person who should be the list admin. Also provide the justification for the list, and indicate if it was explicitly requested by an OSGeo Project PSC or project representative.
• Include any special instructions (such pointers to info if the list is actually being migrated from elsewhere).
• Hopefully, someone in SAC will act on the ticket and create the list. When complete the ticket will be updated and closed, and the proposed admin will receive an email with the admin password for the list.
• Review the list settings, in particular ensuring that the short description gets set (ie. review following points)

Generally it is up to the mailman administrator to judge whether a list is justified for creation. The justification doesn't need to be strong, but the list should be OSGeo related, reasonably likely to generate some use, or be requested on behalf of a project.

All lists

• Please set "A terse phrase identifying this list." to something brief and meaningful as this is what is shown on the main lists directory. It is on the General tab.
• Set "Who can view subscription list?" on the Privacy tab to "List Admin Only", otherwise anyone can subscribe and then harvest the mailing list.
• Set generic_nonmember_action on the Privacy->Sender filters tab to reject as most lists are not actively administered so we prefer to reject rather than holding posts from unsubscribed folks.
• Ensure the "Prefix for subject line of list postings." on the General tab is set to something meaningful. If the list name is generic (such as "announce") it may be helpful to prefix it with OSGeo-, eg. "[OSGeo-Announce]"
• Be sure to choose a name that does not already have an existing alias, such as: sales, support, www, security. If you need to use one of these names someone will have to edit the /etc/aliases file for you manually.

Other options

• For announce lists (ie. moderated lists) turn on "Emergency moderation of all list traffic" on. It is on the General tab.

Outstanding Issues

1. Administrator/moderator permissions. Currently this is a hodgepodge. Some administrators have set the passwords but then others can't help administrate the list. The default admin password is not very secure. It has been suggested that we use some sort of LDAP/http authentication to verify that folks trying to do list admin are in some sort of mail administration group.
2. List setting policies. Do we want to mandate more policies than the ones listed above as "Notes for List Administrators"?
3. Search. Do we want to do anything special to enable searching the archive? Jason has setup an OSGeo.org grouping on nabble (http://www.nabble.com/OSGeo.org-f18127.html, http://osgeo-org.1803224.n2.nabble.com/) that makes searching all osgeo related mailing lists effective. We might want to expand on that.

System Tasks

New Lists

• Create a new list through the web admin login.

Renaming Lists

• create new list

  $sudo /usr/lib/mailman/bin/newlist listname admins@email lists_passwd

• move original lists archive to newlists archive location

  $ sudo cp /var/lib/mailman/archives/private/oldlist.mbox/oldlist.mbox \
    /var/lib/mailman/archives/private/newlist.mbox/

• create archive

  $ sudo /usr/lib/mailman/bin/arch --wipe newlist

• export subscribers from old list regular and digest members

  $ sudo /usr/lib/mailman/bin/list_members -r oldlistname > listname-regular.txt
  $ sudo /usr/lib/mailman/bin/list_members -d oldlistname > listname-digest.txt

• import subscribers into new list

  $ sudo /usr/lib/mailman/bin/add_members --regular-members=listname-regular.txt --welcome-msg=y newlistname
  $ sudo /usr/lib/mailman/bin/add_members --digest-members=listname-digest.txt --welcome-msg=y newlistname

• remove old list

  $ sudo /usr/lib/mailman/bin/rmlist oldlistname

• update aliases and check that proper permissions are set

  $ sudo /usr/lib/mailman/bin/genaliases
  $ sudo /usr/lib/mailman/bin/check_perms -f

• edit postfix aliases - /etc/aliases

  oldlist:               newlist@lists.osgeo.org
  oldlist-request:       newlist-request@lists.osgeo.org
  oldlist-admin:         newlist-admin@lists.osgeo.org
  oldlist-owner:         newlist-owner@lists.osgeo.org

• update postfix with new aliases

  $ sudo /usr/bin/newaliases
  $ sudo /sbin/service postfix reload

• copy / clone user addresses across lists:
  • add -r to remove the old address

  $ sudo clone_member old@address.com new@address.com

Migrate

When migrating from another system to osgeo, get the .mbox files, and the whole /var/lib/mailman/lists/<listname> directory, and follow roughly the following steps as root (or sudo each step):

  # newlist -q grass-announce warmerdam@pobox.com junk 
  # cd /var/lib/mailman/lists/grass-announce
  # cp ~warmerdam/grass-announce/* . 
  # withlist -l -r fix_url grass-announce
  # /usr/lib/mailman/bin/arch grass-announce ~warmerdam/grass-announce.mbox
  # chown -R mailman.mailman /var/lib/mailman/lists/grass-announce
  # chown -R mailman.mailman /var/lib/mailman/archives/private/grass-announce*
  # genaliases
  # /sbin/service postfix restart
  # mailmanctl restart

If the list name on the new system is different than on the old system, it may be prudent to fix this up on the mail page of the web admin interface after the withlist command has been used, and before the archives are imported. (FrankW)

References

• Here is another approach to renaming, though our web admin interface won't actually allow you to do the final rename step

Remove Mails from Archive

Sometimes users send mails with contentious content to the list and ask for it to be removed. As a courtesy this should be done asap. In general it is better to not remove the mail altogether but just remove the content. Instructions:

• http://wiki.list.org/pages/viewpage.action?pageId=4030681

Archives live in:

/var/lib/mailman/archives/

To recreate archives use:

/usr/lib/mailman/bin/arch [listname]

List Statistics

From time to time it is nice to get a sense of the number of lists, users, etc. The following may be helpful:

all subscribers of all osgeo lists hosted on osgeo.org:

 sudo list_lists -b | xargs -n 1 sudo list_members | wc -l

unique list of email addresses across all lists:

 sudo list_lists -b | xargs -n 1 sudo list_members | sort |  uniq | wc -l

Note: since 2009 there is Markus Neteler's cronjob running for gathering these statistics and sending them to "info AT osgeo org". For a time series, see statistics

Nabble

There are a variety of web forum oriented interfaces for mailing lists that can be connected to existing mailing lists. One of the most widely used is Nabble and OSGeo makes an effort to provide some administrative support for it.

The Nabble liasons for OSGeo are Jorge Sanz with Mateusz Loskot as a backup.

Technical details to follow.