Difference between revisions of "SAC:Mailing Lists"
(49 intermediate revisions by 10 users not shown) | |||
Line 1: | Line 1: | ||
− | Mailing lists are administered with Mailman on | + | Mailing lists are administered with Mailman on lists.osgeo.org (see http://lists.osgeo.org for a list of available lists), physically this is [[osgeo6]]. Jorge Sanz (jsanz) does most of the mailman work, with Frank Warmerdam (warmerdam) and Martin Spott (martin) being the server side people if there are super technical issues. |
= Notes for List Administrators = | = Notes for List Administrators = | ||
− | == | + | == Requesting a new list == |
− | * Please set "A terse phrase identifying this list." to something brief and meaningful as this is what is shown on the main lists directory. It is on the General tab. | + | If your project or activity would like to have a mailing list at lists.osgeo.org please take the following steps: |
− | * Set "Who can view subscription list?" on the Privacy tab to "List Admin Only", otherwise anyone can subscribe and then harvest the mailing list. | + | |
− | * | + | * First, confirm that there isn't already an appropriate list for this activity, and consider whether OSGeo is the right home for the proposed list. |
+ | * File a ticket at http://trac.osgeo.org/osgeo (use your OSGeo Userid to login) and make sure the component is set to "Mails & Mailing Lists". This is the request to the System Administration Committee to create the list. | ||
+ | * Please include the exact name you would like for the list (ie. foo-announce), and the email address of the person who should be the list admin. Also provide the justification for the list, and indicate if it was explicitly requested by an OSGeo Project PSC or project representative. | ||
+ | * Include any special instructions (such pointers to info if the list is actually being migrated from elsewhere). | ||
+ | * Hopefully, someone in SAC will act on the ticket and create the list. When complete the ticket will be updated and closed, and the proposed admin will receive an email with the admin password for the list. | ||
+ | * Review the list settings, in particular ensuring that the short description gets set (ie. review following points) | ||
+ | |||
+ | Generally it is up to the mailman administrator to judge whether a list is justified for creation. The justification doesn't need to be strong, but the list should be OSGeo related, reasonably likely to generate some use, or be requested on behalf of a project. | ||
+ | |||
+ | == Configuring the mailing list == | ||
+ | |||
+ | * Please set "A terse phrase identifying this list." to something brief and meaningful as this is what is shown on the main lists directory. It is on the ''General'' tab. | ||
+ | * Set "Who can view subscription list?" on the ''Privacy'' tab to "List Admin Only", otherwise anyone can subscribe and then harvest the mailing list. | ||
+ | * Set ''generic_nonmember_action'' on the ''Privacy->Sender filters'' tab to reject as most lists are not actively administered so we prefer to reject rather than holding posts from unsubscribed folks. | ||
+ | * Be sure to choose a name that does not already have an existing alias, such as: sales, support, www, security. If you need to use one of these names someone will have to edit the /etc/aliases file for you manually. | ||
+ | * Configure to not tamper the original incoming message in any way to avoid breaking DKIM signatures, see https://trac.osgeo.org/osgeo/ticket/3011#comment:23 - you may find the /osgeo/mailman-tools/recommended_setup script useful to do this (installed on the list server host) | ||
== Other options == | == Other options == | ||
− | * For announce lists (ie. moderated lists) turn on "Emergency moderation of all list traffic" on the General | + | * For announce lists (ie. moderated lists) turn on "Emergency moderation of all list traffic" on. It is on the ''General'' tab. |
= Outstanding Issues = | = Outstanding Issues = | ||
Line 17: | Line 32: | ||
# Administrator/moderator permissions. Currently this is a hodgepodge. Some administrators have set the passwords but then others can't help administrate the list. The default admin password is not very secure. It has been suggested that we use some sort of LDAP/http authentication to verify that folks trying to do list admin are in some sort of mail administration group. | # Administrator/moderator permissions. Currently this is a hodgepodge. Some administrators have set the passwords but then others can't help administrate the list. The default admin password is not very secure. It has been suggested that we use some sort of LDAP/http authentication to verify that folks trying to do list admin are in some sort of mail administration group. | ||
# List setting policies. Do we want to mandate more policies than the ones listed above as "Notes for List Administrators"? | # List setting policies. Do we want to mandate more policies than the ones listed above as "Notes for List Administrators"? | ||
− | # Search. Do we want to do anything special to enable searching the archive? | + | # Search. Do we want to do anything special to enable searching the archive? Jason has setup an OSGeo.org grouping on nabble (<strike>http://www.nabble.com/OSGeo.org-f18127.html</strike>, http://osgeo-org.1803224.n2.nabble.com/) that makes searching all osgeo related mailing lists effective. We might want to expand on that. |
+ | |||
+ | = System Tasks = | ||
+ | |||
+ | == Mailing List Admin tasks when moving list to discourse == | ||
+ | |||
+ | Modify the listinfo.html of the mailing list, navigating from the administration panel | ||
+ | |||
+ | '''Edit the public HTML pages and text files''' -> '''General list information page''' | ||
+ | |||
+ | This | ||
+ | * will not show about new subscriptions | ||
+ | * will allow the subscribers to unsubscribe. | ||
+ | |||
+ | <pre> | ||
+ | <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> | ||
+ | <!-- $Revision: 5865 $ --> | ||
+ | <HTML> | ||
+ | <HEAD> | ||
+ | <TITLE><MM-List-Name> Info Page</TITLE> | ||
+ | |||
+ | </HEAD> | ||
+ | <BODY BGCOLOR="#ffffff"> | ||
+ | |||
+ | <P> | ||
+ | <TABLE BORDER="0" CELLSPACING="4" CELLPADDING="5"> | ||
+ | <TR> | ||
+ | <TD COLSPAN="2" WIDTH="100%" BGCOLOR="#99CCFF" ALIGN="CENTER"> | ||
+ | <B><FONT COLOR="#000000" SIZE="+1"><MM-List-Name> -- | ||
+ | <MM-List-Description></FONT></B> | ||
+ | </TD> | ||
+ | </TR> | ||
+ | <tr> | ||
+ | <td colspan="2"> | ||
+ | <p> | ||
+ | </td> | ||
+ | </tr> | ||
+ | <tr> | ||
+ | <TD COLSPAN="1" WIDTH="100%" BGCOLOR="#FFF0D0"> | ||
+ | <B><FONT COLOR="#000000">About <MM-List-Name></FONT></B> | ||
+ | </TD> | ||
+ | <TD COLSPAN="1" WIDTH="100%" BGCOLOR="#FFF0D0"> | ||
+ | <MM-lang-form-start><MM-displang-box> <MM-list-langs> | ||
+ | <MM-form-end> | ||
+ | </TD> | ||
+ | </TR> | ||
+ | <tr> | ||
+ | <td colspan="2"> | ||
+ | <P><MM-List-Info></P> | ||
+ | <p> To see the collection of prior postings to the list, | ||
+ | visit the <MM-Archive><MM-List-Name> | ||
+ | Archives</MM-Archive>. | ||
+ | <MM-Restricted-List-Message> | ||
+ | </p> | ||
+ | </TD> | ||
+ | </TR> | ||
+ | <tr> | ||
+ | <TD COLSPAN="2" WIDTH="100%"> | ||
+ | <p> | ||
+ | <MM-Options-Form-Start> | ||
+ | <MM-Editing-Options> | ||
+ | <MM-Form-End> | ||
+ | </td> | ||
+ | </tr> | ||
+ | </table> | ||
+ | <MM-Mailman-Footer> | ||
+ | </BODY> | ||
+ | </HTML> | ||
+ | </pre> | ||
+ | |||
+ | |||
+ | == New Lists == | ||
+ | |||
+ | |||
+ | '''Not accepting new lists''' | ||
+ | * Create a new list through the [http://lists.osgeo.org/mailman/create web admin login]. | ||
+ | |||
+ | == Renaming Lists == | ||
+ | |||
+ | * create new list | ||
+ | $sudo /usr/lib/mailman/bin/newlist listname admins@email lists_passwd | ||
+ | * move original lists archive to newlists archive location | ||
+ | $ sudo cp /var/lib/mailman/archives/private/oldlist.mbox/oldlist.mbox \ | ||
+ | /var/lib/mailman/archives/private/newlist.mbox/newlist.mbox | ||
+ | * create archive | ||
+ | $ sudo /usr/lib/mailman/bin/arch --wipe newlist | ||
+ | * export subscribers from old list regular and digest members | ||
+ | $ sudo /usr/lib/mailman/bin/list_members -r oldlistname > listname-regular.txt | ||
+ | $ sudo /usr/lib/mailman/bin/list_members -d oldlistname > listname-digest.txt | ||
+ | * import subscribers into new list | ||
+ | $ sudo /usr/lib/mailman/bin/add_members --regular-members=listname-regular.txt --welcome-msg=y newlistname | ||
+ | $ sudo /usr/lib/mailman/bin/add_members --digest-members=listname-digest.txt --welcome-msg=y newlistname | ||
+ | * remove old list | ||
+ | $ sudo /usr/lib/mailman/bin/rmlist oldlistname | ||
+ | * check that proper permissions are set (takes 10+ minutes!) | ||
+ | $ sudo /usr/lib/mailman/bin/check_perms -f | ||
+ | * edit postfix aliases - /etc/aliases | ||
+ | oldlist: newlist@lists.osgeo.org | ||
+ | oldlist-request: newlist-request@lists.osgeo.org | ||
+ | oldlist-admin: newlist-admin@lists.osgeo.org | ||
+ | oldlist-owner: newlist-owner@lists.osgeo.org | ||
+ | * update postfix with new aliases | ||
+ | $ sudo /usr/bin/newaliases | ||
+ | $ sudo /usr/sbin/service postfix reload | ||
+ | |||
+ | == copy / clone user addresses across lists == | ||
+ | * add '''-r''' to remove the old address | ||
+ | $ sudo clone_member old@address.com new@address.com | ||
+ | |||
+ | == Migrate == | ||
+ | |||
+ | When migrating from another system to osgeo, get the .mbox files, and the whole /var/lib/mailman/lists/<listname> directory, and follow roughly the following steps as root (or sudo each step): | ||
+ | |||
+ | # newlist -q grass-announce warmerdam@pobox.com junk | ||
+ | # cd /var/lib/mailman/lists/grass-announce | ||
+ | # cp ~warmerdam/grass-announce/* . | ||
+ | # withlist -l -r fix_url grass-announce | ||
+ | # /usr/lib/mailman/bin/arch grass-announce ~warmerdam/grass-announce.mbox | ||
+ | # chown -R list.list /var/lib/mailman/lists/grass-announce | ||
+ | # chown -R list.list /var/lib/mailman/archives/private/grass-announce* | ||
+ | # /usr/lib/mailman/bin/genaliases | ||
+ | # /usr/sbin/service postfix restart | ||
+ | # /usr/lib/mailman/bin/mailmanctl restart | ||
+ | |||
+ | If the list name on the new system is different than on the old system, it may be prudent to fix this up on the mail page of the web admin interface after the ''withlist'' command has been used, and before the archives are imported. (''FrankW'') | ||
+ | |||
+ | == References == | ||
+ | * Here is [http://www.chris-lamb.co.uk/2007/10/02/renaming-a-mailman-list/ another approach to renaming], though our web admin interface won't actually allow you to do the final rename step | ||
+ | |||
+ | |||
+ | == Remove Mails from Archive == | ||
+ | Sometimes users send mails with contentious content to the list and ask for it to be removed. As a courtesy this should be done asap. In general it is better to not remove the mail altogether but just remove the content. Instructions: | ||
+ | * http://wiki.list.org/pages/viewpage.action?pageId=4030681 | ||
+ | Archives live in: | ||
+ | /var/lib/mailman/archives/ | ||
+ | To recreate archives use: | ||
+ | /usr/lib/mailman/bin/arch [listname] | ||
+ | |||
+ | |||
+ | == List Statistics == | ||
+ | |||
+ | From time to time it is nice to get a sense of the number of lists, users, etc. The following may be helpful: | ||
+ | |||
+ | all subscribers of all osgeo lists hosted on osgeo.org: | ||
+ | |||
+ | sudo list_lists -b | xargs -n 1 sudo list_members | wc -l | ||
+ | |||
+ | unique list of email addresses across all lists: | ||
+ | |||
+ | sudo list_lists -b | xargs -n 1 sudo list_members | sort | uniq | wc -l | ||
+ | |||
+ | Note: since 2009 there is [[User:Neteler|Markus Neteler]]'s cronjob running for gathering these statistics and sending them to "info AT osgeo org". For a time series, [[VisibilityStats#Mailing_list_subscribers|see statistics]] | ||
+ | |||
+ | = Hints for subscribers = | ||
+ | |||
+ | Being subscribed to one or more mailing lists may result in a lot of incoming mail so it may be useful to configure your Mail User Agent (MUA) so that it makes it easier for you to tell which mail comes from which mailing list. How to do this depends on your system setup. | ||
+ | |||
+ | == Procmail users == | ||
+ | |||
+ | If you use [https://en.wikipedia.org/wiki/Procmail Procmail] to filter your incoming email a good recipe to automatically | ||
+ | put mailing list traffic each in its own folder could be this: | ||
+ | |||
+ | :0 | ||
+ | * ^List-Id: *<\/[^.]* | ||
+ | .list-$MATCH/ | ||
+ | |||
+ | == Mutt users == | ||
+ | |||
+ | If you use [http://mutt.org/ Mutt] MUA you may find these useful: | ||
+ | |||
+ | * '''subjectrx''' directive to tweak the Subject line (e.g. drop or add a prefix) - http://www.mutt.org/doc/manual/#display-munging | ||
+ | * '''list-reply''' command to reply to the mailing list (OSGeo Mailing Lists contain '''List-Post''' header) - http://www.mutt.org/doc/manual/#list-reply | ||
+ | |||
+ | = Nabble (deprecated) = | ||
+ | |||
+ | |||
+ | Since August 2021 Nabble is not anymore an archive for OSGeo. More details in the [https://www.osgeo.org/foundation-news/osgeo-nabble-forum-is-shutting-down/ OSGeo news item]. | ||
+ | |||
+ | [[Category:Infrastructure]] | ||
+ | [[Category:Services]] |
Latest revision as of 11:12, 7 October 2024
Mailing lists are administered with Mailman on lists.osgeo.org (see http://lists.osgeo.org for a list of available lists), physically this is osgeo6. Jorge Sanz (jsanz) does most of the mailman work, with Frank Warmerdam (warmerdam) and Martin Spott (martin) being the server side people if there are super technical issues.
Notes for List Administrators
Requesting a new list
If your project or activity would like to have a mailing list at lists.osgeo.org please take the following steps:
- First, confirm that there isn't already an appropriate list for this activity, and consider whether OSGeo is the right home for the proposed list.
- File a ticket at http://trac.osgeo.org/osgeo (use your OSGeo Userid to login) and make sure the component is set to "Mails & Mailing Lists". This is the request to the System Administration Committee to create the list.
- Please include the exact name you would like for the list (ie. foo-announce), and the email address of the person who should be the list admin. Also provide the justification for the list, and indicate if it was explicitly requested by an OSGeo Project PSC or project representative.
- Include any special instructions (such pointers to info if the list is actually being migrated from elsewhere).
- Hopefully, someone in SAC will act on the ticket and create the list. When complete the ticket will be updated and closed, and the proposed admin will receive an email with the admin password for the list.
- Review the list settings, in particular ensuring that the short description gets set (ie. review following points)
Generally it is up to the mailman administrator to judge whether a list is justified for creation. The justification doesn't need to be strong, but the list should be OSGeo related, reasonably likely to generate some use, or be requested on behalf of a project.
Configuring the mailing list
- Please set "A terse phrase identifying this list." to something brief and meaningful as this is what is shown on the main lists directory. It is on the General tab.
- Set "Who can view subscription list?" on the Privacy tab to "List Admin Only", otherwise anyone can subscribe and then harvest the mailing list.
- Set generic_nonmember_action on the Privacy->Sender filters tab to reject as most lists are not actively administered so we prefer to reject rather than holding posts from unsubscribed folks.
- Be sure to choose a name that does not already have an existing alias, such as: sales, support, www, security. If you need to use one of these names someone will have to edit the /etc/aliases file for you manually.
- Configure to not tamper the original incoming message in any way to avoid breaking DKIM signatures, see https://trac.osgeo.org/osgeo/ticket/3011#comment:23 - you may find the /osgeo/mailman-tools/recommended_setup script useful to do this (installed on the list server host)
Other options
- For announce lists (ie. moderated lists) turn on "Emergency moderation of all list traffic" on. It is on the General tab.
Outstanding Issues
- Administrator/moderator permissions. Currently this is a hodgepodge. Some administrators have set the passwords but then others can't help administrate the list. The default admin password is not very secure. It has been suggested that we use some sort of LDAP/http authentication to verify that folks trying to do list admin are in some sort of mail administration group.
- List setting policies. Do we want to mandate more policies than the ones listed above as "Notes for List Administrators"?
- Search. Do we want to do anything special to enable searching the archive? Jason has setup an OSGeo.org grouping on nabble (
http://www.nabble.com/OSGeo.org-f18127.html, http://osgeo-org.1803224.n2.nabble.com/) that makes searching all osgeo related mailing lists effective. We might want to expand on that.
System Tasks
Mailing List Admin tasks when moving list to discourse
Modify the listinfo.html of the mailing list, navigating from the administration panel
Edit the public HTML pages and text files -> General list information page
This
- will not show about new subscriptions
- will allow the subscribers to unsubscribe.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <!-- $Revision: 5865 $ --> <HTML> <HEAD> <TITLE><MM-List-Name> Info Page</TITLE> </HEAD> <BODY BGCOLOR="#ffffff"> <P> <TABLE BORDER="0" CELLSPACING="4" CELLPADDING="5"> <TR> <TD COLSPAN="2" WIDTH="100%" BGCOLOR="#99CCFF" ALIGN="CENTER"> <B><FONT COLOR="#000000" SIZE="+1"><MM-List-Name> -- <MM-List-Description></FONT></B> </TD> </TR> <tr> <td colspan="2"> <p> </td> </tr> <tr> <TD COLSPAN="1" WIDTH="100%" BGCOLOR="#FFF0D0"> <B><FONT COLOR="#000000">About <MM-List-Name></FONT></B> </TD> <TD COLSPAN="1" WIDTH="100%" BGCOLOR="#FFF0D0"> <MM-lang-form-start><MM-displang-box> <MM-list-langs> <MM-form-end> </TD> </TR> <tr> <td colspan="2"> <P><MM-List-Info></P> <p> To see the collection of prior postings to the list, visit the <MM-Archive><MM-List-Name> Archives</MM-Archive>. <MM-Restricted-List-Message> </p> </TD> </TR> <tr> <TD COLSPAN="2" WIDTH="100%"> <p> <MM-Options-Form-Start> <MM-Editing-Options> <MM-Form-End> </td> </tr> </table> <MM-Mailman-Footer> </BODY> </HTML>
New Lists
Not accepting new lists
- Create a new list through the web admin login.
Renaming Lists
- create new list
$sudo /usr/lib/mailman/bin/newlist listname admins@email lists_passwd
- move original lists archive to newlists archive location
$ sudo cp /var/lib/mailman/archives/private/oldlist.mbox/oldlist.mbox \ /var/lib/mailman/archives/private/newlist.mbox/newlist.mbox
- create archive
$ sudo /usr/lib/mailman/bin/arch --wipe newlist
- export subscribers from old list regular and digest members
$ sudo /usr/lib/mailman/bin/list_members -r oldlistname > listname-regular.txt $ sudo /usr/lib/mailman/bin/list_members -d oldlistname > listname-digest.txt
- import subscribers into new list
$ sudo /usr/lib/mailman/bin/add_members --regular-members=listname-regular.txt --welcome-msg=y newlistname $ sudo /usr/lib/mailman/bin/add_members --digest-members=listname-digest.txt --welcome-msg=y newlistname
- remove old list
$ sudo /usr/lib/mailman/bin/rmlist oldlistname
- check that proper permissions are set (takes 10+ minutes!)
$ sudo /usr/lib/mailman/bin/check_perms -f
- edit postfix aliases - /etc/aliases
oldlist: newlist@lists.osgeo.org oldlist-request: newlist-request@lists.osgeo.org oldlist-admin: newlist-admin@lists.osgeo.org oldlist-owner: newlist-owner@lists.osgeo.org
- update postfix with new aliases
$ sudo /usr/bin/newaliases $ sudo /usr/sbin/service postfix reload
copy / clone user addresses across lists
- add -r to remove the old address
$ sudo clone_member old@address.com new@address.com
Migrate
When migrating from another system to osgeo, get the .mbox files, and the whole /var/lib/mailman/lists/<listname> directory, and follow roughly the following steps as root (or sudo each step):
# newlist -q grass-announce warmerdam@pobox.com junk # cd /var/lib/mailman/lists/grass-announce # cp ~warmerdam/grass-announce/* . # withlist -l -r fix_url grass-announce # /usr/lib/mailman/bin/arch grass-announce ~warmerdam/grass-announce.mbox # chown -R list.list /var/lib/mailman/lists/grass-announce # chown -R list.list /var/lib/mailman/archives/private/grass-announce* # /usr/lib/mailman/bin/genaliases # /usr/sbin/service postfix restart # /usr/lib/mailman/bin/mailmanctl restart
If the list name on the new system is different than on the old system, it may be prudent to fix this up on the mail page of the web admin interface after the withlist command has been used, and before the archives are imported. (FrankW)
References
- Here is another approach to renaming, though our web admin interface won't actually allow you to do the final rename step
Remove Mails from Archive
Sometimes users send mails with contentious content to the list and ask for it to be removed. As a courtesy this should be done asap. In general it is better to not remove the mail altogether but just remove the content. Instructions:
Archives live in:
/var/lib/mailman/archives/
To recreate archives use:
/usr/lib/mailman/bin/arch [listname]
List Statistics
From time to time it is nice to get a sense of the number of lists, users, etc. The following may be helpful:
all subscribers of all osgeo lists hosted on osgeo.org:
sudo list_lists -b | xargs -n 1 sudo list_members | wc -l
unique list of email addresses across all lists:
sudo list_lists -b | xargs -n 1 sudo list_members | sort | uniq | wc -l
Note: since 2009 there is Markus Neteler's cronjob running for gathering these statistics and sending them to "info AT osgeo org". For a time series, see statistics
Hints for subscribers
Being subscribed to one or more mailing lists may result in a lot of incoming mail so it may be useful to configure your Mail User Agent (MUA) so that it makes it easier for you to tell which mail comes from which mailing list. How to do this depends on your system setup.
Procmail users
If you use Procmail to filter your incoming email a good recipe to automatically put mailing list traffic each in its own folder could be this:
:0 * ^List-Id: *<\/[^.]* .list-$MATCH/
Mutt users
If you use Mutt MUA you may find these useful:
- subjectrx directive to tweak the Subject line (e.g. drop or add a prefix) - http://www.mutt.org/doc/manual/#display-munging
- list-reply command to reply to the mailing list (OSGeo Mailing Lists contain List-Post header) - http://www.mutt.org/doc/manual/#list-reply
Nabble (deprecated)
Since August 2021 Nabble is not anymore an archive for OSGeo. More details in the OSGeo news item.