Difference between revisions of "SAC:Mailing Lists"

From OSGeo
Jump to navigation Jump to search
(→‎New Lists: no longer an error on list creation.)
 
(24 intermediate revisions by 8 users not shown)
Line 1: Line 1:
−
Mailing lists are administered with Mailman on the virtual host lists.osgeo.org. Physically this is the "Mail" VM running on osgeo4.  Frank Warmerdam (warmerdam) and Alan Boudreault (aboudreault) do most of the mailman work.
+
Mailing lists are administered with Mailman on lists.osgeo.org (see http://lists.osgeo.org for a list of available lists), physically this is [[osgeo6]]. Jorge Sanz (jsanz) does most of the mailman work, with Frank Warmerdam (warmerdam) and Martin Spott (martin) being the server side people if there are super technical issues.
  
 
= Notes for List Administrators =  
 
= Notes for List Administrators =  
Line 8: Line 8:
  
 
* First, confirm that there isn't already an appropriate list for this activity, and consider whether OSGeo is the right home for the proposed list.  
 
* First, confirm that there isn't already an appropriate list for this activity, and consider whether OSGeo is the right home for the proposed list.  
−
* File a ticket at http://trac.osgeo.org/osgeo (use your OSGeo Userid to login) and make sure the component is set to "SAC".  This is the request to the System Administration Committee to create the list.  
+
* File a ticket at http://trac.osgeo.org/osgeo (use your OSGeo Userid to login) and make sure the component is set to "Mails & Mailing Lists".  This is the request to the System Administration Committee to create the list.  
 
* Please include the exact name you would like for the list (ie. foo-announce), and the email address of the person who should be the list admin.  Also provide the justification for the list, and indicate if it was explicitly requested by an OSGeo Project PSC or project representative.  
 
* Please include the exact name you would like for the list (ie. foo-announce), and the email address of the person who should be the list admin.  Also provide the justification for the list, and indicate if it was explicitly requested by an OSGeo Project PSC or project representative.  
 
* Include any special instructions (such pointers to info if the list is actually being migrated from elsewhere).  
 
* Include any special instructions (such pointers to info if the list is actually being migrated from elsewhere).  
Line 14: Line 14:
 
* Review the list settings, in particular ensuring that the short description gets set (ie. review following points)
 
* Review the list settings, in particular ensuring that the short description gets set (ie. review following points)
  
−
Generally it is up to the mailman administrator to judge whether a list is justified for creation.  The justification doesn't need to be strong, but the list should be OSGeo related, reasonably likely to generate some use, or be requested on behalf of a project.
+
Generally it is up to the mailman administrator to judge whether a list is justified for creation.  The justification doesn't need to be strong, but the list should be OSGeo related, reasonably likely to generate some use, or be requested on behalf of a project.
  
−
== All lists ==
+
== Configuring the mailing list ==
  
 
* Please set "A terse phrase identifying this list." to something brief and meaningful as this is what is shown on the main lists directory.  It is on the ''General'' tab.  
 
* Please set "A terse phrase identifying this list." to something brief and meaningful as this is what is shown on the main lists directory.  It is on the ''General'' tab.  
 
* Set "Who can view subscription list?" on the ''Privacy'' tab to "List Admin Only", otherwise anyone can subscribe and then harvest the mailing list.  
 
* Set "Who can view subscription list?" on the ''Privacy'' tab to "List Admin Only", otherwise anyone can subscribe and then harvest the mailing list.  
 
* Set ''generic_nonmember_action'' on the ''Privacy->Sender filters'' tab to reject as most lists are not actively administered so we prefer to reject rather than holding posts from unsubscribed folks.
 
* Set ''generic_nonmember_action'' on the ''Privacy->Sender filters'' tab to reject as most lists are not actively administered so we prefer to reject rather than holding posts from unsubscribed folks.
−
* Ensure the "Prefix for subject line of list postings." on the ''General'' tab is set to something meaningful.  If the list name is generic (such as "announce") it may be helpful to prefix it with OSGeo-, eg. "[OSGeo-Announce]"
 
 
* Be sure to choose a name that does not already have an existing alias, such as: sales, support, www, security.  If you need to use one of these names someone will have to edit the /etc/aliases file for you manually.
 
* Be sure to choose a name that does not already have an existing alias, such as: sales, support, www, security.  If you need to use one of these names someone will have to edit the /etc/aliases file for you manually.
 +
* Configure to not tamper the original incoming message in any way to avoid breaking DKIM signatures, see https://trac.osgeo.org/osgeo/ticket/3011#comment:23 - you may find the /osgeo/mailman-tools/recommended_setup script useful to do this (installed on the list server host)
  
 
== Other options ==
 
== Other options ==
Line 35: Line 35:
  
 
= System Tasks =  
 
= System Tasks =  
 +
 +
== Mailing List Admin tasks when moving list to discourse ==
 +
 +
Modify the listinfo.html of the mailing list, navigating from the administration panel
 +
 +
'''Edit the public HTML pages and text files''' -> '''General list information page'''
 +
 +
This
 +
* will not show about new subscriptions
 +
* will allow the subscribers to unsubscribe.
 +
 +
<pre>
 +
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 +
<!-- $Revision: 5865 $ -->
 +
<HTML>
 +
  <HEAD>
 +
    <TITLE><MM-List-Name> Info Page</TITLE>
 +
 
 +
  </HEAD>
 +
  <BODY BGCOLOR="#ffffff">
 +
 +
    <P>
 +
      <TABLE BORDER="0" CELLSPACING="4" CELLPADDING="5">
 +
<TR>
 +
  <TD COLSPAN="2" WIDTH="100%" BGCOLOR="#99CCFF" ALIGN="CENTER">
 +
    <B><FONT COLOR="#000000" SIZE="+1"><MM-List-Name> --
 +
<MM-List-Description></FONT></B>
 +
  </TD>
 +
</TR>
 +
<tr>
 +
    <td colspan="2">
 +
      <p>&nbsp;
 +
    </td>
 +
  </tr>
 +
  <tr>
 +
    <TD COLSPAN="1" WIDTH="100%" BGCOLOR="#FFF0D0">
 +
      <B><FONT COLOR="#000000">About <MM-List-Name></FONT></B>
 +
    </TD>
 +
            <TD COLSPAN="1" WIDTH="100%" BGCOLOR="#FFF0D0">
 +
              <MM-lang-form-start><MM-displang-box> <MM-list-langs>
 +
              <MM-form-end>
 +
            </TD>
 +
  </TR>
 +
    <tr>
 +
      <td colspan="2">
 +
<P><MM-List-Info></P>
 +
  <p> To see the collection of prior postings to the list,
 +
      visit the <MM-Archive><MM-List-Name>
 +
  Archives</MM-Archive>.
 +
      <MM-Restricted-List-Message>
 +
  </p>
 +
</TD>
 +
      </TR>
 +
    <tr>
 +
      <TD COLSPAN="2" WIDTH="100%">
 +
  <p>
 +
<MM-Options-Form-Start>
 +
<MM-Editing-Options>
 +
<MM-Form-End>
 +
      </td>
 +
    </tr>
 +
  </table>
 +
<MM-Mailman-Footer>
 +
</BODY>
 +
</HTML>
 +
</pre>
  
  
 
== New Lists ==
 
== New Lists ==
  
−
* Create a new list through the [http://lists.osgeo.org/mailman/create web admin login]. You may need to do the following to make the lists operational:
+
 
−
# sudo /usr/lib/mailman/bin/genaliases
+
'''Not accepting new lists'''
−
# sudo /sbin/service postfix restart
+
* Create a new list through the [http://lists.osgeo.org/mailman/create web admin login].
  
 
== Renaming Lists ==
 
== Renaming Lists ==
Line 49: Line 115:
 
* move original lists archive to newlists archive location
 
* move original lists archive to newlists archive location
 
   $ sudo cp /var/lib/mailman/archives/private/oldlist.mbox/oldlist.mbox \
 
   $ sudo cp /var/lib/mailman/archives/private/oldlist.mbox/oldlist.mbox \
−
     /var/lib/mailman/archives/private/newlist.mbox/
+
     /var/lib/mailman/archives/private/newlist.mbox/newlist.mbox
 
* create archive
 
* create archive
 
   $ sudo /usr/lib/mailman/bin/arch --wipe newlist
 
   $ sudo /usr/lib/mailman/bin/arch --wipe newlist
Line 60: Line 126:
 
* remove old list
 
* remove old list
 
   $ sudo /usr/lib/mailman/bin/rmlist oldlistname
 
   $ sudo /usr/lib/mailman/bin/rmlist oldlistname
−
* update aliases and check that proper permissions are set
+
* check that proper permissions are set (takes 10+ minutes!)
−
  $ sudo /usr/lib/mailman/bin/genaliases
 
 
   $ sudo /usr/lib/mailman/bin/check_perms -f
 
   $ sudo /usr/lib/mailman/bin/check_perms -f
 
* edit postfix aliases - /etc/aliases
 
* edit postfix aliases - /etc/aliases
Line 70: Line 135:
 
* update postfix with new aliases
 
* update postfix with new aliases
 
   $ sudo /usr/bin/newaliases
 
   $ sudo /usr/bin/newaliases
−
   $ sudo /sbin/service postfix reload
+
   $ sudo /usr/sbin/service postfix reload
−
* copy / clone user addresses across lists:
+
 
−
** add '''-r''' to remove the old address
+
== copy / clone user addresses across lists ==
 +
* add '''-r''' to remove the old address
 
   $ sudo clone_member old@address.com new@address.com
 
   $ sudo clone_member old@address.com new@address.com
  
Line 84: Line 150:
 
   # withlist -l -r fix_url grass-announce
 
   # withlist -l -r fix_url grass-announce
 
   # /usr/lib/mailman/bin/arch grass-announce ~warmerdam/grass-announce.mbox
 
   # /usr/lib/mailman/bin/arch grass-announce ~warmerdam/grass-announce.mbox
−
   # chown -R mailman.mailman /var/lib/mailman/lists/grass-announce
+
   # chown -R list.list /var/lib/mailman/lists/grass-announce
−
   # chown -R mailman.mailman /var/lib/mailman/archives/private/grass-announce*
+
   # chown -R list.list /var/lib/mailman/archives/private/grass-announce*
−
   # genaliases
+
   # /usr/lib/mailman/bin/genaliases
−
   # /sbin/service postfix restart
+
   # /usr/sbin/service postfix restart
−
   # mailmanctl restart
+
   # /usr/lib/mailman/bin/mailmanctl restart
  
 
If the list name on the new system is different than on the old system, it may be prudent to fix this up on the mail page of the web admin interface after the ''withlist'' command has been used, and before the archives are imported.  (''FrankW'')
 
If the list name on the new system is different than on the old system, it may be prudent to fix this up on the mail page of the web admin interface after the ''withlist'' command has been used, and before the archives are imported.  (''FrankW'')
  
−
 
 
 
== References ==
 
== References ==
 
* Here is [http://www.chris-lamb.co.uk/2007/10/02/renaming-a-mailman-list/ another approach to renaming], though our web admin interface won't actually allow you to do the final rename step
 
* Here is [http://www.chris-lamb.co.uk/2007/10/02/renaming-a-mailman-list/ another approach to renaming], though our web admin interface won't actually allow you to do the final rename step
 +
 +
 +
== Remove Mails from Archive ==
 +
Sometimes users send mails with contentious content to the list and ask for it to be removed. As a courtesy this should be done asap. In general it is better to not remove the mail altogether but just remove the content. Instructions:
 +
* http://wiki.list.org/pages/viewpage.action?pageId=4030681
 +
Archives live in:
 +
/var/lib/mailman/archives/
 +
To recreate archives use:
 +
/usr/lib/mailman/bin/arch [listname]
  
  
Line 109: Line 183:
 
   sudo list_lists -b | xargs -n 1 sudo list_members | sort |  uniq | wc -l
 
   sudo list_lists -b | xargs -n 1 sudo list_members | sort |  uniq | wc -l
  
−
= Nabble =  
+
Note: since 2009 there is [[User:Neteler|Markus Neteler]]'s cronjob running for gathering these statistics and sending them to "info AT osgeo org". For a time series, [[VisibilityStats#Mailing_list_subscribers|see statistics]]
 +
 
 +
= Hints for subscribers =
 +
 
 +
Being subscribed to one or more mailing lists may result in a lot of incoming mail so it may be useful to configure your Mail User Agent (MUA) so that it makes it easier for you to tell which mail comes from which mailing list. How to do this depends on your system setup.
 +
 
 +
== Procmail users ==
 +
 
 +
If you use [https://en.wikipedia.org/wiki/Procmail Procmail] to filter your incoming email a good recipe to automatically
 +
put mailing list traffic each in its own folder could be this:
 +
 
 +
    :0 
 +
    * ^List-Id: *<\/[^.]*
 +
    .list-$MATCH/
 +
 
 +
== Mutt users ==
 +
 
 +
If you use [http://mutt.org/ Mutt] MUA you may find these useful:
 +
 
 +
* '''subjectrx''' directive to tweak the Subject line (e.g. drop or add a prefix) - http://www.mutt.org/doc/manual/#display-munging
 +
* '''list-reply''' command to reply to the mailing list (OSGeo Mailing Lists contain '''List-Post''' header) - http://www.mutt.org/doc/manual/#list-reply
  
−
There are a variety of web forum oriented interfaces for mailing lists that can be connected to existing mailing lists.  One of the most widely used is Nabble and OSGeo makes an effort to provide some administrative support for it. 
+
= Nabble (deprecated) =
  
−
The Nabble liasons for OSGeo are [[Jorge Sanz]] with [[Mateusz Loskot]] as a backup.
 
  
−
Technical details to follow.
+
Since August 2021 Nabble is not anymore an archive for OSGeo. More details in the [https://www.osgeo.org/foundation-news/osgeo-nabble-forum-is-shutting-down/ OSGeo news item].
  
 
[[Category:Infrastructure]]
 
[[Category:Infrastructure]]
 +
[[Category:Services]]

Latest revision as of 11:12, 7 October 2024

Mailing lists are administered with Mailman on lists.osgeo.org (see http://lists.osgeo.org for a list of available lists), physically this is osgeo6. Jorge Sanz (jsanz) does most of the mailman work, with Frank Warmerdam (warmerdam) and Martin Spott (martin) being the server side people if there are super technical issues.

Notes for List Administrators

Requesting a new list

If your project or activity would like to have a mailing list at lists.osgeo.org please take the following steps:

  • First, confirm that there isn't already an appropriate list for this activity, and consider whether OSGeo is the right home for the proposed list.
  • File a ticket at http://trac.osgeo.org/osgeo (use your OSGeo Userid to login) and make sure the component is set to "Mails & Mailing Lists". This is the request to the System Administration Committee to create the list.
  • Please include the exact name you would like for the list (ie. foo-announce), and the email address of the person who should be the list admin. Also provide the justification for the list, and indicate if it was explicitly requested by an OSGeo Project PSC or project representative.
  • Include any special instructions (such pointers to info if the list is actually being migrated from elsewhere).
  • Hopefully, someone in SAC will act on the ticket and create the list. When complete the ticket will be updated and closed, and the proposed admin will receive an email with the admin password for the list.
  • Review the list settings, in particular ensuring that the short description gets set (ie. review following points)

Generally it is up to the mailman administrator to judge whether a list is justified for creation. The justification doesn't need to be strong, but the list should be OSGeo related, reasonably likely to generate some use, or be requested on behalf of a project.

Configuring the mailing list

  • Please set "A terse phrase identifying this list." to something brief and meaningful as this is what is shown on the main lists directory. It is on the General tab.
  • Set "Who can view subscription list?" on the Privacy tab to "List Admin Only", otherwise anyone can subscribe and then harvest the mailing list.
  • Set generic_nonmember_action on the Privacy->Sender filters tab to reject as most lists are not actively administered so we prefer to reject rather than holding posts from unsubscribed folks.
  • Be sure to choose a name that does not already have an existing alias, such as: sales, support, www, security. If you need to use one of these names someone will have to edit the /etc/aliases file for you manually.
  • Configure to not tamper the original incoming message in any way to avoid breaking DKIM signatures, see https://trac.osgeo.org/osgeo/ticket/3011#comment:23 - you may find the /osgeo/mailman-tools/recommended_setup script useful to do this (installed on the list server host)

Other options

  • For announce lists (ie. moderated lists) turn on "Emergency moderation of all list traffic" on. It is on the General tab.

Outstanding Issues

  1. Administrator/moderator permissions. Currently this is a hodgepodge. Some administrators have set the passwords but then others can't help administrate the list. The default admin password is not very secure. It has been suggested that we use some sort of LDAP/http authentication to verify that folks trying to do list admin are in some sort of mail administration group.
  2. List setting policies. Do we want to mandate more policies than the ones listed above as "Notes for List Administrators"?
  3. Search. Do we want to do anything special to enable searching the archive? Jason has setup an OSGeo.org grouping on nabble (http://www.nabble.com/OSGeo.org-f18127.html, http://osgeo-org.1803224.n2.nabble.com/) that makes searching all osgeo related mailing lists effective. We might want to expand on that.

System Tasks

Mailing List Admin tasks when moving list to discourse

Modify the listinfo.html of the mailing list, navigating from the administration panel

Edit the public HTML pages and text files -> General list information page

This

  • will not show about new subscriptions
  • will allow the subscribers to unsubscribe.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<!-- $Revision: 5865 $ -->
<HTML>
  <HEAD>
    <TITLE><MM-List-Name> Info Page</TITLE>
  
  </HEAD>
  <BODY BGCOLOR="#ffffff">

    <P>
      <TABLE BORDER="0" CELLSPACING="4" CELLPADDING="5">
	<TR>
	  <TD COLSPAN="2" WIDTH="100%" BGCOLOR="#99CCFF" ALIGN="CENTER">
	    <B><FONT COLOR="#000000" SIZE="+1"><MM-List-Name> --
	<MM-List-Description></FONT></B>
	  </TD>
	</TR>
	<tr>
	    <td colspan="2">
	      <p> 
	    </td>
	  </tr>
	  <tr>
	    <TD COLSPAN="1" WIDTH="100%" BGCOLOR="#FFF0D0">
	      <B><FONT COLOR="#000000">About <MM-List-Name></FONT></B>
	    </TD>
            <TD COLSPAN="1" WIDTH="100%" BGCOLOR="#FFF0D0">
              <MM-lang-form-start><MM-displang-box> <MM-list-langs>
              <MM-form-end>
            </TD>
	  </TR>
	    <tr>
	      <td colspan="2">
		<P><MM-List-Info></P>
	  <p> To see the collection of prior postings to the list,
	      visit the <MM-Archive><MM-List-Name>
		  Archives</MM-Archive>.
	      <MM-Restricted-List-Message>
	  </p>
	</TD>
      </TR>
    <tr>
      <TD COLSPAN="2" WIDTH="100%">
	  <p>
	<MM-Options-Form-Start>
	<MM-Editing-Options>
		<MM-Form-End>
      </td>
    </tr>
  </table>
<MM-Mailman-Footer>
</BODY>
</HTML>


New Lists

Not accepting new lists

Renaming Lists

  • create new list
  $sudo /usr/lib/mailman/bin/newlist listname admins@email lists_passwd
  • move original lists archive to newlists archive location
  $ sudo cp /var/lib/mailman/archives/private/oldlist.mbox/oldlist.mbox \
    /var/lib/mailman/archives/private/newlist.mbox/newlist.mbox
  • create archive
  $ sudo /usr/lib/mailman/bin/arch --wipe newlist
  • export subscribers from old list regular and digest members
  $ sudo /usr/lib/mailman/bin/list_members -r oldlistname > listname-regular.txt
  $ sudo /usr/lib/mailman/bin/list_members -d oldlistname > listname-digest.txt
  • import subscribers into new list
  $ sudo /usr/lib/mailman/bin/add_members --regular-members=listname-regular.txt --welcome-msg=y newlistname
  $ sudo /usr/lib/mailman/bin/add_members --digest-members=listname-digest.txt --welcome-msg=y newlistname
  • remove old list
  $ sudo /usr/lib/mailman/bin/rmlist oldlistname
  • check that proper permissions are set (takes 10+ minutes!)
  $ sudo /usr/lib/mailman/bin/check_perms -f
  • edit postfix aliases - /etc/aliases
  oldlist:               newlist@lists.osgeo.org
  oldlist-request:       newlist-request@lists.osgeo.org
  oldlist-admin:         newlist-admin@lists.osgeo.org
  oldlist-owner:         newlist-owner@lists.osgeo.org
  • update postfix with new aliases
  $ sudo /usr/bin/newaliases
  $ sudo /usr/sbin/service postfix reload

copy / clone user addresses across lists

  • add -r to remove the old address
  $ sudo clone_member old@address.com new@address.com

Migrate

When migrating from another system to osgeo, get the .mbox files, and the whole /var/lib/mailman/lists/<listname> directory, and follow roughly the following steps as root (or sudo each step): 

  # newlist -q grass-announce warmerdam@pobox.com junk 
  # cd /var/lib/mailman/lists/grass-announce
  # cp ~warmerdam/grass-announce/* . 
  # withlist -l -r fix_url grass-announce
  # /usr/lib/mailman/bin/arch grass-announce ~warmerdam/grass-announce.mbox
  # chown -R list.list /var/lib/mailman/lists/grass-announce
  # chown -R list.list /var/lib/mailman/archives/private/grass-announce*
  # /usr/lib/mailman/bin/genaliases
  # /usr/sbin/service postfix restart
  # /usr/lib/mailman/bin/mailmanctl restart

If the list name on the new system is different than on the old system, it may be prudent to fix this up on the mail page of the web admin interface after the withlist command has been used, and before the archives are imported. (FrankW)

References


Remove Mails from Archive

Sometimes users send mails with contentious content to the list and ask for it to be removed. As a courtesy this should be done asap. In general it is better to not remove the mail altogether but just remove the content. Instructions:

Archives live in: 

/var/lib/mailman/archives/

To recreate archives use: 

/usr/lib/mailman/bin/arch [listname]


List Statistics

From time to time it is nice to get a sense of the number of lists, users, etc. The following may be helpful:

all subscribers of all osgeo lists hosted on osgeo.org: 

 sudo list_lists -b | xargs -n 1 sudo list_members | wc -l

unique list of email addresses across all lists: 

 sudo list_lists -b | xargs -n 1 sudo list_members | sort |  uniq | wc -l

Note: since 2009 there is Markus Neteler's cronjob running for gathering these statistics and sending them to "info AT osgeo org". For a time series, see statistics

Hints for subscribers

Being subscribed to one or more mailing lists may result in a lot of incoming mail so it may be useful to configure your Mail User Agent (MUA) so that it makes it easier for you to tell which mail comes from which mailing list. How to do this depends on your system setup.

Procmail users

If you use Procmail to filter your incoming email a good recipe to automatically put mailing list traffic each in its own folder could be this: 

   :0  
   * ^List-Id: *<\/[^.]*
   .list-$MATCH/

Mutt users

If you use Mutt MUA you may find these useful:

Nabble (deprecated)

Since August 2021 Nabble is not anymore an archive for OSGeo. More details in the OSGeo news item.

Retrieved from "http://wiki.osgeo.org/w/index.php?title=SAC:Mailing_Lists&oldid=132694"