Difference between revisions of "FOSS4G 2009 Workshop Selection Criteria"

From OSGeo
Jump to navigation Jump to search
(Added reality.)
Line 1: Line 1:
−
== Criteria used for ranking workshops and tutorials ==
+
== Process ==
−
 
 
−
The following criteria will be scored:
 
−
 
 
−
{|
 
−
|-
 
−
| 1.
 
−
|
 
−
| Basic themes
 
−
|-
 
−
|
 
−
| 1.1
 
−
| Meets the conference theme, "User Driven"
 
−
|-
 
−
|
 
−
| 1.2
 
−
| Makes use of FOSS software
 
−
|-
 
−
|
 
−
| 1.3
 
−
| Promotes FOSS software
 
−
|-
 
−
| 2.
 
−
|
 
−
| Workshop
 
−
 
 
−
|-
 
−
|
 
−
| 2.1
 
−
| Demonstrates practical application
 
−
|-
 
−
|
 
−
| 2.2
 
−
| Feedback / evaluations from previous conferences for similar workshop topic
 
−
|-
 
−
| 3.
 
−
|
 
−
| Instructor(s)
 
−
|-
 
−
|
 
−
| 3.1
 
−
| Expertise in the topics discussed
 
−
|-
 
−
|
 
−
| 3.2
 
−
| Previous presentation experience
 
−
|-
 
−
|
 
−
| 3.3
 
−
| Feedback / evaluations from previous conferences
 
−
|-
 
−
| 4.
 
−
|
 
−
| Booleans
 
−
|-
 
−
|
 
−
| 4.1
 
−
| Beginner User
 
−
|-
 
−
|
 
−
| 4.2
 
−
| Technical requirements can be met by the conference facilities
 
−
|}
 
−
 
 
−
 
 
−
== Scoring ==
 
  
−
Each criteria above will be rated on a scale from 0 to 5 as follows:
+
From my email to the list:
  
−
  0 = does not meet the criteria
+
  I promised a quick overview of the process we will be running through for
−
  1 = pays lip-service to the criteria
+
workshop and tutorial submissions, so here it is.
−
  3 = fundamentally meets the criteria
+
 
−
  5 = exceeds all of the criteria
+
After the call closes (tonight for me) I will be munching through all the  
 +
  submissions and producing two pdf docs, one for workshops one for tutorials. 
 +
These will be made available for people to review, likely by posting them
 +
here.  What I will need is everybody to send me a list of their favorite
 +
10 workshops (or fewer), in order of preference, and their favorite 12
 +
tutorials (or fewer).  If there are workshops that you would really like
 +
to see run as tutorials, make a write-in vote and we can pester them to
 +
change formats if needed.  These should be sent to me (email@deleted.com
 +
use "FOSS4G Workshop Review" as the subject please), not to a public list.
 +
 
 +
  I will then to one of two things.  I can create a private list if there are
 +
sufficient people reviewing (only two have answered by call thus far) and we
 +
can move discussion to that, or I'll just email everyone directly.  I will
 +
aggregate the votes and tell everyone the results identifying clear winners
 +
and which selections need to be fought for. I'm not going to impose any
 +
  'conflict of interest' restrictions here, you may vote for your own, but
 +
please flag it as your own during voting and discussions.
 +
 
 +
The final decision on the recommendation rests with me, which I will make
 +
from the discussion in a week or so.  This recommendation will be passed
 +
to the OC and they will make adjustments as they require.
  
−
Section 4 (Boolean) are ranked simply as satisfying (1) or not satisfying (0) the criteria.
+
We ended up needing a second pass for Workshops that ranked close together.  I asked people to justify their rankings, and at Jeff's suggestion did a 0-5 numerical ranking.  Justifications were all sensible, mostly required to ensure people actually thought about it, and the rankings showed a clear preference so no 'executive decisions' were required.
  
−
== Weighting ==
 
−
 
−
Here's where things get tricky.  The criteria points do not hold even weight, nor do they hold the same weight in workshops as tutorials.
 
−
 
−
For example, it has been observed during past conferences that beginner workshops are much more important than advanced or developer oriented workshops.  As such, beginner workshops should be rated higher.  This is not the case with tutorials, which would benefit from a more balanced distribution.
 
−
 
−
== Process ==
 
  
−
As we have not defined a separate Workshops committee, the selection of workshops will fall to the Local Organising Committee.  All members that are not submitting workshops or tutorials of their own will be ''encouraged'' to rank the workshops and [[User:Mleslie|someone]] will coalate the rankings and report the selections.  The LOC will have the opportunity to alter the selection at that stage in order to ensure we are presenting the kind of program we want, but will need to do so in a public forum.
+
[[Category:FOSS4G2009]]
 +
[[Category:FOSS4G]]

Revision as of 18:04, 26 March 2009

Process

From my email to the list: 

I promised a quick overview of the process we will be running through for 
workshop and tutorial submissions, so here it is.
 
After the call closes (tonight for me) I will be munching through all the 
submissions and producing two pdf docs, one for workshops one for tutorials.  
These will be made available for people to review, likely by posting them 
here.  What I will need is everybody to send me a list of their favorite 
10 workshops (or fewer), in order of preference, and their favorite 12 
tutorials (or fewer).  If there are workshops that you would really like 
to see run as tutorials, make a write-in vote and we can pester them to 
change formats if needed.  These should be sent to me (email@deleted.com
use "FOSS4G Workshop Review" as the subject please), not to a public list.
  
I will then to one of two things.  I can create a private list if there are 
sufficient people reviewing (only two have answered by call thus far) and we 
can move discussion to that, or I'll just email everyone directly.  I will 
aggregate the votes and tell everyone the results identifying clear winners 
and which selections need to be fought for. I'm not going to impose any 
'conflict of interest' restrictions here, you may vote for your own, but 
please flag it as your own during voting and discussions.
  
The final decision on the recommendation rests with me, which I will make 
from the discussion in a week or so.  This recommendation will be passed 
to the OC and they will make adjustments as they require.

We ended up needing a second pass for Workshops that ranked close together. I asked people to justify their rankings, and at Jeff's suggestion did a 0-5 numerical ranking. Justifications were all sensible, mostly required to ensure people actually thought about it, and the rankings showed a clear preference so no 'executive decisions' were required.

Retrieved from "http://wiki.osgeo.org/w/index.php?title=FOSS4G_2009_Workshop_Selection_Criteria&oldid=37073"