FOSS4G2007 Lessons Learned

Back to Conference Committee

This page is a collection of information, comments, and suggestions, based on the planning, organizing, and hosting of the FOSS4G 2007 conference. The intent is to provide a resource that can be used by organizers of future FOSS4G conferences. This is not a forum for conference attendees to comment on the conference. Use FOSS4G2007_Comments for that.

Bidding

If you approach your local conference center, or tourist bureau, you may find a wealth of support for your bid. Local governments often fund promotion to bring in conferences, and will get behind you with free assistance in preparing your bid. A conference of only 500 people can represent a couple million dollars for a local economy, so it is worth their time to help you win the conference.

Victoria 2007 engaged with both the local conference center and a local organizing company, who helped on the bid preparation pro bono in return for the opportunity to work the conference in the event we won. The conference centre provided lots of pre-made content about the facility, accommodations, and so on, and the organizer helped prepare material for the bid web site.

Take advantage of your local government to assist your bid. FOSS4G is a large enough event now that localities receive a significant financial benefit when we locate in their city.

Timetable

The most important piece of timetable information is that you can not make any predictions about response/attendance until deadlines. The number of workshop submissions in 2007 doubled on the deadline day; the number of presentation submissions doubled in the last two days. The 2004 organizers reported having registration go from 100 to 200 in the last month. The 2006 organizers reported having registration go from 150 to over 500 in the last six weeks.

2007 attempted to force the registration timetable backwards somewhat by having an early-bird rate ($395CAD) substantially lower than the regular rate ($565CAD) with the early-bird deadline two months in advance.

Lesson: The steeper Early Bird price schedule, combined with a large associated marketing campaign around the Early Bird deadline did change the traditional registration pattern. 2007 registered 450 people prior to early bird, about 2/3 of the final attendance, far better than in previous years. Remember, the pricing and date alone will not be enough, people have to actually know the deadline is coming, so an associated marketing blitz will also be required.

Lesson: Where there is no financial motivation or other counterbalance, people will put things off to the last minute. Over 2/3 of the workshop submissions were received on the last day. Same basic pattern for the presentations, over 1/2 of the submissions received in the last two days.

Money

Sponsorships

Starting sponsorship solicitation as early as possible is very important. People will register at the last moment, so it is very difficult to predict registration revenues, but sponsorship revenues can be locked up very early through aggressive work on getting commitments from sponsors. First write out a list of potential sponsors, then assign each one to a committee member to pursue. We used the Google Spreadsheet to keep a shared page of information about who had been contacted, who had committed, how many contacts had been made, who at the sponsoring company had responded, etc.

A large number of the sponsorships were obtained through the direct connections of the conference chair with people in the sponsoring organizations (ILMB, Safe, Refractions, Leica, Google, ESRI, GeoConnections, Timberline, Sierra). Some of the international companies (Google, ESRI, Leica) may sponsor again, for longer term strategic reasons, however the lesson we draw from this is that it's important to have connections and be prepared to use them if you're planning on landing a large sponsorship dossier. You can't count on the previous years sponsors simply showing up again the following year. Note that 20% of the ultimate sponsorship total was from Canadian/BC government (ILMB and GeoConnections), so having good government contacts is also important.

Complimentary Registrations and Volunteers

When budgeting, remember that a proportion of your attendance is going to be complimentary. These attendees are hard because while they don't pay, they still do inconvenient things like eat the food and sit in chairs.

We had approximately 80 complimentary registrations. (The exact number should show up in the final report.) That was slightly over 10% of the total attendance.

We used volunteers for room monitors, security, basically everything that required a body, about 7 people at any one time. In return for 8 hours of labor (two 4-hour shifts) they got complimentary registration for the rest of the conference (and a cool orange shirt). The cost of every budget item that had to be scaled on a "per person" basis (food, shirts, bags, etc) was about $230, so those 8 hours of volunteer labour cost us dearly. In 2006, the conference was at a university and they were having a reading break at the time of the conference, so the organizers just hired 8 students to work the conference full time. Even if they paid them $20 per hour, they probably came out far ahead of us, financially. Simply hiring staff for things like room monitoring, manning the registration desk, security, may be more fiscally effective than using volunteers as we did.

Workshop Pricing

The price differential between "Full Conference" and "Full Conference + Workshops" was low enough that (it turned out) the workshops were financially a net neutral activity. We could have cancelled the whole workshop day (theoretically) and the financial outcome of the event would have been identical. Of course, the workshops were a huge draw, so canceling them would be stupid, but given that they are a huge draw, it might be wiser to price them so they turn a bit of a profit.

To reiterate: the cost of renting the center for a the day, computers for the workshops, feeding the delegates, and other sundry expenses for the day was about the same as the revenue represented by the price difference between Full Conference and Workshops times the number of workshop delegates.

Registration

Support For Some Attendees

You will get lots and lots of requests for support. Travel subsidies, free registration, free lodging, you name it. Establish your policy early, and work within that policy. Because we had no policy and no budget item for this, our policy was "no". However, if you think ahead, you can probably ensure that a couple worthy attendees who would otherwise be unable to come can make the conference. Establish your budget number, and how you will decide who gets support, and go from there.

Scams

We had a number of attempts to scam the conference in various ways. The most popular were:

• "I want to attend, please write me a VISA support letter", from folks with no obvious motivation to attend.
• "I paid with a credit card, but I want a re-fund by wire."

We also had a few people pay the conference in full in order to establish their VISA credentials and then attempt to get a VISA on that basis alone. They eventually asked for and received a refund, less the cancellation fee ($50). The cancellation fee sounds draconian, but it probably cut down on the number of these scams by making a failure potentially expensive.

Discounts

Everyone wants a discount, and everyone feels they have a right to a discount. As with your policy for travel support, figure out your discount policy well in advance. The FOSS4G2007_Discounts page is a good start in determining the classes of people who will be looking for discounts.

Breakdown by Country

The registration breakdown by country was:

• North America, 521, 72%
  • USA (265), Canada (253), Mexico (3)
• Europe, 139, 19%
  • Italy (21), France (18), United Kingdom (15), Spain (12), The Netherlands (12), Germany (11), Switzerland (11), Norway (6), Belgium (5), Austria (5), Czech Republic (4), Sweden (4), Finland (3), Portugal (2), Poland (2), Romania (2), Denmark (2), Hungary (1), Luxembourg (1), Latvia (1), Slovenia (1)
• Australasia, 40, 5%
  • Korea (11), Japan (11), Australia (4), Indonesia (3), China (3), New Zealand (3), Fiji (2), Taiwan (1), Thailand (1), Vietnam (1)
• South America, 10, 1.4%
  • Brazil (4), Venezuela (3), Chile (2), Cuba (1)
• Africa, 7, 0.9%
  • South Africa (5), Namibia (2)
• Near East, 4, 0.5%
  • India (2), Iran (1), Afghanistan (1)

The registration breakdown by distance was:

• Less than 10km: 97, 13%
• Between 10km and 100km: 42, 6%
• Between 100km and 1000km: 91, 12%
• Between 1000km and 5000km: 290, 40%
• Between 5000km and 10000km: 175, 24%
• More than 10000km: 28, 4%

Attendance

At the Conference

The recording of attendance has to be done on site. Just because someone registers doesn't mean they attended the conference. There also may be last-minute changes such as one person (perhaps even from another country) attending in place of someone else. The actual attendance at the conference can be tracked by the registration system used at the conference desk where attendees check in and receive their conference pass. Attendance numbers for FOSS4G2007 will be in the final report.

Within the Conference

Attendance at specific venues or events within the conference may be easy to measure, such as at a ticketed social event, or harder to measure, such as estimating the crowd in a ballroom. For FOSS4G2007 some of the attendance recording was done by the conference secretariat as part of their role while on site.

Lesson: We had a 'room monitor' (one of the volunteers) assigned to each room used for Workshops, Labs, and Presentations. For Workshops, one of their roles was to use a registration list to ensure that only people who had paid for the Workshop attended the Workshop. Labs and Presentations were 'first come first served', so there was no such list. It would have been easy to have had a system to have 'room monitors' keep track of attendance.

Within the Code Sprint

50 people showed up at the peak. It was a sort of conference room with maybe 15 round tables - most were used and worked like a good focal point for groups of 10 or less working together.

We supplied some power cords, but not nearly enough, so the tables around the edge of the room were sought after.

Wireless internet was provided. The AV wasn't good, would have been better if we'd stayed at the conference center, since we had them trained by that point.

Paper flip charts were available.

Coffee and snacks were provided in the morning, along with lunch.

Cost about $1500, room+food+AV.

The section is a summary of emails from Paul Ramsey and Tyler Mitchell http://lists.osgeo.org/pipermail/aust-nz/2007-December/thread.html

Marketing

The Mail-out

We attempted some old-fashioned physical marketing, to see what the effect would be. 10,000 postcards were printed, with attractive designs and some basic information and prominent web URL 1 2. Mailing lists were purchased from two North American geospatial magazines (GeoWorld and Geospatial Solutions) for states with good physical proximity to the conference (WA, OR, CA) or strong GIS communities (VA, CO). The entire exercise (designing and printing card, list purchasing, postage) cost about $12,000.

Based on the results below, it does not appear to have worked very well at all.

How People Heard about FOSS4G2007

The registration form on the web site included a question about "how did you hear about the conference". Here are the responses:

• 223 (36%), colleague
• 160 (26%), weblink
• 134 (20%), previous conference
• 60 (10%), email
• 43 (7%), other
• 4 (1%), mailout

On the basis of these numbers, the physical mail-out seems to have been a poor expense. The email spamming activities were highly effective, and worth doing again, both bulk-email to harvested addresses and emails to project mailing lists. The web banners on the project sites were also highly effective. Note that the purchased mailing lists did include e-mail addresses for a couple thousand people, and those were used for direct email campaigns.

Lesson: Harvest and purchase mass email lists. Get emails onto lists of associations and other organizations with lists. Make spam a top marketing priority. Spam more frequently.

Lesson: Ensure as many project sites are badged as possible. Note that the top referrers are Mapserver, QGIS, PostGIS, OSGeo, and GRASS. Paul Ramsey's personal blog (part of the PlanetGeospatial blog roll) was the fifth highest driver of traffic. Get a FOSS4G08 blog into the PlanetGS.com blog roll and keep it live and interesting.

Local / Regional / International

You are going to attract people from three geographic segments, and they have very different characteristics:

• Local people will find it cheap to attend the conference. Therefore, even marginally interested people might go, just to "see what it is about". So your local marketing campaign should be as broad as possible. Hit everyone, don't just target at "GIS, open source", target all scientists, all computer people, all development people. You never know who might have an interest, and because the threshold for them to come is relatively low, it is worth it to be indiscriminate.
• Regional people will have to travel to the conference, but not far. These are folks who might be in the field, but not have heard about open source GIS before. It will still be a learning experience for them. A broad approach is still not a bad idea, but gets harder to execute and more expensive over a larger regional territory. Buy mailing lists from local trade magazines for your targetted regions (2007 did the US states, CA, WA, OR, CO, VA primarily, as well as Canada).
• International people will be committing to a major expense to attend. These are true believers. Getting web badges on the open source project sites, and doing mail-outs to the project lists is enough to keep these folks informed of deadlines and connected to the beat of the conference.

Use Proxies

Wherever possible, identify people to do your marketing for you:

• Emails to project lists should come from project leaders.
• Get local OSGeo chapters to promote the conference to their members and to local GIS organizations.
• Get academics to carry the torch to their professional bodies.

Local Marketing is Key

One thing we failed at in 2007 was really hitting the local market as hard as possible. As a result, we had people who should have known about the conference saying at the last minute "oh, there's a conference coming?". Word of mouth is not enough, you need to spam local people indiscriminately. Build a list of every remotely technical person in the city, and ensure they get three emails about the conference.

Lesson: Don't be afraid to spam. Web traffic analysis from when we did spam showed that we logged way more hits after spam runs on local institutions. We should have done more.

Lesson: Don't take anything for granted. We thought that local people would sort of just "show up". Just because FOSS4G is consuming your every waking moment does not mean that anyone else will have heard about it, even your office mate three cubicles down. Get into local general media, like the local paper. Make sure that every local tech group knows about it (Linux users groups, Java users groups, Perl mongers, etc).

Web Site & Program

The web site is the public face of your conference. It it's not on the web site, it doesn't exist.

People make the "buying decision" for your conference based on your web site. It had better

• Look smart and professional
• Include as much information about the content of the conference as possible

In order to include lots of content, you need to get your program firmed up as quickly as possible. The earlier you have your program set, the more information you can provide to prospective attendees on the web site.

Lists are Important

You will be able to easily mail all the active community members by putting together a list-of-lists (FOSS4G2007_Communication#E-Mail_Marketing) and getting the list owners to help propagate the FOSS4G message.

However.

Since 80% of your attendance will be local and regional people, not necessarily the international community you communicate with on the lists, you must compile lists of local people who may have an interest in the conference. You must compile these lists early in the process so that the marketing message can be put out early for people who need lead-time to get approvals for conference attendance.

Here are some ways to get lists:

• Regional technology magazines sell their subscription lists
  • Get the physical addresses and mail postcards
  • Get the e-mail addresses for your mass-email list
• Screen-scrape the phone directories of government agencies
  • Mail everyone you can, particular people in the 1-2 hour travel radius of your event
• Get contact lists from sponsors if they will provide them
• Professional groups have lists of potentially interested parties
  • Surveyors, geologists, ecologists, biologists, foresters
• Academics are often interested and also connected
  • Computer science, geography, ecology, forestry

Remember that if people do not know the event is coming, they'll never purchase a registration, and you cannot count on the international community for your attendance, you need local attendees.

2007 purchased lists from magazines and did a 10,000 card mail-out. Also compiled addresses from local organization web sites that might be interested, and added them to the main physical/email distribution lists. We compiled the secondary lists manually, just surfing to county and city site and gathering the names of GIS coordinators.

Press Releases

Press releases get wide distribution and repetition via the less-industrious GIS "news" outlets and "journalists". Doing formal press releases and making sure they hit all the GIS web sites and so on for each important milestone (call for workshops, call for papers, early bird, etc) will add to the out-of-community marketing reach.

During the conference, press releases can be used to control the message coming out of the conference. Much of the conference "coverage" will come from these same "journalists" re-packaging the press release material they are receiving. Hence, the coverage of 2007 hinged a lot on the Ingres announcement, the Autodesk/Mentor announcement, and some of the releases Safe Software sent out, all of which were transmitted to the "media" in the traditional PR-oriented way. Having an "in house journalist" to create content, a couple releases a day, and fire it out to the outlets will improve the quality of the "press" the event receives.

Lesson: Coordinate the media your vendors are going to create via press releases, and make sure you add in some of your own. Use press releases as part of the lead-in to important events, like call-for-papers, early bird deadline, and one-week-to-go.

Social Venues

The Sticky Wicket was a great venue in terms of capacity (1200 seats), location (one block from the conference centre) and service options (food and beverage, table service and bar service).

Lesson: A number of people felt short changed when the "Welcome Reception" was really just a mixer at the Wicket. Bad choice of name, should have been "Ice Breaker" or "Get Together". Easier to change the name than to put on an actual reception.

Lesson: The Wicket actually closed for a private function on Tuesday, our "free evening" when ordinarily everyone would have met at the Wicket. Ooops! Check the event schedule for the venue, don't make assumptions about the schedule.

The BC Museum was a nice venue, in terms of visual interest and location. Being indoors also meant we were not affected by weather. A few attendees felt short changed when the reception was a stand-up rather than a full sit-down dinner. However, the stand-up worked very well, allowing people to move about and talk with lots of different folks rather than being planted in one location at a table. Having the venue close to the hotels meant that people could stay for as much or as little as they liked. The IMAX movie was good in theory, but bad in practice -- since we did not pre-screen it, we didn't know just how bad it was going to be! Never believe the trailers.

Lesson: The tickets to the reception cost $50, but the reception cost over $80 for venue, food, drink and entertainment. In general, we seem to have a history of subsidizing the event, but it's a dangerous and potentially unsustainable habit. Why should people who don't go to the event pay for people who do?

Conference Venue

The biggest single expense category for 2007 was the food (it was actually broken into several categories in the budget, but if you put them together it was the easy winner). That was because the Conference Centre had an exclusive catering arrangement with the associated hotel, and the food was expensive. $5 cokes, $4 coffees, that sort of thing. This will be very common for most conference venues. Be prepared, be unsurprised.

One pleasant surprise at the 2007 venue was the price of wireless internet. Conference centres and hotels traditionally charge a huge amount for internet access, but the quotation we received from the in-house provider was 30% of our expected budget number. However, be prepared to pay a lot for internet.

On the exhibition floor, the conference centre might expect you to leave your exhibitors to "fend for themselves", paying extra fees for internet, power, and so on. 2007 opted to pay those charges directly, so they were included in the overall sponsor/exhibition fees and everyone had access. It seems silly to consider power or internet an "optional" expense at a technology trade show -- it is not like we are running a home-and-garden exhibition, where an exhibitor might reasonably run a booth without them.

Audio-Visual

Have laser pointers be part of what is provided for Workshop / Lab Instructors.

During the conference planning, have the conference centre provide a full classroom mockup, including a computer connected to a projector, and a screen. For 2007, we didn't see this complete setup until the conference was starting, and there were comments on the evaluations that the projected images weren't bright enough, or, for larger rooms, big enough.

For 2007, part of the volunteer staff were for 'room monitors' for the Workshops and Labs. Part of that was 'security', to control access to Workshops only for those who had registered, and for Labs to limit the capacity to the size of the room. The other part of the 'room monitor' duties was to be able to assist the Instructors(e.g. handing out and collecting evaluations). When doing the pre-conference orientation for 'room monitors'(or others who may be assisting Instructors/Presenters), make sure they are trained in how to use any handheld or hands-free microphone systems.

Quirks

• FOSS4G attendees have an unusual willingness to sit on the floor during lunch (or really, any time at all). This surprised the venue managers quite a bit. They provided more stand-up tables by day two.
• FOSS4G attendees pack laptops. Almost all of them. That means (a) lots of demand for places to use laptops, like chairs and (b) huge wireless load. VCC had good physical infrastructure in terms of access points and outgoing bandwidth (dual 100Mbit fibre) but had not sufficiently tested their log-in system prior to the conference. The result was a melt-down on day two, as they ran out of logins in their security box, then ongoing issues afterwards from machines with "bad logins" from the melt-down the day prior. This will be hard to avoid -- the conference centre will always be sure they can handle anything right up until you prove them wrong.
• FOSS4G attendees are predominantly male. That means that standardized food orders, that expect more gender balance, will tend to under-budget for how much each attendee eats. This was especially noticeable on "pizza day", when, despite the food services accidentally over-providing 20 pizzas, we still consumed every single piece.

Accommodations

Getting a hotel room block reserved ahead of time was wise. The kind of contract you have to agree to will vary, depending on the demand for rooms from ordinary customers. We got very good contracts, with three different hotels at different rates, with no penalties for not filling the blocks.

• 50 rooms at $100 night
• 50 rooms at $110 night
• 100 rooms at $135 night

The final room block was filled on August 23.

Providing a very complete list of local accommodation options on the web site will save your attendees the trouble of Googling it all up themselves. Plus you'll know and be able to find some that are very hard to find independently. List every reasonable hotel you can think of, with a web link directly to the hotel site, so that folks don't have to hunt them down.

Lesson: FOSS4G attendees are cheap. The cheapest room block ($100 / night) filled up right away, followed by the second cheapest, and the most expensive block ($135 /night) filled up last. Attendees booked hotels relatively late in the process, in the final month before the conference.

Transportation

Try to be explicit about transportation options and publish that information on the web site well in advance. In particular anything that is not normally part of the airline experience, like how to get downtown, is useful for people to visualize their trip and feel comfortable about it.

FOSS4G 2007 kept all activities downtown, and had no transportation logistics. Previous conferences all had at least one event that required hundreds of people to get onto buses (2004, hotels were downtown but conference was at Carleton, and everyone had to be bussed; 2005, the event was a barbecue at an old fort 30 minutes out of town; 2006, after the castle dinner everyone had to be bussed back to Lausanne).

Workshops and Labs

• Initially, the conference organizing committee dealt with "workshops", and that was primarily done by a couple of people
• A Workshop Committee was then formed, to deal with "workshops", within the parameters set out by the conference organizing committee (e.g. the number of time slots for "workshops")
• It was decided that Workshops were to be 3 hours long, with no planned break. On 'the Workshops day'(Monday) there were to be 12 Workshops, with 6 in the morning and 6 in the afternoon.
• In addition to the 12 slots for 3 hour workshops (for the 'workshop day'), there were 16 slots for 1.5 hour Labs within the conference timetable.

Workshop Committee

When the Workshop Committee was formed, there was no clear set of roles and responsibilities, or expectations. This lead two two unclear situations.

One was that once the Workshop Committee ranked the workshop/lab submissions, some members of the committee felt that those results should determine which workshops/labs were selected for the conference program, not that the local organizing committee would make the final decision.

Lesson: Make it clear right from the start what role the Workshop Committee will play in the selection of the workshops/labs.

The other situation was that as the organization of the conference evolved, some tasks that needed to be done fell to the Workshop Committee, and some committee members didn't feel those were things that "they had signed up for". For example, it appeared to be a given that committee members expected to be involved in the process of evaluating workshop/lab submissions, but not involved in the loading and configuring of software on the PCs used for the workshop/labs.

Lesson: Make it clear right from the start the roles and responsibilities of Workshop Committee members. This may include preparation of the Call for Workshops, communications with people during the call, evaluation of the workshop/lab submissions, working with selected Instructors to refine requirements, loading and configuring software on the classroom PCs, communications with Instructors regarding things such as deadlines for printed materials etc., helping with the on site setup and tear down of computers, and assisting with monitoring the workshop/lab classrooms during the conference.

Terminology

The intent for 2007 was that "workshops" would be 'hands on' classes with computers. The term "Workshops" was used to refer to computer classes that were 3 hours long, and for which there was a separate charge from the conference fee. To distinguish the shorter 1.5 hour computer classes, which were within the conference, they were called "Labs". That terminology wasn't decided until after the 'Call for Workshops' was over. In the future, it would be better to use consistent terminology right from the start.

Despite the intent to have the "workshops" be 'hands on', there were a couple of classes in 2007 that had minimal 'hands on', and there were complaints about that in the evaluations. If there is a desire to have something like "workshops", but without a 'hands on' component, perhaps using a different name would be appropriate, such as "Classes".

Call for Workshops

Timing

If the "workshops" are to be a distinct item, for which attendees pay extra, then the "workshops" must be selected prior to the opening of registration. Workshops/Labs are also a good piece of "web site content" that help convey to potential attendees "what the conference is about". As with every other piece of conference content, the sooner these things are selected and published, the easier you will find it to attract non-traditional attendees, who form their opinions based on the content you provide on the web site.

In 2007, the Call for Workshops went out in early February (8 months prior to conference) and closed in early March (7 months prior to conference). Workshops/Labs were selected by the end of March (6 months prior to conference).

Format of the Call

Workshop submissions were accepted by email, with the submitters having to fill in a standard document template. The information received was entered into a summary spreadsheet (for title, name, short abstract) and document (for all information). Submitters were asked to indicate what kind of physical infrastructure they required, and whether their submission was a 3 hour or 1.5 hour submission. Some submitters indicated they could do both formats, others only indicated one.

Lesson: It was not made clear to submitters that "both" was an acceptable answer, which led to a bad selectivity situation later on. If the conference is going to have multiple formats, such as Workshops, Labs, Classes, then make that clear in the document template or instructions, and encourage people to make submissions for all formats that apply to their material.

The document template included a section for 'User Level'. There were issues both with the various levels to choose from, and what they meant. The 'User Level' was used, to some extent, by the Workshop Committee in the selection process to try and make sure the program was balanced. It may be worthwhile to display the 'User Level' in the registration system for classes for which people have to pay extra (i.e. for 2007, for Workshops).

Lesson: The options for 'User Level' need to be broad enough to describe the various types of levels (in 2007, we forgot "intermediate"), and just as importantly, there needs to be a definition for what the levels mean.

Lesson: The 'User Level' was displayed on the website's page for Workshops/Labs. Especially for Labs, where people might decide 'at the last minute' which room to go into, the User Level should be indicated in the conference program and on the signage for the rooms, to help avoid comments on the evaluation forms of people who felt the material was too hard or too easy.

Results of the Call

52 workshop submissions were received. 33 submissions indicated they could do 3 hour formats, which made almost 3-to-1 demand versus supply (12 slots). 22 submissions indicated they could do 1.5 hour formats, which was much closer to the 16 slots available in the lab format.

Selection Process

Ranking

The ranking of the results of the Call for Workshops was done by the Workshop Committee. "workshop" submissions were ranked using a multi-criteria system, scored from 1-5, and each "workshop" was supposed to be given scores for all criteria by all Workshop Committee members. In the end, some members found the process too onerous and only gave one score for each workshop, based on a holistic understanding of all the criteria. Some members ranked in a range of 3-5, others used the entire range of 1-5. In general, because of the different methodologies, the amount of 'information' pulled from the ranking process was not as high as it could have been. Ideally, each committee member would provide equal 'information' to the decision, but in 2007, those members who used the whole range of their scores, and only provided one score, had higher influence than members who ranked more judiciously and provided all the criteria separately.

Lesson #1: Before publishing the Call for Workshops, agree on the decision criteria, and publish them along with the Call.

Lesson #2: Do not attempt to gather individual scores for each criterion. Have members score the submissions holistically, keeping all the criteria in mind as they do so.

Lesson #3: Do not use a 1-X scoring system, but instead use an ordered sorting system, where each member returns a sorted list of submissions, from most to least desirable. This approach maximizes the amount of information gathered about each submission, and sidesteps "plumping" strategies (where a member gives all the submissions a "0" except for the two she really likes which she gives a "5", thereby accentuating her affect on the overall average).

Lesson #4: Potentially, remove the Workshop Committee ranking process entirely and move to a community scoring model. However, given the limited number of time slots, and the benefits for workshop presenters (free admission), a community model might be a tempting target for vote pooling and other forms of influence. Also, because Workshops (i.e. for which there is an extra fee) must be selected far far in advance of the conference date, it will not be possible to bring in the opinions of conference registrants who are not members of the "usual" OSS community of interest.

Decisions

While the Workshop Committee did the ranking of the Workshop/Lab submissions, for 2007 the final decision as to which Workshops/Labs would be in the conference was made by the conference organizing committee. There was some feeling in the Workshop Committee that once the Workshop Committee made it's selections, they should stand, and not be modified by the conference committee.

The Workshop Committee first selected the twelve 3-hour workshops, via a ranking process (see above). Submissions that were accepted as 3-hour Workshops and that also had entries in the 1.5 hour list were removed from that list. Then the 1.5 hour Labs were selected using the same ranking process.

The submissions that were only in the 3-hour list had a much harder time making the cut than the ones that were dual-listed. Some submitters would probably be "OK" with that, since their content would not fit into a 1.5 hour format. Others would have been happy to present in either format, if they knew they had the option when submitting.

Communication with Instructors

For 2007, after the Workshops/Labs were selected, Instructors were contacted. Shortly after that, communication between the Workshop Committee and the Instructors started, using a combination of the Wiki and a mailing list. As things evolved, certain deadlines for submission of materials, etc. were communicated to the Instructors, typically via an email on the mailing list, with additional details on the Wiki. Some Instructors missed some of the communications and/or deadlines due to summer holidays, while others just didn't participate much, or even pay attention to the communications.

Lesson: Have a clear plan, with dates, and communicate it to the selected Instructors as part of the first communication with those Instructors.

Lesson: Some evaluations felt that there wasn't enough "help" in the classroom. Perhaps have people that submit proposals for Workshops/Labs indicate their preferred number of attendees. As soon as it is known (for Workshops, you won't know until well into the registration process), communicate to Instructors the size of the room they will be in, in case they need to recruit additional 'helpers'.

Computers

In the document template used for the Call for Workshops, the submitters had the following choices:

• Provided Windows XP workstations
• Provided Linux VMWare machine
• Instructor-provided LiveCD
• Participants bring their own Laptop
• None

The majority of submitters selected 'Windows XP', and for those that selected 'Linux VMWare', we provided VMWare, and the Instructors provided the virtual machine. It was decided that 'bring your own laptop' wasn't going to be an option, as that might exclude those who didn't have a laptop to bring, so all classrooms were configured to maximize seating for the provided computers. Submissions that selected 'None' were typically not selected, because if they didn't need computers, they didn't fit the 'hands on' requirement we had for Workshops and Labs. The 'LiveCD' option caused two problems:

• most of the computers didn't have DVD drives, so when a the use of a LiveDVD was requested, we had to plan for that class to be in a specific room
• for some reason, one of the Instructor's LiveCDs would not work on the computers in the assigned classroom, even though a prior version had been tested, and worked - that necessitated the swapping of rooms with another class

While the initial plans were modest, as registrations for the Workshops grew, we expanded the computer requirements until we ended up reaching the capacity of the rooms we had selected for the Workshops. For the 6 simultaneous Workshops on 'workshops day', we had rooms with 42,42,20,20,20 and 15 computers, with 2 attendees per computer, for a total of 159 computers (plus 6 more - one for each Workshop Instructor) for a total of 318 Workshop attendees.

Almost all the computers were rented, with one room's worth being an in-kind contribution from one of the conference sponsors. The net result was that we rented 145 computers, and the rental agency could not supply that order using a single model of computer. They supplied three different (but similar) computer models. That necessitated the installation and configuration of the software needed by all the Workshops/Labs be done three times (on three 'master' computers), as well as one more time on the in-kind computers.

Lesson: Try and reduce the work of 'loading the software'. For 2007, there was a lot of time spent on doing this. One approach would be to only use LiveCD/DVDs, but that may not be practical.

Lesson: For 2007, we had two rooms of computers that, once setup, stayed as computer classrooms throughout the week. They were used for Workshops on Monday, and for Labs Tuesday - Thursday. The rest of the computers(about 125) had to be unpacked and setup on Sunday afternoon, and disassembled and packed up on Monday afternoon. It's a lot of work for just one day's classes. Asking the attendees to 'stay 5 extra minutes' at the end of the day's classes to disconnect the mouse, keyboard, and monitor helped save time.

Lesson: Demand for workshops was so high, that we could have probably run two days of workshops, running some classes twice, and had people buy as many workshop slots as they wanted (1 to 4). Doing so would have also substantially increased our workshop profitability, since the main rental cost was in getting the machines, not keeping them for extra days.

Lesson: Workshop profitability was probably close to zero. Taking the difference between Workshops & Conference price and Conference price, and multiplying by the number of participants gives about a $50K differential. But computer rental cost about $25K, and the conference center for the day cost about $8K, and lunch cost about $9K. Throw in other grab-bag expenses (insurance, security, etc) associated with having the conference open for that day, and the profit-making of the workshop day was relatively small, or nonexistant. Either increasing the price differential, or increasing the utilization of assets like rental computers, would improve the financial side of offering workshops.

Printed Handouts

• If the conference organizing committee is going to provide a service to Instructors to print their handouts, tell the Instructors the requirements (e.g. formats, deadlines) in the acceptance notification. Many Instructors will wait until very close to, or past, whatever deadline you provide.
• Print all handouts using both sides of the page, to conserve paper, and mandate the same for handouts that Instructors provide themselves.

Presentations

213 presentation abstracts were received before the submission deadline. Of those 73 were received on the closing day, 39 the day before, and 13 the day before that. Basically a slow dribble until the last days, then a rushing torrent.

Submission Process

The submission form and database worked acceptably. Having the coordinates of submitters made for fun, with the KML outputs and so on. One item missing from the form that would have made it easier to build presentations into sessions would be a "category" that indicated whether the talk was "software / technical", or "business / case study". For example, a good case study of how Geoserver is used in MassGIS was tracked with technical talks on Geoserver.

Community Ranking Process

The ranking process was universally hailed as a good idea, and the results were considered good. However, it is hard to know how representative of the final attendance the opinions of the community were. The final program ended up fairly technical and software oriented, and in general the case study style of presentations did not receive prominent billing in the process.

The ranking numbers were used by the committee to choose the final 120 talks for presentation. Some redundant talks (topic or speaker) were demoted and some talks that appeared to be of special interest were promoted. In addition to the 120 accepted talks, 10 stand-by talks were selected.

Building Sessions

The ranking process gave us the raw information that we used to build the list of 120 talks that would be presented. However, they still needed to be grouped into sets of three for the planned 90-minute presentation sessions.

Every talk was printed (id, title, author, abstract) onto a half-sheet of paper, and given a color-code to indicate how "popular" it had been in the ranking process, and annotated with its ranking scores. Then the committee shuffled the papers around the board-room table, trying to create groups of three that were both (a) topically similar and (b) of similar popularity. The idea being to keep very popular topics together so they could be tracked into larger rooms, and less popular topics together for smaller rooms.

Each set of three was stapled together when the process was complete.

Cancellations

Speakers will cancel. In 2007, we had 7 speakers cancel before publication of the program, and 2 cancel on-site. As part of the final presentation selection process, we picked 10 talks to use as stand-by talks. When speakers were notified of acceptance/rejection, the stand-by talks were also asked if they were willing to serve as stand-bys. Being stand-by meant confirming that you were going to attend the conference, and would have your talk ready in the event of a cancellation.

Posters

The poster session was not planned in advance, but kept around as an option, and then brought forward as a salve for folks who were rejected as speakers. Most of the posters were based on abstracts that were submitted as talks but not accepted. We took every poster submission that was received before the poster deadline.

We had space for 40 posters but only received 28 submissions before the deadline.

Wiki

The use of the OSGeo Wiki helped with various aspects of the conference organization. For example, the Workshop Committee used it for various tasks related to Workshops and Labs, and it was used to self-organize the people interested in having BOF sessions at the conference, and for people looking to share accommodation.

Lesson: Make sure all wiki pages related to the conference have a Category, so that there is a handy 'index page', such as this one for the FOSS4G 2007 conference

Conference Evaluation

As it says at the top of this page, "This is not a forum for conference attendees to comment on the conference.". Conference attendees had the opportunity to fill out a conference evaluation form, and the results of the tabulation of those forms will be in the FOSSG2007 final report. What this doesn't capture is feedback from people who didn't attend the conference. To capture that information, future conference organizers may wish to ask for feedback from such people, such as asking "if you attended prior FOSS4G conferences, why did you not attend the FOSS4G2007 conference".

Workshop Evaluations

Are all here: FOSS4G2007_Workshop/Lab_Evaluations

Onsite Staff

For 2007, the meeting management company, Sea To Sky, had three of their own staff onsite. In addition to that, there were volunteers that were assigned specific tasks. For example, Sea To Sky managed the Registration Desk, but always with volunteer help. Aside from the volunteers, we had some other 'staff' planned, such as the Volunteer Coordinator, the Workshop Committee chairperson, Workshop Committee members who had offered to 'help out as needed', people that were running the Demo Theatre, etc.

Lesson: Regardless of a lack of any specific plan to do so, in practice the Volunteer Coordinator and Workshop Committee Chair spent the entire conference as 'staff', dealing both with things that they had planned to do, as well as lots of other issues. It is almost impossible for such people to participate in the conference as attendees, so that should be taken into account when planning 'how to staff the conference'.

Sea To Sky provided radios for use by their staff and key conference staff such as the Volunteer Coordinator and Workshop Committee Chair, and these proved to be an invaluable communications tool.

The setup of the over 150 computers on Sunday (to be used for 'Workshop Monday') was done both by scheduled volunteers (i.e. was one of their "shifts"), as well as by other people who were able to help (e.g. some Workshop Committee members, some volunteers who weren't "on shift", and even some conference attendees who were already in town). On 'Workshop Monday' the tear down of the majority of the computers (leaving just what was needed for the Labs for the rest of the week) was accomplished in a similar fashion, however, it went faster than expected, because at the end of the afternoon workshops, we asked the workshop attendees to stay for an extra 5 minutes and to shutdown their PCs, and to disconnect the cables.