FOSS4G 2012 Lessons Learned

The international FOSS4G 2012 conference, which was scheduled to be held in Beijing in Sept 2012, was cancelled. This has been a disappointing setback for our OSGeo community, and here I capture some of the key events which lead up to this cancellation, and with our hind site perspective, identify areas we can change to make future conferences more resilient and successful.

This conversation started on the OSGeo Conference email list, then moved to the FOSS4G 2012 Lessons Learned wiki page as ideas consolidated.

The following discussion ground rules applied:

• Avoid letting the discussion break into a witch hunt, or blame game. Remember that almost all people involved in FOSS4G 2012 were volunteers, giving of their precious time freely.
• Instead, identify an event or decision, discuss the implications of the event, and ideally follow up with some recommendations on what we can do in future.

Host City Selection

Prior to 2012, OSGeo's Conference Committee had agreed to a 3 year rotation for the location of FOSS4G conferences, which went:

• Europe (2010)
• North America (2011)
• Rest of the world (2012)
• Europe ...

The bid process involves cities providing a light, 2 page, "Letter of Intent", followed by a comprehensive bid if the "Letter of Intent" was approved. However by Letter of Intent deadline for FOSS4G 2012 there were no Letters of Intent. The deadline was extended, and Letters of intent were received from Rome (Europe), Prague (Europe), Hanoi (Asia), and a late entry from Beijing (Asia).

This was summarised by OSGeo Conference chair,

What happened is that we did not receive any submissions before the initial deadline, and then we opened the bidding to all areas, and then we received 1 submission from the desired region and 2 from Europe, and then a second late submission from the desired region.

My opinion is that the stated desired region is in fact still the desired region, and that all OSGeo conference committee members should keep this information in their head as they vote. (meaning: all 4 letters are an option for this voting stage, but the preferred region is 'anywhere other than NA or Europe')

In the end, only Prague [Europe] and Beijing [desired region] submitted a full FOSS4G bid, and when it came to a final vote, the OSGeo Conference committee was split between a bid from a more experienced team in Prague, and following OSGeo's established rotation with Beijing. In retrospect, we should have put more emphasis on selecting the experienced FOSS4G team.

European and North American international FOSS4G events have traditionally attracted more delegates and sponsors, which makes these conferences:

1. More financially profitable
2. Less financially risky
3. Reach more people (although not necessarily reaching more regions)

As we move forward, we may wish to favour selection of committees and cities with prior experience of holding local or regional FOSS4G events before being awarded an international event.

Competing regional conferences

In 2011, major regional conferences started in both Europe and North America, which competed for international FOSS4G attendance, along with some FOSS4G conferences from the region. It was debated whether OSGeo should support and encourage these new regional conferences, knowing that they would have an impact on attendance at Beijing.

As explained in a post by the Chair of the OSGeo Board:

From all that I can tell, now FOSS4G Beijing will become a local conference with support from "OSGeo international". This and no more. It will not be the Global or World conference that FOSS4G was before because we will have a FOSS4G CEE and FOSS4G North America event (plus the regular local ones) in the same year. There is no chance at all that Beijing can attract the same vibrant global participation that we had at the last global FOSS4G conferences.

The question is not whether we will have a FOSS4G in Beijing or CEE or North America. From all that I can tell we will have them all. There is no reason (and probably no way) to stop the North American or CEE initiative or both. Instead it is great to see so much interest and momentum - and we would be stupid to stifle it.

The competing regional conferences are listed here and here.

Local Organising Committee experience

Lack of Professional Conference Organiser

The Local Organising Committee (LOC) had teamed with a Professional Conference Organiser (PCO), starting from the bidding for the FOSS4G 2011 conference. However, it seems that at some point the PCO stopped helping with FOSS4G. The LOC were then unsuccessful in trying to sign up a new PCO. This was a significant setback for FOSS4G 2011, as PCO's bring significant experience in running a conference. They have experience with local venues and business, and they manage many of the day-to-day tasks which takes workload off the LOC.

From what I can gather, the PCO were not contractually engaged with the PCO up front, which allowed them to disengage later. What are the lessons? The LOC should contractually engage with the PCO very early in the conference cycle, and OSGeo oversight should ensure this happens.

Losing key LOC members

One of the key Chinese OSGeo community members, Professor Yu, passed away shortly after Beijing was awarded the conference. This was very unfortunate, both on a personal level, and organisation level.

Loss of key committee members is reasonably common (although usually people step down for various reasons, rather than pass away). For instance, a key FOSS4G-Sydney evangelist, who promoted the Sydney event at prior FOSS4G conferences, stepped back and didn't attend Sydney's FOSS4G 2009. The original FOSS4G-Devar 2011 chair had to stand down for personal reasons shortly after the bid was accepted. These examples highlight the need for organizing committees to have strength in depth, and in particular to have a backup plan if the conference chair has to step down. This was a question that was asked of the Nottingham FOSS4G 2013 contenders, who have two backups to the conference chair, as well as a committee with strength in depth overall.

Decision Making

A conference chair is asked to make many decisions related to the conference, and the majority of the time, there is no clear understanding about the benefits or downsides of each option. Usually the only sure thing is that not making a decision will be detrimental to the conference. Consequently, it is important for LOCs to become quick and efficient at analyzing possibilities and then making decisions.

From what I can gather, the Beijing LOC would have benefited from being more efficient in making decisions. For instance, in mid-November 2012 the OSGeo-Live community asked the LOC to commit to distributing OSGeo-Live DVDs at the Beijing conference. The LOC took almost 3 months to confirm they would support this. Other conferences usually provide such confirmation within a week, often within a day or two.

I suspect delays related to decisions would have contributed to schedule slipages. The lesson here is that LOC's should be structured and resourced such that they can make decisions efficiently. A prior conference chair extended this observation to note the importance of the conference chair:

[A key lesson is the] importance of an active LOC and even more importantly an active CHAIR. Committees don't move, they can't communicate, they can't move. People can, so an active CHAIR is the single critical ingredient. And the more that person in invested in both organizing and communicating the event, the better it will be.

Schedule slip

As the deadline for the FOSS4G conference approached, there was significant schedule slip on key milestones, such as the ability to accept conference papers. This was providing a visible indication of some of the other issues listed in this analysis.

I think the lesson here is quite simple. Make sure there is an appropriately resourced project manager responsible for managing the conference schedule. (This task is usually provided by a PCO).

OSGeo Oversight

A second issue is that although OSGeo had identified concerns with FOSS4G Beijing's progress reasonably early, intervention from OSGeo was late in coming. A prior FOSS4G chair noted:

We need to put harder stops in place to short circuit failure. If you don't have a call for workshops out by February, [serious questions are asked, such as should the conference be cancelled?]. If you don't have $30K in sponsorship in place by April, [serious questions]. If you don't have a call for papers out by May, [serious questions]. This [FOSS4G 2012 conference] dragged out longer than it should of because there were no hard stop points.

During the build up to FOSS4G Beijing, one of the key volunteers on OSGeo conference committee, who had previously been very active for personal reasons contributed much less (see also "Loosing key LOC members" above). This left a noticeable hole in the OSGeo conference committee which was not filled by another volunteer. The OSGeo Conference committee had previously provided checks on conferences, such as reviewing and approving the conference's budget and submitting to the OSGeo board for approval, however this didn't happen for the FOSS4G Beijing conference.

What are the lesson's here? It may be that the critical role of approving finances should be covered by a paid position, funded by profits of FOSS4G conferences. Something like this was considered as described under the No mentor section below.

Is Failure Acceptable?

There has been discussion over the level of OSGeo Oversight that should be applied to a LOC. An OSGeo board member noted:

It is still desirable to give the local organizers quite a bit of freedom and that we should accept that occasional failure is not a disaster.

Which a prior FOSS4G chair responded with:

On the contrary, I'd say that random failures are a disaster and will actually contribute to more failures. ...The success of a conference is tied to the perceived expectations. Throwing a conference is like throwing a party. Do you want to go to a lame party? No, you want to go to a rocking party. If PartyPete throws awesome parties every Thursday, you'll clear your schedule as next Thursday rolls around. If LameLou throws passable parties sometimes, and sometimes cancels them, you'll start going to PartyPete's instead. Consistency is very important.

The same thing will go double for sponsors: are you going to commit to early sponsorship and send a cheque to an event that was cancelled last year? Or will you hold on to your cheque until the last minute just in case? The uncertainty effect is going to make the financial situation of future conferences more precarious as sponsors and registrants hedge their bets until later in the calendar. This will only get worse if we embrace failure as an occasionally acceptable mode.

I'd argue that while it may be ok to experiment with some fringe elements of a FOSS4G program, occasional failure of the international FOSS4G conference should be considered unacceptable.

No mentor

A proposal was put to the OSGeo board, which was eventually approved, to have an experienced FOSS4G mentor support the Beijing Local Organising Committee. (A funded mentor was not provided to previous conferences). This proposal fell through, and although some prior FOSS4G chairs were approached (and others?), a replacement mentor was not found.

This left the Beijing FOSS4G LOC committee without some key expertise which could have been very valuable.

What is the lesson here? I think this was a good idea which fell through, and is worth pursuing again in future.

Communication

Language barrier

From what I understand, Beijing LOC were most comfortable speaking in Chinese, and had varying levels of experience with English. I observed that finding the right English words to support a conversation and convey important messages was a time consuming task, often involving decisions being made in Chinese, then translated to English. This communication overhead would have produced a significant workload on the LOC, who were already working on the difficult and time consuming task of running a FOSS4G conference.

I believe this communication gap also contributed to many of the other symptoms discussed here. Slow communication between the LOC and community would have:

• Contributed toward slow responses to community queries, hindering the international community contributing prior experience toward the LOC,
• Slowed decisions from the LOC resulting in schedule slip,
• Caused difficulties getting the quality control of the website correct,
• and reduced marketing and communication to potential international delegates.

Cultural Differences

I question whether cultural differences contributed to communication shortfalls. From my observations, it seems Chinese are more circumspect about sending public communication, often using face-to-face meetings, and waiting for review from a superior before making a public statement. This contrasts with open source communities I've observed, where many opinions are discussed publicly, both amongst senior and junior developers, until a rough consensus is reached.

The OSGeo board representative to FOSS4G 2011 noted:

For a while I attempted to play [the FOSS4G board representative] role ... Also, while I was nominally involved, in practice there was never any discussion on any mailing list I was on ...

To me, a lesson of the Beijing effort is that it is hard to be involved remotely if the LOC won't communicate by email/irc/etc. I presume all the discussions that did happen were done by private email or in person but I was left with no visibility or ability to assist.

Collective Knowledge

I believe our experience with this conference highlights how much of our collective FOSS4G knowledge is stored in volunteers' heads, and is passed between different events through our various communication channels. When we constrict information flow by introducing a language barrier, we have also constricted access to our knowledge on how to run a conference.

A few suggestions on ways to address this include:

1. Collect our conference running knowledge in a central source, that can be handed on without the high level of communication currently being used. In particular, I'm suggesting starting to collect our processes in a FOSS4G Cookbook or similar.
2. Set up a permanent FOSS4G coordinator role (one person, or an international PCO, or similar) who are responsible for coordinating conferences and personally remembering lessons learned between conferences. (Note the risk of this person resigning and loosing all collected knowledge)
3. Additionally, ensure key members in the LOC can communicate fluently with the rest of the OSGeo community. In most cases at the moment, this would mean speaking fluently in English.

Response to emails

There were a number of comments that I was privately CCed on which indicated that the international community were not receiving responses after emailing the LOC. Here are some examples:

As I've told you before it has been frustrating to me to not receive any feedback from the LOC on my offer to sponsor the event. I basically had the plan to come with my whole team (5 people now), but can't afford such investment considering the state the conference and participation levels are at now. In fact we have moved focus to the Nottingham event just after Beijing because it appears to be (1) better organized (but that may just appear like it due to the lack of communication from Beijing, (2) an audience that is of interest to [company name] and (3) cheaper / closer to home.

Another from the academic lead, who later stepped down:

... [regarding email responses] from two "important players" I have had no feedback, namely from the local organizers and from OSGeo.

I think the lesson here is that the LOC and PCO should be suitably motivated and resourced, and be provided with enough delegation to respond to all community queries promptly. Every query should be responded to within one working day, even if the response is "we will have an answer to you after the LOC meets next week".

Website out of date

A conference's website is the primary form of communication with potential delegates. For FOSS4G 2012, the website took an excessively long time to be developed and brought online, and then when it was brought online, it contained incorrect information and broken links (mainly cut and paste from the prior FOSS4G website). People were having significant issues with submitting papers and registering to attend.

The FOSS4G LOC had hired an external web developer to create the website, who had done a poor job of development. It seemed that there was a lack of quality control from both the web developer, and LOC. In the past, development of the website has either been managed by technically experienced developers (as was the case in 2009), or by the PCO.

The lesson here is that the website needs to be made a priority and suitably resourced. There is the potential for website management software to be passed on from one conference to the next. (We considered this option in 2009 but found the Open Source conference management software used by FOSS4G 2008 was not going to integrate easily with the software our PCO was using). It would be worth future FOSS4G conferences revisiting this question.

Minimal "buzz"

To a certain extent, a conference is successful because the LOC says it is going to be successful (and potential attendees and sponsors believe the statement). Presenters and sponsors attend the conference because they believe there will be lots of delegates, and delegates attend because they believe there will be lots of quality presenters and sponsors. And one of the most effective ways for everyone to be convinced of the conference's success is to create lots of "buzz". I.e., lots of press releases, articles, blogs, twitter discussion and more talking about how good the conference is going to be.

FOSS4G 2009 possibly went a little too far by putting out 41 press releases. However, FOSS4G Beijing could certainly have benefited from more "Buzz", as the OSGeo Board Chair noted:

on the website at http://2012.foss4g.org/ there is still no option for submitting abstracts although the submission has been opened - apparently without notice to any of the regular OSGeo channels. Workshops submission ends in two weeks.

No international speakers have been announced and there are only Chinese sponsors listed (although interest by regulars was documented as early as December 2012).

Engaging international organisers

Compared to prior international FOSS4G events, there was minimal international involvement in organising the FOSS4G event. Of particular concern was that the international academic track lead resigned, saying:

... I regret [the LOC] did not fully support the setup I proposed. Specifically, the LOC insists on using their own deadlines and reviewing and publication plan. Of course they have every right to do so, because it is in fact their conference...

There is a significant amount of work involved in organising a conference, and it is very valuable to share tasks with the international community. This has two key benefits:

• It allows the LOC to focus on the local issues (like sorting out the venue)
• It facilitates knowledge transfer between years, as roles like the Academic track lead are often coordinated by the same core people over a number of years.

So lesson here is look for opportunities to make use of the international community to coordinate specific areas of the conference.

Weekly meetings

Less than 3 months before FOSS4G 2012 was due, weekly meetings were started between volunteers from the international community and the LOC. I understand that the LOC were having meetings internally, but there was little visibility of them from the international community. The extra meetings facilitated transparency from the international community into the progress of the LOC, which in turn provided opportunities for the international community to volunteer to help. Eventually, with the help of these weekly meetings it was assessed that the level of effort required to bring the conference back on track, along with the likely outcome, resulted in a decision to cancel the conference.

In retrospect, these meetings should have started much earlier, ideally from the start of the conference planning a year or so earlier such that support from the international community could have made a better impact in the earlier stages. So lesson hear is start having periodic meetings from early in the planning cycle, and invite the international community to participate if you can.

Who wrote the original version of this Lesson's Learned?

The telling of history is always influenced by the perspective of the narrator, and as such it is useful to know who the narrator was. This narration was by myself, Cameron Shorter. I was the chair of FOSS4G 2009, I'm a member of the OSGeo Conference Committee, and I voted for FOSS4G 2012 to be held in Beijing. I helped volunteers add Chinese translations to the OSGeo-Live DVD, and commit to handing the DVD out at FOSS4G Beijing. I later attended weekly meetings with FOSS4G Beijing's LOC and other international volunteers, until it was decided by the LOC that the conference should be cancelled.