Benchmarking 2010

Basic Premise

Following up on last year's exercise, the performance shoot-out presentation at FOSS4G2010 will test how long each Web mapping server takes to generate a map image, from a common set of spatial data, on a common platform. The data will be served by each Web mapping server through the WMS standard, which will serve exactly the same set of LAYERS. A JMeter load will be run on the testing box to measure various aspects of those layers.

Timeline

January 1st, 2010	begin contacting all mapping servers
June 1st, 2010	commitment due by all interested mapping servers
June 2nd, 2010	exercise begins (and weekly meetings)
August 1st, 2010	final testing begins
August 23rd, 2010	no further testing
September 6-9, 2010	present results at FOSS4G2010

Communication

Coordination/communication is primarily via the Benchmarking mailing list: http://lists.osgeo.org/mailman/listinfo/benchmarking

Weekly meetings will occur through IRC chat in the #foss4g channel on irc.freenode.net

Next IRC meeting

• Wed June 2nd @ 15:00:00 UTC
  • Agenda:
    • team representative introductions
    • Establish this year's 'Rules of Engagement' (previous RoE)
    • Discuss dataset to be used (including how to share the large dataset, for running local tests)
    • Discuss server infrastructure
    • Discuss possible changes to the test script parameters
    • Set next meeting date

Participants

Mapping Server	Development Team Leader
Cadcorp GeognoSIS	Martin Daly	Martin Daly
Constellation-SDI	Adrian Custer	Cédric Briançon
Erdas Apollo	Anne-Sophie Collignon	Dimitri Monie
GeoServer	Andrea Aime	Andrea Aime
MapGuide Open Source	Daniel Morissette	Zac Spitzer
Mapnik	Dane Springmeyer	Dane Springmeyer
MapServer	Jeff McKenna	Jeff McKenna
Oracle MapViewer	Michael Smith	Michael Smith
QGIS mapserver	Marco Hugentobler	Marco Hugentobler

External Related Links

• FOSS4G WMS Benchmark
• WMSTester - tool for testing not from OSGeo

---