Benchmarking 2009

Participants

• Geoserver
  • Andrea Aime
• MapServer
  • Danielle Morissette
  • Jeff McKenna

Basic Premise

Each WMS will be set up on the server box to serve exactly the same set of LAYERS. A JMeter load will be run on the testing box to exercise various aspect of those layers.

Layers

• texas_roads_unstyled This test exercises raw data access speed. The lines are black, the background is white, there are a hundred or so lines in each output image. The output is a two-color PNG8 image.
• texas_roads_styled This test exercises the renderer a bit more. The lines will be "pipe-styled" and different road types will have different styles. TODO(Define the road classes and expected styles.)
• texas_roads_styled_labeled This test exercises the render and the label algorithms. Combine the above style with road labels. Labels will follow curves. TODO(Define the expected labeling look. We know that exact matches between engines will be impossible, this will be a "best efforts" test)
• point_layer_styled This test exercises the ability of the renderer to build a map in which points are symbolized with a certain number of externally provided icons (png/svg) depending on some point attribute.
• single big ECW file (actual data TBD). This test exercises the ability to read a single wavelet compressed file and return it in JPEG format.
• file system mosaic of TIFF tiles. Generated out of the ECW file by splitting it into a sizeable amount of tiles, this test checks the ability of the server to efficiently retrieve data from a large collection of images and deal the associated file handling issues (ulimit and the like). It can also serve as a comparison with ECW results.