WPS benchmark 2014/Test

WPS Validity testing description

Reference page

The general description of each tests mention for each test the time response will also be evaluated in parallel by using the Apache Benchmark tool for the specified requests. During the tests run, ressources consuptions will be also evaluated, but the way it will be evaluated is not well defined by now (any idea?).

The following default services are supposed to be available on the Server Instances for testing purpose (they are basically list of functions from table 2 and 3 from GeoXACML) :

• Equals
• Disjoint
• Touches
• Crosses
• Within
• Contains
• Overlaps
• Intersects
• Buffer
• Boundary
• Union
• Intersection
• Difference
• SymDifference
• Centroid
• ConvexHull

The script you can use to run tests over the list of required services is hosted here.

GetCapabilities

Basic KVP testing

Send simple valid GetCapabilities requests to the server using GET method, verify the validity of returned XML responses against OGC schemas.

Send wrong GetCapabilities requests and check that returned answer are ExceptionRepport validating against the corresponding OGC schema and the server return code in the headers with not 200. Sample of wrong GetCapabilities requests :

• ?request=GetCapabilities so without specifying the service required parameter, make sure the exceptionCode is MissingParameterValue and locator is corresponding to the missing required parameter
• ?request=GetCapabilities&service=WXS so by specifying wrong service value, make sure the exceptionCode is InvalidParameterValue and locator is corresponding to the wrongly valued required parameter

For each valid and wrong GetCapabilities requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100).

XML/POST testing

Send simple valid GetCapabilities XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas. For each valid XML GetCapabilities requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100).

DescribeProcess

Basic KVP testing

Send simple valid DescribeProcess requests to the server using GET method, verify the validity of returned XML responses against OGC schemas.

Send wrong DescribeProcess requests and check that returned answer are ExceptionRepport validating against the corresponding OGC schema and the server return code in the headers with not 200.

Sample of wrong DescribeProcess requests :

• ?request= DescribeProcess so without specifying the service required parameter, make sure the exceptionCode is MissingParameterValue and locator is corresponding to the missing required parameter
• ?request=DescribeProcess&service=WXS&version=1.0.0&Identifier=XXX so by specifying wrong service value (value which may vary are : version, Identifier), make sure the exceptionCode is InvalidParameterValue and locator is corresponding to the wrongly valued required parameter

For each valid and wrong DescribeProcess requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100).

XML/POST testing

Send simple valid DescribeProcess XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas.

For each valid XML DescribeProcess requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100).

Execute

For XML Execute requests, we will use requests templates which are publically available from here: http://zoo-project.org/trac/browser/trunk/testing/requests . Note that is the future this link may be updated to the current github account when updated.

Synchronous requests

Basic KVP testing

Send simple valid Execute requests to the server using GET method, verify the validity of returned XML responses against OGC schemas.

Send wrong Execute requests and check that returned answer are ExceptionRepport validating against the corresponding OGC schema and the server return code in the headers with not 200.

Sample of wrong Execute requests :

• ?request= Execute&service=WPS&version=1.0.0&Identifier=XXX so without specifying a required parameters (for each required parameters, so request, version, service, Identifier, DataInputs), make sure the exceptionCode is MissingParameterValue and locator is corresponding to the missing required parameter
• ?request=Execute&service=WXS&version=1.0.0&Identifier=XXX&DataInputs=a=1 so by specifying wrong value for a required parameter (for each required parameters, value will vary and will be defined), make sure the exceptionCode is

InvalidParameterValue and locator is corresponding to the wrongly valued required parameter For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 500 -c 10).

XML/POST synchronous testing

Send simple valid Execute XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas.

For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 500 -c 10).

Asynchronous requests

Basic KVP testing

Send simple valid Execute requests to the server using GET method, verify the validity of returned XML responses against OGC schemas. Poll statusLocation until obtaining a final wps:ExecuteResponse containing the computed result, check that for every poll requests we obtain a valid XML wps:ExecuteResponse. Send wrong Execute requests and check that returned answer are ExceptionRepport validating against the corresponding OGC schema and the server return code in the headers with not 200. Sample of wrong Execute requests :

• ?request= Execute&service=WPS&version=1.0.0&Identifier=XXX&storeExecuteResponse=true&status=true so without specifying a required parameters (for each required parameters, so request, version, service, Identifier, DataInputs), make sure the exceptionCode is MissingParameterValue and locator is corresponding to the missing required parameter
• ?request=Execute&service=WXS&version=1.0.0&Identifier=XXX&DataInputs=a=1&storeExecuteResponse=true&status=true so by specifying wrong value for a required parameter (for each required parameters, value will vary and will be defined), make sure the exceptionCode is InvalidParameterValue and locator is corresponding to the wrongly valued required parameter

For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 500 -c 10).

XML/POST synchronous testing

Send simple valid Execute XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas. For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 500 -c 10).