Difference between revisions of "WPS benchmark 2014/Test"
m |
|||
| Line 36: | Line 36: | ||
* ?request=GetCapabilities so without specifying the service required parameter, make sure the exceptionCode is MissingParameterValue and locator is corresponding to the missing required parameter | * ?request=GetCapabilities so without specifying the service required parameter, make sure the exceptionCode is MissingParameterValue and locator is corresponding to the missing required parameter | ||
* ?request=GetCapabilities&service=WXS so by specifying wrong service value, make sure the exceptionCode is InvalidParameterValue and locator is corresponding to the wrongly valued required parameter | * ?request=GetCapabilities&service=WXS so by specifying wrong service value, make sure the exceptionCode is InvalidParameterValue and locator is corresponding to the wrongly valued required parameter | ||
| − | For each valid and wrong GetCapabilities requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n | + | For each valid and wrong GetCapabilities requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100). |
=== XML/POST testing === | === XML/POST testing === | ||
Send simple valid GetCapabilities XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas. | Send simple valid GetCapabilities XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas. | ||
| − | For each valid XML GetCapabilities requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n | + | For each valid XML GetCapabilities requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100). |
== DescribeProcess == | == DescribeProcess == | ||
| Line 53: | Line 53: | ||
* ?request=DescribeProcess&service=WXS&version=1.0.0&Identifier=XXX so by specifying wrong service value (value which may vary are : version, Identifier), make sure the exceptionCode is InvalidParameterValue and locator is corresponding to the wrongly valued required parameter | * ?request=DescribeProcess&service=WXS&version=1.0.0&Identifier=XXX so by specifying wrong service value (value which may vary are : version, Identifier), make sure the exceptionCode is InvalidParameterValue and locator is corresponding to the wrongly valued required parameter | ||
| − | For each valid and wrong DescribeProcess requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n | + | For each valid and wrong DescribeProcess requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100). |
=== XML/POST testing === | === XML/POST testing === | ||
| Line 59: | Line 59: | ||
Send simple valid DescribeProcess XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas. | Send simple valid DescribeProcess XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas. | ||
| − | For each valid XML DescribeProcess requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n | + | For each valid XML DescribeProcess requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100). |
== Execute == | == Execute == | ||
| Line 77: | Line 77: | ||
* ?request=Execute&service=WXS&version=1.0.0&Identifier=XXX&DataInputs=a=1 so by specifying wrong value for a required parameter (for each required parameters, value will vary and will be defined), make sure the exceptionCode is | * ?request=Execute&service=WXS&version=1.0.0&Identifier=XXX&DataInputs=a=1 so by specifying wrong value for a required parameter (for each required parameters, value will vary and will be defined), make sure the exceptionCode is | ||
InvalidParameterValue and locator is corresponding to the wrongly valued required parameter | InvalidParameterValue and locator is corresponding to the wrongly valued required parameter | ||
| − | For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n | + | For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 500 -c 10). |
==== XML/POST synchronous testing ==== | ==== XML/POST synchronous testing ==== | ||
| Line 83: | Line 83: | ||
Send simple valid Execute XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas. | Send simple valid Execute XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas. | ||
| − | For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n | + | For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 500 -c 10). |
=== Asynchronous requests === | === Asynchronous requests === | ||
Revision as of 07:40, 10 July 2014
WPS Validity testing description
The general description of each tests mention for each test the time response will also be evaluated in parallel by using the Apache Benchmark tool for the specified requests. During the tests run, ressources consuptions will be also evaluated, but the way it will be evaluated is not well defined by now (any idea?).
The following default services are supposed to be available on the Server Instances for testing purpose (they are basically list of functions from table 2 and 3 from GeoXACML) :
- Equals
- Disjoint
- Touches
- Crosses
- Within
- Contains
- Overlaps
- Intersects
- Buffer
- Boundary
- Union
- Intersection
- Difference
- SymDifference
- Centroid
- ConvexHull
The script you can use to run tests over the list of required services is hosted here.
GetCapabilities
Basic KVP testing
Send simple valid GetCapabilities requests to the server using GET method, verify the validity of returned XML responses against OGC schemas.
Send wrong GetCapabilities requests and check that returned answer are ExceptionRepport validating against the corresponding OGC schema and the server return code in the headers with not 200. Sample of wrong GetCapabilities requests :
- ?request=GetCapabilities so without specifying the service required parameter, make sure the exceptionCode is MissingParameterValue and locator is corresponding to the missing required parameter
- ?request=GetCapabilities&service=WXS so by specifying wrong service value, make sure the exceptionCode is InvalidParameterValue and locator is corresponding to the wrongly valued required parameter
For each valid and wrong GetCapabilities requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100).
XML/POST testing
Send simple valid GetCapabilities XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas. For each valid XML GetCapabilities requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100).
DescribeProcess
Basic KVP testing
Send simple valid DescribeProcess requests to the server using GET method, verify the validity of returned XML responses against OGC schemas.
Send wrong DescribeProcess requests and check that returned answer are ExceptionRepport validating against the corresponding OGC schema and the server return code in the headers with not 200.
Sample of wrong DescribeProcess requests :
- ?request= DescribeProcess so without specifying the service required parameter, make sure the exceptionCode is MissingParameterValue and locator is corresponding to the missing required parameter
- ?request=DescribeProcess&service=WXS&version=1.0.0&Identifier=XXX so by specifying wrong service value (value which may vary are : version, Identifier), make sure the exceptionCode is InvalidParameterValue and locator is corresponding to the wrongly valued required parameter
For each valid and wrong DescribeProcess requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100).
XML/POST testing
Send simple valid DescribeProcess XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas.
For each valid XML DescribeProcess requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100).
Execute
For XML Execute requests, we will use requests templates which are publically available from here: http://zoo-project.org/trac/browser/trunk/testing/requests . Note that is the future this link may be updated to the current github account when updated.
Synchronous requests
Basic KVP testing
Send simple valid Execute requests to the server using GET method, verify the validity of returned XML responses against OGC schemas.
Send wrong Execute requests and check that returned answer are ExceptionRepport validating against the corresponding OGC schema and the server return code in the headers with not 200.
Sample of wrong Execute requests :
- ?request= Execute&service=WPS&version=1.0.0&Identifier=XXX so without specifying a required parameters (for each required parameters, so request, version, service, Identifier, DataInputs), make sure the exceptionCode is MissingParameterValue and locator is corresponding to the missing required parameter
- ?request=Execute&service=WXS&version=1.0.0&Identifier=XXX&DataInputs=a=1 so by specifying wrong value for a required parameter (for each required parameters, value will vary and will be defined), make sure the exceptionCode is
InvalidParameterValue and locator is corresponding to the wrongly valued required parameter For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 500 -c 10).
XML/POST synchronous testing
Send simple valid Execute XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas.
For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 500 -c 10).
Asynchronous requests
Basic KVP testing
Send simple valid Execute requests to the server using GET method, verify the validity of returned XML responses against OGC schemas. Poll statusLocation until obtaining a final wps:ExecuteResponse containing the computed result, check that for every poll requests we obtain a valid XML wps:ExecuteResponse. Send wrong Execute requests and check that returned answer are ExceptionRepport validating against the corresponding OGC schema and the server return code in the headers with not 200. Sample of wrong Execute requests :
- ?request= Execute&service=WPS&version=1.0.0&Identifier=XXX&storeExecuteResponse=true&status=true so without specifying a required parameters (for each required parameters, so request, version, service, Identifier, DataInputs), make sure the exceptionCode is MissingParameterValue and locator is corresponding to the missing required parameter
- ?request=Execute&service=WXS&version=1.0.0&Identifier=XXX&DataInputs=a=1&storeExecuteResponse=true&status=true so by specifying wrong value for a required parameter (for each required parameters, value will vary and will be defined), make sure the exceptionCode is InvalidParameterValue and locator is corresponding to the wrongly valued required parameter
For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 500 -c 10).
XML/POST synchronous testing
Send simple valid Execute XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas. For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 500 -c 10).