WPS benchmark 2014/Test - Revision history

Djay: /* WPS Validity testing description */

2014-08-21T08:03:26Z

WPS Validity testing description

Djay at 14:40, 10 July 2014

2014-07-10T14:40:20Z

Djay at 14:39, 10 July 2014

2014-07-10T14:39:18Z

Djay at 08:22, 3 June 2014

2014-06-03T08:22:44Z

Djay: Created page with "= WPS Validity testing description = The general description of each tests mention for each test the time response will also be evaluated in parallel by using the Apache Benchm..."

2014-05-30T17:10:34Z

Created page with "= WPS Validity testing description = The general description of each tests mention for each test the time response will also be evaluated in parallel by using the Apache Benchm..."

New page

= WPS Validity testing description =

The general description of each tests mention for each test the time response will also be evaluated in parallel by using the Apache Benchmark tool for the specified requests. During the tests run, ressources consuptions will be also evaluated, but the way it will be evaluated is not well defined by now (any idea?).

The following default services are supposed to be available on the Server Instances for testing purpose (they are basically list of functions from table 2 and 3 from GeoXACML) :

* Equals
* Disjoint
* Touches
* Crosses
* Within
* Contains
* Overlaps
* Intersects
* Buffer
* Boundary
* Union
* Intersection
* Difference
* SymDifference
* Centroid
* ConvexHull

The script you can use to run tests over the list of required services is hosted [http://zoo-project.org/trac/browser/trunk/testing/run.sh here].

== GetCapabilities ==

=== Basic KVP testing ===

Send simple valid GetCapabilities requests to the server using GET method, verify the validity of returned XML responses against OGC schemas.

Send wrong GetCapabilities requests and check that returned answer are ExceptionRepport validating against the corresponding OGC schema and the server return code in the headers with not 200.
Sample of wrong GetCapabilities requests :
* ?request=GetCapabilities so without specifying the service required parameter, make sure the exceptionCode is MissingParameterValue and locator is corresponding to the missing required parameter
* ?request=GetCapabilities&service=WXS so by specifying wrong service value, make sure the exceptionCode is InvalidParameterValue and locator is corresponding to the wrongly valued required parameter
For each valid and wrong GetCapabilities requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 10000 -c 100).
=== XML/POST testing ===
Send simple valid GetCapabilities XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas.
For each valid XML GetCapabilities requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 10000 -c 100).

== DescribeProcess ==

=== Basic KVP testing ===

Send simple valid DescribeProcess requests to the server using GET method, verify the validity of returned XML responses against OGC schemas.

Send wrong DescribeProcess requests and check that returned answer are ExceptionRepport validating against the corresponding OGC schema and the server return code in the headers with not 200.

Sample of wrong DescribeProcess requests :
* ?request= DescribeProcess so without specifying the service required parameter, make sure the exceptionCode is MissingParameterValue and locator is corresponding to the missing required parameter
* ?request=DescribeProcess&service=WXS&version=1.0.0&Identifier=XXX so by specifying wrong service value (value which may vary are : version, Identifier), make sure the exceptionCode is InvalidParameterValue and locator is corresponding to the wrongly valued required parameter

For each valid and wrong DescribeProcess requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 10000 -c 100).

=== XML/POST testing ===

Send simple valid DescribeProcess XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas.

For each valid XML DescribeProcess requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 10000 -c 100).

== Execute ==

For XML Execute requests, we will use requests templates which are publically available from here: http://zoo-project.org/trac/browser/trunk/testing/requests . Note that is the future this link may be updated to the current github account when updated.

=== Synchronous requests ===

==== Basic KVP testing ====

Send simple valid Execute requests to the server using GET method, verify the validity of returned XML responses against OGC schemas.

Send wrong Execute requests and check that returned answer are ExceptionRepport validating against the corresponding OGC schema and the server return code in the headers with not 200.

Sample of wrong Execute requests :
* ?request= Execute&service=WPS&version=1.0.0&Identifier=XXX so without specifying a required parameters (for each required parameters, so request, version, service, Identifier, DataInputs), make sure the exceptionCode is MissingParameterValue and locator is corresponding to the missing required parameter
* ?request=Execute&service=WXS&version=1.0.0&Identifier=XXX&DataInputs=a=1 so by specifying wrong value for a required parameter (for each required parameters, value will vary and will be defined), make sure the exceptionCode is
InvalidParameterValue and locator is corresponding to the wrongly valued required parameter
For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 10000 -c 100).

==== XML/POST synchronous testing ====

Send simple valid Execute XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas.

For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 10000 -c 100).

=== Asynchronous requests ===

==== Basic KVP testing ====
Send simple valid Execute requests to the server using GET method, verify the validity of returned XML responses against OGC schemas. Poll statusLocation until obtaining a final wps:ExecuteResponse containing the computed result, check that for every poll requests we obtain a valid XML wps:ExecuteResponse.
Send wrong Execute requests and check that returned answer are ExceptionRepport validating against the corresponding OGC schema and the server return code in the headers with not 200.
Sample of wrong Execute requests :
* ?request= Execute&service=WPS&version=1.0.0&Identifier=XXX&storeExecuteResponse=true&status=true so without specifying a required parameters (for each required parameters, so request, version, service, Identifier, DataInputs), make sure the exceptionCode is MissingParameterValue and locator is corresponding to the missing required parameter
* ?request=Execute&service=WXS&version=1.0.0&Identifier=XXX&DataInputs=a=1&storeExecuteResponse=true&status=true so by specifying wrong value for a required parameter (for each required parameters, value will vary and will be defined), make sure the exceptionCode is InvalidParameterValue and locator is corresponding to the wrongly valued required parameter
For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 10000 -c 100).

==== XML/POST synchronous testing ====
Send simple valid Execute XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas.
For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 10000 -c 100).

← Older revision		Revision as of 08:22, 3 June 2014
Line 1:		Line 1:
	= WPS Validity testing description =		= WPS Validity testing description =
		+
		+	[http://wiki.osgeo.org/wiki/WPS_benchmark_2014 Reference page]

	The general description of each tests mention for each test the time response will also be evaluated in parallel by using the Apache Benchmark tool for the specified requests. During the tests run, ressources consuptions will be also evaluated, but the way it will be evaluated is not well defined by now (any idea?).		The general description of each tests mention for each test the time response will also be evaluated in parallel by using the Apache Benchmark tool for the specified requests. During the tests run, ressources consuptions will be also evaluated, but the way it will be evaluated is not well defined by now (any idea?).

@@ Line 24: / Line 24: @@
 * ConvexHull
-The script you can use to run tests over the list of required services is hosted [http://zoo-project.org/trac/browser/trunk/testing/run.sh here].
+The script you can use to run tests over the list of required services is hosted [https://github.com/WPS-Benchmarking/cptesting/blob/master/scripts/run.sh here].
 == GetCapabilities ==

@@ Line 36: / Line 36: @@
 * ?request=GetCapabilities so without specifying the service required parameter, make sure the exceptionCode is MissingParameterValue and locator is corresponding to the missing required parameter
 * ?request=GetCapabilities&service=WXS so by specifying wrong service value, make sure the exceptionCode is InvalidParameterValue and locator is corresponding to the wrongly valued required parameter
-For each valid and wrong GetCapabilities requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 10000 -c 100).
+For each valid and wrong GetCapabilities requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100).
 === XML/POST testing ===
 Send simple valid GetCapabilities XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas.
-For each valid XML GetCapabilities requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 10000 -c 100).
+For each valid XML GetCapabilities requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100).
 == DescribeProcess ==
@@ Line 53: / Line 53: @@
 * ?request=DescribeProcess&service=WXS&version=1.0.0&Identifier=XXX so by specifying wrong service value (value which may vary are : version, Identifier), make sure the exceptionCode is InvalidParameterValue and locator is corresponding to the wrongly valued required parameter
-For each valid and wrong DescribeProcess requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 10000 -c 100).
+For each valid and wrong DescribeProcess requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100).
 === XML/POST testing ===
@@ Line 59: / Line 59: @@
 Send simple valid DescribeProcess XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas.
-For each valid XML DescribeProcess requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 10000 -c 100).
+For each valid XML DescribeProcess requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100).
 == Execute ==
@@ Line 77: / Line 77: @@
 * ?request=Execute&service=WXS&version=1.0.0&Identifier=XXX&DataInputs=a=1 so by specifying wrong value for a required parameter (for each required parameters, value will vary and will be defined), make sure the exceptionCode is
 InvalidParameterValue and locator is corresponding to the wrongly valued required parameter
-For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100).
+For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 500 -c 10).
 ==== XML/POST synchronous testing ====
@@ Line 83: / Line 83: @@
 Send simple valid Execute XML requests to the server using POST method, verify the validity of returned XML responses against OGC schemas.
-For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 5000 -c 100).
+For each valid and wrong Execute requests, Apache Benchmark will be run by using 10000 requests with 100 concurent (-n 500 -c 10).
 === Asynchronous requests ===