eValid™ -- The Web Quality Suite Browser-Based Client-Side Functional Testing and Validation Page Timing/Tuning Transaction Monitoring. WebSite Spidering & Analysis and Realistic Server Loading. © Copyright 2000-2011 by Software Research, Inc.

Summary
The eValid test engine can test web services easily and efficiently. It does this by creating a web services testbed from the WSDL file for the service. Tests are fed to the testbed using eValid test data generation capabilities. Results are analyzed using special value and range validation commands on the resulting outputs. The experiment raises some interesting questions.

Demonstration Description
The eValid Web Services Technology Demonstration described here shows how eValid is used to test and validate a web service. The web service used is one that returns basic temperature data based on the supplied USA zip code. The data is provided to the web service through an HTML interface using eValid's eVGenerate engine to enter multiple values (zip codes from a selected range) to test the web service response (i.e. does it return a meaningful temperature).

Benefits
For a web service to be effective it must be predictable, robust, reliable and accurate. Only if all of these quality attributes are true will any web service find widespread adoption. The eValid technology demonstrated here is an illustration of how systematic, automated testing of a web service can identify problems, confirm operation, provide validation, and measure performance systematically and automatically.

Implementation Details
The steps in producing a web services validation experiment are outlined here. Samples of how the screens appear are given whenever possible.

1. Identify Web Service To Be Tested.
   The WSDL file fully describes a web service. The web service being tested in this technology demo is found at: http://www.xmethods.net/sd/2001/TemperatureService.wsdl. This particular web service returns the temperature value as a function of the USA ZipCode.

2. Analyze The WSDL File For Testable Dependencies.
   This step should identify any dependent web services that the web service under test relies upon. In most cases there will be none but if there are dependent web services you will need to arrange for a test stub that simulates that web service if you wish to test the candidate web service independently.

3. Construct the HTML Web Services Driver Interface.
   This process creates an HTML interface that reveals, for the target web service name, the set of arguments and their types, and the set of responses and their types.
   

4. Make The Web Services Driver Interface Available to eValid.
   Install that interface as an HTML link as the eValid current URL, but this may have to be a locally constructed HTML file to have the best effect. This is where eValid feeds data to the web service.
   

5. Make the Web Services Result Interface Available to eValid.
   Here is the result of using the HTML input interface to invoke the web service. Note that in this case the result is expressed as a value appearing also on an HTML page.
   

6. Record the Master Test Script.
   Using the HTML input page, record a master test script that drives the candidate web service. After this script is perfected it can be parameterized for use by eValid's eVGenerate or Data Synthesis for value insertions in production runs. Here is how the eValid test script appears for this example after it has been annotated and edited (parameterized).

   # Recording by eValid V8. # (c) Copyright 2000-2011 by Software Research, Inc.  # # Software Research, Inc. # 1663 Mission St. # San Francisco, CA 94103 # # Recording made on: Microsoft Windows XP # Script Created by: Richard Gella # # This demonstrates the use of eValid to dynamically test a web service # through an HTML interface. ProjectID "Project" GroupID "Web Service Sample" TestID "ZIP" LogID "AUTO" ScreenSize 1280 1024 FontSize 2 # Navigate to the Web Service HTML Interface... InitLink "http://www.soapclient.com/soaptest.html" # Enter the URL of the WSDL file to be tested... InputValue 0 106 "TEXT" "SoapWSDL" "http://www.xmethods.net/sd/TemperatureService.wsdl " "" "" # Access the WSDL File... SubmitClick 0 124 "Retrieve" "" NAV # Parameterize the data by using the eVGen command from the main calling script # to randomly or sequentially generate values as input for the Web Service... InputValue 0 115 "TEXT" "zipcode" "{% one}{% two}{% three}{% four}{% five}" "" "" # Select HTML output from the Web Service... InputValue 0 124 "SELECT-ONE" "template" "HTML" "0" "" # Retrieve results from the Web Service... SubmitClick 0 127 "Invoke" "" NAV # Validate Temperature Report... ValidateSelectedValueRange 0 96 0 "0.00" "100.0" "" # End of Script

7. Create eVGenerate And/Or Data Synthesis Variables and Values.
   This step creates the values to be fed into the web service under test and establishes the criteria for measurement. This and the prior step may have to be repeated until the combinations are correct and the measurements that the combinations indicate are sufficient and appropriate. Here are the values for eVGenerate that are used to specify a five-digit USA zipcode.

   eVGenerate Values.data file contents: one 1 2 3 4 5 6 7 8 9 0 two 1 2 3 4 5 6 7 8 9 0 three 1 2 3 4 5 6 7 8 9 0 four 1 2 3 4 5 6 7 8 9 0 five 1 2 3 4 5 6 7 8 9 0

8. Choose Validation & Measurement values.
   Within the script set up the necessary alarms and timer reads, as well as validation steps, that are used to confirm the web service. For the best example there should be one performance metric and one content metric. The sample script above validates the temperature returned by the web service based on the validation range specified in the script file. For this demonstration, the range specified is from 0 to 100. An invalid zip code requested by the web service returns a temperature of "-999.0",  requiring result validation.

   The script phrase below validates the temperature that the web service recovers.

   ... # Validate Temperature Report ValidateSelectedValueRange 0 96 0 "0.00" "100.0" "" ...

9. Make the Web Services Test Run.
   Run the tests just designed and study the log files. Report on the variations found and identify any anomalous results.

10. Analyze the Results.
    Here is a summary of the results of the runs we made using zip codes sequentially generated by using the eV.Generate engine. For simplicity we limited the run to a total of 100 web service retrievals.
    • Zip Codes Checked: A total of 100 zipcodes as follows: 00011 + nn121, with nn = 00, 01, 02, ..., 99.
    • Number of WebService Test Runs Made: 100.
    • Number of Illegal Zip Codes: 38%.
    • Number of Legal Zip Codes with In-Range Temperature: 54%.
    • Number of Legal Zip Codes With System Errors: 8%.

11. Questions Raised.
    Admittedly this is a very cursory look at the zipcode-to-current temperature web service, but these questions arise:
    • Number of Illegal Zip Codes: We found it unusual that so many zipcodes had no data. Is it really true that 38% [of our small sample] of zipcodes "don't exist?"
    • Number of Legal Zip Codes With System Errors: The fact that 8% of the legal zipcodes returned a system error code does not imply this web service is particularly reliable. One would expect better.