|
|
This table identifies alternative technology approaches to achieve robust testing of WebSites. It describes the pros and cons of the four primary technical approaches.
See also Comparative Capability Summary for an indication of how these factors affect testing of particular features of WebSites. See also this summary of eValid Advantages.
| WebSite Feature | eValid InBrowser Technology | Windows Desktop Based | HTTP Protocol Emulation | Browser Proxy Based |
| Short Technical Description | InBrowser technology solution: functionality entirely inside the browser. | Monitor Windows event loop, drill-down using MFCs, record and play. | Wrapper around HTTP implementation to grap outbound URL data, save selected inbound files. | Program that takes over the HTTP protocol and processes URL's outbound and data inbound. |
| Availability and Status | eValid is a unique patented commercial product | Many commercial products | Some commercial products, some freeware | Some commercial products, some freeware |
| Capability for WebSite, Sub-WebSite Dynamic Analysis | Superior. Recursive-descent search with limits is built in, as are certain fixed filters. Additional user-supplied filters are possible. | Access to information about what to do next is not directly available. | Guiding the search requires detailed parsing of each page as it is downloaded. | Guiding the search requires detailed parsing of each page as it is downloaded. |
| Capability for Functional Testing and Validation | Superior. The approach is powerful flexible and general. | Usually very good. Depends on the methods used to intercept and analyze the events. | Very poor. Tests that require detailed access to page properties generally require parsing the downloaded HTML and then GETing additional pieces. | Very poor. (Same story as with HTTP GETs.) |
| Capability for WebSite Timing, Page Tuning | Superior. 1 msec internal timer make it possible to measure nearly everything about a website. | Limited to Windows timing accuracy, which may be difficult to use reliably. | It is possible to time downloads, but rendering entire pages is very difficult or impossible.
Very difficult to handle secure sessions. See performance measurement warning below. | Timings possible with an agent outside the proxy.
See performance measurement warning below. |
| Capability for WebSite Loading | Superior. Sessions detail selectable are FULL or TEXT or URL level. Very accurate timings. Very realistic behavior. | There's only one event loop, so load simulations do not have the opportunity for expansion. Instead, one has to create some kind of engine that uses recorded parameters to create load. | It is easy to generate a lot of activity, but very difficult
to have that activity be coherent, e.g. preserve state.
See performance measurement warning below. | Similar to HTTP GETs:
Easy to drive around and download URL's individually.
See performance measurement warning below. |
Performance Measurement Warning
Caution is advised when using timing and loading data for WebServer sizing
because of the significant difference between retrieval of a single URL (a basepage)
and retrieval of a complete pages as viewed by a browser.
Typically the basepage for a WebSite, specified by a single URL, e.g. http://www.mysite.com, consists of a body of text plus references to style sheets, JavaScript files, and images. What varies is the total amount of data downloaded to satisfy a request. URL-only retrievals obtain only the single named URL, whereas a browser automatically downloads not only the single named URL but all of the associated files. Browsers do this automatically by making a series of subsidiary URL requests through the HTTP protocol.
| Type of Download | Percentage of FULL | Margin of Error Range | Average Margin of Error |
| FULL | 100% | None | None |
| TEXT | 20%-40% | 5:1 to 3.3:1 | 3:1 |
| URL | 0.5%-20% | 5:1 to 200:1 | 100:1 |
The margin of error is important in this way: if by using URL-only measurements you conclude your WebSite servers can handle 1000 users, you may in fact have a capacity of 1000 / 100 = 10 users.
