|
|
Key Ideas: A very complex AJAX application was tested with nine distinct playback scripts that involve 600+ URLS and requires playback times from 110 to 584 seconds, and which in total download 101+ MBytes of data. All of the scripts required SyncOnText or SyncOnElementProperty commands to assure reliable operation in a LoadTest run that involved 1,000 Browser Users (BUs).
Tested Application Details
The application under test (AUT) in this case study
was a very complex, AJAX-based,
financial analysis and reporting system.
There is an HTTPS login (we were supplied with 1,000 login names/passwords), followed by a variety of operations that generally involve interrogating a database to produce a report. In some tests as many as four (4) sub-windows are open below the main window. All test sequences ended by logging out.
The test scripts -- a total of nine of them -- tended to be rather long, both in number of commands used and total playback time. Some scripts were over 200 lines in length, before addition of DOM synchronization commands. You can get a sense of the complexity if you estimate that 1-2 synchronization commands were needed for each newly arrived-at page.
Some tests require up to ~360 seconds (~6 minutes) total playback time even with the natural recorded Wait times all reduced to zero, i.e. with the tests run at a 99+% duty cycle.
In a few cases we added Delay 100 commands to allow adequate time for the DOM to stabilize when the page was particularly complex. For example, some pages had over 3,500 DOM elements.
The total test data volume -- due to the large size of some of the pages being downloaded -- was comparatively high, ranging from ~5 MBytes up to ~23 MBytes for a single script playback.
Script and Scenario Statistics
Here are the characteristics of each test script in the overall Loadtest scenario.
These results were achieved with a Playback Multiplier of 1.0,
and were run in "singleton mode".
| # | Test Script Name | LoadTest Scenario Percentage Allocation | Original Script Lines | Original Script Actions | DOM-Based Synchronization Commands Added | URLs Visited | Playback Duty Cycle | Downloaded MBytes | Total Playback Time |
| 1 | Quote | 5% | 102 | 59 | 19 | 55 | 77% | 8.02 | 323 secs |
| 2 | Calculator | 8% | 146 | 96 | 29 | 74 | 55% | 11.11 | 364 secs |
| 3 | Charts | 5% | 114 | 65 | 22 | 72 | 70% | 10.48 | 284 secs |
| 4 | Import | 1% | 81 | 57 | 9 | 66 | 71% | 10.57 | 310 secs |
| 5 | Performance | 4% | 85 | 45 | 13 | 80 | 74% | 9.01 | 167 secs |
| 6 | Profiles | 54% | 265 | 200 | 63 | 88 | 59% | 20.54 | 584 secs |
| 7 | Publisher | 8% | 98 | 60 | 15 | 68 | 84% | 13.97 | 579 secs |
| 8 | Queue | 7% | 63 | 39 | 11 | 72 | 71% | 8.36 | 110 secs |
| 9 | Reporter | 8% | 98 | 45 | 17 | 49 | 59% | 9.57 | 269 secs |
| Σ | 100% | 1052 | 666 | 198 | 624 | 101.63 | 2,990 secs |
Script Conversion
The original scripts were recorded from the application "from life"
and all of them worked out of the box using
natural Wait times to assure synchronization.
To optimize their use in a degraded server loading environment,
in which we expected these scripts to encounter de-synchronizations,
we augmented each long-running download in each script with
one or more eValid SyncOn... commands.
Synchronization Methods Used
The Sync commands that were added to the scripts
were either SyncOnText or SyncOnElementProperty commands:
These more powerful commands were used whenever a SyncOnText was found to be insufficient, e.g. when a pulldown list was only propagated well after the name of the list (which appears visibly on the page) appeared.
Ramp Up Schedule
The load on the server was ramped up at the rate of ~10 users per minute,
and an additional machine was added every 10 minutes.
This process went on until a total of 10 machines or 1,000 BUs was achieved.
Cloud Computing Resource
The cloud machine images we used for these tests was an
8-core virtual CPU running at 2.67 GHz, and having 68 GBytes of RAM.
The operating system on all images was
Windows Server Data 2008 Center 64-bit [SP2].
eValid THIN Footprint
During the run we saw the footprint of eValid THIN LoadTest playback engine
grow from a starting value of ~12 MBytes
to as high 900 MBytes in some cases.
The average eValid "THIN" footprint growth was to about 100 MBytes.
Resource Usage
Near the end of the test run, when all 1,000 eValid browsers were emulating
users, the typical
CPU usage rates were near 100%,
and typical memory usage for each 100 eValid browsers
was 38%-40% of the available RAM.
