There seem to be [at least] two different flavors of this: (i) vary the test data to see what effect that has on detections of problems in the unit under test; or, (ii) vary the actual program as a way to find out if the current test set can detect the change. The former generally is called "random testing" and the latter has been called "mutation testing".

The concept of adding randomness to improve the quality of testing has even turned up in a slightly different way in the eValid web testing engine, where there are commands that "randomize" things like the wait time interval: WaitRandom. This command is used so that either server loading scenarios or transaction monitoring tests can appear even more realistic by having the simulated user actions arrive less constantly.

Interest in this area seems to be growing, as suggested by the First International Workshop on Random Testing (RT2006), Portland, Maine, July 2006. (This is co-located with the already-mentioned ISSTA Conference.

For more information you might also check out the notion of Fuzz Testing in Google. Also worthy of note is the succinct Mutation Analysis description in Wikipedia.