Automated Performance Testing

Continuous Integration has become a standard part of the "DevOps" of many teams, and one component of that is usually automated testing of the code at a "stage" or "testing" point before it is released.

Less common is automated performance testing, which is launching a load test at some point in the continuous integration process. While it is more common to monitor performance of the live site, it is rare to test it prior to making changes live.

Automated performance testing has special complications as compared to other testing. It requires more infrastructure; testing on a stage environment is not likely to be representative and testing on live is risky and perhaps too late.

Performance testing is also dependent on the data set (has the site become slow because of new features, or because of user or content growth?) and modeling the nature of the traffic (did the site become slow because of a new theme, or because social media is sending more people to expensive pages ?).

A fairly simple example automated performance test will be described, as well as future directions.

Speakers

Room: 
Track: 
Coding and development
Experience level: 
Advanced
Questions answered by this session: 
How much does it cost to set up automated performance testing ?
What sorts of problems will I avoid with automated performance testing ?
How is performance testing over time different from performance monitoring over time ?
How exactly should a performance testing environment emulate the live environment ?
What sorts of events might cause performance problems that would not be caught by automated performance testing ?
Colorado mountains