Online shopping is one of the most popular online activities and retail e-commerce sales has become a multi-billion-dollar industry worldwide. This activity may increase by an order of magnitude during special marketing promotional events or holiday season. Online retailers are extra cautious during such peak periods and strive extra hard to ensure a smooth customer shopping experience. Performance testing activities play a crucial role in building confidence in the software application and underlying infrastructure systems by recreating pseudo-real customer web traffic at the expected holiday period scale. If you are planning to build and launch an e-commerce website or gearing towards building capacity for anticipated peak sales growth, you need to be prepared to load your production systems up to the anticipated load while sustaining zippy application response times. So how do you exactly do that? What types of tests do you build? When do you execute them and most importantly, where do you look for metrics to provide analysis and recommendations? We will discuss some of the techniques that have worked well at Wayfair. During this presentation, we will understand how to spin up testing infrastructure on demand via automation, execute tests, collect real time metrics, generate reports and spin them down on completion. We will also touch upon system performance benchmarking toolsets and discuss Performance Engineering as a Service (PEAS).