Siddharth Deswal said
10 months, 1 week ago: @jeduchscherer
You should always test your pages to similar audiences in the same time-frame and not the exact same audience because you’d confuse visitors if they see different variations on different visits. For example, “visitors coming from Facebook ad campaign ABC” split 50/50 to test name “Widget X campaign on Facebook” during the week “August 20 to August 26″.
Essentially, this is basic behavioral targeting. The reason is that if you test Control one week and Variation the next, you’ll be skewing the results and not really getting the most optimum insights into what works and what doesn’t.
Also, behavioral targeting allows you to slice and dice your visitors very well and display content / ads / offers with laser-like focus, leading to much better conversion rates.
Here are a couple of links on A/B testing and behavioral targeting that you might find useful:
(I work for Wingify, the company that makes Visual Website Optimizer)