Chad Summerhill wrote a nice post recently about something you should be doing if you manage your own PPC account and something you should be showing your clients if you manage accounts for others - visual statistically significant ad test results by time period.  Check out the "how-to" in that post.

I did this exercise and can see how important it is to look at your ad test segmented out by time period (day or week, whatever is most appropriate).  The main reason for this is that if you simply report the final results of your test, you won't tell the whole story.  Most of the time, we look at the results of tests like this...

Ad Test Table.png

Why is this imperfect?  Well, we know cumulatively that the challenger performed better, but does that mean we should delete the champion?  Maybe, maybe not.  You don't really know until you look at the data in time.  There could be a scenario where the champion really struggled out of the gate in this test, but has actually been beating the challenger in the last week.  If so, how would you know?  Again, by taking a look at the results in time to view the trend.  With this particular ad test, here's how it turned out...

Ad Test Chart By Day.png

This way, you can trend out how the winner actually became the winner.  In this test, we can see that the challenger was consistently better over time, which now gives us supreme confidence when we hit the delete button for the champion.

One other thing.  See those trend lines in the graph?  Not only does it show you which ad was better (by the gap between the two lines), but it also shows you the overall trend, meaning you can see if this metric is getting better or worse for you.  In this example, the ad group is performing better over time regardless of what ad is running.  This shows that whatever optimization efforts have been performed are working.

March 4, 2011

Mike Fleming specializes in Analytics and Paid Search for Pole Position Marketing, a leading search engine optimization and marketing firm helping businesses grow since 1998. You can follow Mike on Twitter at @SEMFlem. Mike enjoys playing, writing and recording music along with playing basketball to get his workout in. He resides in Canton, Ohio with a girl who threw a snowball at him one day…then married him.

Mike and the team at Pole Position are available to help clients expand their online presence and grow their businesses. Contact them via their site or by phone at 866-685-3374.


Hey Mike, thanks for the mention!

Like your approach, especially being able to see if your metrics are improving over time regardless of the test.

Comments closed after 30 days to combat spam.

Search Engine Guide > Mike Fleming > Telling the Story of Your PPC Ad Tests With Time