When analyzing which ads work better you probably don’t break it down by keyword. This means you may come to the wrong conclusions. We recently compared two ads as you can see in the screen shot below (I changed many of the details to protect the privacy of our client).

Google Adwords AB tests must be analyzed by keyword to avoid reaching the wrong conclusions

The ad on the left had a conversion rate of 1.1% as opposed to the ad on the right which had a conversion rate of 1.8%. However when we break it down by keywords we see that Keyword 2 had:

Different Google Ads for Different Keywords

Look at Keyword 4 and you see that the opposite is true—Ad A is best. Therefore Keyword 4 needs ad A. If you didn’t do this analysis by keyword you would come to the wrong conclusions and think Ad B is best for the ad group.

The results that you need to remember are:

You may also ask how these two different keywords ended up in the same ad group to begin with. In a world with time restraints it is not practicable to give each keyword its own group. Even if each keyword had its own group you would still have to do the above analysis as people will use different modifiers for each keyword. That is why the above analysis is based on the actual keywords used by the searchers and not the phrase matched keywords in the ad group.

This is what makes it important to compare ads by keyword.

There are lots of additional insights you can get from the above analysis but I will leave that for another post.