Optimization Pain Point Series: Takes Too Long To Get A/B Test Setup & Running

24 Apr

[This is an ongoing series of business issues I hear from clients or users getting into optimization and testing]

There are lots of tools out there ranging from bare bones to bells & whistles and from testing for dummies easy to dance dance revolution for uncoordinated people tough. The most popular enterprise level testing tool, Adobe Test&Target, packs a big punch but is definitely not the easiest testing tool out there. Newer generation tools like Visual Website Optimizer and Optimizely, though not as powerful as a T&T,  makes getting A/B tests up and running seem all too easy.

Besides coming up with the testing strategy, ideas, creative copy and assets, there are two areas that can frustrate users trying to setup a test campaign. These are tagging the pages and the actual setup of the campaign within in the tool or platform.

Tagging –– almost all testing tools require Javascript tagging. They range from one universal tag to many per page. T&T requires an mbox tagged on all ‘areas’ of the page that need to be tested. For example, if you want to test four different areas of the home page, four mbox tags are required plus the standard mbox.js. For a VWO, only one universal tag needs to be placed on the page.

If you are using a tool like T&T and parts of the page where you want to test are not tagged with mboxes, it can take some time for you to get IT resources to tag the page. If you are using a tool like VWO, as long as all pages are tagged with its universal tag as standard protocol, there is no need to get further IT resources to test an area on the page. This is where the potential for the biggest difference in setup time can take place.

If you plan ahead and coordinate efforts with the IT team to get the pages tagged, tagging will not become the huge snag to getting your test up and running.

Campaign setup — here is where I believe there is a big misconception. If pages are tagged and creative assets are done, I can have a simple A/B campaign setup in T&T in minutes. Sounds silly, but the easy step by step setup process of tools like VWO or Optimizely can slow you down – if there is a timed race for campaign setup.


Bottom line: if you get comfortable with the testing tool, you can have tests setup and running in no time.

Recommendation: plan ahead by building a testing pipeline so if you need IT resources to tag pages, you can be one step ahead.


Related Posts:

The posts here are my opinion. I hope you found it helpful or at least fun to read. My passion is in web analytics, A/B testing, MVT, LPO, CRO, etc. All things related to driving more conversions for you buck!
    About Me.

No comments yet

Leave a Reply