Randomized Tests version 1.3 introduces new feature – custom impression functions. It allows You to decide who and under what conditions participates in A/B split test. In this post I would like to show You how to create custom impression detection function and how to use it to speed up experiments.

The need for custom impression detection functions

Let’s assume that You want to change Your registration page. You hope that new version will ease process of registration and more customers will register/order. You want to include in experiment only those visitors that actually viewed ‘Register’ page and remove from experiment those visitors that never intended to register / make purchase and never visited ‘Register’ page. Why? Because You can get more accurate results more quickly. Let me show it to You on example.

Example

Let’s assume that Your site gets 1000 visits daily (excluding bots). Of those 1000 visitors only 100 intend to buy anything ( other visitors are just surfing, comparing prices or doing anything else – they are not customers). Those 100 visitors get to ‘Register’ page where You  already set up A/B split test with two versions of the page.  You know – from Google Analytics or anywhere else – that old version of the page (version A or control group) has  20% conversion rate. You expect new version of the page to have 40% conversion rate. How quickly will You know that the new page is better? Here’s comparison of default IDF vs custom IDF:

• Default impression detection function (all visitors are included in the experiment)

Group	Visitors	Impressions	Conversions	Conv. Rate	Duration
Group #A	500	500	10	2%	6.41 days
Group #B	500	500	20	4%

• Custom impression detection function (impression occurs only when visitor views ‘Register’ page for the first time)

Group	Visitors	Impressions	Conversions	Conv. Rate	Duration
Group #A	500	50	10	20%	5.66 days
Group #B	500	50	20	40%

As You can see from this simple example there’s 15% decrease in duration of experiment. In other scenarios improvement may be even better.

How to create IDF (Impression Detection Function)

Randomized Tests can make use of any function that:

• is located in ‘includes/functions/extra_functions’ directory
• ends with ‘_rt_idf’ string (e.g. my_custom_rt_idf())
• takes five arguments in following order:
  • $experimentRowID – ID of row in rt_experiments table representing current experiment
  • $experimentID – ID of current experiment You created (e.g. ‘login_test’)
  • $groupRowID – ID of row in rt_experiments_groups table representing group to which customer has been assigned
  • $groupID – ID above group (e.g. ‘control_group’, ‘test_group’)
  • $newlyAssigned – TRUE if visitor has just been assigned to group (e.g. it’s his first visit to Your store)
• returns:
  • TRUE – if impression occured
  • FALSE – otherwise

Ok, so let’s create such function and upload it to /includes/functions/extra_functions:

First, function checks if currently visited page is ‘create_account’ page – We want impression to occur only on registration form page (which in Zen Cart is ‘create_account’ page). Impression should occur only once per visitor so function checks using UTI if visitor already viewed ‘create_account’ page. If he didn’t function does two things:

• saves information that visitor viewed ‘create_account’ page in UTI
• return TRUE – impression occured

If visitor already visited ‘create_account’ page impression does not occur and function returns FALSE.

Now upload the function to includes/functions/extra_functions and create experiment that uses it to detect impression.

Modify ‘Create Account’ page

Let’s modify ‘Create Account’ page. We just want to test two versions of ‘Create Account’ page so I used ‘Standard Groupset’ (Admin->Tools->[RT] Groupsets) as groupset. Standard Groupset has two groups: control group (id ‘cg’) and test group (id ‘tg’).

If visitor has been assigned to control group (‘cg’) old version of page is displayed. In other case new ‘Create Account’ page is used.

How to use IDF in experiment

Go to Admin->Tools-> [RT] Experiments page. Create new experiment as usually. In ‘Impression Detection Function‘ box paste name of Your impression detection function: ‘visited_create_account_page_rt_idf’  (without braces). Click on ‘Create’. That’s it!

Test that Your IDF works

Visit Your site and check that impression only occurs on first visit to ‘Create Account’ page.

That’s it. If anything isn’t clear please post comment.