New version of Randomized Tests for Zen Cart is available. New version introduces few bug fixes and one very important feature – orders filters and customer filter. Order/Customer filters can be used to dynamically remove (not permanently) from statistics certain customers/orders and recalculate statistics without them. Why is it important?

Let’s assume that You have store with many customers. You use Randomized Tests to check how new layout affects sales. Unfortunately after You run Your experiment for few days You notice that one group is biased because it includes one or two top buyers. Those two buyers are different from regular customer – they buy a lot of items, place orders every few days and probably aren’t affected by new layout – they already know Your store and don’t care about it as long as they can place orders.

Group containing these customers is biased since it includes data outliers. You probably would like to remove them from statistics. New version of Randomized Tests can do that using filter functions. Currently there are two types of filter functions:

• order filter functions – executed for each order in group – allows removal certain orders from statistics.
• customer filter functions – executed for each customer in group. Orders from rejected customers will not included in statistics.

Randomized Tests V1.2 contain two sample filter functions:

• ddigers_2std_customer_total_rt_cff() – removes top buyers (formally – customers with total spent in 2+ standard deviation from average) from statistics.
• ddigers_2std_order_total_rt_off() – removes super big orders from statistics.

To check how they’re implemented view admin/includes/functions/extra_functions/randomized_tests.php

You can implement Your own functions by following below pattern.

Custom Customer Filter Functions

Randomized Tests will recognize any function with name ending with “_rt_cff” as customer filter function and allow You to use it. For example following function will be recognized as filter:

Randomized Tests passes to customer filter functions single attribute – id of customer in database. If function returns TRUE customer is accepted in calculation of statistics, and is rejected otherwise.

Order Filter Functions

Order filter functions are similar to customer filter functions – any function with name ending with “_rt_off” will be recognized as order filter function. Example filter function:

Randomized Tests passes few more parameters to order filter functions:

• $eid – experiment row id in database
• $gid – group row id in database
• $cid – customer id in database
• $oid – order id in database
• $itemsBought – number of items in order
• $totalValue – total value of order

As with customer filter functions order filter functions should return TRUE to accept order in statistics and FALSE to reject it.

Making Filter Functions Visible to Randomized Tests Module

Let’s assume that You’ve placed Your custom filter functions in admin/includes/functions/extra_functions/my_custom_filters.php file. The best way to make them visible to RandomizedTests is to include the file through autoloaders interface. Create config.filters.php file in admin/includes/auto_loaders/ and paste there:

That’s it – Randomized Tests module will now be able to use Your filters.

Screenshot

Here’s screenshot of experiment with applied filters:

Download

You can download current version of Randomized Tests here.

A/B Split Tests, Utils, Zen Cart