2

We have recently started using Sitecore A/B Testing and I am getting lots of questions about how the scoring works. I have been through the relevant Sitecore DMS documents but I still am not 100% sure if I understand how the scoring works.

My basic understanding is that the scores are based on Value Per Visit and my assumption is that the the value relates to the whole visit and not just the specific components we may be trying to optimize with the A/B Test.

For example, if option A has a goal associated with it worth 5 points, anyone presented with this option would get 5 points PLUS any other goal values they trigger during that visit to the site. That might add 5, 10, 50 or more to the visit score and then the option A score would be "total visits score/total visits".

Can anyone confirm if my assumptions are correct or explain where I may be off base? Can a user presented with option B change the score for option A?

1 Answer 1

5

By default, the Engagement Value is calculated on a per-visit basis. So your assumption is basically correct - and it does make it hard to test how a particular component variation does against another.

That being said; there are tools to help you.

We're currently implementing SBOS Accellerators into our solution. We have the same issue you are describing, and need a more fine-grained approach to testing.

Basically SBOS accellerators will allow you to track individual personalisation performance, not "just" looking at the overall Engagement Value.

Lars Petersen blogs about it here: http://www.larsdk.dk/2014/01/must-have-marketplace-modules-for-sitecore-digital-marketing-system/

Marketplace link for the module here: http://marketplace.sitecore.net/en/Modules/SBOS_Accelerators.aspx

We found a few issues in testing the module, but none were really severe. I know these issues are being fixed if they haven't already.

3
  • Thanks Mark, that was exactly what I was looking for. I installed the SBOS module and presented the results to our marketing team. I think this was enough to stop them from switching to GA Experiments for now and the report already identified some interesting information. Seems like this should be available out of the box though. Mar 16, 2014 at 20:12
  • Hi Kevin. I agree. But I also know, much Sitecore functionality that is now "in the box" started out as modules and/or experiments. If enough people find it useful, I'm sure the Sitecore product team will pick up on it eventually. Mar 18, 2014 at 9:22
  • I see that this works only before 7.5 version of sitecore.Is there any other version of the same tool which works on 8.0? I was setting the goals for the target pages and it does not work. Variation of the pages works and the goal that is been set on the target item also triggers. While digging into it i found out that Fact_TestConversions,Fact_MvTesting table is not populated with data so my conversions are always zero. Any help could be appreciated. Aug 31, 2016 at 16:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.