NuN February 2016

A/B testing(show new feature only for 50% of users)

I'am creating a new feature for my iOS app. After I publish the app I wants to show the new feature only for 50% of the users, so I can do some testing which version makes more orders. I have no idea how to do it without using some third parties like Optimizely. Also is it possible to do this using Google Tag Manager(GTM). So can someone please help me to figure this out.

Thank you very much for your time.:)

Answers


Forelight February 2016

It’s hard to do it on your own, though not impossible of course: Optimizelys of the world are just programs. You’ll need to solve these problems:

  • Targeting: Some algorithm that will assign user session to either control or (one of) treatment(s). This has to be random, of course, or you may as well stop there.
  • Routing: Send sessios to the targeted experience.
  • Logging: You’ll need to intelligently log events from sessions as they traverse their targeted experience. These may be many, so be careful not to add latency to your app path. Your statistical analysis will be based on these.
  • Experience stability: how do you ensure (if you do) that a returning user sees the same experience he’s already seen.

Note as well, that Optimizely will only help you if all your changes are on the device and not on the server. If you need to instrument server changes as well, you’ll have to look into Sitespect or Variant.


NuN March 2016

I finally figure it out how to do the A/B testing with 'Google Tag Manager'(GTM). In GTM you can create a variable called 'Google Analytics Content Experiment'. With this variable you can select how many percentage of users going to see each Variation(your experiments). You can create up to 10 variations for single experiment.

GTM is so cool and so powerful. GTM contain so many feature that could save lot of your time and I'm totally recommend it for anyone who are going to do A/B testing.

Post Status

Asked in February 2016
Viewed 3,773 times
Voted 8
Answered 2 times

Search




Leave an answer