A/B Test Simulator
Simulate A/B tests with different conversion rates and sample sizes to understand statistical significance.
AB Testing Case Study
Introduction
This is an example to show how to implement a simple A/B test for deciding what colour button to use on a site. During my time working as a Product Manager in numerous companies there has been a general lack of infrastructure to implement and monitor A/B Tests. This has led me to instead create this example to show understanding of both implementation and analysis of the concept.
Implementation
The implementation is simple to demonstrate one of the smallest units of a site that can be tested - a button's colour. The implementation is simply a button on a page that has a 50% chance of showing either blue or purple. There are event listeners and handlers to track the clicks by simply logging the result in the console. In the real world this would be logged into a more permanent database to then be analysed after the experiment has been run.
Analysis
The analysis provided in the Results Document demonstrates how I would interpret the results of an A/B experiment of this type. In this case I have stubbed the data to show a close but significant different in CTR based on the pageviews. The analysis includes a Z-Score and P-Value calculation as well as results and recommendations based on those figures.