Test and launch your experiment | Amplitude Experiment

This article helps you:

QA your experiment before and after rollout
Launch your experiment to your users

Before any users view your experiment, make sure the variants you’ve developed look and function exactly the way you intended. Because Experiment allows you to assign specific variants by user ID, device ID, or cohort, you can ensure that Amplitude serves your test devices the relevant variants when they enter your experiment.

On the Overview page for your specific experiment, review the Overview, Delivery, Variants, and Targeting sections. Make sure you set everything the way you planned it.

Click Test Instrumentation to send the experiment’s variants to the testers you designated when you configured the experiment's audience.

Test Instrumentation and targeting

When you test your instrumentation, Amplitude ignores the target segments you configured in the experiment. Test instrumentation sends variants to only the Testers.

Launch your experiment

When you’re satisfied that your experiment works as you intended, click Start Experiment.

If you want, you can set an end date for the experiment or accept the default Experiment analysis range.

Note

Clicking Start Experiment is the only way to activate your experiment. If you change the start date for the experiment, it won't automatically activate on the new date.

After your experiment is running, you can make a decision on your experiment when it reaches statistical significance or its end date.

Scheduling your experiment

To schedule the experiment for launch at a later time, expand the Start Experiment menu and click Schedule start. In the modal that appears, set the date and time that you want to begin the experiment.

Experiment start and variant delivery

When a scheduled experiment reaches its start time, there may be up to a one hour delay before the experiment begins exposing users to variants.

QA after rollout

After rollout, you can track how many of your users were exposed to each variant on a daily basis.

Go to Experiments > your experiment > Activity tab > Diagnostics to view how many users were exposed to each variant.

This is a useful way to QA the assignment process. If you notice that one variant is enrolling significantly more or significantly fewer users than you expected, it could mean an issue you should investigate.

If you do spot some outliers or anomalies that concern you, click into the chart or information to conduct a deep dive into the potential causes. To learn more about understanding anomalies, review this article on Root Cause Analysis.