A/B testing is a simple and effective way to  transparently compare the performance of Appodeal with other mediation services.

During A/B test, application users randomly split into two groups. The first group uses Appodeal, the second - another mediation service.

To obtain a statistically reliable result, it is necessary to exclude the influence of the groups on each other, i.e. the user (or device) should be strictly assigned to one group.

The percentage of traffic distribution between groups is configured on the server using Firebase Remote Config or another service for A/B testing. This will allow to dynamically adjust the distribution of traffic during the test, as well as direct all traffic to Appodeal, if the test result is satisfactory.

A/B testing using Firebase Remote Config

1. Create a parameter key named "mediation_partner" in Firebase Remote Config. In the "Default value" field specify "appodeal".
2. Create the condition "mediation partner variant" with the setting "User in random percentile" "> 80%". The specified percentage of traffic distribution between mediation services can be changed.
3. For the "mediation partner variant" condition specify the value of the "mediation_partner" key with the name of the second mediation service, for example "mopub".
4. In the application get the value of the key "mediation_partner" and initialize and use only the corresponding mediation service.

Demo projects for implementing the algorithm:

• Android
• iOS (Objective-C / Swift)

A/B testing using your own server

For cases when Firebase Remote Config is not used, the necessary settings for dividing user groups can be set on the server:

1. When the application is launched for the first time, generate a random number for each user in the range from 0 to 100. Let’s denote this value by userPercentage. Store the value of userPercentage on the device and use it in all subsequent application sessions.
2. Get a value from the file on the server, that indicate what percentage of the advertising traffic will work through Appodeal. Let’s denote it by appodealPercentage.
3. If userPercentage < appodealPercentage, initialize and use Appodeal. Otherwise, initialize and use another mediation service.

Recommendations to avoid problems during A/B testing

• Both mediation services should not be initialized in one session.
• To avoid errors during build and run time of the application, the versions of third-party SDKs in the application should coincide with the their versions in Appodeal SDK (check "Third-party SDKs versions" table in SDK documentation). Android SDKs of those networks are available in the archive with the Appodeal SDK distribution. iOS SDKs of those networks are located in the CocoaPods release of the Appodeal SDK.
• When building an Android application, we recommend using MultiDex.
• In the case where only one network account is used for both mediation services, traffic distribution should be organized by using different ad units (placements / zones) for each group of users.
• To get correct DAU you need to label if you labeled each user with user group they belong to in your analytics service

Performance comparison of the mediation services

ARPDAU is used to evaluate performance of advertising, since it is the most objective metric for evaluating results for different impression volumes.

ARPDAU (Average Revenue Per Daily Active User) shows how much you earn on average per day on one active user.

This metric is calculated by the formula:

ARPDAU = Daily Revenue / Daily Active Users (Daily Revenue / Number of Active Users per day)

To calculate ARPDAU for advertising traffic that worked through the Appodeal, use the data from the dashboard page.

To calculate the value of ARPDAU for another mediation service you need revenue and DAU. To calculate revenue you need to pull it for mediation service dashboard or from each ad network account used in that mediation service. You can get DAU in your analytics service (Yandex AppMetrica, Google Analytics).