| Title: | Power Analysis for AB Testing |
|---|---|
| Description: | Power analysis for AB testing. The calculations are based on the Welch's unequal variances t-test, which is generally preferred over the Student's t-test when sample sizes and variances of the two groups are unequal, which is frequently the case in AB testing. In such situations, the Student's t-test will give biased results due to using the pooled standard deviation, unlike the Welch's t-test. |
| Authors: | William Cha [aut, cre] |
| Maintainer: | William Cha <[email protected]> |
| License: | GPL (>= 3) |
| Version: | 0.1.0 |
| Built: | 2026-05-27 10:38:53 UTC |
| Source: | https://github.com/chawm/pwrab |
AB_t2n performs the power analysis for AB testing. It uses the Welch's t-test,
which allows for the standard deviation to vary across groups.
AB_t2n(N = NULL, percent_B = NULL, mean_diff = NULL, sd_A, sd_B, sig_level = NULL, power = NULL, alternative = c("two_sided", "less", "greater"), max_sample = 1e+07)AB_t2n(N = NULL, percent_B = NULL, mean_diff = NULL, sd_A, sd_B, sig_level = NULL, power = NULL, alternative = c("two_sided", "less", "greater"), max_sample = 1e+07)
N |
Total number of observations (sum of observations for groups A and B) |
percent_B |
Percentage of total observations allocated to group B (between 0 and 1 - e.g. input .5 for 50%) |
mean_diff |
Difference in means of the two groups, with mean_B - mean_A |
sd_A |
Standard deviation of group A |
sd_B |
Standard deviation of group B |
sig_level |
Significance level (Type I error probability) |
power |
Power of test (1 minus Type II error probability) |
alternative |
Character string specifying the alternative hypothesis, must be one of "two_sided" (default), "greater" or "less" |
max_sample |
Maximum sample size that is searched for |
Exactly one of the parameters 'N', 'percent_B', 'mean_diff', 'sig_level', and 'power' must be passed as NULL, and the omitted parameter is determined from the others. sd_A and sd_B must be specified. When 'percent_B' is the parameter omitted, two solutions may exist, in which case the smaller value will be returned
Object of class "power.htest", a list of the arguments (including the computed one).
# Search for power given other parameters AB_t2n(N = 3000, percent_B = .3, mean_diff = .15, sd_A = 1, sd_B = 2, sig_level = .05, alternative = 'two_sided') # Search for sample size required to satisfy other parameters AB_t2n(percent_B = .3, mean_diff = .15, sd_A = 1, sd_B = 2, sig_level = .05, power = .8, alternative = 'two_sided')# Search for power given other parameters AB_t2n(N = 3000, percent_B = .3, mean_diff = .15, sd_A = 1, sd_B = 2, sig_level = .05, alternative = 'two_sided') # Search for sample size required to satisfy other parameters AB_t2n(percent_B = .3, mean_diff = .15, sd_A = 1, sd_B = 2, sig_level = .05, power = .8, alternative = 'two_sided')
AB_t2n_prop performs the power analysis for AB testing, and when
dependent variables are proportions (between 0 and 1). It uses the Welch's t-test,
which allows for the standard deviation to vary across groups.
AB_t2n_prop(prop_A = NULL, prop_B = NULL, N = NULL, percent_B = NULL, sig_level = NULL, power = NULL, alternative = c("two_sided", "less", "greater"), max_sample = 1e+07)AB_t2n_prop(prop_A = NULL, prop_B = NULL, N = NULL, percent_B = NULL, sig_level = NULL, power = NULL, alternative = c("two_sided", "less", "greater"), max_sample = 1e+07)
prop_A |
Proportion of successes in group A (between 0 and 1) |
prop_B |
Proportion of successes in group B (between 0 and 1) |
N |
Total number of observations (sum of observations for groups A and B) |
percent_B |
Percentage of total observations allocated to group B (between 0 and 1 - e.g. input .5 for 50%) |
sig_level |
Significance level (Type I error probability) |
power |
Power of test (1 minus Type II error probability) |
alternative |
Character string specifying the alternative hypothesis, must be one of "two_sided" (default), "greater" or "less" |
max_sample |
Maximum sample size that is searched for |
Exactly one of the parameters 'prop_A', 'prop_B', 'N', 'percent_B', 'sig_level', and 'power' must be passed as NULL, and the omitted parameter is determined from the others. The standard deviations for each group are calculated using the formula sqrt(prop * (1 - prop)). When 'percent_B' is the parameter omitted, two solutions may exist, in which case the smaller value will be returned. For two_sided tests, when 'prop_A' or 'prop_B' is omitted, two solutions may exist, in which case both will be reported
Object of class "power.htest", a list of the arguments (including the computed one).
# Search for power given other parameters AB_t2n_prop(prop_A = .2, prop_B = .25, N = 3000, percent_B = .3, sig_level = .05, alternative = 'two_sided') # Search for proportion in group B required to satisfy other parameters AB_t2n_prop(prop_A = .2, N = 3000, percent_B = .3, power = .8, sig_level = .05, alternative = 'two_sided')# Search for power given other parameters AB_t2n_prop(prop_A = .2, prop_B = .25, N = 3000, percent_B = .3, sig_level = .05, alternative = 'two_sided') # Search for proportion in group B required to satisfy other parameters AB_t2n_prop(prop_A = .2, N = 3000, percent_B = .3, power = .8, sig_level = .05, alternative = 'two_sided')