An online shop tests up to four designs of its checkout button. Design i was shown to visitors_i people, of whom buyers_i bought something. With the prior Beta(a, b) for every design's true purchase rate, design i's posterior is Beta(a + buyers_i, b + visitors_i - buyers_i), and the designs are independent. The product team asks one question: for each design, what is the probability that its true rate is the highest of all?
Write chance_best(results, a, b), where results is a list of pairs (buyers, visitors), and return the list of these probabilities in the same order. They add up to 1. Every probability must be accurate to within 1e-6, which rules out estimating them by random draws: compute them.
Examples
Input: results = [(0, 3), (1, 3)], a = 1, b = 1
Output: [0.21428571428571427, 0.7857142857142857]
Explanation: the posteriors are Beta(1, 4) and Beta(2, 3); the first design has the
higher rate with probability exactly 3/14.
Input: results = [(3, 10), (5, 10), (4, 10)], a = 1, b = 1
Output: [0.12045667967973495, 0.5911361322619044, 0.2884071880583606]
Input: results = [(1200, 10000), (1260, 10000)], a = 1, b = 1
Output: [0.0982558303960197, 0.9017441696039803]
Explanation: the second design converts 12.6% against 12.0%; with 10,000 visitors each
it is the better one with probability 0.90. (Values shown to fewer digits.)
Constraints
2 <= len(results) <= 4,0 <= buyers <= visitors <= 10**5a, bare whole numbers with1 <= a, b <= 100- floats are compared with a tolerance of
1e-6; each call must finish well within a tenth of a second
Goals
- Give every variant of an A/B/n test its own Beta posterior
- Express 'variant i has the highest rate' as an integral of densities and cumulative probabilities
- Evaluate that integral accurately, even when the posteriors are very narrow