Problem 655771 · hard · Level 06 Heuristics & Optimization

Order Rates for New Restaurants

empirical Bayes · shrinkage · Poisson model · conjugate prior · mean squared error

A food delivery app has just listed a few hundred new restaurants. Each has been open for only 1 to 5 days, and for each day the app knows how many orders it received. The app wants to predict every restaurant's long-run average orders per day (its true rate), to decide how many couriers to send to each area.

Write estimate_rates(records). records is a list with one entry per restaurant, and each entry is the list of that restaurant's daily order counts. Return a list of the same length with your estimated rate (a float) for each restaurant, in the same order.

Each restaurant's daily counts are Poisson with its own true rate, and the true rates of the restaurants were drawn from one common population distribution, which the tests do not reveal (it differs between tests).

Each test calls rate_trial(estimate_rates, case) from the setup: it builds the restaurants of test case, runs your function once and reports your mean squared error over all restaurants, "mse". You can call restaurant_orders(case) to look at the data of a test, and rate_trial to measure an idea with Run.

Examples

Input:  rate_trial(estimate_rates, 2)
Output: for example {"mse": 1.54, "plain": 2.483, "restaurants": 491}
Explanation: estimating each restaurant by its own average orders per day has a mean
squared error of 2.483 on this test; a better set of estimates reached 1.54.

How this problem is scored

  • Baseline: every restaurant's own average orders per day (the maximum likelihood estimate for that restaurant alone). Its mean squared error is reported as "plain". You pass a test when your mean squared error is no larger.
  • Best known: the mean squared error of the best possible estimates for someone who knows the true population distribution exactly (computed offline).
  • Score: 100 * (plain - yours) / (plain - best), from 0 at the baseline to 100 at (or below) the best known value.

Constraints

  • 300 to 500 restaurants per test, 1 to 5 days each; the counts are whole numbers >= 0
  • the result must not depend on the clock; your function must finish well within a second in the browser

Goals

  • Estimate many related rates at once, each from very little data
  • Learn a prior from the whole population and use it to improve every individual estimate
  • Trade a little bias for a large reduction in variance, and measure the result
Starting Python…