Problem 630009 · medium · Level 06 Heuristics & Optimization

The Baskets Nobody Saw

maximum likelihood · truncated Poisson model · numerical maximisation · bisection

A market stall's till records the number of items in every sale. Visitors who look around and leave without buying are not recorded, so every count in the log is at least 1. The owner believes the number of items a visitor picks up is Poisson with some rate lam, and would like to know lam and what fraction of visitors leave empty-handed.

Given that a count is at least 1, the Poisson model says it equals c with probability

lam**c * exp(-lam) / (c! * (1 - exp(-lam)))        for c = 1, 2, 3, ...

Write fit_baskets(counts) that returns a tuple of three floats:

  1. the value of lam that maximises the likelihood of the recorded counts under this model,
  2. the log-likelihood at that value,
  3. the fitted probability exp(-lam) that a visitor buys nothing.

Your lam must be accurate to at least 7 significant digits.

The setup provides basket_sizes(n, rate, seed), which simulates n recorded sales for a true rate rate (the visitors who bought nothing are dropped, as at the till).

Examples

Input:  counts = [1, 2, 1, 3, 1, 1, 2, 4]
Output: (1.423262352113118, -10.242675528887894, 0.24092674553967042)
Explanation: the recorded mean is 15 / 8 = 1.875, but that mean is inflated because
the zeros are missing. The fitted rate is 1.4233, and the model says about 24% of
visitors bought nothing.

Input:  counts = [1, 2]
Output: (0.8742174657987167, -1.7650780135140562, 0.41718835613418875)

Constraints

  • 2 <= len(counts) <= 10**5, every count is a whole number >= 1, and at least one count is >= 2
  • floats are compared with a tolerance of 1e-6

Goals

  • Write the likelihood of a model whose data cannot include zeros
  • Maximise a log-likelihood numerically when there is no closed-form answer
  • Use the fitted model to estimate something that was never observed
Starting Python…