A data blog collects dozens of daily series (visitors to a museum, the price of cocoa, rainfall in a far-away city, ...) and hunts for surprising links. Pairs of series that both drift up or down over time often show large correlations although nothing connects them.
series is a dictionary from a name to a list of daily values; all lists have the same length. Write strongest_pair(series) that finds the pair of different series whose correlation coefficient has the largest absolute value, and returns a tuple (a, b, r_levels, r_changes):
aandbare the two names witha < b;r_levelsis the correlation coefficient of the two series themselves;r_changesis the correlation coefficient of their day-to-day changes (values[t] - values[t - 1]fort = 1, 2, ...).
Pairs whose abs(r_levels) are equal within 1e-12 are ordered by (a, b), and the first one wins. A correlation coefficient is Sxy / sqrt(Sxx * Syy) (sums of products and squares of deviations from the means) or None when Sxx or Syy is 0. Pairs whose r_levels is None are never chosen; if no pair has one, return None.
The setup provides random_walks(k, n, seed), which returns a dictionary of k series of length n named "s00", "s01", ...; each one starts at 100 and moves by an independent random step every day, so no two series are related in any way.
Examples
Input: series = {"cocoa": [3, 4, 6, 7, 9], "visits": [10, 11, 14, 15, 16],
"rain": [5, 1, 4, 2, 3]}
Output: ("cocoa", "visits", 0.9789951616722312, 0.5773502691896258)
Explanation: cocoa and visits both rise, r is about 0.98. Their changes are
[1, 2, 1, 2] and [1, 3, 1, 1], which agree far less: r is about 0.58.
Constraints
2 <= len(series) <= 40, every list has the same length between3and500- values are integers or floats with absolute value at most
10**6 - floats are compared with a tolerance of
1e-6
Goals
- Search many pairs of series for the strongest correlation
- See that series which merely drift can be strongly correlated by chance
- Check a correlation again on the day-to-day changes