Problem 336096 · medium · Level 03 Linear Management & Searching

A Scaler That Remembers Its Training Data

py-closures · py-higher-order · feature scaling · min-max normalisation

A house-price model wants every feature between 0 and 1. The ranges must come from the training houses, and exactly the same ranges must be used for every house the model sees later.

Write fit_minmax(rows). rows is the training data, a non-empty list of rows of numbers, all of the same length. For each column, find its smallest value lo and its largest value hi. Return a pair of functions (scale, unscale):

  • scale(row) returns a new list with each value x turned into (x - lo) / (hi - lo), clipped to the range [0.0, 1.0] (new houses may lie outside the training range). For a column where lo == hi, the scaled value is 0.0.
  • unscale(row) turns scaled values back: lo + y * (hi - lo), without clipping. For a column where lo == hi it returns lo.

The returned functions must keep working after the caller has changed or emptied the training list, and they are called many times, so fitting happens once.

Helpers available with Run: apply(fn, rows) calls fn on every row, fit_then_forget(fit_minmax, train, rows) fits, empties the training list, then scales rows, and house_rows(n, seed) generates houses.

Examples

Input:  scale, unscale = fit_minmax([[10, 1], [30, 1], [20, 1]])
        apply(scale, [[10, 1], [25, 1], [40, 5]])
Output: [[0.0, 0.0], [0.75, 0.0], [1.0, 0.0]]

Input:  apply(unscale, [[0.5, 0.3], [1.5, 0.0]])
Output: [[20.0, 1], [40.0, 1]]

In the tests the same calls are written apply(fit_minmax(train)[0], rows) and apply(fit_minmax(train)[1], rows).

Constraints

  • Up to 3 * 10**4 training rows with up to 8 columns, and as many calls to scale.
  • Values are compared with a tolerance of 1e-6.

Goals

  • Return functions from a function, each remembering numbers computed once
  • Keep the fitted settings inside the closure so callers cannot pass the wrong ones
  • Write a forward transformation and its inverse that share the same settings
Starting Python…