A cheap temperature logger drifts in a straight line against a reference thermometer, and you want to fit that line reading ≈ w · reference + b. Now and then the logger glitches and records a wildly wrong value. With the squared error, a single glitch drags the whole line towards it, because its error is squared. A gentler loss treats small errors like the squared error and large ones only linearly. For an error e and a threshold delta > 0:
loss(e) = e² / 2 if |e| <= delta
loss(e) = delta · (|e| - delta / 2) otherwise
The two pieces meet smoothly at |e| = delta. The loss of (w, b) is the average of loss(w·x + b - y) over the data.
Write robust_fit(xs, ys, delta, lr, steps) that starts at w = 0.0, b = 0.0, runs steps steps of gradient descent on this loss with learning rate lr (each step computes both partial derivatives at the current point, then updates both), and returns (w, b, loss) with the loss at the final parameters.
Examples
Input: xs = [0, 1, 2, 3], ys = [1, 3, 5, 40], delta = 1.0, lr = 0.1, steps = 1
Output: (0.15, 0.1, 11.42625)
Explanation: at (0, 0) the errors are -1, -3, -5, -40. Only the first is within delta, so the slopes
of the four losses in their errors are -1, -1, -1, -1: every point pulls equally, however wrong.
Input: xs = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], ys = [1.2, 2.9, 5.1, 7.0, 8.8, 11.1, 13.0, 45.0, 17.1, 19.0],
delta = 1.0, lr = 0.05, steps = 2000
Output: (2.0335294117647575, 0.9917647058820367, 2.9437529411764705)
Explanation: the glitch at x = 7 barely moves the line; with the squared error the slope would be 2.9.
Constraints
1 <= len(xs) == len(ys) <= 500,0 < delta,0 < lr <= 0.5,0 <= steps <= 5000- floats are compared with a tolerance of
1e-6
Goals
- Write a loss that is quadratic for small errors and linear for large ones, and its derivative
- Train a line by gradient descent with that loss
- See how the choice of loss decides how much a few wild points pull the model