Problem 649170 · hard · Level 06 Heuristics & Optimization

Catch the Failing Pumps

anomaly detection · F1 score · precision and recall · standardisation · hidden test set

A water company monitors its pumps. Each reading is [temperature in °C, vibration in mm/s, current in A, noise in dB], and in a healthy pump all four rise together with the load it is running at. Faults are rare and come in several kinds: a worn bearing shakes more than the load explains, a cooling fault runs hot, a slipping belt draws too little current, and an overloaded pump runs beyond its normal range. Nobody has labelled examples of faults; you only get a log of healthy readings.

Write detect(normal, readings) that learns from the healthy readings normal and returns one flag per row of readings: True (or 1) for "probably a fault", False (or 0) otherwise.

The tests call judge_detector(detect, n, seed). It gives your function pump_log(n, seed) (healthy readings) and 1000 hidden readings from the same pumps, about 5% of them faulty, and keeps the truth to itself. pump_log(n, seed) is available in your code.

How this problem is scored

A detector is judged by its F1 score on the hidden readings: 2·TP / (2·TP + FP + FN), where TP counts caught faults, FP false alarms and FN missed faults. It passes when its F1 is at least that of the 2.5-sigma rule (flag a reading when any feature is more than 2.5 standard deviations from its healthy mean). Its quality (0 to 100) is the share of the gap between that rule and the rule that knows the generator (it knows exactly how healthy readings depend on the load, and uses the best threshold for these hidden readings) that it closes; 100 at or above it. Match the reference solution's quality (the par in the header) for the third star.

Examples

Input:  pump_log(2, 1)
Output: [[47.9, 4.81, 10.5, 66.9], [53.2, 5.92, 15.5, 71.1]]

Input:  judge_detector(detect, 200, 1)
Output: a summary such as {"tp": 33, "fp": 12, "fn": 26, "total": 1000}
        F1 = 66 / 104 = 0.635; the 2.5-sigma rule scores 0.421 here

Constraints

  • 200 <= len(normal) <= 600, len(readings) = 1000, four features
  • each test must finish in well under a second in your browser
  • your flags must not depend on the clock; if you use randomness, use a random.Random with a fixed seed

Goals

  • Build an anomaly detector from healthy data only
  • Catch faults that show up as unusual combinations of readings, not as extreme single values
  • Trade false alarms against missed faults to maximise F1 on hidden data
Starting Python…