A hidden test failed, and the function under suspicion has a slow but trustworthy twin. Write a tool that finds the smallest input on which they disagree: first_failure(fn, ref, values, max_len).
Try the candidate lists in this order: first by length, from 0 up to max_len; within one length, in the order itertools.product(values, repeat=length) produces them (so it follows the order of values). For each candidate, call fn and ref, each with its own new list of the candidate's items (a function might change its argument), and compare their outcomes. The outcome of a call is the value it returns, or, if it raises an exception, the exception's type. The functions disagree when the outcomes differ: different values, one raises and the other does not, or they raise different exception types.
Return the first candidate on which they disagree, as a list, or None if they agree on every candidate.
The setup holds a few suspects and their twins in CASES: CASES[name] is a pair (fn, ref) for the names "second_largest", "runs", "is_sorted", "mean", "reverse", "window", "unique" and "same" (their docstrings say what they should do). With Run you can call them directly, for example CASES["runs"][0]([1, 1, 2]).
Examples
Input: first_failure(*CASES["is_sorted"], [1, 2], 3)
Output: [1, 1]
Input: first_failure(*CASES["second_largest"], [0, 1], 4)
Output: [0, 0]
Explanation: [], [0] and [1] raise ValueError in both; for [0, 0] the reference raises
ValueError (fewer than two distinct values), the suspect returns 0.
Constraints
1 <= len(values) <= 4,0 <= max_len <= 6, so at most about 5,500 candidates.- The functions return plain values (numbers, lists, booleans,
None) or raise.
Goals
- Generate every small input in a fixed order with `itertools.product`
- Compare two functions' outcomes, treating an exception as an outcome
- Turn "it fails somewhere" into the smallest failing case