A hyperparameter search wants to try every combination of a learning rate and a batch size. Both candidate lists are open-ended generators (smaller and smaller rates, larger and larger batches), so itertools.product cannot be used: it reads its inputs completely before it yields anything. And a nested loop would try the first learning rate with every batch size forever.
Write a generator function pairs(xs, ys) that yields every pair (x, y) with x from xs and y from ys. Number the items of each source from 0, and yield the pair of items i and j in order of i + j, and for equal sums in order of i. Each source may be endless, finite or empty; with finite sources, pairs ends after the last pair.
Each source can be read only once and reading is expensive, so read an item exactly when the first pair that uses it is due, never earlier. At the very first pair, which needs an item of each, read xs first; if a source turns out to be empty, stop without reading any further.
Helpers you can use with Run: first(gen, n), halvings(start) and doublings(start) (endless candidate lists), Counted(items) (an iterator whose reads attribute counts the items handed out) and reading(pairs, xs, ys, k), which takes k pairs and reports how many items of each source were read.
Examples
Input: first(pairs(halvings(0.1), doublings(16)), 6)
Output: [(0.1, 16), (0.1, 32), (0.05, 16), (0.1, 64), (0.05, 32), (0.025, 16)]
Input: reading(pairs, itertools.count(1), "abc", 8)
Output: ([(1, "a"), (1, "b"), (2, "a"), (1, "c"), (2, "b"), (3, "a"), (2, "c"), (3, "b")], 3, 3)
Explanation: after "c" the letters run out, so the pairs (0, 3) and (1, 3) of later sums are skipped.
Input: reading(pairs, [], itertools.count(), 5)
Output: ([], 0, 0)
Constraints
- The tests take at most 20,000 pairs; finite sources have at most 300 items.
- Items can be any values; compare positions, never the items themselves.
Goals
- See why `itertools.product` cannot combine endless iterators, and write a lazy replacement
- Visit every pair of two possibly endless streams in a fair order, reading each item only once
- Handle sources that end at any point, including empty ones, without reading ahead