A new mail service has only a handful of messages that users have marked as spam or not spam ("ham"). The messages are written in an invented language, so you cannot write rules by hand: the model must learn which words matter. Messages are typed carelessly: some words are capitalised or in capitals, and some end in one of ! . , ?.
Write classify(texts, labels, new_texts) that learns from the labelled messages (labels[i] is "spam" or "ham") and returns a list with one label per message of new_texts.
The tests call judge_inbox(classify, n, seed). It trains your function on inbox(n, seed) (a list of (text, label) pairs) and gives it 1000 hidden messages from the same service, keeping their labels to itself. inbox(n, seed) is available in your code for experiments, for example to train on one seed and test on another.
How this problem is scored
A filter passes a test when it is right on more hidden messages than always predicting the most common training label. Its quality (0 to 100) is the share of the gap between that baseline and the rule that knows the generator (it knows the true probability of every word in spam and ham, and still makes a few mistakes) that it closes; 100 at or above that rule. The accuracies are shown next to every test. Match the reference solution's quality (the par in the header) for the third star.
Examples
Input: inbox(3, 1)
Output: [("Dosas dosas lorir kofir ... lavin lolor", "ham"), ("halon koha lavis, bamus", "ham"),
("rivin Lako mupes lorir lolor", "ham")]
Input: judge_inbox(classify, 30, 1)
Output: a summary such as {"correct": 778, "total": 1000, "hidden_counts": {"spam": 306, "ham": 694}}
this one passes: always "ham" would be right on only 694 messages
Constraints
30 <= len(texts) <= 200,len(new_texts) = 1000; messages have 4 to 25 words- each test must finish in well under a second in your browser
- your predictions must not depend on the clock; if you use randomness, use a
random.Randomwith a fixed seed
Goals
- Build a word-based classifier from a small labelled inbox
- Beat the majority baseline on hidden messages
- Improve a model by experiment: text normalisation and the amount of smoothing