An online shop predicts whether a visitor clicks on a banner (label 1) or not (label 0) with a logistic model: for features x, the click probability is p = σ(w·x + b) with σ(z) = 1 / (1 + e^(-z)). The model is trained by gradient descent on the average log loss.
Write logistic_step(X, y, w, b, lr) that performs one step of gradient descent from the parameters (w, b) with learning rate lr, on all the rows of X at once, and returns the new parameters as a tuple (new_w, new_b) (new_w a list). The partial derivatives of the average log loss are
∂L/∂w_j = mean over the rows of (p - y) · x_j ∂L/∂b = mean over the rows of (p - y)
where p is the model's probability for that row at the current parameters. The lists passed in must not be changed.
Examples
Input: X = [[1, 2], [2, 0], [0, 1]], y = [1, 0, 0], w = [0, 0], b = 0, lr = 0.5
Output: ([-0.08333333333333333, 0.08333333333333333], -0.08333333333333333)
Explanation: every p is 0.5, so the errors p - y are -0.5, 0.5 and 0.5. The gradient is
[(-0.5 + 1.0 + 0) / 3, (-1.0 + 0 + 0.5) / 3] = [1/6, -1/6] and 0.5 / 3 = 1/6 for b.
Input: X = [[0.5], [1], [1.5], [2], [2.5], [3], [3.5], [4], [4.5], [5]], y = [0, 0, 0, 1, 0, 1, 0, 1, 1, 1],
w = [0.0], b = 0.0, lr = 0.5
Output: ([0.2375], 0.0)
Constraints
1 <= len(X) <= 5000,1 <= d <= 20, labels are0or1,0 < lr <= 10|w·x + b| <= 30for every row at the given parameters- floats are compared with a tolerance of
1e-6
Goals
- Compute the predicted probabilities of a logistic model with several features
- Use the gradient of the average log loss, the mean of (p - y) times each feature
- Update all weights and the bias together from one gradient