Notes
Empirical Risk Minimization = Minimize loss L as measured (empirically) on the data.
$$ \min_{w,b} \sum_{i=1}^m L(f_{w,b}(x^{(i)}), y^(i)) $$i.e. find parameters w,b that optimize this function.
Generally in regression, we have:
$$ $$And in classification, we have:
$$ (s(y))_j = \frac{e^{y_j}}{\sum_i e^{y_i}} \\ D(s,l) = -\sum_i l_i \log s_i $$