Smooth l1 loss

\[ \ell(x, y) = L = \{l_1, ..., l_N\}^T \]

With

\[ l_n = \begin{cases} 0.5 (x_n - y_n)^2 / beta, & \text{if } |x_n - y_n| < beta \\ |x_n - y_n| - 0.5 * beta, & \text{otherwise } \end{cases} \]

If reduction is not none, then:

\[ \ell(x, y) = \begin{cases} \operatorname{mean}(L), & \text{if reduction} = \text{`mean';}\\ \operatorname{sum}(L), & \text{if reduction} = \text{`sum'.} \end{cases} \]

等有机会 plot 一下。