Stopping conditon of dual CD method for L1- and L2-loss SVM is changed

Currently the stopping condition is M-m <= \epsilon, where

M = max proj_gard(alpha) and m = min proj_grad(alpha).

This condition works well in practice, but we must ensure that the
following situation does not occur.

  M, m >> 0  or M,m << 0
but
  M - m <= \epsilon.

From the optimality condition, |M| and |m| must be close to zero.
For example, because

  grad_i f(0) = -1, for all i.

M=m= -1 and they satisfy M-m <= \epsilon.  Thus we add |M| <=\epsilon
and |m| <= \epsilon$ into the stopping condition.  In fact we can use
only these two inequalities as the stopping condition, but for
historical reasons we keep M-m <= \epsilon as the main condition to
check.

diff --git a/linear.cpp b/linear.cpp
index 86ec3dd9eefc6e33cc90f16a85029c870e5cf97b..5426533064fa9dd6525b27ae46253f3cae5f945b 100644 (file)
--- a/linear.cpp
+++ b/linear.cpp
@@ -1025,7 +1025,8 @@ static int solve_l2r_l1l2_svc(const problem *prob, const parameter *param, doubl
                if(iter % 10 == 0)
                        info(".");
 
-               if(PGmax_new - PGmin_new <= eps)
+               if(PGmax_new - PGmin_new <= eps &&
+                       fabs(PGmax_new) <= eps && fabs(PGmin_new) <= eps)
                {
                        if(active_size == l)
                                break;