xfrm: allow avoiding buffer overflow for key in xfrmnl_sa_get_*_params()

xfrm: allow avoiding buffer overflow for key in xfrmnl_sa_get_*_params()

The previous API of xfrmnl_sa_get_*_params() would always require
a @key buffer, but it was not possible to avoid buffer overflow
because the required size was unknown.

That is not really fixable, because the old API is broken.

Now, allow omitting the @key argument to only request the @key_size.
That allows the caller to ask beforehand how large the @key buffer
must be: ((@key_size + 7) / 8).

Unfortunately, omitting the key against previous versions of libnl
leads to a crash. And passing a key against older versions makes it
impossible to avoid buffer-overflow.

Another option would be to add functions like
xfrmnl_sa_get_crypto_params_keylen() so the user can query the required
buffer size by calling that instead of xfrmnl_sa_get_crypto_params().
However, then the user also requires a backport of the new API
and this will not be possible against older libnl3 versions either.
Thus, if the user already requires the fix, he can just as well
require a backport of this patch and then safely call xfrmnl_sa_get_crypto_params()
without @key argument. This way has the advantage/disadvantage, that
it can detect the presence of the patch at runtime.

The cumbersome way to get it right would be:

    unsiged key_len;
    char *key;
    int r;

    if (!nl_has_capability(17 /*NL_CAPABILITY_XFRM_SA_KEY_SIZE*/)) {
        /* no way to use this API safely. Abort. */
        return -NLE_OPNOTSUPP;
    }

    r = xfrmnl_sa_get_crypto_params(sa, NULL, &key_len, NULL);
    if (r < 0)
         return r;
    key = malloc((key_len + 7) / 8);
    if (!key)
         return -NLE_NOMEM;
    r = xfrmnl_sa_get_crypto_params(sa, NULL, &key_len, &key);
    if (r < 0) {
        free(key);
        return r;
    }
    ...

http://lists.infradead.org/pipermail/libnl/2016-June/002155.html

Signed-off-by: Thomas Haller <thaller@redhat.com>