Guard against incoming rowcount estimate of NaN in cost_mergejoin().

Guard against incoming rowcount estimate of NaN in cost_mergejoin().

Although rowcount estimates really ought not be NaN, a bug elsewhere
could perhaps result in that, and that would cause Assert failure in
cost_mergejoin, which I believe to be the explanation for bug #5977 from
Anton Kuznetsov.  Seems like a good idea to expend a couple more cycles
to prevent that, even though the real bug is elsewhere.  Not back-patching,
though, because we don't encourage running production systems with
Asserts on.