fontselect: fix undefined behavior with calloc

fontselect: fix undefined behavior with calloc

If allocations have the size zero, malloc/calloc implementations
can return a zero-size buffer or NULL. The earlier introduced malloc
checking blows up if an implementation returns NULL. Fix that by only
allocating and checking when it's actually needed.

Also fix a minor problem with iconv deinitialization in an error path.