tests: robustify ipc_msgbuf.test against broken libc

tests: robustify ipc_msgbuf.test against broken libc

glibc between commits glibc-2.25~130 and glibc-2.26~740 had broken
msgctl(IPC_RMID) on hppa: this operation always failed with EINVAL
because of inappropriate use of IPC_64 flag.  Similar issues were
fixed on other niche architectures.  Let's workaround these issues
by skipping the test in case of msgctl(IPC_RMID) failure.

* tests/ipc_msgbuf.c (cleanup): Change return type to int,
return 77 in case of msgctl(IPC_RMID) failure.
(main): Explicitly invoke cleanup() at the end.