emove use of persistent buffer in extokens

emove use of persistent buffer in extokens

This removes a layer of indirection. Previously data was being written to the
persistent scratch space, ex->tmp, and then copied to a further temporary buffer
in tok. We now simply allocate to tok and write directly into it. This also
addresses an issue where the persistent buffer was being unconditionally used,
despite that it may have failed to be opened due to lack of memory.

Related to #1873.