PostgreSQL Bugs

Collected from the PG bugs email list.

Bug ID16233
PG Version12.1
OSUbuntu 18.04.2
Opened2020-01-26 06:46:00+00
Reported byJohann Du Toit
StatusNew

Body of first available message related to this bug follows.

The following bug has been logged on the website:

Bug reference:      16233
Logged by:          Johann Du Toit
Email address:      (redacted)
PostgreSQL version: 12.1
Operating system:   Ubuntu 18.04.2
Description:        

Hi All,

I browsed around and noticed a few people having similar issues with logical
replication.

To my eyes it looks similar to the issue here:
https://www.postgresql.org/message-id/flat/16129-a0c0f48e71741e5f%40postgresql.org
which was fixed here:
https://git.postgresql.org/gitweb/?p=postgresql.git&a=commit&h=a2aa224e

I'm trying to replicate some tables across from a 11.6 publisher to 12.1.
The database is about 250GB in size, and the segfaults start happening about
20GB in. I've retried it several times and it happens on different tables
each time... but it is the larger and probably more active tables - so there
might be inserts and updates happening for that one on the publisher.

I guess I just want someone with a more expert eye to look over the trace
below as well and confirm that it is indeed the same as the above issue.

When is 12.2 due out? Feb 13? Are there any daily Ubuntu builds available
anywhere so I can test the patch for my purposes, before I try to compile
from scratch?

Here's the error log:
2020-01-26 05:24:40 UTC LOG:  logical replication table synchronization
worker for subscription "upgradesub1", table "simpro_recurringinvoice" has
finished
2020-01-26 05:24:40 UTC LOG:  logical replication table synchronization
worker for subscription "upgradesub1", table "federated_banktransactionitem"
has started
2020-01-26 05:31:50 UTC LOG:  background worker "logical replication worker"
(PID 5953) was terminated by signal 11: Segmentation fault
2020-01-26 05:31:50 UTC LOG:  terminating any other active server
processes
2020-01-26 05:31:50 UTC WARNING:  terminating connection because of crash of
another server process
2020-01-26 05:31:50 UTC DETAIL:  The postmaster has commanded this server
process to roll back the current transaction and exit, because another
serve
r process exited abnormally and possibly corrupted shared memory.
2020-01-26 05:31:50 UTC HINT:  In a moment you should be able to reconnect
to the database and repeat your command.
2020-01-26 05:31:50 UTC WARNING:  terminating connection because of crash of
another server process
2020-01-26 05:31:50 UTC DETAIL:  The postmaster has commanded this server
process to roll back the current transaction and exit, because another
serve
r process exited abnormally and possibly corrupted shared memory.
2020-01-26 05:31:50 UTC HINT:  In a moment you should be able to reconnect
to the database and repeat your command.
2020-01-26 05:31:50 UTC LOG:  archiver process (PID 223) exited with exit
code 1
2020-01-26 05:31:50 UTC LOG:  all server processes terminated;
reinitializing
2020-01-26 05:31:51 UTC LOG:  database system was interrupted; last known up
at 2020-01-26 05:26:40 UTC
2020-01-26 05:31:52 UTC LOG:  recovered replication state of node 1 to
928/5A36E320
2020-01-26 05:31:52 UTC LOG:  database system was not properly shut down;
automatic recovery in progress
2020-01-26 05:31:52 UTC LOG:  redo starts at 3/D7CD028
2020-01-26 05:32:02 UTC LOG:  redo done at 3/5F7B3FC0
2020-01-26 05:32:04 UTC LOG:  database system is ready to accept
connections
2020-01-26 05:32:04 UTC LOG:  logical replication apply worker for
subscription "upgradesub1" has started
2020-01-26 05:32:04 UTC LOG:  logical replication table synchronization
worker for subscription "upgradesub1", table "federated_banktransactionitem"
has started


And here are the stack traces from the core dump:

Reading symbols from /usr/lib/postgresql/12/bin/postgres...Reading symbols
from
/usr/lib/debug/.build-id/29/96ede8876d910de930d48171662af5702b63e0.debug...done.
done.
[New LWP 5953]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `postgres: logical replication worker for subscription
34353                   '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  __memcpy_avx_unaligned () at
../sysdeps/x86_64/multiarch/memcpy-avx-unaligned.S:245
245	../sysdeps/x86_64/multiarch/memcpy-avx-unaligned.S: No such file or
directory.
#0  __memcpy_avx_unaligned () at
../sysdeps/x86_64/multiarch/memcpy-avx-unaligned.S:245
#1  0x00005580e72787dc in memcpy (__len=227612305, __src=<optimized out>,
__dest=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/string3.h:53
#2  tts_virtual_materialize (slot=0x5580e8a03810) at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/executor/execTuples.c:235
#3  0x00005580e7279deb in ExecFetchSlotHeapTuple (slot=0x5580e8a03810,
materialize=<optimized out>, shouldFree=0x7ffd07b5b4d7)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/executor/execTuples.c:1624
#4  0x00005580e710ad2c in heapam_tuple_update (relation=0x7f43828885d8,
otid=0x5580e8a03aa0, slot=0x5580e8a03810, cid=0, snapshot=<optimized out>,

    crosscheck=0x0, wait=true, tmfd=0x7ffd07b5b560, lockmode=0x7ffd07b5b55c,
update_indexes=0x7ffd07b5b5d7)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/access/heap/heapam_handler.c:325
#5  0x00005580e713ad56 in table_tuple_update (update_indexes=0x7ffd07b5b5d7,
lockmode=0x7ffd07b5b55c, tmfd=0x7ffd07b5b560, wait=true, 
    crosscheck=0x0, snapshot=0x0, cid=<optimized out>, slot=0x5580e8a03810,
otid=0x5580e8a03aa0, rel=0x7f43828885d8)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/include/access/tableam.h:1261
#6  simple_table_tuple_update (rel=0x7f43828885d8,
otid=otid@entry=0x5580e8a03aa0, slot=slot@entry=0x5580e8a03810,
snapshot=0x0, 
    update_indexes=update_indexes@entry=0x7ffd07b5b5d7)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/access/table/tableam.c:325
#7  0x00005580e7275dd2 in ExecSimpleRelationUpdate
(estate=estate@entry=0x5580e8a438e0, epqstate=epqstate@entry=0x7ffd07b5b6b0,

    searchslot=searchslot@entry=0x5580e8a03a70,
slot=slot@entry=0x5580e8a03810)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/executor/execReplication.c:493
#8  0x00005580e7376194 in apply_handle_update (s=s@entry=0x7ffd07b62ce0)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/replication/logical/worker.c:750
#9  0x00005580e7376f98 in apply_dispatch (s=0x7ffd07b62ce0)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/replication/logical/worker.c:968
#10 LogicalRepApplyLoop (last_received=10068998058648)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/replication/logical/worker.c:1176
#11 ApplyWorkerMain (main_arg=<optimized out>)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/replication/logical/worker.c:1734
#12 0x00005580e7344c08 in StartBackgroundWorker () at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/postmaster/bgworker.c:834
#13 0x00005580e73520f0 in do_start_bgworker (rw=<optimized out>)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/postmaster/postmaster.c:5770
#14 maybe_start_bgworkers () at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/postmaster/postmaster.c:5996
#15 0x00005580e7352d65 in sigusr1_handler (postgres_signal_arg=<optimized
out>)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/postmaster/postmaster.c:5167
#16 <signal handler called>
#17 0x00007f47a46de5b3 in __select_nocancel () at
../sysdeps/unix/syscall-template.S:84
#18 0x00005580e70c1d85 in ServerLoop () at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/postmaster/postmaster.c:1668
#19 0x00005580e7354274 in PostmasterMain (argc=5, argv=<optimized out>)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/postmaster/postmaster.c:1377
#20 0x00005580e70c3909 in main (argc=5, argv=0x5580e8945820) at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/main/main.c:228


==========
bt full

#0  __memcpy_avx_unaligned () at
../sysdeps/x86_64/multiarch/memcpy-avx-unaligned.S:245
No locals.
#1  0x00005580e72787dc in memcpy (__len=227612305, __src=<optimized out>,
__dest=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/string3.h:53
No locals.
#2  tts_virtual_materialize (slot=0x5580e8a03810) at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/executor/execTuples.c:235
        data_length = 227612305
        val = <optimized out>
        natt = 20
        vslot = 0x5580e8a03810
        desc = <optimized out>
        sz = <optimized out>
        data = <optimized out>
#3  0x00005580e7279deb in ExecFetchSlotHeapTuple (slot=0x5580e8a03810,
materialize=<optimized out>, shouldFree=0x7ffd07b5b4d7)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/executor/execTuples.c:1624
No locals.
#4  0x00005580e710ad2c in heapam_tuple_update (relation=0x7f43828885d8,
otid=0x5580e8a03aa0, slot=0x5580e8a03810, cid=0, snapshot=<optimized out>,

    crosscheck=0x0, wait=true, tmfd=0x7ffd07b5b560, lockmode=0x7ffd07b5b55c,
update_indexes=0x7ffd07b5b5d7)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/access/heap/heapam_handler.c:325
        shouldFree = true
        tuple = <optimized out>
        result = <optimized out>
#5  0x00005580e713ad56 in table_tuple_update (update_indexes=0x7ffd07b5b5d7,
lockmode=0x7ffd07b5b55c, tmfd=0x7ffd07b5b560, wait=true, 
    crosscheck=0x0, snapshot=0x0, cid=<optimized out>, slot=0x5580e8a03810,
otid=0x5580e8a03aa0, rel=0x7f43828885d8)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/include/access/tableam.h:1261
No locals.
#6  simple_table_tuple_update (rel=0x7f43828885d8,
otid=otid@entry=0x5580e8a03aa0, slot=slot@entry=0x5580e8a03810,
snapshot=0x0, 
    update_indexes=update_indexes@entry=0x7ffd07b5b5d7)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/access/table/tableam.c:325
        tmfd = {ctid = {ip_blkid = {bi_hi = 40232, bi_lo = 33416}, ip_posid
= 32579}, xmax = 2189986040, cmax = 32579, traversed = 216}
        lockmode = 21888
        __func__ = "simple_table_tuple_update"
#7  0x00005580e7275dd2 in ExecSimpleRelationUpdate
(estate=estate@entry=0x5580e8a438e0, epqstate=epqstate@entry=0x7ffd07b5b6b0,

    searchslot=searchslot@entry=0x5580e8a03a70,
slot=slot@entry=0x5580e8a03810)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/executor/execReplication.c:493
        recheckIndexes = 0x0
        update_indexes = false
        resultRelInfo = 0x5580e8a03490
        rel = <optimized out>
        tid = 0x5580e8a03aa0
#8  0x00005580e7376194 in apply_handle_update (s=s@entry=0x7ffd07b62ce0)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/replication/logical/worker.c:750
        rel = 0x5580e8b3bcb8
        relid = <optimized out>
        idxoid = <optimized out>
        estate = <optimized out>
        epqstate = {parentestate = 0x5580e8a438e0, epqParam = -1,
tuple_table = 0x0, relsubs_slot = 0x5580e8a03ca0, plan = 0x0, arowMarks =
0x0, 
          origslot = 0x5580e8a03810, recheckestate = 0x0, relsubs_rowmark =
0x0, relsubs_done = 0x0, recheckplanstate = 0x0}
        oldtup = {values = {0x0 <repeats 196 times>, 0x27c1434 <error:
Cannot access memory at address 0x27c1434>, 
            0x24002f6249d52 <error: Cannot access memory at address
0x24002f6249d52>, 0x1 <error: Cannot access memory at address 0x1>, 0x0,
0x0, 
            0x7f44965f39a4
"\021COMPANYn\203\032\016\065LN\236\271\373\a]D~\317K\244", 
            0x7f44965f39ac "n\203\032\016\065LN\236\271\373\a]D~\317K\244",
0xa4 <error: Cannot access memory at address 0xa4>, 
            0x0 <repeats 233 times>, 0x5580e75378d4 <dostr+116>
"L\001;E\205\344\177\310H\203\304\b[A\\A]A^A_]\303\017\037D", 0x0, 
            0x5 <error: Cannot access memory at address 0x5>, 0x3b <error:
Cannot access memory at address 0x3b>, 0x7ffd07b5cc10 "j7\240\350\200U", 
            0x0, 0x7ffd07b5cc10 "j7\240\350\200U", 0x7ffd07b5c580 "",
0x5580e7537a22 <fmtint+242> "\213u\214\205\366y\027\367\336L\211\352\277 ",

            0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3538333631000000
<error: Cannot access memory at address 0x3538333631000000>, 0x0, 
            0xe01d63a3a2eb5700 <error: Cannot access memory at address
0xe01d63a3a2eb5700>, 0x0, 0x0, 0x5580e7679dd2 "%u", 
            0x5580e75378d4 <dostr+116>
"L\001;E\205\344\177\310H\203\304\b[A\\A]A^A_]\303\017\037D", 0x7ffd07b5cc00
"P\314\265\a\375\177", 
            0x5 <error: Cannot access memory at address 0x5>, 0x3b <error:
Cannot access memory at address 0x3b>, 0x7ffd07b5ccd0 "t7\240\350\200U", 
            0x0, 0x7ffd07b5ccd0 "t7\240\350\200U", 0x7ffd07b5c640
"esl\347\200U", 
            0x5580e7537a22 <fmtint+242>
"\213u\214\205\366y\027\367\336L\211\352\277 ", 0x0, 
            0x7b5c6b0 <error: Cannot access memory at address 0x7b5c6b0>,
0x7ffd07b5c630 "\230sl\347\200U", 
            0x200000000 <error: Cannot access memory at address
0x200000000>, 0x7f47a6d647c8 "", 0x7ffd07b5c6b0 "", 0x0, 
            0x3435393632000000 <error: Cannot access memory at address
0x3435393632000000>, 0x0, 
            0x3435393632000000 <error: Cannot access memory at address
0x3435393632000000>, 0x7ffd07b5cc90 "\020", 
            0xe01d63a3a2eb5700 <error: Cannot access memory at address
0xe01d63a3a2eb5700>, 0x0, 
            0x5580e75378d4 <dostr+116>
"L\001;E\205\344\177\310H\203\304\b[A\\A]A^A_]\303\017\037D", 0x5580e76c7398
"%u/%u_%s", 0x5580e76c739e "%s", 
            0x5580e76c7365 "fsm", 0x5580e76c7398 "%u/%u_%s", 0x5580e76c739d
"_%s", 0x7ffd07b5ccd0 "t7\240\350\200U", 
            0x7ffd07b5ccc0 "\020\315\265\a\375\177", 0x5580e7537d88
<dopr+616> "A\200\177$", 0x0, 0x7ffd07b5c760 "", 0x7ffd07b5c6e0 "", 
            0x7ffd07b5c770 "", 0x7ffd07b5c6f0 "", 0x207b5c760 <error: Cannot
access memory at address 0x207b5c760>, 0x7f47a6d647c8 "", 
            0x7ffd07b5c770 "", 0x0, 0x0, 0x7b5cd00 <error: Cannot access
memory at address 0x7b5cd00>, 0x0, 0x7ffd07b5cd50 "\a", 
            0x0 <repeats 158 times>, 0xe01d63a3a2eb5700 <error: Cannot
access memory at address 0xe01d63a3a2eb5700>, 0x0, 
            0x80 <error: Cannot access memory at address 0x80>,
0x5580e7679dcd "base/%u", 0x5580e8a03760 "", 
            0x2 <error: Cannot access memory at address 0x2>, 0x1 <error:
Cannot access memory at address 0x1>, 
            0x7ffd07b5cc50 "p\314\265\a\375\177", 0x5580e7538e09
<pg_vsnprintf+89> "H\213E\300\306", 0x5580e8a0376a "", 0x5580e8a03760 "", 
            0x5580e8a037df "l\020", 0x0, 0x550000000000 <error: Cannot
access memory at address 0x550000000000>, 
            0x80 <error: Cannot access memory at address 0x80>, 0x0,
0xe01d63a3a2eb5700 <error: Cannot access memory at address
0xe01d63a3a2eb5700>, 
            0x7ffd07b5cc70 "\220\315\265\a\375\177", 
            0x5580e753e9a2 <pvsnprintf+18>
"\205\300xjHc\320H9\323w\r\215P\001=\376\377\377?Hc\322\177\r[H\211\320A\\]\303\017\037D",

            0x80 <error: Cannot access memory at address 0x80>,
0x7f47a6d647c8 "", 0x7ffd07b5cd90 "\200y\226\350\200U", 
            0x5580e753eb40 <psprintf+256>
"H9\303I\211\307v\230H\213\225\030\377\377\377dH3\024%(", 0x7ffd07b5cc90
"\020", 
            0xe01d63a3a2eb5700 <error: Cannot access memory at address
0xe01d63a3a2eb5700>, 
            0x3000000010 <error: Cannot access memory at address
0x3000000010>, 0x80 <error: Cannot access memory at address 0x80>, 
            0x5580e76c7393 "base/%u/%u_%s", 0x5580e8a03760 "", 0x2 <error:
Cannot access memory at address 0x2>, 
            0x1 <error: Cannot access memory at address 0x1>, 0x7ffd07b5cd10
"P\315\265\a\375\177", 
            0x5580e7538e09 <pg_vsnprintf+89> "H\213E\300\306",
0x5580e8a03774 "", 0x5580e8a03760 "", 0x5580e8a037df "l\020", 0x0, 
            0x550000000000 <error: Cannot access memory at address
0x550000000000>, 0x15 <error: Cannot access memory at address 0x15>, 
            0x1 <error: Cannot access memory at address 0x1>,
0xe01d63a3a2eb5700 <error: Cannot access memory at address
0xe01d63a3a2eb5700>, 
            0x7ffd07b5cd50 "\a", 0x5580e73bd1e2 <LWLockAcquire+82>
"\204\300\017\204\305", 0x80 <error: Cannot access memory at address 0x80>,

            0x5580e74ff066 <hash_search_with_hash_value+214>
"\205\300u\326f\017\037D", 0x200000 <error: Cannot access memory at address
0x200000>, 
            0x3 <error: Cannot access memory at address 0x3>, 0x7ffd07b5ce70
"\240\316\265\a\375\177", 
            0x5580e74ff066 <hash_search_with_hash_value+214>
"\205\300u\326f\017\037D", 
            0x7ffd00000007 <error: Cannot access memory at address
0x7ffd00000007>, 0x0, 
            0x200b5cd70 <error: Cannot access memory at address
0x200b5cd70>, 0x7 <error: Cannot access memory at address 0x7>,
0x7f4700000000 "", 
            0x0, 0x20000694a <error: Cannot access memory at address
0x20000694a>, 0x0, 0x5580e8967980 "\001@", 
            0x14 <error: Cannot access memory at address 0x14>,
0x7ffd07b5cdd0 "\360\315\265\a\375\177", 
            0xffffffff <error: Cannot access memory at address 0xffffffff>,
0x5580e8967980 "\001@", 0x5580e7977ac0 <default_lockmethod> "\b", 
            0x7 <error: Cannot access memory at address 0x7>, 0x7f479a4e7970
"@\216G\232G\177", 0x7ffd07b5cdf0 "p\316\265\a\375\177", 
            0x5580e73b4ab4 <RemoveLocalLock+196>
"H\205\300t([A\\]\303f\220H\213=I\302_", 0x1 <error: Cannot access memory at
address 0x1>, 
            0x5580e8967980 "\001@", 0x7ffd07b5ce70 "\240\316\265\a\375\177",
0x5580e73b6cf2 <LockRelease+594> "\351\336\376\377\377f\017\037\204", 
            0xa00000900000000 <error: Cannot access memory at address
0xa00000900000000>, 0x7f479a478e40 "\362e\001", 
            0x7ffd07b5ce70 "\240\316\265\a\375\177", 0x0, 0x694a00004001
<error: Cannot access memory at address 0x694a00004001>, 
            0x101000000000000 <error: Cannot access memory at address
0x101000000000000>, 
            0x7ffd00000007 <error: Cannot access memory at address
0x7ffd00000007>, 
            0xe01d63a3a2eb5700 <error: Cannot access memory at address
0xe01d63a3a2eb5700>, 0x0, 0x3 <error: Cannot access memory at address 0x3>,

            0x7ffd07b5ceb0 "", 0x7f47a6d40400 "\177\006", 0x3 <error: Cannot
access memory at address 0x3>, 
            0x1 <error: Cannot access memory at address 0x1>, 0x7ffd07b5cea0
"\340\356\265\a\375\177", 
            0x5580e73b3b53 <UnlockRelationForExtension+67>
"H\213E\370dH3\004%(", 
            0x694a00004001 <error: Cannot access memory at address
0x694a00004001>, 
            0x101000000000000 <error: Cannot access memory at address
0x101000000000000>, 0x3 <error: Cannot access memory at address 0x3>, 
            0xe01d63a3a2eb5700 <error: Cannot access memory at address
0xe01d63a3a2eb5700>, 0x7ffd07b5eee0 "\300\065\240\350\200U", 
            0x5580e73a500d <fsm_extend+189> "H\213E\330dH3\004%(", 0x0...},
changed = {false <repeats 13 times>, 127, 2, false, 96, 55, 160, 232, 
            128, 85, false, false, false, 40, 18, 231, 128, 85, false,
false, 16, false, false, false, false, false, false, false, false, 53, 8, 
            false, false, false, false, false, 96, 55, 160, 232, 128, 85,
false, false, false, 87, 235, 162, 163, 99, 29, 224, 94, true, false, 
            false, false, false, false, false, 80, 52, 160, 232, 128, 85,
false, false, 176, 235, 181, 7, 30, 53, 8, false, 200, 14, 170, 130, 67, 
            127, false, false, 208, 54, 160, 232, 128, 85, false, false, 96,
55, 160, 232, true, false <repeats 15 times>, 68, 127, false, false, 
            94, true, 170, 130, 67, 127, false, false, 93, true, false,
false, false, false, false, false, 93, true, false, false, false, false, 
            false, false, 94, true, false, false, false, false, false,
false, 16, 236, 181, 7, 253, 127, false, false, 64, 45, 18, 231, 128, 85, 
            false, false, true, false, false, false, 9, false, false, false,
false, 87, 235, 162, 163, 99, 29, 224, 112, 41, 103, 147, 68, 127, 94, 
            true, 112, 41, 103, 147, 68, 127, false, false...}}
        newtup = {values = {0x5580e8a02c90 "164", 0x0, 0x0, 0x0,
0x5580e8a02ca8 "t", 0x5580e8a02cc0 "t", 0x5580e8a02cd8 "t", 0x0, 
            0x5580e8a02cf0 "2020-01-26 03:31:32.968235+00", 0x5580e8a02d20
"{}", 0x5580e8a02d38 "t", 0x0, 
            0x5580e8a02d50
"EAAVmqC0nGM8BAEfChlxxjUzIRlR0uu0cPZC8KWl3F6ve556GaBYZBCLpD8qVoMx2qRGIZAGvpIPjZAtTJQaxP0oZAZBzufZB0NJt6JPdVQe69iFzlAuZAL2mddvoGyikNr5MLZBDB3bz9OxOQZAG1tmZCz2BM3yIdmyFCC0ZBZB6lywjGBQZDZD",
0x0, 0x0, 0x5580e8a02e60 "2019-03-04 01:48:57.081209+00", 
            0x5580e8a02e90 "2019-08-15 23:16:31.514461+00", 
            0x5580e8a02ec0 "{\"act_1377232949260487\": {\"id\":
\"act_1377232949260487\", \"name\": \"See Member\", \"fail_count\": 0}}", 
            0x5580e8a02f50 "{\"act_1377232949260487\": {\"id\":
\"act_1377232949260487\", \"name\": \"See Member\"},
\"act_1384380421774161\": {\"id\": \"act_1384380421774161\", \"name\":
\"1384380421774161\"}}", 
            0x5580e8a03060 "{\"139196079609435\": {\"id\":
\"139196079609435\", \"name\": \"Ponman Constructions\", \"fail_count\": 0,
\"access_token\":
\"EAAVmqC0nGM8BAOGaCEi1ChQGPTNLTtM4trDTZBZB2KCqbmze5tzJi7IeNxQuZBO8YZCrz7MzJG8GyBeTfyzz5UN"...,
0x0, 0x5580e8a03270 "364", 0x0, 
            0x5580e8a03288 "{}", 0x0, 0x5580e8a032a0 "0", 0x0,
0x5580e8a032b8 "act_1377232949260487", 0x5580e8a032e8 "0", 0x0 <repeats 22
times>, 
            0x5580e8a03350 "", 0x0 <repeats 39 times>, 0x7f47a465c6ce
<__GI__IO_default_xsputn+158> "I\211D$(\353\275\017\037", 
            0x7ffd07b5fa20
"\216Q\222\021-\002j\206m\301,\\\355\307\234.L\346\361\351%7\327&\365\335\245\256:\232\021\203\374\304:\313\322\315P:\243\220b\275^\227\201p\343\230}eY\275_=\310\316/\254\345\021\033\065\t\340*\206\211B\336\064\231\f\022\247h0\331+\vB\334vFN8\224\322\307\255\227\345\342O\324\364`\331",

            0x7ffd07b5fa10
"mNv\355\252\320{o\030aEjENt\305\216Q\222\021-\002j\206m\301,\\\355\307\234.L\346\361\351%7\327&\365\335\245\256:\232\021\203\374\304:\313\322\315P:\243\220b\275^\227\201p\343\230}eY\275_=\310\316/\254\345\021\033\065\t\340*\206\211B\336\064\231\f\022\247h0\331+\vB\334vFN8\224\322\307\255\227\345\342O\324\364`\331",
0x7f47a3d2a6fe "Key table file '%s' not found", 
            0x7ffd07b5fb90
"\222\357\372\215\325\311\207\303\240\303\a\v\265\240\332\347\062_M\365`F\354\246\272P\324%\244.+6\200\245yr[j\026\231\030\241\362\327\360\207k\017\033\276\276i{\r\276/\304%\270\220\276\062\"Y\337G/\266\334\062\034\307\362\070\024,(\267]\257
\023@\346\272E\205V\265p\240\221\033?\031\226\212\335\353\004\300k\272\257]5L\263\261\067\334\bB\002\003\243\230\252\a\330\276opE\001[\203\022C\321\030Gu.\370@\"\r`\377\315\177\221\353\214\262\344N\276\205\061$\342\264\377\325\303}\fU\262\vD\216\213n-\b\314/\033\234>p\225[o\211{\362t]\276r\261\226\026;\376\261\336\200t-q\016\211\350.t"...,
0x0, 
            0x7f47a462e92d <_IO_vfprintf_internal+1981>
"H\213\225h\373\377\377H\213\215\060\373\377\377H\211\326H)\316H9\360uND\213\215(\373\377\377\271\377\377\377\177D)\311Hc\311H9\310\017\217\215",
0x0, 0x7f47a465c6ce <__GI__IO_default_xsputn+158>
"I\211D$(\353\275\017\037", 
            0x7ffd07b5fa60
"\t\340*\206\211B\336\064\231\f\022\247h0\331+\vB\334vFN8\224\322\307\255\227\345\342O\324\364`\331",

            0x7ffd07b5fa50
"\343\230}eY\275_=\310\316/\254\345\021\033\065\t\340*\206\211B\336\064\231\f\022\247h0\331+\vB\334vFN8\224\322\307\255\227\345\342O\324\364`\331",
0x7f47a3d579d3 "%s:%s", 
            0x7ffd07b5fbd8 "\362\070\024,(\267]\257
\023@\346\272E\205V\265p\240\221\033?\031\226\212\335\353\004\300k\272\257]5L\263\261\067\334\bB\002\003\243\230\252\a\330\276opE\001[\203\022C\321\030Gu.\370@\"\r`\377\315\177\221\353\214\262\344N\276\205\061$\342\264\377\325\303}\fU\262\vD\216\213n-\b\314/\033\234>p\225[o\211{\362t]\276r\261\226\026;\376\261\336\200t-q\016\211\350.t\305
\337\204@\204\304\016\065\022\307%\247\006\334\233\024,i\317t\361\233\301c:ye\250g\207\002R\255\304\333\030\372\300-\030\064\305Q\304_`lq\276\314\356\347\345\374\327\376\343\024\357|(#.yj\216\224\246Dnz"...,
0x0, 
            0x7f47a462e92d <_IO_vfprintf_internal+1981>
"H\213\225h\373\377\377H\213\215\060\373\377\377H\211\326H)\316H9\360uND\213\215(\373\377\377\271\377\377\377\177D)\311Hc\311H9\310\017\217\215",
0x0, 0x7f47a465c6ce <__GI__IO_default_xsputn+158>
"I\211D$(\353\275\017\037", 
            0x7ffd07b5faa0
"&\372\320\232\204efa|\321Y\323\226[1\301\250\306\213\305\063\022\237`\247\257",

            0x7ffd07b5fa90 "Y
|A\034~\032\220(\004\004\232PM\004\342&\372\320\232\204efa|\321Y\323\226[1\301\250\306\213\305\063\022\237`\247\257",

            0x7f47a3d2a6d8 "Keytab %s is nonexistent or empty",
0x7f47a461a232 <msort_with_tmp+946> "\205\300\177\312I\213E", 0x0, 0x0, 
            0x8 <error: Cannot access memory at address 0x8>, 
            0x7ffd07b5fc50 "t-q\016\211\350.t\305
\337\204@\204\304\016\065\022\307%\247\006\334\233\024,i\317t\361\233\301c:ye\250g\207\002R\255\304\333\030\372\300-\030\064\305Q\304_`lq\276\314\356\347\345\374\327\376\343\024\357|(#.yj\216\224\246Dnz\t\205\314\266\001\335\016dc\\L\200u\245\343\361XFm\234\335\274\257Z21,]\356\066\367\275\223\303\217\265@\332\064\331\235G\367\331\067\001\350\300\031?\031M\201\337\250\203\060d\222JBm\034M\233O\031\257\244\202?\022\030\201m\332\325^\034\253(\256\340ef\231\322\214\250B\347\247u\201\371\033\240U\242\350\200U",
0x0, 
            0x5580e8a10ba8 "\320\266?\246G\177", 0x7ffd07b5f5f8
"\230;4\203C\177", 0x100000000 <error: Cannot access memory at address
0x100000000>, 
            0x7ffd00000000 <error: Cannot access memory at address
0x7ffd00000000>, 0x2 <error: Cannot access memory at address 0x2>, 
            0xffffffff00000000 <error: Cannot access memory at address
0xffffffff00000000>, 0x8 <error: Cannot access memory at address 0x8>, 
            0x2 <error: Cannot access memory at address 0x2>, 0x2 <error:
Cannot access memory at address 0x2>, 
            0x320 <error: Cannot access memory at address 0x320>,
0x5580e8a108b0 "\220\215?\246G\177", 0x5580e8a10ba8 "\320\266?\246G\177", 
            0x7f47a461a1dc <msort_with_tmp+860>
"\351\b\377\377\377\017\037\200", 
            0xffffffff00000000 <error: Cannot access memory at address
0xffffffff00000000>, 0x0, 0x8 <error: Cannot access memory at address 0x8>,

            0x7f43833436e4 <dostr+116>
"L\001;E\205\344\177\310H\203\304\b[A\\A]A^A_]\303\017\037D", 0x7f47a3d2a6e1
" is nonexistent or empty", 
            0x7f4383346264 "%s", 0x7f4383349a66
".postgresql/postgresql.key", 0x7f4383346261 "%s/%s", 0x7f4383346263 "/%s",

            0x7ffd07b5fc60
"5\022\307%\247\006\334\233\024,i\317t\361\233\301c:ye\250g\207\002R\255\304\333\030\372\300-\030\064\305Q\304_`lq\276\314\356\347\345\374\327\376\343\024\357|(#.yj\216\224\246Dnz\t\205\314\266\001\335\016dc\\L\200u\245\343\361XFm\234\335\274\257Z21,]\356\066\367\275\223\303\217\265@\332\064\331\235G\367\331\067\001\350\300\031?\031M\201\337\250\203\060d\222JBm\034M\233O\031\257\244\202?\022\030\201m\332\325^\034\253(\256\340ef\231\322\214\250B\347\247u\201\371\033\240U\242\350\200U",

            0x7ffd07b5fc50 "t-q\016\211\350.t\305
\337\204@\204\304\016\065\022\307%\247\006\334\233\024,i\317t\361\233\301c:ye\250g\207\002R\255\304\333\030\372\300-\030\064\305Q\304_`lq\276\314\356\347\345\374\327\376\343\024\357|(#.yj\216\224\246Dnz\t\205\314\266\001\335\016dc\\L\200u\245\343\361XFm\234\335\274\257Z21,]\356\066\367\275\223\303\217\265@\332\064\331\235G\367\331\067\001\350\300\031?\031M\201\337\250\203\060d\222JBm\034M\233O\031\257\244\202?\022\030\201m\332\325^\034\253(\256\340ef\231\322\214\250B\347\247u\201\371\033\240U\242\350\200U",

            0x7f4383343b98 <dopr+616> "A\200\177$", 0x4 <error: Cannot
access memory at address 0x4>, 
            0x3 <error: Cannot access memory at address 0x3>, 0x320 <error:
Cannot access memory at address 0x320>, 
            0x7ffd07b5f700 "\370\004\340\246G\177", 0x7ffd07b5f680 "",
0x2a461a232 <error: Cannot access memory at address 0x2a461a232>, 
            0x7f47a6d647c8 "", 0x0, 0x8 <error: Cannot access memory at
address 0x8>, 
            0x7ffd07b5fc00
"B\002\003\243\230\252\a\330\276opE\001[\203\022C\321\030Gu.\370@\"\r`\377\315\177\221\353\214\262\344N\276\205\061$\342\264\377\325\303}\fU\262\vD\216\213n-\b\314/\033\234>p\225[o\211{\362t]\276r\261\226\026;\376\261\336\200t-q\016\211\350.t\305
\337\204@\204\304\016\065\022\307%\247\006\334\233\024,i\317t\361\233\301c:ye\250g\207\002R\255\304\333\030\372\300-\030\064\305Q\304_`lq\276\314\356\347\345\374\327\376\343\024\357|(#.yj\216\224\246Dnz\t\205\314\266\001\335\016dc\\L\200u\245\343\361XFm\234\335\274\257Z21,]\356\066\367\275\223\303\217\265@\332\064\331"...,

            0x7ffd07b5f6b0 "H\371\265\a\375\177", 0x5580e8a10b80
"\030\265?\246G\177", 
            0x7ffd07b5fcb0
"XFm\234\335\274\257Z21,]\356\066\367\275\223\303\217\265@\332\064\331\235G\367\331\067\001\350\300\031?\031M\201\337\250\203\060d\222JBm\034M\233O\031\257\244\202?\022\030\201m\332\325^\034\253(\256\340ef\231\322\214\250B\347\247u\201\371\033\240U\242\350\200U",
0x0, 
            0x0, 0x7 <error: Cannot access memory at address 0x7>, 0x0, 0x8
<error: Cannot access memory at address 0x8>, 
            0x7 <error: Cannot access memory at address 0x7>, 0x6 <error:
Cannot access memory at address 0x6>, 
            0x320 <error: Cannot access memory at address 0x320>,
0x5580e8a108b0 "\220\215?\246G\177", 
            0x7ffd07b5f948
"{L\323$P\374\063\210\064\344Z\330\317\177\214\002\r\221\366\203\242\337\226\224\214\206\025\212\376z\215\353\331\265b\220\317\253\251\266#\371\262\225\343\006<a\315e9\367\064V\335E\037\256\265!0\266c`\344\261v{\r\372\357\aomeT\216$oa\021\226G\357\253\350\037\214\312MF\221zME=\256d&\a[\251Lg;\306&\250\300\003\335'\b\311\351\212\v\263`BF\337\327q\264V\214\312\262\231\023\200ip\344;`\331\335\325Dn1\223.]f$#\304\347\066O\235?\273\325\027h\305l0,\302,\270\026\037\313\070/\325I\240,\255\324\323\306\r\267\345\"x\242c\333\360`\033Pt\307J\215\345V\213z\331"...,

            0x7f47a461a1dc <msort_with_tmp+860>
"\351\b\377\377\377\017\037\200", 0x0, 0x0, 0x8 <error: Cannot access memory
at address 0x8>, 
            0x7ffd07b5fc50 "t-q\016\211\350.t\305
\337\204@\204\304\016\065\022\307%\247\006\334\233\024,i\317t\361\233\301c:ye\250g\207\002R\255\304\333\030\372\300-\030\064\305Q\304_`lq\276\314\356\347\345\374\327\376\343\024\357|(#.yj\216\224\246Dnz\t\205\314\266\001\335\016dc\\L\200u\245\343\361XFm\234\335\274\257Z21,]\356\066\367\275\223\303\217\265@\332\064\331\235G\367\331\067\001\350\300\031?\031M\201\337\250\203\060d\222JBm\034M\233O\031\257\244\202?\022\030\201m\332\325^\034\253(\256\340ef\231\322\214\250B\347\247u\201\371\033\240U\242\350\200U",
0x0, 
            0x7f47a6bede14 <check_match+324>
"\205\300\017\205m\377\377\377\351z\377\377\377\017\037\200", 0x0, 
            0x1e2 <error: Cannot access memory at address 0x1e2>,
0x7f47a6e004f8 "", 0x7f47a5d57b70 "", 0x7f47a5d6de00 "", 
            0x7f47a6bee64b <do_lookup_x+2011>
"H\203\304\060H\205\300\017\205\327\002", 0x1e2 <error: Cannot access memory
at address 0x1e2>, 
            0x7f47a5d6de00 "", 0x7f47a6e004f8 "", 0x7ffd07b5f7d8 "",
0x7ffd07b5f7d4 "", 0x7f47a6bedfe1 <do_lookup_x+369>
"\205\300\017\205\004\b", 
            0x7ffd07b5fd00 "\240U\242\350\200U", 0x7f47a619ccf8
"CRYPTO_set_ex_data", 0x7f47a6198200 "8\035", 0x7ffd07b5f7d8 "", 
            0x89f61ce6 <error: Cannot access memory at address 0x89f61ce6>,
0x227d873 <error: Cannot access memory at address 0x227d873>, 
            0x26 <error: Cannot access memory at address 0x26>, 
            0x7ffd07b5f8b0
"\a\232\316\265\344,\350\244\247\307\214L8X7\016+\v\240\030d\017\302\v9\262n\032K\212\342\341-C\004-KS\344N\315\257^7\004\304a\200\350'p\322\365\220\342\264\004\352\215\356\366B(r\347\346W8\026l\024>\003\241\030u\251\301\315\036\226\322O$\333\302\202G\210i\b\264\250N\246y\311R\001x%\234R0\255\326\314G3\177v\235t\317\f\344\375\260\325\240\247b\261N\215Q\225\205\237R\313@V\330\060\377\276\366eq\346*\316\263\227\217\313D(\315\200\251{L\323$P\374\063\210\064\344Z\330\317\177\214\002\r\221\366\203\242\337\226\224\214\206\025\212\376z\215\353\331\265b\220\317\253\251\266#\371\262\225\343\006<a"...,
0x7f47a5d6de00 "", 0x7f47a5d57b70 "", 0x7ffd07b5f7d4 "", 
            0x7ffd07b5f8a0
"\363\003\201P\221HhZ1\350~\262\335\315e0\a\232\316\265\344,\350\244\247\307\214L8X7\016+\v\240\030d\017\302\v9\262n\032K\212\342\341-C\004-KS\344N\315\257^7\004\304a\200\350'p\322\365\220\342\264\004\352\215\356\366B(r\347\346W8\026l\024>\003\241\030u\251\301\315\036\226\322O$\333\302\202G\210i\b\264\250N\246y\311R\001x%\234R0\255\326\314G3\177v\235t\317\f\344\375\260\325\240\247b\261N\215Q\225\205\237R\313@V\330\060\377\276\366eq\346*\316\263\227\217\313D(\315\200\251{L\323$P\374\063\210\064\344Z\330\317\177\214\002\r\221\366\203\242\337\226\224\214\206\025\212\376z\215\353"...,
0x5580e8988030 "h\261\340\246G\177", 0x0, 0x37 <error: Cannot access memory
at address 0x37>, 0x5580e8a108b0 "\220\215?\246G\177", 
            0x7b5fa68 <error: Cannot access memory at address 0x7b5fa68>,
0x0, 0x0, 0x7f47a6e00358 " \264\340\246G\177", 
            0x7ffd07b5f940
"\227\217\313D(\315\200\251{L\323$P\374\063\210\064\344Z\330\317\177\214\002\r\221\366\203\242\337\226\224\214\206\025\212\376z\215\353\331\265b\220\317\253\251\266#\371\262\225\343\006<a\315e9\367\064V\335E\037\256\265!0\266c`\344\261v{\r\372\357\aomeT\216$oa\021\226G\357\253\350\037\214\312MF\221zME=\256d&\a[\251Lg;\306&\250\300\003\335'\b\311\351\212\v\263`BF\337\327q\264V\214\312\262\231\023\200ip\344;`\331\335\325Dn1\223.]f$#\304\347\066O\235?\273\325\027h\305l0,\302,\270\026\037\313\070/\325I\240,\255\324\323\306\r\267\345\"x\242c\333\360`\033Pt"...,

            0x7f47a6d6a918 "*\343\031\246G\177", 0x710b22270bff2fff <error:
Cannot access memory at address 0x710b22270bff2fff>, 
            0x303a34b4093998ac <error: Cannot access memory at address
0x303a34b4093998ac>, 
            0x246b49573bd17a70 <error: Cannot access memory at address
0x246b49573bd17a70>, 
            0x3439bfde83e46960 <error: Cannot access memory at address
0x3439bfde83e46960>, 
            0xf0157085a6a080a7 <error: Cannot access memory at address
0xf0157085a6a080a7>, 
            0xca11cc132b52c1ad <error: Cannot access memory at address
0xca11cc132b52c1ad>, 
            0xd1b8ef3f1064d051 <error: Cannot access memory at address
0xd1b8ef3f1064d051>, 
            0x8a722414ca0fd3f5 <error: Cannot access memory at address
0x8a722414ca0fd3f5>, 
            0x6121d111b2e9aad2 <error: Cannot access memory at address
0x6121d111b2e9aad2>, 
            0x2576d52883acf6d7 <error: Cannot access memory at address
0x2576d52883acf6d7>, 
            0x5a7ce25fdf371d89 <error: Cannot access memory at address
0x5a7ce25fdf371d89>, 
            0x98d000d2a14d6ffd <error: Cannot access memory at address
0x98d000d2a14d6ffd>, 
            0x89afdf9a2a747ef1 <error: Cannot access memory at address
0x89afdf9a2a747ef1>, 
            0x16756b9404b958b4 <error: Cannot access memory at address
0x16756b9404b958b4>, 
            0x1973ac5b49d4acd4 <error: Cannot access memory at address
0x1973ac5b49d4acd4>, 
            0x31a488f3f735b010 <error: Cannot access memory at address
0x31a488f3f735b010>, 
            0xfc4fc0ab3a24e2c3 <error: Cannot access memory at address
0xfc4fc0ab3a24e2c3>, 
            0x2161964b4f8696bb <error: Cannot access memory at address
0x2161964b4f8696bb>, 
            0x3fa08f7baac26f18 <error: Cannot access memory at address
0x3fa08f7baac26f18>, 
            0x751a9be2bc937451 <error: Cannot access memory at address
0x751a9be2bc937451>, 
            0x5a684891508103f3 <error: Cannot access memory at address
0x5a684891508103f3>, 
            0x3065cdddb27ee831 <error: Cannot access memory at address
0x3065cdddb27ee831>, 
            0xa4e82ce4b5ce9a07 <error: Cannot access memory at address
0xa4e82ce4b5ce9a07>, 
            0xe3758384c8cc7a7 <error: Cannot access memory at address
0xe3758384c8cc7a7>, 
            0xbc20f6418a00b2b <error: Cannot access memory at address
0xbc20f6418a00b2b>, 
            0xe1e28a4b1a6eb239 <error: Cannot access memory at address
0xe1e28a4b1a6eb239>, 
            0x4ee4534b2d04432d <error: Cannot access memory at address
0x4ee4534b2d04432d>, 
            0x8061c404375eafcd <error: Cannot access memory at address
0x8061c404375eafcd>, 
            0xb4e290f5d27027e8 <error: Cannot access memory at address
0xb4e290f5d27027e8>, 
            0x722842f6ee8dea04 <error: Cannot access memory at address
0x722842f6ee8dea04>, 
            0x3e146c163857e6e7 <error: Cannot access memory at address
0x3e146c163857e6e7>, 
            0x1ecdc1a97518a103 <error: Cannot access memory at address
0x1ecdc1a97518a103>, 
            0x4782c2db244fd296 <error: Cannot access memory at address
0x4782c2db244fd296>, 
            0x79a64ea8b4086988 <error: Cannot access memory at address
0x79a64ea8b4086988>, 
            0x30529c25780152c9 <error: Cannot access memory at address
0x30529c25780152c9>...}, changed = {true <repeats 20 times>, false, true, 
            true, true, true, true, true, true, true, false <repeats 1635
times>}}
        has_oldtup = false
        localslot = 0x5580e8a03a70
        remoteslot = 0x5580e8a03810
        found = <optimized out>
        __func__ = "apply_handle_update"
#9  0x00005580e7376f98 in apply_dispatch (s=0x7ffd07b62ce0)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/replication/logical/worker.c:968
        action = 85 'U'
#10 LogicalRepApplyLoop (last_received=10068998058648)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/replication/logical/worker.c:1176
        start_lsn = <optimized out>
        end_lsn = 10068998057328
        send_time = <optimized out>
        c = <optimized out>
        s = {data = 0x5580e8ab8a10 "w", len = 990, maxlen = -1, cursor =
990}
        fd = -1
        len = <optimized out>
        buf = 0x5580e8ab8a10 "w"
        ping_sent = false
        wait_time = <optimized out>
        rc = <optimized out>
        endofstream = false
        last_recv_timestamp = 633331871835472
#11 ApplyWorkerMain (main_arg=<optimized out>)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/replication/logical/worker.c:1734
        worker_slot = <optimized out>
        originname =
"pg_34353\000\213Q\347\200U\000\000\000Y\224\350\200U\000\000\000$\000\000\000\000\000\000\000Y\224\350\200U\000\000\360.\266\a\375\177\000\000\240-\266\a\375\177\000\000a\346Q\347\200U\000"
        origin_startpos = 10068690433128
        myslotname = <optimized out>
        options = {logical = true, slotname = 0x5580e89cfab0 "upgradesub1",
startpoint = 10068690433128, proto = {physical = {startpointTLI = 1}, 
            logical = {proto_version = 1, publication_names =
0x5580e89cfbb0}}}
        __func__ = "ApplyWorkerMain"
#12 0x00005580e7344c08 in StartBackgroundWorker () at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/postmaster/bgworker.c:834
        local_sigjmp_buf = {{__jmpbuf = {94012146213392,
-7351491136948389850, 94012129894936, 0, 140724732833520, 94012146483200, 
              -7351491136981944282, -3674514232863606746}, __mask_was_saved
= 1, __saved_mask = {__val = {18446744066192964103, 140724732833322, 
                94012146227664, 140724732833440, 94012125054965,
94012146227664, 0, 140724732833424, 94012125076945, 94012129894936,
94012129894920, 
                140724732833504, 94012125054965, 140724732833472, 0, 0}}}}
        worker = 0x5580e8946210
        __func__ = "StartBackgroundWorker"
#13 0x00005580e73520f0 in do_start_bgworker (rw=<optimized out>)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/postmaster/postmaster.c:5770
        worker_pid = <optimized out>
#14 maybe_start_bgworkers () at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/postmaster/postmaster.c:5996
        rw = <optimized out>
        num_launched = 0
        now = <optimized out>
        iter = {cur = 0x5580e8987da0, next = 0x5580e8987ff0, prev =
0x5580e79aed00 <BackgroundWorkerList>}
#15 0x00005580e7352d65 in sigusr1_handler (postgres_signal_arg=<optimized
out>)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/postmaster/postmaster.c:5167
        save_errno = 4
        __func__ = "sigusr1_handler"
#16 <signal handler called>
No locals.
#17 0x00007f47a46de5b3 in __select_nocancel () at
../sysdeps/unix/syscall-template.S:84
No locals.
#18 0x00005580e70c1d85 in ServerLoop () at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/postmaster/postmaster.c:1668
        timeout = {tv_sec = 59, tv_usec = 512943}
        rmask = {fds_bits = {48, 0 <repeats 15 times>}}
        selres = <optimized out>
        now = <optimized out>
        readmask = {fds_bits = {48, 0 <repeats 15 times>}}
        last_lockfile_recheck_time = 1580015713
        last_touch_time = 1580014392
        __func__ = "ServerLoop"
#19 0x00005580e7354274 in PostmasterMain (argc=5, argv=<optimized out>)
    at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/postmaster/postmaster.c:1377
        opt = <optimized out>
        status = <optimized out>
        userDoption = <optimized out>
        listen_addr_saved = true
        i = <optimized out>
        output_config_variable = <optimized out>
        __func__ = "PostmasterMain"
#20 0x00005580e70c3909 in main (argc=5, argv=0x5580e8945820) at
/build/postgresql-12-7Qyh90/postgresql-12-12.1/build/../src/backend/main/main.c:228
No locals.

================

Thanks everyone. I still have the core dump available if anyone wants more
detail on anything.

Cheers,
Johann

Messages

DateAuthorSubject
2020-01-26 06:46:00+00PG Bug reporting formBUG #16233: Yet another "logical replication worker" was terminated by signal 11: Segmentation fault
2020-01-26 16:28:18+00Tom LaneRe: BUG #16233: Yet another "logical replication worker" was terminated by signal 11: Segmentation fault
2020-01-27 00:57:39+00Johann du ToitRe: BUG #16233: Yet another "logical replication worker" was terminated by signal 11: Segmentation fault
2020-01-28 09:34:15+00Johann du ToitRe: BUG #16233: Yet another "logical replication worker" was terminated by signal 11: Segmentation fault