OpenZFS 8126 - ztest assertion failed in dbuf_dirty due to dn_nlevels changing

The sync thread is concurrently modifying dn_phys->dn_nlevels
while dbuf_dirty() is trying to assert something about it, without
holding the necessary lock. We need to move this assertion further down
in the function, after we have acquired the dn_struct_rwlock.

Authored by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: Giuseppe Di Natale <dinatale2@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/8126
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/0ef125d
Closes #6314

diff --git a/module/zfs/dbuf.c b/module/zfs/dbuf.c
index f4e24e2099a239ef3ca3446c1d392e4d342a81d7..dc2c00495b54c550668584815802659f8b08441e 100644 (file)
--- a/module/zfs/dbuf.c
+++ b/module/zfs/dbuf.c
@@ -1625,11 +1625,6 @@ dbuf_dirty(dmu_buf_impl_t *db, dmu_tx_t *tx)
            (dmu_tx_is_syncing(tx) ? DN_DIRTY_SYNC : DN_DIRTY_OPEN));
 
        ASSERT3U(dn->dn_nlevels, >, db->db_level);
-       ASSERT((dn->dn_phys->dn_nlevels == 0 && db->db_level == 0) ||
-           dn->dn_phys->dn_nlevels > db->db_level ||
-           dn->dn_next_nlevels[txgoff] > db->db_level ||
-           dn->dn_next_nlevels[(tx->tx_txg-1) & TXG_MASK] > db->db_level ||
-           dn->dn_next_nlevels[(tx->tx_txg-2) & TXG_MASK] > db->db_level);
 
        /*
         * We should only be dirtying in syncing context if it's the
@@ -1747,6 +1742,16 @@ dbuf_dirty(dmu_buf_impl_t *db, dmu_tx_t *tx)
                drop_struct_lock = TRUE;
        }
 
+       /*
+        * We need to hold the dn_struct_rwlock to make this assertion,
+        * because it protects dn_phys / dn_next_nlevels from changing.
+        */
+       ASSERT((dn->dn_phys->dn_nlevels == 0 && db->db_level == 0) ||
+           dn->dn_phys->dn_nlevels > db->db_level ||
+           dn->dn_next_nlevels[txgoff] > db->db_level ||
+           dn->dn_next_nlevels[(tx->tx_txg-1) & TXG_MASK] > db->db_level ||
+           dn->dn_next_nlevels[(tx->tx_txg-2) & TXG_MASK] > db->db_level);
+
        /*
         * If we are overwriting a dedup BP, then unless it is snapshotted,
         * when we get to syncing context we will need to decrement its