OpenZFS 8835 - Speculative prefetch in ZFS not working for misaligned reads

In case of misaligned I/O sequential requests are not detected as such
due to overlaps in logical block sequence:

    dmu_zfetch(fffff80198dd0ae0, 27347, 9, 1)
    dmu_zfetch(fffff80198dd0ae0, 27355, 9, 1)
    dmu_zfetch(fffff80198dd0ae0, 27363, 9, 1)
    dmu_zfetch(fffff80198dd0ae0, 27371, 9, 1)
    dmu_zfetch(fffff80198dd0ae0, 27379, 9, 1)
    dmu_zfetch(fffff80198dd0ae0, 27387, 9, 1)

This patch makes single block overlap to be counted as a stream hit,
improving performance up to several times.

Authored by: Alexander Motin <mav@FreeBSD.org>
Approved by: Gordon Ross <gwr@nexenta.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Allan Jude <allanjude@freebsd.org>
Reviewed by: Gvozden Neskovic <neskovic@gmail.com>
Reviewed by: George Melikov <mail@gmelikov.ru>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/8835
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/aab6dd482a
Closes #7062

diff --git a/module/zfs/dmu_zfetch.c b/module/zfs/dmu_zfetch.c
index ea9afa2826e3ea46cf5603c3a832f8924d73dd55..a2bf80993f9ebb8bd21175bf6db6b4d4cb19e5cf 100644 (file)
--- a/module/zfs/dmu_zfetch.c
+++ b/module/zfs/dmu_zfetch.c
@@ -226,19 +226,33 @@ dmu_zfetch(zfetch_t *zf, uint64_t blkid, uint64_t nblks, boolean_t fetch_data)
 
        rw_enter(&zf->zf_rwlock, RW_READER);
 
+       /*
+        * Find matching prefetch stream.  Depending on whether the accesses
+        * are block-aligned, first block of the new access may either follow
+        * the last block of the previous access, or be equal to it.
+        */
        for (zs = list_head(&zf->zf_stream); zs != NULL;
            zs = list_next(&zf->zf_stream, zs)) {
-               if (blkid == zs->zs_blkid) {
+               if (blkid == zs->zs_blkid || blkid + 1 == zs->zs_blkid) {
                        mutex_enter(&zs->zs_lock);
                        /*
                         * zs_blkid could have changed before we
                         * acquired zs_lock; re-check them here.
                         */
-                       if (blkid != zs->zs_blkid) {
-                               mutex_exit(&zs->zs_lock);
-                               continue;
+                       if (blkid == zs->zs_blkid) {
+                               break;
+                       } else if (blkid + 1 == zs->zs_blkid) {
+                               blkid++;
+                               nblks--;
+                               if (nblks == 0) {
+                                       /* Already prefetched this before. */
+                                       mutex_exit(&zs->zs_lock);
+                                       rw_exit(&zf->zf_rwlock);
+                                       return;
+                               }
+                               break;
                        }
-                       break;
+                       mutex_exit(&zs->zs_lock);
                }
        }