]> granicus.if.org Git - zfs/log
zfs
10 years agoDrive database update
Richard Yao [Wed, 13 Nov 2013 14:30:21 +0000 (09:30 -0500)]
Drive database update

Added:

Adata S396 (obtained from drive_id)
Apple MacBookAir3,1 SSD (obtained from drive_id)
Apple MacBookPro10,1 SSD (obtained from drive_id)
Intel 510 (obtained from drive_id)
Intel 710 (obtained from drive_id)
Intel DC S3500 (obtained from drive_id)
Netapp LUN (obtained from illumos user's sd.conf)
OCZ Agility 3 (obtained from drive_id)
OCZ Vertex (obtained from drive_id)
Samsung PM800 (obtained from drive_id)
Sandisk U100 (obtained from drive_id)
Sun Comstar (obtained from illumos user's sd.conf)

Notes:

1. The entries for the Intel DC S3500 were extrapolated from the 800GB
model's entry, which is "ATA     INTEL SSDSC2BB80".

2. The entires for the Intel 710 were extrapolated from the 120GG
model's entry, which is "ATA     INTEL SSDSA2BZ12".

3. The entires for the Intel 510 were extrapolated from the 250GB
model's entry, which is "ATA     INTEL SSDSC2MH25".

4. The entires for the Apple MacBookPro10,1 SSD were extrapolated from
the 512GB model's entry, which is "ATA     APPLE SSD SM512E". Google
searches suggest that this is a rebadged Samsung 830.

5. The entires for the Apple MacBookAir3,1 SSD were extrapolated from
the 128GB model's entry, which is "ATA     APPLE SSD TS128C". Google
searches suggest that this is a rebadged Kingston SSDNow V+ 100 (based
on Toshiba).

6. Sun Comstar is an iSCSI Target, so we cannot tell what the correct
sector size is through this method. We list it only for reference
purposes, but it is commented out. Similarly, it is not clear what the
right thing to do for Netapp is, so we comment it out.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1907

10 years agoSome nvlist allocations in hold processing need to use KM_PUSHPAGE.
Tim Chase [Wed, 20 Nov 2013 13:56:56 +0000 (07:56 -0600)]
Some nvlist allocations in hold processing need to use KM_PUSHPAGE.

This should hopefully catch the rest of the allocations in the
user hold/release processing that were missed by commit
65c67ea86e9f112177f1ad32de8e780f10798a64.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1852
Closes #1855

10 years agoOnly commit the ZIL once in zpl_writepages() (msync() case).
Etienne Dechamps [Sun, 10 Nov 2013 15:00:11 +0000 (15:00 +0000)]
Only commit the ZIL once in zpl_writepages() (msync() case).

Currently, using msync() results in the following code path:

    sys_msync -> zpl_fsync -> filemap_write_and_wait_range -> zpl_writepages -> write_cache_pages -> zpl_putpage

In such a code path, zil_commit() is called as part of zpl_putpage().
This means that for each page, the write is handed to the DMU, the ZIL
is committed, and only then do we move on to the next page. As one might
imagine, this results in atrocious performance where there is a large
number of pages to write: instead of committing a batch of N writes,
we do N commits containing one page each. In some extreme cases this
can result in msync() being ~700 times slower than it should be, as well
as very inefficient use of ZIL resources.

This patch fixes this issue by making sure that the requested writes
are batched and then committed only once. Unfortunately, the
implementation is somewhat non-trivial because there is no way to run
write_cache_pages in SYNC mode (so that we get all pages) without
making it wait on the writeback tag for each page.

The solution implemented here is composed of two parts:

 - I added a new callback system to the ZIL, which allows the caller to
   be notified when its ITX gets written to stable storage. One nice
   thing is that the callback is called not only in zil_commit() but
   in zil_sync() as well, which means that the caller doesn't have to
   care whether the write ended up in the ZIL or the DMU: it will get
   notified as soon as it's safe, period. This is an improvement over
   dmu_tx_callback_register() that was used previously, which only
   supports DMU writes. The rationale for this change is to allow
   zpl_putpage() to be notified when a ZIL commit is completed without
   having to block on zil_commit() itself.

 - zpl_writepages() now calls write_cache_pages in non-SYNC mode, which
   will prevent (1) write_cache_pages from blocking, and (2) zpl_putpage
   from issuing ZIL commits. zpl_writepages() will issue the commit
   itself instead of relying on zpl_putpage() to do it, thus nicely
   batching the writes. Note, however, that we still have to call
   write_cache_pages() again in SYNC mode because there is an edge case
   documented in the implementation of write_cache_pages() whereas it
   will not give us all dirty pages when running in non-SYNC mode. Thus
   we need to run it at least once in SYNC mode to make sure we honor
   persistency guarantees. This only happens when the pages are
   modified at the same time msync() is running, which should be rare.
   In most cases there won't be any additional pages and this second
   call will do nothing.

Note that this change also fixes a bug related to #907 whereas calling
msync() on pages that were already handed over to the DMU in a previous
writepages() call would make msync() block until the next TXG sync
instead of returning as soon as the ZIL commit is complete. The new
callback system fixes that problem.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1849
Closes #907

10 years agoChange zfs-dkms requirement
Trey Dockendorf [Fri, 15 Nov 2013 19:36:24 +0000 (13:36 -0600)]
Change zfs-dkms requirement

Version 2.2.0.3-20 of dkms in the EPEL/Fedora repositories added the
necessary patches to support ZoL,  Therefore, the zfs-dkms requirement
on dkms is set to match that version or higher.  This allows us to
drop the custom dkms build in the ZoL EPEL/Fedora repositories.

References:
  https://bugzilla.redhat.com/show_bug.cgi?id=1023598

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1873

10 years agoIllumos #2583
Yuri Pankov [Tue, 19 Nov 2013 15:41:37 +0000 (16:41 +0100)]
Illumos #2583

2583 Add -p (parsable) option to zfs list

References:
  https://www.illumos.org/issues/2583
  illumos/illumos-gate@43d68d68c1ce08fb35026bebfb141af422e7082e

Ported-by: Gregor Kopka <gregor@kopka.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes: #937
10 years agoAdd I/O Read/Write Accounting
Brian Behlendorf [Fri, 15 Nov 2013 17:59:09 +0000 (09:59 -0800)]
Add I/O Read/Write Accounting

Because ZFS bypasses the page cache we don't inherit per-task I/O
accounting for free.  However, the Linux kernel does provide helper
functions allow us to perform our own accounting.  These are most
commonly used for direct IO which also bypasses the page cache, but
they can be used for the common read/write call paths as well.

Signed-off-by: Pavel Snajdr <snajpa@snajpa.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #313
Closes #1275

10 years agoDocument ZFS module parameters.
Turbo Fredriksson [Sat, 16 Nov 2013 06:52:54 +0000 (06:52 +0000)]
Document ZFS module parameters.

This is a first draft of a zfs-module-parameters(5) man page. I have
just extracted the parameter name and its description with modinfo,
then checked the source what type it is and its default value.

This will need more work, preferably someone that actually know these
values and what to use them for.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1856

10 years agoAdd missing libzfs_core to Makefiles
Maximilian Mehnert [Sun, 17 Nov 2013 10:47:50 +0000 (11:47 +0100)]
Add missing libzfs_core to Makefiles

On some platforms symbols provided by libzfs_core and used by
libzfs were not available to the linker.  To avoid this issue
libzfs_core has been added to the list of required libraries
when building utilities which depend on libzfs.  This should
have been handled properly by libtool and it's still not
entirely clear why it wasn't on all platforms.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1841

10 years agoIllumos #4322
Steven Hartland [Tue, 19 Nov 2013 13:48:28 +0000 (00:48 +1100)]
Illumos #4322

4322 ZFS deadlock on dp_config_rwlock
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Ilya Usvyatsky <ilya.usvyatsky@nexenta.com>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/4322
  illumos/illumos-gate@c50d56f667f119d78fa3d94d6bef2c298ba556f6

Ported by: Chris Dunlop <chris@onthe.net.au>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1886

10 years agoFix typos in commit b83e3e48c9b183a80dd00eb6c7431a1cbc7d89c9
DHE [Wed, 20 Nov 2013 00:00:43 +0000 (19:00 -0500)]
Fix typos in commit b83e3e48c9b183a80dd00eb6c7431a1cbc7d89c9

There's a missing semicolon and equals sign in the first hunk of this
commit in config/kernel-bdi.m4. This results in the test always
failing. The effects were noticed when rrdtool, a tool which modifies
files by mmap() and msync(), would have data never get saved to disk
in spite of the files working while the mounted filesystem remains
mounted.

Signed-off-by: DHE <git@dehacked.net>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Richard Yao <ryao@gentoo.org>
Closes #1889

11 years agoRemove the slog restriction on bootfs pools
Brian Behlendorf [Thu, 14 Nov 2013 22:22:52 +0000 (14:22 -0800)]
Remove the slog restriction on bootfs pools

Under Linux this restriction does not apply because we have access
to all the required devices.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1631

11 years agoTighten zfs dependency on zfs-kmod
Cyril Plisko [Mon, 26 Aug 2013 06:04:38 +0000 (09:04 +0300)]
Tighten zfs dependency on zfs-kmod

Make zfs depend on the same version of zfs-kmod, rather than on same or
better. When yum repository contains a number of versions the dependency
resolution breaks on trying to install non-latest version.

Signed-off-by: Cyril Plisko <cyril.plisko@mountall.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1677

11 years agoFixes (extends) support for selinux xattrs to more inode types
Matthew Thode [Wed, 6 Nov 2013 21:54:54 +0000 (15:54 -0600)]
Fixes (extends) support for selinux xattrs to more inode types

Properly initialize SELinux xattrs for all inode types.  The
initial implementation accidentally only did this for files.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1832

11 years agoReduce stack for traverse_visitbp() recursion
Brian Behlendorf [Wed, 13 Nov 2013 19:05:17 +0000 (11:05 -0800)]
Reduce stack for traverse_visitbp() recursion

During pool import stack overflows may still occur due to the
potentially deep recursion of traverse_visitbp().  This is most
likely to occur when additional layers are added to the block
device stack such as DM multipath.  To minimize the stack usage
for this call path the following changes were made:

1) Added the keywork 'noinline' to the vdev_*_map_alloc() functions
   to prevent them from being inlined by gcc.  This reduced the
   stack usage of vdev_raidz_io_start() from 208 to 128 bytes, and
   vdev_mirror_io_start() from 144 to 128 bytes.

2) The 'saved_poolname' charater array in zfsdev_ioctl() was moved
   from the stack to the heap.  This reduced the stack usage of
   zfsdev_ioctl() from 368 to 112 bytes.

3) The major saving came from slimming down traverse_visitbp() from
   from 224 to 144 bytes.  Since this function is called recursively
   the 80 bytes saved per invokation adds up.  The following changes
   were made:

  a) The 'hard' local variable was replaced by a TD_HARD() macro.

  b) The 'pd' local variable was replaced by 'td->td_pfd' references.

  c) The zbookmark_t was moved to the heap.  This does cost us an
     additional memory allocation per recursion by that cost should
     still be minimal.  The cost could be further reduced by adding
     a dedicated zbookmark_t slab cache.

  d) The variable declarations in 'if (BP_GET_LEVEL()) { }' were
     restructured to use the minimum amount of stack.  This includes
     removing the 'cbp' local variable.

Overall for the offending use case roughly 1584 of total stack space
has been saved.  This is enough to avoid overflowing the stack on
stock kernels with 8k stacks.  See #1778 for additional details.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ned Bass <bass6@llnl.gov>
Closes #1778

11 years agoSome nvlist allocations in hold processing need to use KM_PUSHPAGE.
Tim Chase [Sun, 10 Nov 2013 15:00:54 +0000 (09:00 -0600)]
Some nvlist allocations in hold processing need to use KM_PUSHPAGE.

Commit 95fd54a1c5b93bb2aa3e7dffc28c784b1e21a8bb restructured the
hold/release processing and moved some of the work into the sync task.
A number of nvlist allocations now need to use KM_PUSHPAGE.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1852
Closes #1855

11 years agoFix rollback of mounted filesystem regression
Tim Chase [Sun, 10 Nov 2013 01:22:06 +0000 (19:22 -0600)]
Fix rollback of mounted filesystem regression

The Illumos #3875 patch reverted a part of ZoL's 7b3e34b which added
special-case error handling for zfs_rezget().  The error handling dealt
with the case in which an all-ones object number ended up being passed
to dnode_hold() and causing an EINVAL to be returned from zfs_rezget().

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1859
Closes #1861

11 years agoPython 3 fixes
Matthew Thode [Fri, 8 Nov 2013 21:53:54 +0000 (15:53 -0600)]
Python 3 fixes

Future proofing for compatibility with newer versions of Python.

Signed-off-by: Matthew Thode <prometheanfire@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1838

11 years agopep8 code readability changes
Matthew Thode [Fri, 8 Nov 2013 21:52:06 +0000 (15:52 -0600)]
pep8 code readability changes

Update the code to follow the pep8 style guide.

References:
  http://www.python.org/dev/peps/pep-0008/

Signed-off-by: Matthew Thode <prometheanfire@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1838

11 years agoExplain 'zfs list -t snap -o name -s name' speedup
Bassu [Fri, 8 Nov 2013 21:16:38 +0000 (02:16 +0500)]
Explain 'zfs list -t snap -o name -s name' speedup

Commit 0cee240 from FreeBSD dramatically speeds up 'zfs list'
performance assuming you're only interested in the dataset
names.  This optimization should be mentioned in the man page
to allow end users to take advantage of it.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1847

11 years agoUpdating init scripts to have more robust grepping
Matthew Thode [Wed, 6 Nov 2013 22:56:50 +0000 (16:56 -0600)]
Updating init scripts to have more robust grepping

The previous pattern could accidentally match on things like
'real_root=ZFS=node02-zp00/ROOT/rootfs' due to the 'ZFS=no'
substring.

Signed-off-by: Matthew Thode <mthode@mthode.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1837

11 years agoHandle concurrent snapshot automounts failing due to EBUSY.
Tim Chase [Thu, 7 Nov 2013 05:55:18 +0000 (23:55 -0600)]
Handle concurrent snapshot automounts failing due to EBUSY.

In the current snapshot automount implementation, it is possible for
multiple mounts to attempted concurrently.  Only one of the mounts will
succeed and the other will fail.  The failed mounts will cause an EREMOTE
to be propagated back to the application.

This commit works around the problem by adding a new exit status,
MOUNT_BUSY to the mount.zfs program which is used when the underlying
mount(2) call returns EBUSY.  The zfs code detects this condition and
treats it as if the mount had succeeded.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1819

11 years agoDocument the dedupditto pool property.
Tim Chase [Fri, 8 Nov 2013 04:45:39 +0000 (22:45 -0600)]
Document the dedupditto pool property.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1839

11 years agoHonor CONFIG_FS_POSIX_ACL kernel option
Massimo Maggi [Sat, 2 Nov 2013 23:40:26 +0000 (00:40 +0100)]
Honor CONFIG_FS_POSIX_ACL kernel option

The required Posix ACL interfaces are only available for kernels
with CONFIG_FS_POSIX_ACL defined.  Therefore, only enable Posix
ACL support for these kernels.  All major distribution kernels
enable CONFIG_FS_POSIX_ACL by default.

If your kernel does not support Posix ACLs the following warning
will be printed at ZFS module load time.

  "ZFS: Posix ACLs disabled by kernel"

Signed-off-by: Massimo Maggi <me@massimo-maggi.eu>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1825

11 years ago26126 panic system rather than corrupting pool if we hit bug 26100
Matthew Ahrens [Mon, 12 Aug 2013 16:53:33 +0000 (12:53 -0400)]
26126 panic system rather than corrupting pool if we hit bug 26100

References:
  delphix/delphix-os@931c8aaab74b6412933d299890894262e2ef8380

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1650

11 years agoSwitch allocations from KM_SLEEP to KM_PUSHPAGE
Brian Behlendorf [Tue, 5 Nov 2013 18:32:39 +0000 (10:32 -0800)]
Switch allocations from KM_SLEEP to KM_PUSHPAGE

A couple of kmem_alloc() allocations were using KM_SLEEP in
the sync thread context.  These were accidentally introduced
by the recent set of Illumos patches.  The solution is to
switch to KM_PUSHPAGE.

dsl_dataset_promote_sync() -> promote_hold() -> snaplist_make() ->
kmem_alloc(sizeof (*snap), KM_SLEEP);

dsl_dataset_user_hold_sync() -> dsl_onexit_hold_cleanup() ->
kmem_alloc(sizeof (*ca), KM_SLEEP)

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3995
Saso Kiselkov [Mon, 14 Oct 2013 22:29:45 +0000 (18:29 -0400)]
Illumos #3995

3995 Memory leak of compressed buffers in l2arc_write_done

References:
  https://illumos.org/issues/3995

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1688
Issue #1775

11 years agoIllumos #4168, #4169, #4170
George Wilson [Fri, 4 Oct 2013 22:13:23 +0000 (14:13 -0800)]
Illumos #4168, #4169, #4170

4168 ztest assertion failure in dbuf_undirty
4169 verbatim import causes zdb to segfault
4170 zhack leaves pool in ACTIVE state
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/4168
  https://www.illumos.org/issues/4169
  https://www.illumos.org/issues/4170
  illumos/illumos-gate@7fdd916c474ea52896c671bbe7b56ba34a1ca132

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #4082
Matthew Ahrens [Fri, 30 Aug 2013 09:19:35 +0000 (01:19 -0800)]
Illumos #4082

4082 zfs receive gets EFBIG from dmu_tx_hold_free()
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/4082
  illumos/illumos-gate@5253393b09789ec67bec153b866d7285a1cf1645

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3954, #4080, #4081
George Wilson [Thu, 29 Aug 2013 18:56:49 +0000 (10:56 -0800)]
Illumos #3954, #4080, #4081

3954 metaslabs continue to load even after hitting zfs_mg_alloc_failure limit
4080 zpool clear fails to clear pool
4081 need zfs_mg_noalloc_threshold
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3954
  https://www.illumos.org/issues/4080
  https://www.illumos.org/issues/4081
  illumos/illumos-gate@22e30981d82a0b6dc89253596ededafae8655e00

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #4046
Matthew Ahrens [Thu, 22 Aug 2013 17:51:47 +0000 (09:51 -0800)]
Illumos #4046

4046 dsl_dataset_t ds_dir->dd_lock is highly contended
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>

References:
  https://www.illumos.org/issues/4046
  illumos/illumos-gate@b62969f868a827f0823a084bc0af9c7d8b76c659

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

Porting notes:

1. This commit removed dsl_dataset_namelen in Illumos, but that
   appears to have been removed from ZFSOnLinux in an earlier commit.

11 years agoIllumos #4061
Marcel Telka [Fri, 16 Aug 2013 02:33:42 +0000 (22:33 -0400)]
Illumos #4061

4061 libzfs: memory leak in iter_dependents_cb()
Reviewed by: Jeffry Molanus <jeffry.molanus@nexenta.com>
Reviewed by: Boris Protopopov <boris.protopopov@nexenta.com>
Reviewed by: Andy Stormont <andyjstormont@gmail.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/4061
  illumos/illumos-gate@2fbdf8dbf01ec1c85fcd3827cdf9e9f5f46c4c8a

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #4047
Matthew Ahrens [Wed, 21 Aug 2013 04:11:52 +0000 (20:11 -0800)]
Illumos #4047

4047 panic from dbuf_free_range() from dmu_free_object() while
     doing zfs receive
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/4047
  illumos/illumos-gate@713d6c208802cfbb806329ec0d154b641b80c355

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

Porting notes:

1. The exported symbol dmu_free_object() was renamed to
   dmu_free_long_object() in Illumos.

11 years agoIllumos #3996
Matthew Ahrens [Wed, 14 Aug 2013 19:42:31 +0000 (11:42 -0800)]
Illumos #3996

3996 want a libzfs_core API to rollback to latest snapshot
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Andy Stormont <andyjstormont@gmail.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3996
  illumos/illumos-gate@a7027df17fad220a20367b9d1eb251bc6300d203

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3956, #3957, #3958, #3959, #3960, #3961, #3962
George Wilson [Wed, 7 Aug 2013 20:16:22 +0000 (12:16 -0800)]
Illumos #3956, #3957, #3958, #3959, #3960, #3961, #3962

3956 ::vdev -r should work with pipelines
3957 ztest should update the cachefile before killing itself
3958 multiple scans can lead to partial resilvering
3959 ddt entries are not always resilvered
3960 dsl_scan can skip over dedup-ed blocks if physical birth != logical birth
3961 freed gang blocks are not resilvered and can cause pool to suspend
3962 ztest should print out zfs debug buffer before exiting
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3956
  https://www.illumos.org/issues/3957
  https://www.illumos.org/issues/3958
  https://www.illumos.org/issues/3959
  https://www.illumos.org/issues/3960
  https://www.illumos.org/issues/3961
  https://www.illumos.org/issues/3962
  illumos/illumos-gate@b4952e17e8858d3225793b28788278de9fe6038d

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Porting notes:

1. zfs_dbgmsg_print() is only used in userland. Since we do not have
   mdb on Linux, it does not make sense to make it available in the
   kernel. This means that a build failure will occur if any future
   kernel patch depends on it. However, that is unlikely given that
   this functionality was added to support zdb.

2. zfs_dbgmsg_print() is only invoked for -VVV or greater log levels.
   This preserves the existing behavior of minimal noise when running
   with -V, and -VV.

3. In vdev_config_generate() the call to nvlist_alloc() was not
   changed to fnvlist_alloc() because we must pass KM_PUSHPAGE in
   the txg_sync context.

11 years agoIllumos #3949, #3950, #3952, #3953
George Wilson [Wed, 7 Aug 2013 18:24:34 +0000 (10:24 -0800)]
Illumos #3949, #3950, #3952, #3953

3949 ztest fault injection should avoid resilvering devices
3950 ztest: deadman fires when we're doing a scan
3951 ztest hang when running dedup test
3952 ztest: ztest_reguid test and ztest_fault_inject don't place nice together
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3949
  https://www.illumos.org/issues/3950
  https://www.illumos.org/issues/3951
  https://www.illumos.org/issues/3952
  illumos/illumos-gate@2c1e2b44148432fb7a509dd216a99299b6740250

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

Porting notes:

1. The deadman thread was removed from ztest during the original
   port because it depended on Solaris thr_create() interface.
   This functionality should be reintroduced using the more
   portable pthreads.

11 years agoIllumos #3955
Matthew Ahrens [Wed, 7 Aug 2013 18:32:46 +0000 (10:32 -0800)]
Illumos #3955

3955 ztest failure: assertion refcount_count(&tx->tx_space_written) +
     delta <= tx->tx_space_towrite
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3955
  illumos/illumos-gate@be9000cc677e0a8d04e5be45c61d7370fc8c7b54

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3973
Steven Hartland [Tue, 6 Aug 2013 17:50:40 +0000 (09:50 -0800)]
Illumos #3973

3973 zfs_ioc_rename alters passed in zc->zc_name
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3973
  illumos/illumos-gate@a0c1127b147dc6a0372b141deb8c0c2b0195b8ea

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3909
Steven Hartland [Mon, 29 Jul 2013 19:36:31 +0000 (11:36 -0800)]
Illumos #3909

3909 "zfs send -D" does not work
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3909
  illumos/illumos-gate@36f7455d36b60be70d7aae5959fa19e71954678e

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3834
Matthew Ahrens [Mon, 29 Jul 2013 18:58:53 +0000 (10:58 -0800)]
Illumos #3834

3834 incremental replication of 'holey' file systems is slow
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3834
  illumos/illumos-gate@ca48f36f20f6098ceb19d5b084b6b3d4b8eca9fa

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3836
Matthew Ahrens [Wed, 3 Jul 2013 16:13:38 +0000 (08:13 -0800)]
Illumos #3836

3836 zio_free() can be processed immediately in the common case
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/3836
  illumos/illumos-gate@9cb154a3c9f170904dce9bad5bd5a7d256b922a4

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3112, #3113, #3114
Matthew Ahrens [Thu, 16 May 2013 21:18:06 +0000 (14:18 -0700)]
Illumos #3112, #3113, #3114

3112 ztest does not honor ZFS_DEBUG
3113 ztest should use watchpoints to protect frozen arc bufs
3114 some leaked nvlists in zfsdev_ioctl

Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Matt Amdur <Matt.Amdur@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Christopher Siden <chris.siden@delphix.com>
Approved by: Eric Schrock <eric.schrock@delphix.com>

References:
  https://www.illumos.org/issues/3112
  https://www.illumos.org/issues/3113
  https://www.illumos.org/issues/3114
  illumos/illumos-gate@cd1c8b85eb30b568e9816221430c479ace7a559d

The /proc/self/cmd watchpoint interface is specific to Solaris.
Therefore, the #3113 implementation was reworked to use the more
portable mprotect(2) system call.  When the pages are watched they
are marked read-only for protection.  Any write to the protected
address range immediately trigger a SIGSEGV.  The pages are marked
writable again when they are unwatched.

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1489

11 years agoIllumos #3236
George Wilson [Fri, 10 May 2013 19:47:54 +0000 (12:47 -0700)]
Illumos #3236

3236 zio nop-write
Reviewed by: Matt Ahrens <matthew.ahrens@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Christopher Siden <chris.siden@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>

References:
  illumos/illumos-gate@80901aea8e78a2c20751f61f01bebd1d5b5c2ba5
  https://www.illumos.org/issues/3236

Porting Notes

1. This patch is being merged dispite an increased instance of
   https://www.illumos.org/issues/3113 being triggered by ztest.

Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1489

11 years agoIllumos #3875
Keith M Wesolowski [Sat, 27 Jul 2013 17:50:07 +0000 (10:50 -0700)]
Illumos #3875

3875 panic in zfs_root() after failed rollback
Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Gordon Ross <gwr@nexenta.com>

References:
  https://www.illumos.org/issues/3875
  illumos/illumos-gate@91948b51b8e978ddc88a36b2bc3ae83c20cdc9aa

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3888
Matthew Ahrens [Mon, 29 Jul 2013 18:55:16 +0000 (10:55 -0800)]
Illumos #3888

3888 zfs recv -F should destroy any snapshots created since
     the incremental source
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Peng Dai <peng.dai@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3888
  illumos/illumos-gate@34f2f8cf94052481c81be2e134b94a00b501bf21

Porting notes:

1. Commit 1fde1e37208c2f56c72c70a06676676f04b65998 wrapped a
   declaration in dsl_dataset_modified_since_lastsnap in ASSERTV().
   The ASSERTV() and local variable have been removed to avoid an
   unused variable warning.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: Richard Yao <ryao@gentoo.org>
Issue #1775

11 years agoIllumos #3894
Keith M Wesolowski [Sat, 27 Jul 2013 17:51:50 +0000 (10:51 -0700)]
Illumos #3894

3894 zfs should not allow snapshot of inconsistent dataset
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Gordon Ross <gwr@nexenta.com>

References:
  https://www.illumos.org/issues/3894
  illumos/illumos-gate@ca48f36f20f6098ceb19d5b084b6b3d4b8eca9fa

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3829
Matthew Ahrens [Thu, 20 Jun 2013 22:43:17 +0000 (14:43 -0800)]
Illumos #3829

3829 fix for 3740 changed behavior of zfs destroy/hold/release ioctl
Reviewed by: Matt Amdur <matt.amdur@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3829
  illumos/illumos-gate@bb6e70758d0c30c09f148026d6e686e21cfc8d18

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3818
Steven Hartland [Wed, 19 Jun 2013 06:36:40 +0000 (22:36 -0800)]
Illumos #3818

3818 zpool status -x should report pools with removed l2arc devices
Reviewed by: Saso Kiselkov <skiselkov.ml@gmail.com>
Reviewed by: George Wilson <gwilson@zfsmail.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3818
  illumos/illumos-gate@7f2416ef64fb43dab18d9b36c0da64bea37c0df3

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3740
Steven Hartland [Sat, 25 May 2013 02:06:23 +0000 (02:06 +0000)]
Illumos #3740

3740 Poor ZFS send / receive performance due to snapshot
     hold / release processing
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3740
  illumos/illumos-gate@a7a845e4bf22fd1b2a284729ccd95c7370a0438c

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

Porting notes:

1. 13fe019870c8779bf2f5b3ff731b512cf89133ef introduced a merge conflict
   in dsl_dataset_user_release_tmp where some variables were moved
   outside of the preprocessor directive.

2. dea9dfefdd747534b3846845629d2200f0616dad made the previous merge
   conflict worse by switching KM_SLEEP to KM_PUSHPAGE. This is notable
   because this commit refactors the code, adding a new KM_SLEEP
   allocation. It is not clear to me whether this should be converted
   to KM_PUSHPAGE.

3. We had a merge conflict in libzfs_sendrecv.c because of copyright
   notices.

4. Several small C99 compatibility fixed were made.

11 years agoIllumos #3745, #3811
Will Andrews [Tue, 11 Jun 2013 17:13:47 +0000 (09:13 -0800)]
Illumos #3745, #3811

3745 zpool create should treat -O mountpoint and -m the same
3811 zpool create -o altroot=/xyz -O mountpoint=/mnt ignores
     the mountpoint option
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3745
  https://www.illumos.org/issues/3811
  illumos/illumos-gate@8b713775314bbbf24edd503b4869342d8711ce95

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3744
Will Andrews [Tue, 11 Jun 2013 17:13:43 +0000 (09:13 -0800)]
Illumos #3744

3744 zfs shouldn't ignore errors unmounting snapshots
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3744
  illumos/illumos-gate@fc7a6e3fefc649cb65c8e2a35d194781445008b0

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

Porting notes:

1. There is no clear way to distinguish between a failure when we
   tried to unmount the snapdir of a zvol (which does not exist)
   and the failure when we try to unmount a snapdir of a dataset,
   so the changes to zfs_unmount_snap() were dropped in favor of
   an altered Linux function that unconditionally returns 0.

11 years agoIllumos #3743
Will Andrews [Tue, 11 Jun 2013 17:13:38 +0000 (09:13 -0800)]
Illumos #3743

3743 zfs needs a refcount audit
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3743
  illumos/illumos-gate@b287be1ba86043996f49b1cc34c80cc620f9b841

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3742
Will Andrews [Tue, 11 Jun 2013 17:12:34 +0000 (09:12 -0800)]
Illumos #3742

3742 zfs comments need cleaner, more consistent style
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3742
  illumos/illumos-gate@f7170741490edba9d1d9c697c177c887172bc741

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

Porting notes:

1. The change to zfs_vfsops.c was dropped because it involves
   zfs_mount_label_policy, which does not exist in the Linux port.

11 years agoIllumos #3741
Will Andrews [Tue, 11 Jun 2013 17:12:34 +0000 (09:12 -0800)]
Illumos #3741

3741 zfs needs better comments
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3741
  illumos/illumos-gate@3e30c24aeefdee1631958ecf17f18da671781956

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3699, #3739
Martin Matuska [Thu, 23 May 2013 17:07:25 +0000 (13:07 -0400)]
Illumos #3699, #3739

3699 zfs hold or release of a non-existent snapshot does not output error
3739 cannot set zfs quota or reservation on pool version < 22
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Eric Shrock <eric.schrock@delphix.com>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/3699
  https://www.illumos.org/issues/3739
  illumos/illumos-gate@013023d4ed2f6d0cf75380ec686a4aac392b4e43

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3582, #3584
Adam Leventhal [Wed, 28 Aug 2013 23:05:48 +0000 (16:05 -0700)]
Illumos #3582, #3584

3582 zfs_delay() should support a variable resolution
3584 DTrace sdt probes for ZFS txg states

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Dan McDonald <danmcd@nexenta.com>
Reviewed by: Richard Elling <richard.elling@dey-sys.com>
Approved by: Garrett D'Amore <garrett@damore.org>

References:
    https://www.illumos.org/issues/3582
    illumos/illumos-gate@0689f76

Ported by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years ago6977619 NULL pointer deference in sa_handle_get_from_db()
Mark Shellenbaum [Wed, 18 Aug 2010 19:59:31 +0000 (13:59 -0600)]
6977619 NULL pointer deference in sa_handle_get_from_db()

References:
  illumos/illumos-gate@44bffe012cad6481c82ad67bacd6b40bd29def2b

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years ago6939941 problem with moving files in zfs
Mark Shellenbaum [Tue, 6 Apr 2010 01:59:44 +0000 (19:59 -0600)]
6939941 problem with moving files in zfs

References:
  illumos/illumos-gate@d39ee142a97a7c58f60f7b52c62409f2ff64b234

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

Porting notes:

1. This commit was so old that only two lines applied to the modern
   code base.

11 years agoIllumos #3642, #3643
George Wilson [Tue, 23 Apr 2013 17:31:42 +0000 (09:31 -0800)]
Illumos #3642, #3643

3642 dsl_scan_active() should not issue I/O to determine if async
     destroying is active
3643 txg_delay should not hold the tc_lock
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Approved by: Gordon Ross <gwr@nexenta.com>

References:
  https://www.illumos.org/issues/3642
  https://www.illumos.org/issues/3643
  illumos/illumos-gate@4a92375985c37d61406d66cd2b10ee642eb1f5e7

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

Porting Notes:

1. The alignment assumptions for the tx_cpu structure assume that
   a kmutex_t is 8 bytes.  This isn't true under Linux but tc_pad[]
   was adjusted anyway for consistency since this structure was
   never carefully aligned in ZoL.  If careful alignment does impact
   performance significantly this should be reworked to be portable.

11 years agoIllumos #3645, #3692
Matthew Ahrens [Wed, 10 Apr 2013 21:54:56 +0000 (13:54 -0800)]
Illumos #3645, #3692

3645 dmu_send_impl: possibilty of pool hold leak
3692 Panic on zfs receive of a recursive deduplicated stream
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Dan McDonald <danmcd@nexenta.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3645
  https://www.illumos.org/issues/3692
  illumos/illumos-gate@de8d9cff565e928d0ace86f3ea0e2b15094d61df

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1792
Issue #1775

11 years agoIllumos #3598
Matthew Ahrens [Fri, 8 Mar 2013 18:41:28 +0000 (10:41 -0800)]
Illumos #3598

3598 want to dtrace when errors are generated in zfs
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>

References:
  https://www.illumos.org/issues/3598
  illumos/illumos-gate@be6fd75a69ae679453d9cda5bff3326111e6d1ca

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

Porting notes:

1. include/sys/zfs_context.h has been modified to render some new
   macros inert until dtrace is available on Linux.

2. Linux-specific changes have been adapted to use SET_ERROR().

3. I'm NOT happy about this change.  It does nothing but ugly
   up the code under Linux.  Unfortunately we need to take it to
   avoid more merge conflicts in the future.  -Brian

11 years agoIllumos #3517
Yuri Pankov [Thu, 7 Mar 2013 01:57:09 +0000 (17:57 -0800)]
Illumos #3517

3517 importing pool with autoreplace=on and "hole" vdevs crashes syseventd
Reviewed by: Albert Lee <trisk@nexenta.com>
Reviewed by: Jeffry Molanus <jeffry.molanus@nexenta.com>
Reviewed by: George Wilson <gwilson@zfsmail.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3517
  illumos/illumos-gate@efb4a871d8fd510a833bdca610528dde5ed69e42

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoIllumos #3603, #3604: bobj improvements
Matthew Ahrens [Fri, 5 Jul 2013 19:37:16 +0000 (15:37 -0400)]
Illumos #3603, #3604: bobj improvements

3603 panic from bpobj_enqueue_subobj()
3604 zdb should print bpobjs more verbosely
3871 GCC 4.5.3 does not like issue 3604 patch
Reviewed by: Henrik Mattson <henrik.mattson@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/3603
  https://www.illumos.org/issues/3604
  https://www.illumos.org/issues/3871
  illumos/illumos-gate@d04756377ddd1cf28ebcf652541094e17b03c889

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

Note that the patch from Illumos issue 3871 is not accepted into Illumos
at the time of this writing. It is something that I wrote when porting
this. Documentation is in the Illumos issue.

11 years agoIllumos #3588
Matthew Ahrens [Fri, 22 Feb 2013 09:23:09 +0000 (01:23 -0800)]
Illumos #3588

3588 provide zfs properties for logical (uncompressed) space
     used and referenced
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Dan McDonald <danmcd@nexenta.com>
Reviewed by: Richard Elling <richard.elling@dey-sys.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3588
  illumos/illumos-gate@77372cb0f35e8d3615ca2e16044f033397e88e21

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
11 years agoIllumos #3578, #3579
George Wilson [Wed, 20 Feb 2013 21:30:36 +0000 (13:30 -0800)]
Illumos #3578, #3579

3578 transferring the freed map to the defer map should be constant time
3579 ztest trips assertion in metaslab_weight()
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Richard Elling <richard.elling@dey-sys.com>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/3578
  https://www.illumos.org/issues/3579
  illumos/illumos-gate@9eb57f7f3fbb970d4b9b89dcd5ecf543fe2414d5

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
11 years agoIllumos #3561, #3116
George Wilson [Sun, 17 Feb 2013 20:00:54 +0000 (12:00 -0800)]
Illumos #3561, #3116

3561 arc_meta_limit should be exposed via kstats
3116 zpool reguid may log negative guids to internal SPA history
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Gordon Ross <gordon.ross@nexenta.com>
Approved by: Garrett D'Amore <garrett@damore.org>

References:
  https://www.illumos.org/issues/3561
  https://www.illumos.org/issues/3116
  illumos/illumos-gate@20128a0826f9c53167caa9215c12f08beee48e30

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Porting Notes:

1. The spa change was accidentally included in the libzfs_core merge.

2. "Add missing arcstats" (1834f2d8b715d25bafbb0e4a099994f45c3211ae)
   already implemented these kstats a few years ago.

11 years agoIllumos #3537
Matthew Ahrens [Tue, 27 Aug 2013 00:09:29 +0000 (17:09 -0700)]
Illumos #3537

3537 want pool io kstats

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Sa?o Kiselkov <skiselkov.ml@gmail.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Reviewed by: Brendan Gregg <brendan.gregg@joyent.com>
Approved by: Gordon Ross <gwr@nexenta.com>

References:
  http://www.illumos.org/issues/3537
  illumos/illumos-gate@c3a6601

Ported by: Cyril Plisko <cyril.plisko@mountall.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Porting Notes:

1. The patch was restructured to take advantage of the existing
   spa statistics infrastructure.  To accomplish this the kstat
   was moved in to spa->io_stats and the init/destroy code moved
   to spa_stats.c.

2. The I/O kstat was simply named <pool> which conflicted with the
   pool directory we had already created.  Therefore it was renamed
   to <pool>/io

3. An update handler was added to allow the kstat to be zeroed.

11 years agoIllumos #3522
George Wilson [Mon, 11 Feb 2013 06:21:05 +0000 (22:21 -0800)]
Illumos #3522

3522 zfs module should not allow uninitialized variables
Reviewed by: Sebastien Roy <seb@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>

References:
  https://www.illumos.org/issues/3522
  illumos/illumos-gate@d5285cae913f4e01ffa0e6693a6d8ef1fbea30ba

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Porting notes:

1. ZFSOnLinux had already addressed many of these issues because of
   its use of -Wall. However, the manner in which they were addressed
   differed. The illumos fixes replace the ones previously made in
   ZFSOnLinux to reduce code differences.

2. Part of the upstream patch made a small change to arc.c that might
   address zfsonlinux/zfs#1334.

3. The initialization of aclsize in zfs_log_create() differs because
   vsecp is a NULL pointer on ZFSOnLinux.

4. The changes to zfs_register_callbacks() were dropped because it
   has diverged and needs to be resynced.

11 years agoAdd cstyle.pl utility and cstyle.1 man page
Brian Behlendorf [Wed, 30 Oct 2013 18:19:53 +0000 (11:19 -0700)]
Add cstyle.pl utility and cstyle.1 man page

Cstyle is the C source style checker used by Illumos.  Since the
original ZFS source was written using these style guidelines they
must also be followed by ZoL for consistency.

The checker has been added to the scripts directory and may be
run on a per file basis.  New patches should be careful to avoid
introducing new style warnings.

Additionally, the 'checkstyle' target has been added to the top
level Makefile and can be used to check the entire source tree.
While Zol has historically attempted to follow the SunOS style
guide the lack of a rigorous style checker has allowed various
warning to be introduced.  Currently there are 2211 reported
style violations and we want to gradually eliminate these from
the tree.

Note the cstyle.1 man page is provided under man/man1/cstyle.1
but since it is a developer utility it is not installed along
with the other man pages.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
11 years agoAdd missing code to zfs_debug.{c,h}
Richard Yao [Wed, 9 Oct 2013 02:37:38 +0000 (22:37 -0400)]
Add missing code to zfs_debug.{c,h}

This is required to make Illumos 3962 merge.

Signed-off-by: Richard Yao <ryao@gentoo.org>
11 years agoAdd missing copyright notices from Illumos
Richard Yao [Mon, 7 Oct 2013 10:53:58 +0000 (06:53 -0400)]
Add missing copyright notices from Illumos

This resolves merge conflicts when merging Illumos #3588 and Illumos #4047.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoFix incorrect usage of strdup() in zfs_unmount_snap()
Richard Yao [Tue, 8 Oct 2013 21:59:42 +0000 (17:59 -0400)]
Fix incorrect usage of strdup() in zfs_unmount_snap()

Modifying the length of a string returned by strdup() is incorrect
because strfree() is allowed to use strlen() to determine which slab
cache was used to do the allocation.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoFix order of function calls in zio_free_sync()
Richard Yao [Mon, 7 Oct 2013 11:30:22 +0000 (07:30 -0400)]
Fix order of function calls in zio_free_sync()

The resolution of a merge conflict when merging Illumos #3464 caused us
to invert the order couple of function calls in zio_free_sync() versus
what they are in Illumos.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoReintroduce uio_prefaultpages()
Richard Yao [Mon, 2 Sep 2013 04:22:30 +0000 (00:22 -0400)]
Reintroduce uio_prefaultpages()

This was accidentally removed by overzealous commenting.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1775

11 years agoPosix ACL Support
Massimo Maggi [Mon, 28 Oct 2013 16:22:15 +0000 (09:22 -0700)]
Posix ACL Support

This change adds support for Posix ACLs by storing them as an xattr
which is common practice for many Linux file systems.  Since the
Posix ACL is stored as an xattr it will not overwrite any existing
ZFS/NFSv4 ACLs which may have been set.  The Posix ACL will also
be non-functional on other platforms although it may be visible
as an xattr if that platform understands SA based xattrs.

By default Posix ACLs are disabled but they may be enabled with
the new 'aclmode=noacl|posixacl' property.  Set the property to
'posixacl' to enable them.  If ZFS/NFSv4 ACL support is ever added
an appropriate acltype will be added.

This change passes the POSIX Test Suite cleanly with the exception
of xacl/00.t test 45 which is incorrect for Linux (Ext4 fails too).

  http://www.tuxera.com/community/posix-test-suite/

Signed-off-by: Massimo Maggi <me@massimo-maggi.eu>
Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #170

11 years agoImprove xattr property documentation
Brian Behlendorf [Mon, 28 Oct 2013 18:57:15 +0000 (11:57 -0700)]
Improve xattr property documentation

Extend the xattr property section of zfs(8) such that it covers
both styles of supported xattr.  A short discussion of the benefits
and drawbacks of each type is presented to allow users to make an
informed choice.

Signed-off-by: Massimo Maggi <me@massimo-maggi.eu>
Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #170

11 years agoPrevent xattr remove from creating xattr directory
Brian Behlendorf [Mon, 28 Oct 2013 16:07:00 +0000 (09:07 -0700)]
Prevent xattr remove from creating xattr directory

Attempting to remove an xattr from a file which does not contain
any directory based xattrs would result in the xattr directory
being created.  This behavior is non-optimal because it results
in write operations to the pool in addition to the expected error
being returned.

To prevent this the CREATE_XATTR_DIR flag is only passed in
zpl_xattr_set_dir() when setting a non-NULL xattr value.  In
addition, zpl_xattr_set() is updated similarly such that it will
return immediately if passed an xattr name which doesn't exist
and a NULL value.

Signed-off-by: Massimo Maggi <me@massimo-maggi.eu>
Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #170

11 years agoAdd script to fix file names in upstream patches
Prakash Surya [Mon, 26 Aug 2013 16:23:09 +0000 (09:23 -0700)]
Add script to fix file names in upstream patches

Added a simple sed script to do a search and replace on the Illumos
ZFS file names and replace them with the ZFS on Linux equivalent.

Example usage:

    # Replace Illumos paths with Linux paths
    $ ./scripts/zfs2zol-patch.sed arc.c.patch > arc.c.patch.linux

    # Ensure the script worked as expected
    $ diff arc.c.patch arc.c.patch.linux

    # Apply the patch using Linux paths
    $ patch -p1 < arc.c.patch.linux

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Prakash Surya <surya1@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1679

11 years agoRestructure zfs_readdir() to fix regressions
Richard Yao [Wed, 2 Oct 2013 15:22:53 +0000 (11:22 -0400)]
Restructure zfs_readdir() to fix regressions

This does the following:

1. It creates a uint8_t type value, which is initialized to DT_DIR on
dot directories and ZFS_DIRENT_TYPE(zap.za_first_integer) otherwise.
This resolves a regression where we return unintialized values as the
directory entry type on dot directories. This was accidentally
introduced by commit 8170d281263e52ff33d7fba93ab625196844df36.

2. It restructures zfs_readdir() code to use `uint64_t offset` like
Illumos instead of `loff_t *pos`. This resolves a regression where
negative ZAP cursors were treated as if they were dot directories.

3. It restructures the function to more closely match the structure of
zfs_readdir() on Illumos and removes the unused variable outcount, which
was only used on Illumos.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1750

11 years agoAdd -p switch to "zpool get"
Ralf Ertzinger [Wed, 23 Oct 2013 08:50:48 +0000 (10:50 +0200)]
Add -p switch to "zpool get"

This works the same as the -p switch to "zfs get", displaying full
resolution values for appropriate attributes.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1813

11 years agoIntroduce zpool_get_prop_literal interface
Ralf Ertzinger [Wed, 23 Oct 2013 08:33:33 +0000 (10:33 +0200)]
Introduce zpool_get_prop_literal interface

This change introduces zpool_get_prop_literal. It's an expanded version
of zpool_get_prop taking one additional boolean parameter. With this
parameter set to B_FALSE it will behave identically to zpool_get_prop.
Setting it to B_TRUE will return full precision numbers for the
following properties:

ZPOOL_PROP_SIZE
ZPOOL_PROP_ALLOCATED
ZPOOL_PROP_FREE
ZPOOL_PROP_FREEING
ZPOOL_PROP_EXPANDSZ
ZPOOL_PROP_ASHIFT

Also introduced is a wrapper function for zpool_get_prop making it
use zpool_get_prop_literal in the background.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1813

11 years agoCorrected "zfs list -t <type>" syntax
Steven Hartland [Wed, 23 Oct 2013 23:45:45 +0000 (00:45 +0100)]
Corrected "zfs list -t <type>" syntax

in man page and in command help.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1805

11 years agoMerge branch 'kstat'
Brian Behlendorf [Fri, 25 Oct 2013 22:22:34 +0000 (15:22 -0700)]
Merge branch 'kstat'

This branch updates several of the zfs kstats to take advantage
of the improved raw kstat functionality.  In addition, two new
kstats and a script called dbufstat.py are introduced.

Updated+New Kstats
* dbufs        - Stats for all dbufs in the dbuf_hash
* <pool>/txgs  - Stats for the last N txgs synced to disk
* <pool>/reads - Stats for rhe last N reads issues by the ARC
* <pool>/dmu_tx_assign - Histogram of tx assign times

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
11 years agoAdd dbufstat.py command
Brian Behlendorf [Fri, 25 Oct 2013 20:58:45 +0000 (13:58 -0700)]
Add dbufstat.py command

The dbufstat.py command was added to provide a conveniant way to
easily determine what ZFS is caching.  The script consumes the
raw /proc/spl/kstat/zfs/dbufs kstat data can consolidates it in
to a more human readable form.  This was designed primarily as
a tool to aid developers but it may also be useful for advanced
users who want more visibility in to what the ARC is caching.

When run without options dbufstat.py will default to showing a
list of all objects with at least one buffer present in the
cache.  The total cache space consumed by that object will be
printed on the right along with the object type.  Similar to the
arcstats.py command the -x option may used to display additional
fields.

Two other modes of operation are also supported by dbufstat.py
and the expectation is additional display modes may be added as
needed.  The -t option will summerize the total number of bytes
cached for each object type, and the -b option will show every
dbuf currently cached.

The script was designed to be consistent with arcstat.py and
includes most of the same options and funcationality.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
11 years agoAdd visibility in to cached dbufs
Brian Behlendorf [Thu, 3 Oct 2013 00:11:19 +0000 (17:11 -0700)]
Add visibility in to cached dbufs

Currently there is no mechanism to inspect which dbufs are being
cached by the system.  There are some coarse counters in arcstats
by they only give a rough idea of what's being cached.  This patch
aims to improve the current situation by adding a new dbufs kstat.

When read this new kstat will walk all cached dbufs linked in to
the dbuf_hash.  For each dbuf it will dump detailed information
about the buffer.  It will also dump additional information about
the referenced arc buffer and its related dnode.  This provides a
more complete view in to exactly what is being cached.

With this generic infrastructure in place utilities can be written
to post-process the data to understand exactly how the caching is
working.  For example, the data could be processed to show a list
of all cached dnodes and how much space they're consuming.  Or a
similar list could be generated based on dnode type.  Many other
ways to interpret the data exist based on what kinds of questions
you're trying to answer.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Prakash Surya <surya1@llnl.gov>
11 years agoAdd visibility in to dmu_tx_assign times
Brian Behlendorf [Wed, 2 Oct 2013 18:43:52 +0000 (11:43 -0700)]
Add visibility in to dmu_tx_assign times

This change adds a new kstat to gain some visibility into the
amount of time spent in each call to dmu_tx_assign. A histogram
is exported via the new dmu_tx_assign file. The information
contained in this histogram is the frequency dmu_tx_assign
took to complete given an interval range.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
11 years agoAdd visibility in to txg sync behavior
Brian Behlendorf [Tue, 1 Oct 2013 16:50:50 +0000 (09:50 -0700)]
Add visibility in to txg sync behavior

This change is an attempt to add visibility in to how txgs are being
formed on a system, in real time. To do this, a list was added to the
in memory SPA data structure for a pool, with each element on the list
corresponding to txg. These entries are then exported through the kstat
interface, which can then be interpreted in userspace.

For each txg, the following information is exported:

 * Unique txg number (uint64_t)
 * The time the txd was born (hrtime_t)
   (*not* wall clock time; relative to the other entries on the list)
 * The current txg state ((O)pen/(Q)uiescing/(S)yncing/(C)ommitted)
 * The number of reserved bytes for the txg (uint64_t)
 * The number of bytes read during the txg (uint64_t)
 * The number of bytes written during the txg (uint64_t)
 * The number of read operations during the txg (uint64_t)
 * The number of write operations during the txg (uint64_t)
 * The time the txg was closed (hrtime_t)
 * The time the txg was quiesced (hrtime_t)
 * The time the txg was synced (hrtime_t)

Note that while the raw kstat now stores relative hrtimes for the
open, quiesce, and sync times.  Those relative times are used to
calculate how long each state took and these deltas and printed by
output handlers.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
11 years agoAdd visibility in to arc_read
Prakash Surya [Fri, 6 Sep 2013 23:09:05 +0000 (16:09 -0700)]
Add visibility in to arc_read

This change is an attempt to add visibility into the arc_read calls
occurring on a system, in real time. To do this, a list was added to the
in memory SPA data structure for a pool, with each element on the list
corresponding to a call to arc_read. These entries are then exported
through the kstat interface, which can then be interpreted in userspace.

For each arc_read call, the following information is exported:

 * A unique identifier (uint64_t)
 * The time the entry was added to the list (hrtime_t)
   (*not* wall clock time; relative to the other entries on the list)
 * The objset ID (uint64_t)
 * The object number (uint64_t)
 * The indirection level (uint64_t)
 * The block ID (uint64_t)
 * The name of the function originating the arc_read call (char[24])
 * The arc_flags from the arc_read call (uint32_t)
 * The PID of the reading thread (pid_t)
 * The command or name of thread originating read (char[16])

From this exported information one can see, in real time, exactly what
is being read, what function is generating the read, and whether or not
the read was found to be already cached.

There is still some work to be done, but this should serve as a good
starting point.

Specifically, dbuf_read's are not accounted for in the currently
exported information. Thus, a follow up patch should probably be added
to export these calls that never call into arc_read (they only hit the
dbuf hash table). In addition, it might be nice to create a utility
similar to "arcstat.py" to digest the exported information and display
it in a more readable format. Or perhaps, log the information and allow
for it to be "replayed" at a later time.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
11 years agoRevert "Add txgs-<pool> kstat file"
Brian Behlendorf [Mon, 30 Sep 2013 18:51:20 +0000 (11:51 -0700)]
Revert "Add txgs-<pool> kstat file"

This reverts commit e95853a331529a6cb96fdf10476c53441e59f4e1.

11 years agoRevert "Add new kstat for monitoring time in dmu_tx_assign"
Brian Behlendorf [Mon, 30 Sep 2013 18:45:58 +0000 (11:45 -0700)]
Revert "Add new kstat for monitoring time in dmu_tx_assign"

This reverts commit 92334b14ec378b1693573b52c09816bbade9cf3e.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
11 years agoIncrease default udev wait time
Brian Behlendorf [Fri, 11 Oct 2013 21:24:18 +0000 (14:24 -0700)]
Increase default udev wait time

When creating a new pool, or adding/replacing a disk in an existing
pool, partition tables will be automatically created on the devices.
Under normal circumstances it will take less than a second for udev
to create the expected device files under /dev/.  However, it has
been observed that if the system is doing heavy IO concurrently udev
may take far longer.  If you also throw in some cheap dodgy hardware
it may take even longer.

To prevent zpool commands from failing due to this the default wait
time for udev is being increased to 30 seconds.  This will have no
impact on normal usage, the increase timeout should only be noticed
if your udev rules are incorrectly configured.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1646

11 years agoLinux 3.11 compat: Rename LZ4 symbols
Richard Yao [Sat, 5 Oct 2013 21:55:24 +0000 (17:55 -0400)]
Linux 3.11 compat: Rename LZ4 symbols

Linus Torvalds merged LZ4 into Linux 3.11. This causes a conflict
whenever CONFIG_LZ4_DECOMPRESS=y or CONFIG_LZ4_COMPRESS=y are set in the
kernel's .config. We rename the symbols to avoid the conflict.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1789

11 years agoDedup-related documentation additions for zpool and zdb.
Tim Chase [Sun, 13 Oct 2013 16:36:15 +0000 (11:36 -0500)]
Dedup-related documentation additions for zpool and zdb.

Document the "-D" and "-T" options and the optional interval
and count or "zpool status".

Also for zpool's man page, use a consistent order for the
various "-T" options to match the program's help output.

Document the effect of additional "-D" options for zdb.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1786

11 years agoAdd missing dsl pool configuration lock
Tim Chase [Sat, 12 Oct 2013 22:33:28 +0000 (17:33 -0500)]
Add missing dsl pool configuration lock

The semantics introduced by the restructured sync task of illumos
3464 require this lock when calling dmu_snapshot_list_next().
The pool is locked/unlocked for each iteration to reduce the
chance of long-running locks.

This was accidentally missed when doing the original port because
ZoL's control directory code is Linux-specific and is in a
different file than in illumos.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1785

11 years agoIllumos #3552
George Wilson [Thu, 21 Feb 2013 21:58:29 +0000 (13:58 -0800)]
Illumos #3552

3552 condensing one space map burns 3 seconds of CPU in spa_sync()
     thread (fix race condition)

References:
  https://www.illumos.org/issues/3552
  illumos/illumos-gate@03f8c366886542ed249a15d755ae78ea4e775d9d

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Porting notes:

This fixes an upstream regression that was introduced in commit
zfsonlinux/zfs@e51be06697762215dc3b679f8668987034a5a048, which
ported the Illumos 3552 changes. This fix was added to upstream
rather quickly, but at the time of the port, no one spotted it and
the race was rare enough that it passed our regression tests. I
discovered this when comparing our metaslab.c to the illumos
metaslab.c.

Without this change it is possible for metaslab_group_alloc() to
consume a large amount of cpu time.  Since this occurs under a
mutex in a rcu critical section the kernel will log this to the
console as a self-detected cpu stall as follows:

  INFO: rcu_sched self-detected stall on CPU { 0}
  (t=60000 jiffies g=11431890 c=11431889 q=18271)

Closes #1687
Closes #1720
Closes #1731
Closes #1747

11 years agoFix libzfs_core changes to follow GNU libtool guidelines
Richard Yao [Thu, 26 Sep 2013 17:44:10 +0000 (13:44 -0400)]
Fix libzfs_core changes to follow GNU libtool guidelines

The GNU libtool documentation states to start with a version of 0:0:0,
rather than 1:1:0. Illumos uses the name libzfs_core.so.1, so to be
consistent, we should go with 1:0:0.

http://www.gnu.org/software/libtool/manual/libtool.html#Updating-version-info

The GNU libtool documentation also provides guidence on how the version
information should be incremented. Doing this does a SONAME bump of the
libzfs and libzpool libraries. This is particularly important on Gentoo
because a SONAME bump enables portage to retain the older libraries
until any packages that link to them are rebuilt. The main example of
this is GRUB2's grub2-mkconfig, which will break unless it is rebuilt
against the new libraries.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1751

11 years agoGenerate libraries with correct DT_NEEDED entries
Richard Yao [Thu, 26 Sep 2013 17:42:41 +0000 (13:42 -0400)]
Generate libraries with correct DT_NEEDED entries

Libraries that depend on other libraries should list them in ELF's
DT_NEEDED field so that programs linking to them do not need to specify
those libraries unless they depend on them as well. This is not the case
in the current code and the consequence is that anything that needs a
library must know its dependencies. This is fragile and caused GRUB2's
configure script to break when a dependency was added on libblkid in
libzfs.

This resolves that problem by using LIBADD/LDADD to specify libraries in
Makefile.am instead of LDFLAGS. This ensures that proper DT_NEEDED
entries are generated and prevents GRUB2's configure script from
breaking in the presence of a libblkid dependency. This also removes
unneeded dependencies from various files.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1751

11 years agoFix libblkid support
Richard Yao [Wed, 28 Aug 2013 20:17:47 +0000 (16:17 -0400)]
Fix libblkid support

libblkid support is dormant because the autotools check is broken and
liblkid identifies ZFS vdevs as "zfs_member", not "zfs". We fix that
with a few changes:

First, we fix the libblkid autotools check to do a few things:

1. Make a 64MB file, which is the minimum size ZFS permits.
2. Make 4 fake uberblock entries to make libblkid's check succeed.
3. Return 0 upon success to make autotools use the success case.
4. Include stdlib.h to avoid implicit declration of free().
5. Check for "zfs_member", not "zfs"
6. Make --with-blkid disable autotools check (avoids Gentoo sandbox violation)
7. Pass '-lblkid' correctly using LIBS not LDFLAGS.

Second, we change the libblkid support to scan for "zfs_member", not
"zfs".

This makes --with-blkid work on Gentoo.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #1751

11 years agoUpdate detach section of zpool(8)
Neil Stockbridge [Wed, 9 Oct 2013 05:58:30 +0000 (18:58 +1300)]
Update detach section of zpool(8)

The detach section of the zpool(8) man page now suggests the
offline command.  Using offline may be more appropriate for
certain situations.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1776

11 years agoExport symbols dsl_pool_config_{enter,exit}
Ned Bass [Mon, 30 Sep 2013 23:29:37 +0000 (16:29 -0700)]
Export symbols dsl_pool_config_{enter,exit}

These are needed by consumers (i.e. Lustre) who wish to use the
dsl_prop_register() interface to register callbacks when pool
properties of interest change.  This interface requires that the
DSL pool configuration lock is held when called.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #1762

11 years agoFix memory leak false positive in log_internal()
Brian Behlendorf [Wed, 2 Oct 2013 17:00:04 +0000 (10:00 -0700)]
Fix memory leak false positive in log_internal()

When building the spl with --enable-debug-kmem-tracking a memory
leak is detected in log_internal().  This happens to be a false
positive because the memory was freed using strfree() instead of
kmem_free().  All kmem_alloc()'s must be released with kmem_free()
to ensure correct accounting.

  SPL: kmem leaked 135/5641311 bytes
  address          size  data             func:line
  ffff8800cba7cd80 135   ZZZZZZZZZZZZZZZZ log_internal:456

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>