]> granicus.if.org Git - zfs/log
zfs
7 years agoOpenZFS 7736 - ZFS Performance tests should log FIO summary output
Ahmed G [Wed, 18 Jan 2017 00:53:31 +0000 (16:53 -0800)]
OpenZFS 7736 - ZFS Performance tests should log FIO summary output

Authored by: Ahmed G <ahmedg@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Stephen Blinick <stephen.blinick@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Ported-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Porting Notes:
- Using $FIO until 7290 is ported.

OpenZFS-issue: https://www.illumos.org/issues/7736
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/7a61309
Closes #5827

7 years agoOpenZFS 7812 - Remove gender specific language
Daniel Hoffman [Fri, 17 Feb 2017 19:48:20 +0000 (11:48 -0800)]
OpenZFS 7812 - Remove gender specific language

Authored by: Daniel Hoffman <dj.hoffman@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: Chris Williamson <chris.williamson@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Reviewed by: Dan McDonald <danmcd@omniti.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
This change removes all gendered language that did not refer specifically
to an individual person or pet. The convention taken was to use
variations on "they" when referring to users and/or human beings, while
using "it" when referring to code, functions, and/or libraries.
Additionally, we took the liberty to fix up any whitespace issues that
were found in any files that were already being modified.

OpenZFS-issue: https://www.illumos.org/issues/7812
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/ad626db
Closes #5822

7 years agoOpenZFS 7761 - bootfs_005_neg's pool destruction must handle EBUSY
Prakash Surya [Thu, 12 Jan 2017 00:36:58 +0000 (16:36 -0800)]
OpenZFS 7761 - bootfs_005_neg's pool destruction must handle EBUSY

Authored by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Yuri Pankov <yuri.pankov@gmail.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7761
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/ad309d3
Closes #5818

7 years agoOpenZFS 7199 - dsl_dataset_rollback_sync may try to free already free blocks
Andriy Gapon [Mon, 21 Nov 2016 23:09:54 +0000 (15:09 -0800)]
OpenZFS 7199 - dsl_dataset_rollback_sync may try to free already free blocks

7200 no blocks must be born in a txg after a snaphot is created
Authored by: Andriy Gapon <andriy.gapon@clusterhq.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Approved by: Gordon Ross <gordon.w.ross@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7199
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/bfaed0b
Closes #5817

7 years agoOpenZFS 7337 - inherit_001_pos occasionally times out
Matthew Ahrens [Sat, 24 Sep 2016 03:44:15 +0000 (20:44 -0700)]
OpenZFS 7337 - inherit_001_pos occasionally times out

Authored by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7337
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/b021ac0
Closes #5800

Porting notes:
- Additional code refactor for better Zol and OpenZFS codebase sync

7 years agoAllow zfs-tests to run a single test
Giuseppe Di Natale [Fri, 24 Feb 2017 18:59:24 +0000 (10:59 -0800)]
Allow zfs-tests to run a single test

Add a -t flag to zfs-tests to allow a user
to run a single test by providing the path
to the test relative to STF_SUITE.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Closes #5775

7 years agoFix incorrect spare vdev state after replacing
Isaac Huang [Thu, 23 Feb 2017 18:32:15 +0000 (11:32 -0700)]
Fix incorrect spare vdev state after replacing

After a hot spare replaces an OFFLINE vdev, the new
parent spare vdev state is set incorrectly to OFFLINE.
The correct state should be DEGRADED. The incorrect
OFFLINE state will prevent top-level vdev from reading
the spare vdev, thus causing unnecessary reconstruction.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Don Brady <don.brady@intel.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Isaac Huang <he.huang@intel.com>
Closes #5766
Closes #5770

7 years agoRetry setting LED
Christopher Voltz [Thu, 16 Feb 2017 21:41:48 +0000 (15:41 -0600)]
Retry setting LED

If the LED is being accessed by another process when we try to update
it, the update will be lost. Add a retry loop which will read the state
of the LED and update it until the LED is in the correct state. The
number of times this will occur is limited to ensure that the ZEDlet
won't hang ZED.

Refactor to remove duplication so setting of the LED occurs in only one
place.

Cleanup a couple of the warnings generated by shellcheck which weren't
the result of specific choices by the author. Several notes and warnings
are still present but removing them would make the code less clear or
require adding lines to tell shellcheck to ignore the warning.

Remove ",i" from the documentation at the top of the file which appears
to be a typographic error.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Signed-off-by: Christopher Voltz <christopher.voltz@hpe.com>
Closes #5795

7 years agoOpenZFS 7001 - zvol_misc tests should not depend on /sbin or /usr/sbin being in PATH
Hans Rosenfeld [Thu, 26 May 2016 15:32:26 +0000 (17:32 +0200)]
OpenZFS 7001 - zvol_misc tests should not depend on /sbin or /usr/sbin being in PATH

Authored by: Hans Rosenfeld <hans.rosenfeld@nexenta.com>
Reviewed by: Toomas Soome <tsoome@me.com>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7001
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/258c9c7
Closes #5801

Porting notes:
- Most of the changes were previously applied.

7 years agoOpenZFS 7248 - large block support breaks rsend_009_pos
John Wren Kennedy [Tue, 30 Aug 2016 19:01:41 +0000 (12:01 -0700)]
OpenZFS 7248 - large block support breaks rsend_009_pos

7249 rsend_015_pos produces false failures due to race
7250 testrunner can miss options specific to individual tests in runfiles

Authored by: John Wren Kennedy <john.kennedy@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7248
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/f9a78bf
Closes #5799

7 years agoOpenZFS 7762 - avoid division by zero in property_alias_001_pos
Prakash Surya [Wed, 18 Jan 2017 01:08:12 +0000 (17:08 -0800)]
OpenZFS 7762 - avoid division by zero in property_alias_001_pos

Authored by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: Giuseppe Di Natale <dinatale2@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/7762
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/ebaf15cb
Closes #5798

7 years agozfs_arc_num_sublists_per_state should be common to all multilists
Matthew Ahrens [Wed, 15 Feb 2017 23:49:33 +0000 (15:49 -0800)]
zfs_arc_num_sublists_per_state should be common to all multilists

The global tunable zfs_arc_num_sublists_per_state is used by the ARC and
the dbuf cache, and other users are planned. We should change this
tunable to be common to all multilists.  This tuning may be overridden
on a per-multilist basis.

Reviewed-by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed-by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #5764

7 years agoOpenZFS 6404 - zvol_swap_006_pos can occasionally fail due to swaplen being < 16
Matthew Ahrens [Sun, 15 May 2016 15:09:13 +0000 (08:09 -0700)]
OpenZFS 6404 - zvol_swap_006_pos can occasionally fail due to swaplen being < 16

Authored by: Matthew Ahrens <mahrens@delphix.com>
6405 zvol test setup is non deterministic
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Will Andrews <will@freebsd.org>
Approved by: Dan McDonald <danmcd@omniti.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/6404
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/24e268f
Closes #5792

Porting notes:
- Converted zfs to $ZFS until OpenZFS 7290 is ported.
  openzfs/openzfs@1d32ba6

7 years agoFix broken URL
Christopher Voltz [Wed, 15 Feb 2017 19:15:38 +0000 (13:15 -0600)]
Fix broken URL

Google moved their style guides to GitHub. Update the shell style guide
URL to the new location.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Christopher Voltz <christopher.voltz@hpe.com>
Closes #5797

7 years agoOpenZFS 7260 - disable libdiskmgmt in zfstest unless it's required
John Wren Kennedy [Mon, 8 Aug 2016 16:41:07 +0000 (09:41 -0700)]
OpenZFS 7260 - disable libdiskmgmt in zfstest unless it's required

Authored by: John Wren Kennedy <john.kennedy@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7260
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/447b1e1
Closes #5794

Porting notes:
- The library libdiskmgmt is specific to illumos so these changes
  currently have no impact under Linux.  This mechanism could be
  potentially leveraged in the future.

7 years agoUse file-based pools for zpool_expand test 002 and enable it
Olaf Faaland [Mon, 13 Feb 2017 23:30:22 +0000 (15:30 -0800)]
Use file-based pools for zpool_expand test 002 and enable it

Use -pH flags in get_pool_prop so that numeric properties such as size
can be compared.  The zpool_expand test suite is currently the only one
which uses get_pool_prop for a numeric property.

Add TEMPFILE and TEMPFILE{0,1,2} to default.cfg for tests that must
build pools on top of files, such as this one where expansion is
necessary but the entries in DISKS may not point to entities that can be
expanded.

Base the pool used for testing on file-type VDEVs instead of using zvols
within an underlying pool, to avoid issues that come up when pools are
backed by other pools.

Remove shell variables EX_1GB and EX_2GB used to recognize correct expansion,
and instead calculate the appropriate values based on the variables used to
control file or volume size, org_size and exp_size.  This change is also
made in test 001 although that test is not enabled because it depends on
FMA.

Finally, enable zpool_expand_002_pos.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Reviewed-by: Don Brady <don.brady@intel.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Closes #5757

7 years agoOpenZFS 7027 - zfs_written_property_001_pos makes unreasonable assumptions about...
Akash Ayare [Fri, 3 Jun 2016 04:00:23 +0000 (21:00 -0700)]
OpenZFS 7027 - zfs_written_property_001_pos makes unreasonable assumptions about metadata space usage

Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Approved by: Gordon Ross <gordon.ross@nexenta.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/7027
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/48cb8b9
Issue #2441
Closes #5778

7 years agoOpenZFS 6754 - zfs-tests: get_substr() function is redundant
Yuri Pankov [Fri, 18 Mar 2016 21:11:23 +0000 (00:11 +0300)]
OpenZFS 6754 - zfs-tests: get_substr() function is redundant

Authored by: Yuri Pankov <yuri.pankov@gmail.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Approved by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/6754
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/01ff411
Closes #5787

Porting notes:
- Only the ACL test cases used this function and they have been
  dropped from ZoL since Linux used POSIX style ACLs.

  - functional/acl/nontrivial/zfs_acl_chmod_aclmode_001_pos.ksh
  - functional/acl/nontrivial/zfs_acl_chmod_inherit_003_pos.ksh
  - functional/acl/nontrivial/zfs_acl_chmod_rwx_002_pos.ksh

7 years agoDisable racy snapshot_008_pos
George Melikov [Mon, 13 Feb 2017 20:02:22 +0000 (23:02 +0300)]
Disable racy snapshot_008_pos

Sometimes zfstests check freed space just after
`zfs destroy snapshot` and get wrong output, because the
space being freed asynchronously in the background.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Issue #5740
Issue #5784
Closes #5785

7 years agoOpenZFS 7036 - zvol_swap_004_pos test failed
Igor Kozhukhov [Thu, 9 Jun 2016 20:38:35 +0000 (16:38 -0400)]
OpenZFS 7036 - zvol_swap_004_pos test failed

Authored by: Igor Kozhukhov <ikozhukhov@gmail.com>
Reviewed by: - John Kennedy <john.kennedy@delphix.com>
Approved by: - Dan McDonald <danmcd@omniti.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7036
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/3f85b2a
Closes #5783

7 years agoOpenZFS 7398 - zfs test zfs_get_005_neg does not work as expected
Marcel Telka [Thu, 22 Sep 2016 13:36:32 +0000 (15:36 +0200)]
OpenZFS 7398 - zfs test zfs_get_005_neg does not work as expected

Authored by: Marcel Telka <marcel@telka.sk>
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7398
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/4220fdc
Closes #5782

7 years agoOpenZFS 7103 - failed test cli_root/zfs_snapshot/zfs_snapshot_009_pos
Marcel Telka [Wed, 19 Oct 2016 17:50:39 +0000 (10:50 -0700)]
OpenZFS 7103 - failed test cli_root/zfs_snapshot/zfs_snapshot_009_pos

Authored by: Marcel Telka <marcel@telka.sk>
Reviewed by: - Igor Kozhukhov <ikozhukhov@gmail.com>
Reviewed by: - Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: - John Kennedy <john.kennedy@delphix.com>
Approved by: - Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7103
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/3bfdbb4
Closes #5780

7 years agoOpenZFS 7162 - Intermittent failures from ro_props_001_pos
Matthew Ahrens [Tue, 19 Jul 2016 23:10:27 +0000 (16:10 -0700)]
OpenZFS 7162 - Intermittent failures from ro_props_001_pos

Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/7162
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/9ec0cbeb
Closes #5511
Closes #5779

7 years agoFix zfs_compressed_arc_enabled parameter description
Tim Chase [Mon, 13 Feb 2017 18:59:05 +0000 (12:59 -0600)]
Fix zfs_compressed_arc_enabled parameter description

A likely cut/paste error caused the description to be applied to
zfs_arc_average_blocksize.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tim Chase <tim@chase2k.com>
Closes #5788

7 years agoOpenZFS 6580 - zfs-tests use undefined variable WRAPPER
Yuri Pankov [Sat, 9 Apr 2016 16:36:54 +0000 (09:36 -0700)]
OpenZFS 6580 - zfs-tests use undefined variable WRAPPER

Authored by: Yuri Pankov <yuri.pankov@gmail.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Approved by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/6580
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/9498083
Closes #5789

Porting notes:
- deleted in ZoL: functional/utils_test/utils_test_007_pos.ksh

7 years agoOpenZFS 7496 - cmp_ds_cont has never worked
John Wren Kennedy [Sun, 6 Nov 2016 03:32:44 +0000 (20:32 -0700)]
OpenZFS 7496 - cmp_ds_cont has never worked

Authored by: John Wren Kennedy <john.kennedy@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Approved by: Gordon Ross <gordon.w.ross@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7496
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/5dc1fd7
Closes #5781

7 years agoOpenZFS 6642 - testrunner output can be displayed in the wrong order
John Wren Kennedy [Thu, 25 Feb 2016 05:13:41 +0000 (21:13 -0800)]
OpenZFS 6642 - testrunner output can be displayed in the wrong order

6643 zfstest should enforce the required privileges before running.
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Jonathan Mackenzie <jonathan.mackenzie@delphix.com>
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov
OpenZFS-issue: https://www.illumos.org/issues/6642
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/412db4e
Closes #5777

Porting notes:
- The 6643 changes were dropped a different version of this script
  is used to configure the environment under Linux.

7 years agoFix off by one in zpl_lookup
Chunwei Chen [Sat, 11 Feb 2017 20:42:17 +0000 (12:42 -0800)]
Fix off by one in zpl_lookup

Doing the following command would return success with zfs creating an orphan
object.

touch $(for i in $(seq 256); do printf "n"; done)

The funny thing is that this will only work once for each directory, because
after upgraded to fzap, zfs_lookup would fail properly since it has additional
length check.

Signed-off-by: Chunwei Chen <david.chen@osnexus.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #5768

7 years agoEnclosure LED fixes
Tony Hutter [Sat, 11 Feb 2017 00:09:45 +0000 (16:09 -0800)]
Enclosure LED fixes

- Pass $VDEV_ENC_SYSFS_PATH to 'zpool [iostat|status] -c' to include
  enclosure LED sysfs path.

- Set LEDs correctly after import.  This includes clearing any erroniously
  set LEDs prior to the import, and setting the LED for any UNAVAIL drives.

- Include symlink for vdev_attach-led.sh in Makefile.am.

- Print the VDEV path in all-syslog.sh, and fix it so the pool GUID actually
  prints.

Reviewed-by: Don Brady <don.brady@intel.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tony Hutter <hutter2@llnl.gov>
Closes #5716
Closes #5751

7 years agoClear enclosure sysfs path from VDEV label when sysfs path isn't present
Tony Hutter [Fri, 10 Feb 2017 23:18:31 +0000 (15:18 -0800)]
Clear enclosure sysfs path from VDEV label when sysfs path isn't present

This clears vdev_enc_sysfs_path from the label if the VDEV's
/sys/class/block/<dev>/device/enclosure_device path isn't present.

This is important in the case where a disk that is labeled with
vdev_enc_sysfs_path is pulled out and put into another enclosure.
In that case, it's possible that the old sysfs path would be used to
turn on the fault LED for the disk's old slot postion, assuming the
new slot didn't have a LED sysfs entry.

Reviewed-by: Don Brady <don.brady@intel.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tony Hutter <hutter2@llnl.gov>
Closes #5524
Closes #5773

7 years agoOpenZFS 5704 - libzfs can only handle 255 file descriptors
Simon Klinkert [Wed, 11 Mar 2015 10:02:08 +0000 (11:02 +0100)]
OpenZFS 5704 - libzfs can only handle 255 file descriptors

Authored by: Simon Klinkert <simon.klinkert@gmail.com>
Reviewed by: Josef 'Jeff' Sipek <josef.sipek@nexenta.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Ned Bass <bass6@llnl.gov>
Reviewed-by: Chunwei Chen <david.chen@osnexus.com>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/5704
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/bde3d61
Closes #5767

7 years agoOpenZFS 7104 - increase indirect block size
Matthew Ahrens [Wed, 13 Jul 2016 23:48:01 +0000 (16:48 -0700)]
OpenZFS 7104 - increase indirect block size

Authored by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Dan McDonald <danmcd@omniti.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7104
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/4b5c8e9
Closes #5679

7 years agopanic in bpobj_space(): null pointer dereference
Matthew Ahrens [Thu, 9 Feb 2017 18:19:12 +0000 (10:19 -0800)]
panic in bpobj_space(): null pointer dereference

This is a race condition in the deadlist code.

A thread executing an administrative command that uses
dsl_deadlist_space_range() holds the lock of the whole deadlist_t to
protect the access of all its entries that the deadlist contains in an
avl tree.

Sync threads trying to insert a new entry in the deadlist (through
dsl_deadlist_insert() -> dle_enqueue()) do not hold the deadlist lock at
that moment.  If the dle_bpobj is the empty bpobj (our sentinel value),
we close and reopen it.  Between these two operations, it is possible
for the dsl_deadlist_space_range() thread to dereference that bpobj
which is NULL during that window.

Threads should hold the a deadlist's dl_lock when they manipulate its
internal data so scenarios like the one above are avoided.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #5762

7 years agoMove ziltest.sh to the ZTS framework
Brian Behlendorf [Wed, 8 Feb 2017 01:12:53 +0000 (20:12 -0500)]
Move ziltest.sh to the ZTS framework

The ziltest.sh script is a test case designed to verify the correct
functioning of the ZIL.  For historical reasons it was never added
to the test suite and was always run independantly.

This change rectifies that.  The existing ziltest.sh has been
translated in to `slog_015_pos.ksh` and added to the existing
slog test cases.

Reviewed-by: Don Brady <don.brady@intel.com>
Reviewed-by: Chunwei Chen <david.chen@osnexus.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #5758

7 years agoFix iput() calls within a tx
Brian Behlendorf [Tue, 7 Feb 2017 23:57:50 +0000 (18:57 -0500)]
Fix iput() calls within a tx

As explicitly stated in section 2 of the 'Programming rules'
comments at the top of zfs_vnops.c.

  If you must call iput() within a tx then use zfs_iput_async().

Move iput() calls after dmu_tx_commit() / dmu_tx_abort when
possible.  When not possible convert the iput() calls to
zfs_iput_async().

Reviewed-by: Don Brady <don.brady@intel.com>
Reviewed-by: Chunwei Chen <david.chen@osnexus.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #5758

7 years agoAdd `wait_freeing` helper function to ZTS
George Melikov [Wed, 8 Feb 2017 23:27:37 +0000 (02:27 +0300)]
Add `wait_freeing` helper function to ZTS

Sometimes the ZTS checks freed space just after `zfs destroy snapshot` and
gets an unexpected value because of space being freed asynchronously.
For cases like this add a `wait_freeing` function which blocks until the
pools `freeing` property drops to zero.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes #5740

7 years agoFix zfs-mount.service failure on boot
Brian Behlendorf [Wed, 8 Feb 2017 18:17:29 +0000 (10:17 -0800)]
Fix zfs-mount.service failure on boot

The mount(8) command will helpfully try to resolve any device name
which is passed in.  It does this by applying some simple heuristics
before passing it along to the registered mount helper.

Normally this fine.  However, one of these heuristics is to prepend
the current working directory to the passed device name.  If that
resulting directory name exists mount(8) will perform the mount(2)
system call and never invoke the helper utility.

Since the cwd for systemd when running as the system instance is
the root directory the default mount points created by zfs(8) can
cause a mount failure.

This change avoids the issue by explicitly setting the cwd to
a different path when performing the mount.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #5719

7 years agoFix coverity defects: CID 155964, 155965
George Melikov [Wed, 8 Feb 2017 18:06:02 +0000 (21:06 +0300)]
Fix coverity defects: CID 155964, 155965

CID 155964: Missing break in switch
CID 155965: Missing break in switch

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes #5759

7 years agoAllow c99 code to compile
Matthew Ahrens [Wed, 8 Feb 2017 17:27:48 +0000 (09:27 -0800)]
Allow c99 code to compile

Add the appropriate compiler flags to accept c99 code.  This will help to
minimize differences with upstream, and aid porting changes.  One change was
necessary in zvol.c because the DEFINE_IDA() macro does not work with the new
compiler flags.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Chunwei Chen <david.chen@osnexus.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #5756

7 years agoOpenZFS 6931 - lib/libzfs: cleanup gcc warnings
George Melikov [Tue, 7 Feb 2017 22:02:27 +0000 (01:02 +0300)]
OpenZFS 6931 - lib/libzfs: cleanup gcc warnings

Authored by: Igor Kozhukhov <ikozhukhov@gmail.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/6931
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/88f61de
Closes #5741

7 years agoAdd missing module_param for zfs_per_txg_dirty_frees_percent
David Quigley [Tue, 7 Feb 2017 17:44:03 +0000 (10:44 -0700)]
Add missing module_param for zfs_per_txg_dirty_frees_percent

When the code was added this tunable was not exposed via module params. Also it
was not documented. This patch changes the type from a uint32 to a ulong as
done with other percentage tunables and also documents it in the
zfs-module-parameters man page.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: David Quigley <david.quigley@intel.com>
Closes #5750

7 years agoFix coverity defects: CID 155928
Don Brady [Tue, 7 Feb 2017 17:29:47 +0000 (10:29 -0700)]
Fix coverity defects: CID 155928

CID 155928: Integer handling issues (DIVIDE_BY_ZERO)

In the current vdev label, the leaf count is always non-zero
but it doesn't hurt to check the count for future proofing.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Don Brady <don.brady@intel.com>
Closes #5749

7 years agoFix ZFS Test Suite failures caused by ksh brace expansion feature
LOLi [Mon, 6 Feb 2017 18:08:10 +0000 (19:08 +0100)]
Fix ZFS Test Suite failures caused by ksh brace expansion feature

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Signed-off-by: loli10K <ezomori.nozomu@gmail.com>
Closes #5669
Closes #5743

7 years agoOpenZFS 7448 - ZFS doesn't notice when disk vdevs have no write cache
George Melikov [Sat, 4 Feb 2017 17:23:50 +0000 (20:23 +0300)]
OpenZFS 7448 - ZFS doesn't notice when disk vdevs have no write cache

Authored by: Hans Rosenfeld <hans.rosenfeld@nexenta.com>
Reviewed by: Dan Fields <dan.fields@nexenta.com>
Reviewed by: Alek Pinchuk <alek.pinchuk@nexenta.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Reviewed-by: Don Brady <don.brady@intel.com>
Reviewed-by: Tim Chase <tim@chase2k.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7448
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/295438b
Closes #5737

7 years agoOpenZFS 7504 - kmem_reap hangs spa_sync and administrative tasks
George Melikov [Sat, 4 Feb 2017 17:21:25 +0000 (20:21 +0300)]
OpenZFS 7504 - kmem_reap hangs spa_sync and administrative tasks

Authored by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Tim Chase <tim@chase2k.com>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7504
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/405a5a0
Closes #5736

7 years agoOpenZFS 7247 - zfs receive of deduplicated stream fails
George Melikov [Sat, 4 Feb 2017 17:10:24 +0000 (20:10 +0300)]
OpenZFS 7247 - zfs receive of deduplicated stream fails

Authored by: Chris Williamson <chris.williamson@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: loli10K <ezomori.nozomu@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7247
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/2ad25b4
Closes #5689

Porting notes:
- tests/zfs-tests/tests/functional/cli_root/zfs_receive/zfs_receive_013_pos.ksh
  renamed as zfs_receive_015_pos.ksh, zfs_receive_013_pos.ksh is now
  used for OpenZFS test.
- libzfs_sendrecv.c: SMALLEST_POSSIBLE_MAX_DDT_MB is always used
  for all 32-bit builds.

7 years agoDisable sparse_001_pos / truncate_001_pos.ksh on 32-bit systems
Brian Behlendorf [Sat, 4 Feb 2017 03:00:00 +0000 (19:00 -0800)]
Disable sparse_001_pos / truncate_001_pos.ksh on 32-bit systems

Commit 539d33c seems to have significantly increased the run time
of the sparse_001_pos.ksh and truncate_001_pos.ksh test cases on
32-bit systems.  This is likely due to dirty blocks from frees
being deferred to later txgs.

At the moment this is resulting in frequent failures on the
32-bit builders.  Disable this test case until the issue can be
analyzed and resolved.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #5727
Closes #5728

7 years agoOpenZFS 6866 - zdb -l non-zero status if no label
Don Brady [Fri, 3 Feb 2017 22:18:28 +0000 (15:18 -0700)]
OpenZFS 6866 - zdb -l non-zero status if no label

Authored by: Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Ported-by: Don Brady <don.brady@intel.com>
OpenZFS-issue: https://www.illumos.org/issues/6866
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/3e4fae5
Closes #5730

Porting Notes:
- Omitted the illumos-specific `/dev/dsk` and `/dev/rdsk`
path conversions since they don't apply on linux.

7 years agoAdd tmpfile_003_pos to .gitignore
Olaf Faaland [Fri, 3 Feb 2017 21:42:49 +0000 (13:42 -0800)]
Add tmpfile_003_pos to .gitignore

Earlier commit added a test that created tmpfile_003_pos,
but did not add it to the appropriate .gitignore.

ace1eae Add support for O_TMPFILE

Fix the omission.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Closes #5735

7 years agoOpenZFS 4521 - zfstest is trying to execute evil "zfs unmount -a"
Giuseppe Di Natale [Fri, 3 Feb 2017 21:24:44 +0000 (13:24 -0800)]
OpenZFS 4521 - zfstest is trying to execute evil "zfs unmount -a"

Authored by: Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Reviewed by: Dan McDonald <danmcd@omniti.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Ported-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Porting Notes:
- Correctly set __ZFS_POOL_RESTRICT in inherit_001_pos

OpenZFS-issue: https://www.illumos.org/issues/4521
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/8808ac5
Closes #5674

7 years agoOpenZFS 1300 - filename normalization doesn't work for removes
George Melikov [Thu, 2 Feb 2017 22:13:41 +0000 (01:13 +0300)]
OpenZFS 1300 - filename normalization doesn't work for removes

Authored by: Kevin Crowe <kevin.crowe@nexenta.com>
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/1300
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/8f1750d
Closes #5725

Porting notes:
- zap_micro.c: all `MT_EXACT` are replaced by `0`

7 years agoAdd nvlist payload stats for zdb -ll dump
Don Brady [Thu, 2 Feb 2017 22:03:48 +0000 (15:03 -0700)]
Add nvlist payload stats for zdb -ll dump

When dumping the ZFS vdev label with 'zdb -ll', also dump
some nvlist stats to help analyze the current footprint.

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Don Brady <don.brady@intel.com>
Closes #5724

7 years agoFix write(2) returns zero bug from 933ec99
Chunwei Chen [Thu, 2 Feb 2017 17:43:42 +0000 (09:43 -0800)]
Fix write(2) returns zero bug from 933ec99

For generic_write_checks with 2 args, we can exit when it returns zero because
it means count is zero. However this is not the case for generic_write_checks
with 4 args, where zero means no error.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Haakan T Johansson <f96hajo@chalmers.se>
Signed-off-by: Chunwei Chen <david.chen@osnexus.com>
Closes #5720
Closes #5726

7 years agoRemove lint ifdef checks in zdb and dbuf
Giuseppe Di Natale [Thu, 2 Feb 2017 00:47:04 +0000 (16:47 -0800)]
Remove lint ifdef checks in zdb and dbuf

This is effectively dead code for the Linux implementation which can
be removed to improve readability.  We want to linter to check the
real production/debug build as much as possible.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Closes #5722

7 years agoFix reservation test cases for large disks
Brian Behlendorf [Thu, 2 Feb 2017 00:28:01 +0000 (16:28 -0800)]
Fix reservation test cases for large disks

Convert explicit `typeset -i` and `typeset -l` declarations to
`typeset` in order to prevent 32-bit overflow from occurs with
disks >2G.

TEST_ZFSTESTS_DISKSIZE=4G

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #5715
Closes #5714

7 years agoOpenZFS 7072 - zfs fails to expand if lun added when os is in shutdown state
George Melikov [Wed, 1 Feb 2017 21:14:02 +0000 (00:14 +0300)]
OpenZFS 7072 - zfs fails to expand if lun added when os is in shutdown state

Authored by: George Wilson <george.wilson@delphix.com>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7072
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/c39a2aa
Closes #5694

Porting notes:
- vdev.c: 'vdev_get_stats' changes are moved to 'vdev_get_stats_ex'.
- vdev_disk.c: ignored, Linux specific code is different.

7 years agoFix atomic_sub_64() i386 assembly implementation
Brian Behlendorf [Wed, 1 Feb 2017 17:36:47 +0000 (09:36 -0800)]
Fix atomic_sub_64() i386 assembly implementation

The atomic_sub_64() should use sbbl instead of adcl.  In user
space these atomics are used for statistics tracking and aren't
critical which explain how this was overlooked.  The kernel
space implementation of these atomics are layered on the
architecture specific implementations provided by the kernel.

Reviewed by: Stefan Ring <stefanrin@gmail.com>
Reviewed-by: Gvozden Neskovic <neskovic@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #5671
Closes #5717

7 years agoUse fletcher_4 routines natively with `abd_iterate_func()`
David Quigley [Wed, 1 Feb 2017 17:34:22 +0000 (10:34 -0700)]
Use fletcher_4 routines natively with `abd_iterate_func()`

This patch adds the necessary infrastructure for ABD to make use
of the vectorized fletcher 4 routines.

- export ABD compatible interface from fletcher_4
- add ABD fletcher_4 tests for data and metadata ABD types.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Original-patch-by: Gvozden Neskovic <neskovic@gmail.com>
Signed-off-by: David Quigley <david.quigley@intel.com>
Closes #5589

7 years agoMatch on word for excluding pools from tests
Tim Crawford [Tue, 31 Jan 2017 22:47:32 +0000 (17:47 -0500)]
Match on word for excluding pools from tests

Running tests locally were failing on cleanup scripts due to having a
pool named "pool". Match on word so the cleanup logic will cleanup
"testpool.*" while ignoring "pool".

Reviewed-by: loli10K <ezomori.nozomu@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Tim Crawford <tcrawford@datto.com>
Closes #5703

7 years agoOpenZFS 6569 - large file delete can starve out write ops
George Melikov [Tue, 31 Jan 2017 22:44:03 +0000 (01:44 +0300)]
OpenZFS 6569 - large file delete can starve out write ops

Authored by: Alek Pinchuk <alek@nexenta.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Sanjay Nadkarni <sanjay.nadkarni@nexenta.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
Tested-by: kernelOfTruth <kerneloftruth@gmail.com>
OpenZFS-issue: https://www.illumos.org/issues/6569
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1bf4b6f2
Closes #5706

7 years agoOpenZFS 7742 - zfs send wrong error message with invalid long opts
George Melikov [Tue, 31 Jan 2017 22:41:23 +0000 (01:41 +0300)]
OpenZFS 7742 - zfs send wrong error message with invalid long opts

There are two cases:

    1. if an invalid flag is passed, and
    2. if a valid flag is not given a parameter.

In the case of (1), the flag is either short or long. For short flags,
optopt contains the character of the flag. For long, it contains zero,
and we can access the long flag using argv and optind.

In the case of (2), if the flag is short, optopt contains the character
of the flag. If the flag is long, the value in the 4th column of the
long_options table, for that flag, is returned.

We could case over all those values, or we could simply use argv and
optind again.

Note that in the case of something like `--resume`, which is also `-t`,
"t" will be returned if an argument is not provided; so the error
message will say `'t': argument not provided` or similar. This could be
fixed by making it so long and short options don't use the same
character flag, and then combining them in the switch/case statement,
but I didn't think the ugliness of the code would be worth the small
usability enhancement.

Authored by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7742
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/6d69b40
Closes #5702

7 years agoOpenZFS 7545 - zdb should disable reference tracking
Giuseppe Di Natale [Tue, 31 Jan 2017 22:36:35 +0000 (14:36 -0800)]
OpenZFS 7545 - zdb should disable reference tracking

Authored by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Porting Notes: Moved reference_tracking_enable and
reference_history outside of ZFS_DEBUG.

OpenZFS-issue: https://www.illumos.org/issues/7545
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/4dd77f9
Closes #5701

7 years agocontrib/dracut: fix syntax error in module-setup.sh
Aron Xu [Tue, 31 Jan 2017 22:25:49 +0000 (06:25 +0800)]
contrib/dracut: fix syntax error in module-setup.sh

The dracut/02zfsexpandknowledge/module-setup.sh.in has a syntax error
which makes the script unusable by a POSIX compliant shell like Dash
on Debian based systems.

Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Aron Xu <happyaron.xu@gmail.com>
Closes #5712

7 years agoUpdate deadman operation to better align with upstream OpenZFS
Tim Chase [Tue, 31 Jan 2017 22:19:08 +0000 (16:19 -0600)]
Update deadman operation to better align with upstream OpenZFS

The deadman in ZoL didn't behave quite as it did in upstream
OpenZFS.  In addition to the 2 purposes for which OpenZFS used the
zfs_deadman_synctime_ms parameter, ZoL also used it to determine how
frequently the deadman would fire once it has been triggered.

This patch adds the zfs_deadman_checktime_ms parameter to control how
frequently the subsequent checks are performed.

The deadman is now disabled for suspended pools.

As had been the case, unlike upstream OpenZFS, ZoL will not panic when
a hung IO is detected.

The module parameter documentation has been upated to include the new
parameter and to better describe the operation of the deadmen.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Tim Chase <tim@chase2k.com>
Closes #5695

7 years agoOpenZFS 7745 - print error if lzc_* is called before libzfs_core_init
George Melikov [Tue, 31 Jan 2017 18:48:45 +0000 (21:48 +0300)]
OpenZFS 7745 - print error if lzc_* is called before libzfs_core_init

The problem is that consumers of `libZFS_Core` that forget to call
`libzfs_core_init()` before calling any other function of the library
are having a hard time realizing their mistake. The library's internal
file descriptor is declared as global static, which is ok, but it is not
initialized explicitly; therefore, it defaults to 0, which is a valid
file descriptor. If `libzfs_core_init()`, which explicitly initializes
the correct fd, is skipped, the ioctl functions return errors that do
not have anything to do with `libZFS_Core`, where the problem is
actually located.

Even though assertions for that existed within `libZFS_Core` for debug
builds, they were never enabled because the `-DDEBUG` flag was missing
from the compiler flags.

This patch applies the following changes:

    1. It adds `-DDEBUG` for debug builds of `libZFS_Core` and `libzfs`,
       to enable their assertions on debug builds.

    2. It corrects an assertion within `libzfs`, where a function had
       been spelled incorrectly (`zpool_prop_unsupported()`) and nobody
       knew because the `-DDEBUG` flag was missing, and the preprocessor
       was taking that part of the code away.

    3. The library's internal fd is initialized to `-1` and `VERIFY`
       assertions have been placed to check that the fd is not equal to
       `-1` before issuing any ioctl. It is important here to note, that
       the `VERIFY` assertions exist in both debug and non-debug builds.

    4. In `libzfs_core_fini` we make sure to never increment the
       refcount of our fd below 0, and also reset the fd to `-1` when no
       one refers to it. The reason for this, is for the rare case that
       the consumer closes all references but then calls one of the
       library's functions without using `libzfs_core_init()` first, and
       in the mean time, a previous call to `open()` decided to reuse
       our previous fd. This scenario would have passed our assertion in
       non-debug builds.

    5. Once the `ASSERTION` macros were enabled again, two tests from
       the test suite were failing in `libzfs_sendrecv.c` at a
       `ZIO_CHECKSUM_IS_ZERO` check within `dump_record()`. We now zero
       the kernel filled checksums in all `dmu_replay_record`s that we
       read in `cksummer()`, except the ones that are of type
       `DRR_BEGIN`.

I considered making all assertions available for both debug and
non-debug builds, but I figured that it would not be appropriate if, for
example, an outside consumer of `libZFS_Core` suddenly triggers an
assertion failure because they happened to call `libzfs_core_fini()`,
even if previously the reference counter was `0`. Therefore, all the
reference counter related assertions are only enabled for debug builds,
and fd related assertions are enabled for debug and non-debug builds.

Porting notes:
- `ASSERT3S(g_refcount, >, 0);` added to `recv_impl` in
  lib/libzfs_core/libzfs_core.c .

Authored by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7745
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/7e3139a
Closes #5698

7 years agoOpenZFS 7019 - zfsdev_ioctl skips secpolicy when FKIOCTL is set
George Melikov [Tue, 31 Jan 2017 18:24:23 +0000 (21:24 +0300)]
OpenZFS 7019 - zfsdev_ioctl skips secpolicy when FKIOCTL is set

Authored by: Alex Wilson <alex.wilson@joyent.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Richard Lowe <richlowe@richlowe.net>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7019
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/45b1747
Closes #5709

7 years agoOpenZFS 7136 - ESC_VDEV_REMOVE_AUX ought to always include vdev information
George Melikov [Tue, 31 Jan 2017 18:19:36 +0000 (21:19 +0300)]
OpenZFS 7136 - ESC_VDEV_REMOVE_AUX ought to always include vdev information

Authored by: Alan Somers <asomers@gmail.com>
7115 6922 generates ESC_ZFS_VDEV_REMOVE_AUX a bit too often
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7136
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/b72b6bb
Closes #5691

Porting notes:
- Functionally this patch behaves the same as the OpenZFS
  version but it was adapted because because ZoL doesn't
  have the same illumos sysevent_t infrastructure and functionality.

7 years agoOpenZFS 7280 - Allow changing global libzpool variables in zdb and ztest through...
George Melikov [Tue, 31 Jan 2017 18:13:10 +0000 (21:13 +0300)]
OpenZFS 7280 - Allow changing global libzpool variables in zdb and ztest through command line

Authored by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7280
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/0e60744
Closes #5676

7 years agoOpenZFS 7490 - real checksum errors are silenced when zinject is on
George Melikov [Tue, 31 Jan 2017 01:12:58 +0000 (04:12 +0300)]
OpenZFS 7490 - real checksum errors are silenced when zinject is on

Authored by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7490
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/6cedfc3
Closes #5693

7 years agoOpenZFS 6922 - Emit ESC_ZFS_VDEV_REMOVE_AUX after removing an aux device
George Melikov [Mon, 30 Jan 2017 23:33:46 +0000 (02:33 +0300)]
OpenZFS 6922 - Emit ESC_ZFS_VDEV_REMOVE_AUX after removing an aux device

Authored by: Alan Somers <asomers@gmail.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/6922
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/63364b0
Closes #5690

7 years agoOpenZFS 7541 - zpool import/tryimport ioctl returns ENOMEM
George Melikov [Mon, 30 Jan 2017 21:20:55 +0000 (00:20 +0300)]
OpenZFS 7541 - zpool import/tryimport ioctl returns ENOMEM

Authored by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>

The refresh_config() calls into the kernel with ZFS_IOC_POOL_TRYIMPORT.
This ioctl returns the config of the pool in a buffer pre-allocated in
userland. The original estimate for the size is too conservative since
it doesn't account for the large size of vdev stats that are added to
the config before returning.
This fix simply increases the size of the buffer passed. This results in
a speed up of the zpool import process, and less spam in zfs_dbgmsg.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7541
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/a3c7690
Closes #5704

7 years agoOpenZFS 7502 - ztest should run zdb with -G (debug mode)
George Melikov [Mon, 30 Jan 2017 19:34:36 +0000 (22:34 +0300)]
OpenZFS 7502 - ztest should run zdb with -G (debug mode)

Authored by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Approved by: Gordon Ross <gordon.w.ross@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7502
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/c3c65d1
Closes #5677

7 years agoOpenZFS 7277 - zdb should be able to print zfs_dbgmsg's
George Melikov [Sat, 28 Jan 2017 20:16:43 +0000 (23:16 +0300)]
OpenZFS 7277 - zdb should be able to print zfs_dbgmsg's

Porting notes:
- 'zfs_dbgmsg_print()' reintroduced to userspace.

Authored by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7277
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/29bdd2f
Closes #5684

7 years agoFix suspend Godfather I/Os io_reexecute bits
Brian Behlendorf [Sat, 28 Jan 2017 20:13:34 +0000 (12:13 -0800)]
Fix suspend Godfather I/Os io_reexecute bits

After resuming a pool the godfather zio could have both the
ZIO_REEXECUTE_NOW and ZIO_REEXECUTE_SUSPEND bits set.  This
can occur if some child zios set ZIO_REEXECUTE_NOW while
other set ZIO_REEXECUTE_SUSPEND.  The godfather zio can
inherit both flags in zio_notify_parent().

The child zios which assigned the ZIO_REEXECUTE_SUSPEND flag
will be removed from the godfather's child list and added to
the spa->spa_suspend_zio_root child list.   While child zios
with the ZIO_REEXECUTE_NOW bit set remain being monitored
by the godfather zio.

When the godfather zio executes zio_done() the presence of
the ZIO_REEXECUTE_SUSPEND bit results in all io_reexecute
being cleared.  These child zios will then not be re-executed
and instead will be destroyed and lost.

The most straight forward way to address this situation is
to only clear the ZIO_REEXECUTE_SUSPEND bit and leave the
ZIO_REEXECUTE_NOW bit set.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: yuxiang <guo.yong33@zte.com.cn>
7 years agoAdd -Wno-declaration-after-statement to KERNELCPPFLAGS
Brian Behlendorf [Sat, 28 Jan 2017 20:12:25 +0000 (12:12 -0800)]
Add -Wno-declaration-after-statement to KERNELCPPFLAGS

Disable the warnings regarding ISO C90 forbidding mixed
declarations and code.  While this functionality was
introduced as part of C99 gcc does allow this in C90
mode as an extension.

https://gcc.gnu.org/onlinedocs/gcc/Mixed-Declarations.html#Mixed-Declarations

Allowing this usage helps minimize the changes required
when porting patches from OpenZFS.  The downside here is
that this functionality is specific to gcc.

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #5686

7 years agoOpenZFS 7580 - ztest failure in dbuf_read_impl
George Melikov [Sat, 28 Jan 2017 20:11:09 +0000 (23:11 +0300)]
OpenZFS 7580 - ztest failure in dbuf_read_impl

Authored by: George Wilson <george.wilson@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7580
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/3105d95
Closes #5678

7 years agoOpenZFS 7340 - receive manual origin should override automatic origin
George Melikov [Fri, 27 Jan 2017 19:47:54 +0000 (22:47 +0300)]
OpenZFS 7340 - receive manual origin should override automatic origin

Authored by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7340
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/ed4e7a6
Closes #5681

7 years agoOpenZFS 7301 - zpool export -f should be able to interrupt file freeing
George Melikov [Fri, 27 Jan 2017 19:46:39 +0000 (22:46 +0300)]
OpenZFS 7301 - zpool export -f should be able to interrupt file freeing

Authored by: Alek Pinchuk <alek@nexenta.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Sanjay Nadkarni <sanjay.nadkarni@nexenta.com>
Reviewed by: Saso Kiselkov <saso.kiselkov@nexenta.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Approved by: Gordon Ross <gordon.ross@nexenta.com>
Reviewed-by: Tim Chase <tim@chase2k.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7301
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/eb72182
Closes #5680

7 years agoOpenZFS 7254 - ztest failed assertion in ztest_dataset_dirobj_verify: dirobjs + 1...
George Melikov [Fri, 27 Jan 2017 19:43:42 +0000 (22:43 +0300)]
OpenZFS 7254 - ztest failed assertion in ztest_dataset_dirobj_verify: dirobjs + 1 == usedobjs

Authored by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7254
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/c166b69
Closes #5670

7 years agoAdd realloc() success check in zpool_history_unpack()
George Melikov [Fri, 27 Jan 2017 19:10:10 +0000 (22:10 +0300)]
Add realloc() success check in zpool_history_unpack()

Correctly handle the unlikely case where the memory buffer cannot be resized.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Signed-off-by: George Melikov <mail@gmelikov.ru>
Closes #5575

7 years agoRetire .write/.read file operations
Chunwei Chen [Fri, 27 Jan 2017 18:43:39 +0000 (10:43 -0800)]
Retire .write/.read file operations

The .write/.read file operations callbacks can be retired since
support for .read_iter/.write_iter and .aio_read/.aio_write has
been added.  The vfs_write()/vfs_read() entry functions will
select the correct interface for the kernel.  This is desirable
because all VFS write/read operations now rely on common code.

This change also add the generic write checks to make sure that
ulimits are enforced correctly on write.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Chunwei Chen <david.chen@osnexus.com>
Closes #5587
Closes #5673

7 years agoOpenZFS 5561 - support root pools on EFI/GPT partitioned disks
Brian Behlendorf [Fri, 27 Jan 2017 18:40:02 +0000 (10:40 -0800)]
OpenZFS 5561 - support root pools on EFI/GPT partitioned disks

Reviewed by: Jean McCormack <jean.mccormack@nexenta.com>
Reviewed by: Josef 'Jeff' Sipek <josef.sipek@nexenta.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/5561
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1a902ef
Closes #5672

7 years agoOpenZFS 7613 - ms_freetree[4] is only used in syncing context
Tim Chase [Thu, 26 Jan 2017 23:27:19 +0000 (17:27 -0600)]
OpenZFS 7613 - ms_freetree[4] is only used in syncing context

metaslab_t:ms_freetree[TXG_SIZE] is only used in syncing context. We
should replace it with two trees: the freeing tree (ranges that we are
freeing this syncing txg) and the freed tree (ranges which have been
freed this txg).

Authored by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: Tim Chase <tim@chase2k.com>
OpenZFS-issue: https://www.illumos.org/issues/7613
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/a8698da2
Closes #5598

7 years agoOpenZFS 7500 - Simplify dbuf_free_range by removing dn_unlisted_l0_blkid
George Melikov [Thu, 26 Jan 2017 23:15:48 +0000 (02:15 +0300)]
OpenZFS 7500 - Simplify dbuf_free_range by removing dn_unlisted_l0_blkid

Authored by: Stephen Blinick <stephen.blinick@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Approved by: Gordon Ross <gordon.w.ross@gmail.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7500
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/653af1b
Closes #5639

7 years agoOpenZFS 6676 - Race between unique_insert() and unique_remove() causes ZFS fsid change
George Melikov [Thu, 26 Jan 2017 22:43:28 +0000 (01:43 +0300)]
OpenZFS 6676 - Race between unique_insert() and unique_remove() causes ZFS fsid change

Authored by: Josef 'Jeff' Sipek <josef.sipek@nexenta.com>
Reviewed by: Saso Kiselkov <saso.kiselkov@nexenta.com>
Reviewed by: Sanjay Nadkarni <sanjay.nadkarni@nexenta.com>
Reviewed by: Dan Vatca <dan.vatca@gmail.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/6676
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/40510e8
Closes #5667

7 years agoOpenZFS 7386 - zfs get does not work properly with bookmarks
George Melikov [Thu, 26 Jan 2017 22:42:15 +0000 (01:42 +0300)]
OpenZFS 7386 - zfs get does not work properly with bookmarks

Authored by: Marcel Telka <marcel@telka.sk>
Reviewed by: Simon Klinkert <simon.klinkert@gmail.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Approved by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7386
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/edb901a
Closes #5666

7 years agoOpenZFS 5142 - libzfs support raidz root pool (loader project)
George Melikov [Thu, 26 Jan 2017 20:47:40 +0000 (23:47 +0300)]
OpenZFS 5142 - libzfs support raidz root pool (loader project)

Authored by: Toomas Soome <tsoome@me.com>
Reviewed by: - George Wilson <george.wilson@delphix.com>
Reviewed by: - Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: - Andrew Stormont <andyjstormont@gmail.com>
Reviewed by: - Albert Lee <trisk@omniti.com>
Approved by: - Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/5142
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/d5f26ad
Closes #5663

7 years agoOpenZFS 7606 - dmu_objset_find_dp() takes a long time while importing pool
George Melikov [Thu, 26 Jan 2017 20:46:02 +0000 (23:46 +0300)]
OpenZFS 7606 - dmu_objset_find_dp() takes a long time while importing pool

When importing a pool with a large number of filesystems within the same
parent filesystem, we see that dmu_objset_find_dp() takes a long time.
It is called from 3 places: spa_check_logs(), spa_ld_claim_log_blocks(),
and spa_load_verify().

There are several ways to improve performance here:

    1. We don't really need to do spa_check_logs() or
       spa_ld_claim_log_blocks() if the pool was closed cleanly.

    2. spa_load_verify() uses dmu_objset_find_dp() to check that no
       datasets have too long of names.

    3. dmu_objset_find_dp() is slow because it's doing
       zap_value_search() (which is O(N sibling datasets)) to determine
       the name of each dsl_dir when it's opened. In this case we
       actually know the name when we are opening it, so we can provide
       it and avoid the lookup.

This change implements fix #3 from the above list; i.e. make
dmu_objset_find_dp() provide the name of the dataset so that we don't
have to search for it.

Authored by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prashanth Sreenivasa <prashksp@gmail.com>
Reviewed-by: David Quigley <david.quigley@intel.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7606
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/cac6bab
Closes #5662

7 years agoOpenZFS 6412 - zfs receive: -u can be ignored sometimes
George Melikov [Thu, 26 Jan 2017 20:42:48 +0000 (23:42 +0300)]
OpenZFS 6412 - zfs receive: -u can be ignored sometimes

Authored by: Andriy Gapon <andriy.gapon@clusterhq.com>
Reviewed by: - Matthew Ahrens <mahrens@delphix.com>
Reviewed by: - Paul Dagnelie <pcd@delphix.com>
Approved by: - Richard Lowe <richlowe@richlowe.net>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/6412
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/9185393
Closes #5659

7 years agoOpenZFS 7348 - cstyle can't handle ellipsis on continuation line
Giuseppe Di Natale [Thu, 26 Jan 2017 20:40:22 +0000 (12:40 -0800)]
OpenZFS 7348 - cstyle can't handle ellipsis on continuation line

Authored by: Hans Rosenfeld <hans.rosenfeld@nexenta.com>
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: Jason King <jason.brian.king@gmail.com>
Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Ported-by: Giuseppe Di Natale <dinatale2@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/7348
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/c667d21
Closes #5655

7 years agoOpenZFS 6999 - fix 'Use of uninitialized value $picky in numeric eq (==)' in cstyle
Giuseppe Di Natale [Thu, 26 Jan 2017 20:36:56 +0000 (12:36 -0800)]
OpenZFS 6999 - fix 'Use of uninitialized value $picky in numeric eq (==)' in cstyle

Authored by: Richard PALO <richard@NetBSD.org>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: Giuseppe Di Natale <dinatale2@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/6999
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/002ec3e
Closes #5653

7 years agoOpenZFS 7163 - ztest failures due to excess error injection
George Melikov [Thu, 26 Jan 2017 20:34:29 +0000 (23:34 +0300)]
OpenZFS 7163 - ztest failures due to excess error injection

Authored by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7163
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/f34284d
Closes #4484
Closes #5661

7 years agoOpenZFS 7253 - ztest failure: dsl_destroy_head(name) == 0 (0x10 == 0x0), file .....
George Melikov [Thu, 26 Jan 2017 20:32:36 +0000 (23:32 +0300)]
OpenZFS 7253 - ztest failure: dsl_destroy_head(name) == 0 (0x10 == 0x0), file ../ztest.c, line 3235 (#5660)

Authored by: Chris Williamson <chris.williamson@delphix.com>
Reviewed by: - Matthew Ahrens <mahrens@delphix.com>
Reviewed by: - Paul Dagnelie <pcd@delphix.com>
Approved by: - Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7253
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/754998c
Closes #5660

7 years agoOpenZFS 7147 - ztest: ztest_ddt_repair fails with ztest_pattern_match assertion
Brian Behlendorf [Thu, 26 Jan 2017 20:30:43 +0000 (12:30 -0800)]
OpenZFS 7147 - ztest: ztest_ddt_repair fails with ztest_pattern_match assertion

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/7147
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/aab80726
Closes #5652

7 years agoOpenZFS 7336 - vfork and O_CLOEXEC causes zfs_mount EBUSY
George Melikov [Thu, 26 Jan 2017 20:28:29 +0000 (23:28 +0300)]
OpenZFS 7336 - vfork and O_CLOEXEC causes zfs_mount EBUSY

Porting notes:
- statvfs64 is replaced by statfs64.
- ZFS_SUPER_MAGIC definition moved in include/sys/fs/zfs.h
  to share it between user and kernel space.

Authored by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7336
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/dd862f6d
Closes #5651

7 years agoRefresh Linux test suite runfile
Brian Behlendorf [Thu, 26 Jan 2017 20:25:35 +0000 (12:25 -0800)]
Refresh Linux test suite runfile

Associate disabled test cases with existing open issues, update
comments to be consistent, disable a few additional test cases.
The goal is for all enabled test to pass 100% reliably.

The following test cases have been disabled due to infrequent
failures during automated testing.  Several of these test cases
were previous disabled only for the kmemleak builder but have
subsequently been observed on other automated builders.

- zfs_destroy_001_pos - https://github.com/zfsonlinux/zfs/issues/5635
- zfs_rename_006_pos  - https://github.com/zfsonlinux/zfs/issues/5647
- zfs_rename_009_neg  - https://github.com/zfsonlinux/zfs/issues/5648
- zpool_clear_001_pos - https://github.com/zfsonlinux/zfs/issues/5634
- zfs_allow_010_pos   - https://github.com/zfsonlinux/zfs/issues/5646
- reservation_018_pos - https://github.com/zfsonlinux/zfs/issues/5642
- snapused_004_pos    - https://github.com/zfsonlinux/zfs/issues/5513
- rsend_022_pos       - https://github.com/zfsonlinux/zfs/issues/5654
- rsend_024_pos       - https://github.com/zfsonlinux/zfs/issues/5665
- history_008_pos     - https://github.com/zfsonlinux/zfs/issues/5658
- history_006_neg     - https://github.com/zfsonlinux/zfs/issues/5657
- history_008_pos     - https://github.com/zfsonlinux/zfs/issues/5658
- zfs_inherit_003_pos - https://github.com/zfsonlinux/zfs/issues/5669

Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #5649

7 years agoOpenZFS 6872 - zfs libraries should not allow uninitialized variables
George Melikov [Tue, 24 Jan 2017 17:23:34 +0000 (20:23 +0300)]
OpenZFS 6872 - zfs libraries should not allow uninitialized variables

Porting notes:
- Many changes were already made in ZoL (for ex. in d4ed66734).

Authored by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/6872
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/f83b46b
Closes #5640

7 years agoOpenZFS 7571 - non-present readonly numeric ZFS props do not have default value
George Melikov [Tue, 24 Jan 2017 17:15:35 +0000 (20:15 +0300)]
OpenZFS 7571 - non-present readonly numeric ZFS props do not have default value

Authored by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov <mail@gmelikov.ru>
OpenZFS-issue: https://www.illumos.org/issues/7571
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/ad2760a
Closes #5638

7 years agoOpenZFS 6871 - libzpool implementation of thread_create should enforce length is 0
George Melikov [Tue, 24 Jan 2017 17:13:49 +0000 (20:13 +0300)]
OpenZFS 6871 - libzpool implementation of thread_create should enforce length is 0

Porting notes:
- Several direct callers of zk_thread_create() are passing TS_RUN for the
length.  The `len` and `state` were inverted,this commit fixes them.

Authored by: Eli Rosenthal <eli.rosenthal@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: George Melikov mail@gmelikov.ru
OpenZFS-issue: https://www.illumos.org/issues/6871
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/8fc9228
Closes #5621

7 years agoMinor man-page formatting fixes
Håkan Johansson [Tue, 24 Jan 2017 17:09:02 +0000 (18:09 +0100)]
Minor man-page formatting fixes

fB -> \fB in zpool.8 (Properties -> cachefile)
\fN -> \fB in zfs-module-parameters.5 (zfs_dirty_data_max_max_percent)
Three | -> \fR|\fI fixes for arguments of diff and inherit in zfs.8.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: loli10K <ezomori.nozomu@gmail.com>
Signed-off-by: Haakan T Johansson <f96hajo@chalmers.se>
Closes #5645