This patch makes changes to support new RHEL 7.6 release.

Test-Parameters: clientdistro=el7.6 ossdistro=el7.6 mdsdistro=el7.6

Change-Id: I1c6f1d065404fdfecf9ea978a2b161b4f90d3ab2
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33621


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>

Update SLES12 SP3 kernel to 4.4.155-94.50.

Test-Parameters: envdefinitions=CONF_SANITY_EXCEPT=103 \
mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs \
clientdistro=sles12sp3 ossdistro=sles12sp3 mdsdistro=sles12sp3 \
testlist=sanity,conf-sanity,sanity-sec

Lustre-change: https://review.whamcloud.com/33236


Lustre-commit: 937d72ac

Change-Id: I47b72420664fe614a2da7c864c401a9729d96c55
Signed-off-by: Jian Yu <yujian@whamcloud.com>
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/33313


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>

Update target, kernel_config, and ldiskfs files for new version
One ldiskfs patch revised for ext4 changes.
Old unchanged ldiskfs patch kept to use for sles12sp2.

Test-Parameters: clientdistro=sles12sp3 testgroup=review-ldiskfs \
  mdsdistro=sles12sp3 ossdistro=sles12sp3 \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ic6d0219a7133825d1dba0b2bfadf8354442cddb3
Reviewed-on: https://review.whamcloud.com/32600


Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

Backport following upstream patches:

------
ext4: cleanup goto next group

avoid duplicated codes, also we need goto
next group in case we found reserved inode.
------

ext4: reduce lock contention in __ext4_new_inode

While running number of creating file threads concurrently,
we found heavy lock contention on group spinlock:

FUNC                           TOTAL_TIME(us)       COUNT        AVG(us)
ext4_create                    1707443399           1440000      1185.72
_raw_spin_lock                 1317641501           180899929    7.28
jbd2__journal_start            287821030            1453950      197.96
jbd2_journal_get_write_access  33441470             73077185     0.46
ext4_add_nondir                29435963             1440000      20.44
ext4_add_entry                 26015166             1440049      18.07
ext4_dx_add_entry              25729337             1432814      17.96
ext4_mark_inode_dirty          12302433             5774407      2.13

most of cpu time blames to _raw_spin_lock, here is some testing
numbers with/without patch.

Test environment:
Server : SuperMicro Sever (2 x E5-2690 v3@2.60GHz, 128GB 2133MHz
         DDR4 Memory, 8GbFC)
Storage : 2 x RAID1 (DDN SFA7700X, 4 x Toshiba PX02SMU020 200GB
          Read Intensive SSD)

format command:
        mkfs.ext4 -J size=4096

test command:
        mpirun -np 48 mdtest -n 30000 -d /ext4/mdtest.out -F -C \
                -r -i 1 -v -p 10 -u #first run to load inode

        mpirun -np 48 mdtest -n 30000 -d /ext4/mdtest.out -F -C \
                -r -i 3 -v -p 10 -u

Kernel version: 4.13.0-rc3

Test  1,440,000 files with 48 directories by 48 processes:

Without patch:

File Creation   File removal
79,033          289,569 ops/per second
81,463          285,359
79,875          288,475

With patch:
File Creation   File removal
810669          301694
812805          302711
813965          297670

Creation performance is improved more than 10X with large
journal size. The main problem here is we test bitmap
and do some check and journal operations which could be
slept, then we test and set with lock hold, this could
be racy, and make 'inode' steal by other process.

However, after first try, we could confirm handle has
been started and inode bitmap journaled too, then
we could find and set bit with lock hold directly, this
will mostly gurateee success with second try.

Lustre-commit: 3f0a7241
Lustre-change: https://review.whamcloud.com/29032



Signed-off-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I234ff3027c8d96155d374c56b12aab7c4dc0dafd
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Gu Zheng <gzheng@ddn.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/32295


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

With this mod we switch our supported el7 version to RHEL 7.5


Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Iedcea9498591d15eab69187274e4c32c57879e4e
Reviewed-on: https://review.whamcloud.com/32371


Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

Update target, kernel_config, and ldiskfs files for new version
ldiskfs for sles12sp2 and sles12sp3 are no longer identical,
so revise some build files for sles12sp2 here too.

Test-Parameters: clientdistro=sles12sp3 testgroup=review-ldiskfs \
  mdsdistro=sles12sp3 ossdistro=sles12sp3 \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: If554b24644f763462002d81309f8edfa016210e8
Reviewed-on: https://review.whamcloud.com/32146


Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

On one Customer site, we hit following deadlock:

Thread 1:
ofd_object_punch
 osd_punch
  ldiskfs_truncate
   ldiskfs_inode_attach_jinode
     ...
     do_try_to_free_pages
      lu_cache_shrink
       mutex_lock -->try to hold @lu_sites_guard

kswapd thread2:
kthread
 shrink_slab
  lu_cache_shrink
    mutex_lock ---->hold already.
     ...
     dqget
      ldiskfs_acquire_dquot
       jbd2__journal_start-->blocked to wait for more credits.

Thread3:
kthread
 kjournald2
  jbd2_journal_commit_transaction-->blocked to wait Thread2 finished,
				 since Thread1 add a handle into transaction.

So deadlock happens because of Thread1 wait Thread2, Thread2 wait Thread3..
but Thread3 wait Thread1....

This problem still exists even we have switched @lu_sites_guard
into a read/write lock, sine we hold write lock at lu_cahce_shrink().

Fixed the problem by making ldiskfs_inode_attach_jinode() use
GFP_NOFS.

Test-Parameters: testgroup=review-ldiskfs \
	mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Lustre-change: https://review.whamcloud.com/31806


Lustre-commit: 0506e1bd

Change-Id: I0ab143fc0cdb8e1b0c490c2c25e8af483c491a81
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/32058


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

See comments in ext4_free_clusters_after_init:
/* Return the number of free blocks in a block group.  It is used when
 * the block bitmap is uninitialized, so we can't just count the bits
 * in the bitmap. */
So extra check we enhanced here is wrong if this block group
bitmap is uninitialized, since we only check bitmaps here.

Further, Looking at EXT4_BG_BLOCK_UNINIT clear codes, Kernel
will reinit free_clusters_count when tried to clear the flag, so
extra check for uninited block bitmaps dosen't make much sense.

Let's skip uninited block bitmap check if EXT4_BG_BLOCK_UNINIT
is set, whatever free count group desc recorded is untrustable somehow

Lustre-change: https://review.whamcloud.com/31720


Lustre-commit: f7672143

Change-Id: I845f2e0e17e53b7e3073399bd8b0a85e3db66ef8
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/31957


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

Since kstrtol needs a null-terminor string so we
back to use simple_strtol in prealloc patches.

Lustre-change: https://review.whamcloud.com/28553


Lustre-commit: 685ef61f

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I331308d830fbeef9c00156bb8c14b43651d66420
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Alexey Lyashkov <c17817@cray.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/29733


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

When mb prealloc table item value is not valid, the
s_mb_prealloc_table_size should be adjust accordingly.
Port patch to rhel7 & sles12.

Lustre-change: https://review.whamcloud.com/27748


Lustre-commit: b9ecf267

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alexander Boyko <alexander_boyko@xyratex.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I5aa3a32a6a3aedc70160409c6443746fd2ccbbc9
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28228


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

Minor linux version upgrade, but SP2 and SP3 use linux 4.4 versions.
Some new kernel patches, a few revised ldiskfs patches.
All new target and config files.
Some autoconf changes to adapt to new or altered kernel APIs.

Test-Parameters: clientdistro=sles12sp3 testgroup=review-ldiskfs   mdsdistro=sles12sp3 ossdistro=sles12sp3   mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I99e2b6848197ea19402fa415fdb562d03e87d947

In some execution paths jinode data structures are not
getting initialized. Add an extra init call to fix that.

Test-Parameters: clientdistro=sles12sp2 testlist=conf-sanity \
  mdsdistro=sles12sp2 ossdistro=sles12sp2 testgroup=review-ldiskfs \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Lustre-change: https://review.whamcloud.com/28665


Lustre-commit: 84f690ee

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I087eb06b9c5122be1cfd0aabbc04ea1db7ec765a
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Hongchao Zhang <hongchao.zhang@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28968


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

With this mod we switch our supported el7 version to RHEL 7.4

Lustre-change: https://review.whamcloud.com/#/c/28301/


Lustre-commit: b2a39555d1bd4a75017ee6ca2a9a6920a30bafa8

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ib0bcb3547fcf220e1b4665229930c6ff28c6906c
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28532


Tested-by: Jenkins
Tested-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

When initializes (reformat) the filesystem, the number of
free blocks in the group descriptor is calculated via the
ext2fs_reserve_super_and_bgd() (e2fsprogs). As commented
in such function: "This is not necessarily the case when
the flex_bg feature is enabled, so callers should take care!".

So it is normal that we may find the block group descriptor
that has LDISKFS_BG_BLOCK_UNINIT flag but with 0 free blocks.
The ldiskfs_mb_check_ondisk_bitmap() should NOT report error
for such block group, instead, skip the check directly.

Lustre-change: https://review.whamcloud.com/28566


Lustre-commit: 5506c15a

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iba0fb2bf0632a6e54222472bc724a8ea0478e9ae
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Signed-off-by: Minh Diep <minh.diep@intel.com>
Reviewed-on: https://review.whamcloud.com/28765


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs   mdsdistro=sles12sp2 ossdistro=sles12sp2   mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I42107acb5a12f5200d3cb58121d10ffdc1dbc6d2
Reviewed-on: https://review.whamcloud.com/28043


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12 testgroup=review-ldiskfs \
  mdsdistro=sles12 ossdistro=sles12 mdtfilesystemtype=ldiskfs \
  ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ie0cb65603504da3b7dd38ab4962198142b1629f6
Reviewed-on: https://review.whamcloud.com/27222


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Port a patch from upstream for changing extra size:

commit  887a9730614727c4fff7cb756711b190593fc1df
	"ext4: keep existing extra fields when inode expands"

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I9555cb4efd1cbca257536ec699627e8fdb5a5b46
Reviewed-on: https://review.whamcloud.com/27285


Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

Port 3 extra isize patches for sles12sp2 as below:

commit e3014d14a81edde488d9a6758eea8afc41752d2d
       "ext4: fixup free space calculations when expanding inodes"
commit 94405713889d4a9d341b4ad92956e4e2ec8ec2c2
       "ext4: replace bogus assertion in ext4_xattr_shift_entries()"
commit 887a9730614727c4fff7cb756711b190593fc1df
       "ext4: keep existing extra fields when inode expands"

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Iab0914d0010436aa26d99b78b2ae4acea783fb7d
Reviewed-on: https://review.whamcloud.com/27244


Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

Update SLES11 SP4 kernel to 3.0.101-100
Refresh one ldiskfs patch for ext4 context changes.

Test-Parameters: mdsdistro=sles11sp4 ossdistro=sles11sp4 \
  clientdistro=sles11sp4 mdtfilesystemtype=ldiskfs \
  ostfilesystemtype=ldiskfs testgroup=review-ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ia392004c12dda9d49c0f596b166f333e7fcae590
Reviewed-on: https://review.whamcloud.com/27132


Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

few cleanups to support gcc6

Change-Id: Ibe664e87ff67eb3d34ce09c9a1892e55a63ab2cb
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/26584


Tested-by: Jenkins
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

When ldiskfs runs in failover mode with read-only disk, it
may lose part of allocation updates and fail while mounting
fs due to group descriptor checks before journal replay.
Don't produce panics with on-disk checks in read-only mode.

Include ext4-dont-check-before-replay and ext4-dont-check-in-ro
patches in the RHEL7.3 series.

The check-ro patch should be ported in to RHEL7.3.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Icd4b899f9c48040c2453c4d759149e323fa33e18
Reviewed-on: https://review.whamcloud.com/27077


Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Port upstream 5 patches for changing extra isize as below:

commit d0141191a20289f8955c1e03dad08e42e6f71ca9
	"ext4: fix xattr shifting when expanding inodes"
commit 418c12d08dc64a45107c467ec1ba29b5e69b0715
	"ext4: fix xattr shifting when expanding inodes part 2"
commit 443a8c41cd49de66a3fda45b32b9860ea0292b84
	"ext4: properly align shifted xattrs when expanding inodes"
commit e3014d14a81edde488d9a6758eea8afc41752d2d
	"ext4: fixup free space calculations when expanding inodes"
commit 94405713889d4a9d341b4ad92956e4e2ec8ec2c2
	"ext4: replace bogus assertion in ext4_xattr_shift_entries()"

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I01414bcc91d8f57ca72281916d35536d3926e570
Reviewed-on: https://review.whamcloud.com/27045


Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Port failover mode patches to other distro and
fix failure path in replay patch.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I51f5ca0b906a3cbd7554fabb8b447cda4096c781
Reviewed-on: https://review.whamcloud.com/26854


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

include/linux/quota.h undefine @PRJQUOTA and make it
enum, ldiskfs macro check here will be negative, fix
it to use @HAVE_PROJECT_QUOTA, also wrap ext4-export-*.patch
to ext4-projid-feature-support.patch.

Change-Id: I415e366423408413dfb5b99dee98f7a30513f83f
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Reviewed-on: https://review.whamcloud.com/26769


Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Update ldiskfs patches to ensure buildability on unpatched rhel7.2/7.3
Also update lustre_user.h definition of LL_IOC_FSSETXATTR in the face
of missing define of FS_IOC_FSSETXATTR

Change-Id: I518e46e727086ed5e10b4820c4e69b2ea896fd43
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: https://review.whamcloud.com/26647


Tested-by: Jenkins
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>

Export this helper so that lustre can save project id to
MDS and OST objects

Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I5c374e6dc7a509c0cbd46ac383b8d99f7f67d6b6
Reviewed-on: https://review.whamcloud.com/23945


Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Li Xi <lixi@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Pick following patchs from upstream:

commit 847aac644e92e5624f2c153bab409bf713d5ff9a
vfs: Add general support to enforce project quota limits
This patch adds support for a new quota type PRJQUOTA for
project quota enforcement. Also a new method get_projid()
is added into dquot_operations structure.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Jan Kara <jack@suse.cz>

commit 334e580a6f97e2e84d1c19a8679603956acaa622
fs: XFS_IOC_FS[SG]SETXATTR to FS_IOC_FS[SG]ETXATTR promotion
Hoist the ioctl definitions for the XFS_IOC_FS[SG]SETXATTR
API from fs/xfs/libxfs/xfs_fs.h to include/uapi/linux/fs.h
so that the ioctls can be used by all filesystems,
not just XFS. This enables (initially) ext4 to use
the ioctl to set project IDs on inodes.

Based-on-patch-from: Li Xi <lixi@ddn.com>
Signed-off-by: Dave Chinner <dchinner@redhat.com>

Above two chages are wrapped into vfs-project-quotas-rhel7.patch

commit 040cb3786d9b25293b8b0b05b90da0f871e1eb9b
ext4: adds project ID support

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Reviewed-by: Jan Kara <jack@suse.cz>

commit	689c958cbe6be4f211b40747951a3ba2c73b6715
ext4: add project quota support
This patch adds mount options for enabling/disabling project quota
accounting and enforcement. A new specific inode is also used for
project quota accounting.

[Includes fix from Dan Carpenter to correct error checking from dqget(). ]

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Reviewed-by: Jan Kara <jack@suse.cz>

commit	9b7365fc1c82038faa52d56173b20221cf422cbe
ext4: add FS_IOC_FSSETXATTR/FS_IOC_FSGETXATTR interface support
This patch adds FS_IOC_FSSETXATTR/FS_IOC_FSGETXATTR ioctl interface
support for ext4. The interface is kept consistent with
XFS_IOC_FSGETXATTR/XFS_IOC_FSGETXATTR.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Reviewed-by: Jan Kara <jack@suse.cz>

This patch add ldiskfs project quota support
Did following adjustment from original patches:

1.adds padding structure to superblock, so that @s_prj_num is
in the same position of superblock from upstream.

2.Patch aslo disable kabi check for rhel7, since
any of vfs patch's include files change will break the check

Also fix conf-sanity: test_55, because now debugfs output
Project ID, we should output file size.

Signed-off-by: Li Xi <lixi@ddn.com>
Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: I2392f402d9853af969e4cd37318a83f2c4941c3e
Reviewed-on: https://review.whamcloud.com/17770


Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Changes to support new kernel version in new el6.9 release distro
New kernel-config and target files.
Revised base kernel and ldiskfs patches.
Small adjustments to lbuild and autoconf.

Test-Parameters: clientdistro=el6.9 mdsdistro=el6.9 ossdistro=el6.9

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Idd341e4db98c3e6e91c9d8696a9a5e7f03ded6fe
Reviewed-on: https://review.whamcloud.com/26111


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
  mdsdistro=sles12sp2 ossdistro=sles12sp2 mdsfilesystemtype=ldiskfs \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I210b548c1f27a9e444626a29dfd61953850d60ff
Reviewed-on: https://review.whamcloud.com/25688


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Update target and kernel_config files for new version
Add an extra ldiskfs patch to adapt to changes from recent landing
of LU-9031.  These changes were required due to new features
in upstream ext4 code seen in both linux 4.4 and linux 3.12.

Test-Parameters: clientdistro=sles12 testgroup=review-ldiskfs \
  mdsdistro=sles12 ossdistro=sles12 mdsfilesystemtype=ldiskfs \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I0c7f01dfaa087e0799a300531aff45841ba87c68
Reviewed-on: https://review.whamcloud.com/25556


Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

The ext4_mark_inode_dirty() may trigger new block
alloction. ext4_xattr_block_set() will be invoked
and trying to grab i_data_sem. It may cause deadlock
in some time. We backport 3 patches to resolve this
issue.

Backport from upstream:
	2e81a4eeedcaa66e35f58b81e0755b87057ce392
	e3cf5d5d9a86df1c5e413bdd3725c25a16ff854c
	a521100231f816f8cdd9c8e77da14ff1e42c2b17

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: I2e1bc10bc0e03ad760d113a785d43c06f85fcfa3
Reviewed-on: https://review.whamcloud.com/25595


Tested-by: Jenkins
Reviewed-by: Wang Shilong <wshilong@ddn.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
  mdsdistro=sles12sp2 ossdistro=sles12sp2 mdsfilesystemtype=ldiskfs \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ia72bf9e6542627efdd946c7213dee2c77fa73e57
Reviewed-on: https://review.whamcloud.com/24938


Tested-by: Jenkins
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

We need take care of jinode for ldiskfs. Since we
didn't got inode from syscall like sys_open(). So
have to initailize it in OSD by ourselves.

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Iec6db290c3779a8f7c98e5d1356b71fd928d7c88
Reviewed-on: https://review.whamcloud.com/24941


Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Update target and kernel_config files for new version

Test-Parameters: clientdistro=sles12 testgroup=review-ldiskfs \
  mdsdistro=sles12 ossdistro=sles12 mdsfilesystemtype=ldiskfs \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I6c474a939e3d6e8853388d645d82dbfe3038edee
Reviewed-on: https://review.whamcloud.com/24427


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Major linux version upgrade, from linux 3.12 to linux 4.4.
All new ldiskfs and kernel patches.
All new target and config files.

Test-Parameters: clientdistro=sles12sp2 testgroup=review-ldiskfs \
  mdsdistro=sles12sp2 ossdistro=sles12sp2 mdsfilesystemtype=ldiskfs \
  mdtfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I354197d5426ebde3da6c814bfb7f97f0f04113eb
Reviewed-on: https://review.whamcloud.com/24257


Tested-by: Jenkins
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

when ldiskfs runs in failover mode with read-only disk,
it may loose part of allocation updates and fail while
mounting fs due to group descriptor checks before journal
replay.
don't produce panic's with on disk checks in read-only mode.

Seagate-bug-id: MRP-797
Change-Id: I54bee3a0aeb9a15f5ee2a79f7a2a2a905f19af1a
Signed-off-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Signed-off-by: Lokesh Nagappa Jaliminche <lokesh.jaliminche@seagate.com>
Reviewed-on: https://morpheus.xyus.xyratex.com:8443/gerrit/239


Reviewed-by: Andrew Perepechko <Andrew_Perepechko@xyratex.com>
Reviewed-by: Alexander Zarochentsev <alexander_zarochentsev@xyratex.com>
Tested-by: Alexander Lezhoev <Alexander_Lezhoev@xyratex.com>
Reviewed-by: Vitaly Fertman <Vitaly_Fertman@xyratex.com>
Reviewed-on: https://review.whamcloud.com/21141


Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>

Add the new ldiskfs patch series file ldiskfs-3.10-rhel7.3.series which
supports the RHEL7.3 kernel.  Three patch files needed contextual updates
to allow them to apply.

Note that the new RHEL7.3 kernel contains a backport of the
upstream linux kernel commit 923ae0ff9250430133b3310fe62c47538cf1cbc1,
which introduces DAX to ext4.  This adds the flag EXT4_MOUNT_DAX
with value 0x00200.  This conflicted with ext4-data-in-dirent.patch's
EXT4_MOUNT_DIRDATA flag value.  Therefore, for RHEL7.3 the value of the
EXT4_MOUNT_DIRDATA flag is changed to 0x00002.

The ext4-corrupted-inode-block-bitmaps-handling-patches.patch needed
updating for two problems:

In ext4_validate_block_bitmap(), the patch removes the
struct ext4_group_info *grp declaration.  The upstream kernel now
has the following at the beginning of the function:

        if (buffer_verified(bh) || EXT4_MB_GRP_BBITMAP_CORRUPT(grp))
                return;

The declaration/definion of grp is reintroduced to address that
use.

Change-Id: Ia1a2455c1f353b59202b48ce6cdaad801a7f42d2
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/22113


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

RHEL7.2 ext4 code leak to release bh in make_indexed_dir.
It was cause by upstream 6050d47adcadbb53582434d919ed7f038d936712

Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Change-Id: Ie27225cfbfb2e43f1ee48799a9836f873fefacc9
Reviewed-on: http://review.whamcloud.com/22738


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

With RH 6.5 old API was deprecated and was removed.
Backport a new API from ext4 upstream in opposite to copy-paste
older buggy code as FIEMAP now uses in current code.

Kernel upstream commit is 91dd8c114499e9818f2d5919ef0b9eee61810220
ext4: prevent race while walking extent tree for fiemap.

Signed-off-by: Alexey Lyashkov <alexey.lyashkov@seagate.com>
Change-Id: I7790c9e1a9cbfbd2cc429292aa764250e0525e21
Reviewed-on: http://review.whamcloud.com/21603


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

When we hit Lustre crash, e2fsck always complains of Quota
accounting mismatch, and sometimes differences are huge.

The problem is that we don't journal quota updates properly, and
with current codes quota files will be only journaled when syncing
the filesystem, which only happens by sync() call or unmount.

RHEL6.8 version of the patch http://review.whamcloud.com/20503



Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Id5b580083197555029442338f7feaf59b1ad154c
Reviewed-on: http://review.whamcloud.com/20791


Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>