Pack parent FID in getattr request if OBD_CONNECT2_GETATTR_PFID is
enabled, otherwise fill it with target FID for backward compatibility.

Fixes: f9a2da63

 ("LU-13437 mdt: don't fetch LOOKUP lock for remot...")
Test-Parameters: clientversion=2.12 testlist=sanity env=SANITY_EXCEPT="27M 151 156"
Test-Parameters: serverversion=2.12 testlist=sanity env=SANITY_EXCEPT="56 165 205b"
Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Idcf8388b65dee1f0a09a53b240ce8303f3c6ff75
Reviewed-on: https://review.whamcloud.com/39290


Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Neil Brown <neilb@suse.de>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

Pack parent FID in getattr by FID, which will be used to check whether
child is remote object on parent. The helper function is called
mdt_is_remote_object(). NB, directory shard is not treated as remote
object, because if so, client needs to revalidate shards when dir is
accessed, which will hurt performance much.

For getattr by FID, if object is remote file on parent, don't fetch
LOOKUP lock, otherwise client may see stale dir entries.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Id181ecc053579ee394080381a82706334503ced0
Reviewed-on: https://review.whamcloud.com/38561


Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yingjin Qian <qian@ddn.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

Handle layout change internally when mapping name to stripe:
* Move layout changing related code into lmv name to stripe mapping
  so callers doesn't need to take care of the internals.
* lmv_name_to_stripe_index() maps name in new layout, and
  lmv_name_to_stripe_index_old() in old layout.
* rename lmv_migrate_existence_check() to lmv_old_layout_lookup()
  to support directory restripe in the future.
* support layout changing directory in LFSCK.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Icf8bda5db884784f761a2d373a6f81d7e13f525f
Reviewed-on: https://review.whamcloud.com/37711


Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

Similar to file OST object allocation, introduce directory stripe
allocation by space usage, but they don't share the same code because
of the many differences between them: file has mirrors, PFL, object
precreation; while for directory, the first stripe is always on the
same MDT where its master object is on. The changes include:
* add lod_mdt_alloc_qos() to allocate stripes by space/inode usage.
* add lod_mdt_alloc_rr() to allocate stripes round-robin.
* add lod_mdt_alloc_specific() to allocate stripes in the old way.
* add sysfs support for lmv_desc field in LOD structure, and move
  those remain in procfs to sysfs.

This patch also changes LMV QoS code:
* mkdir by QoS if user mkdir by command 'lfs mkdir -i -1 ...', or the
  parent directory default LMV starting MDT index is -1.
* with the above change, 'space' hash flag is useless, remove all
  related code.
* previously 'lfs mkdir -i -1' QoS code is in lfs_setdirstripe(),
  but now it's done in LMV, remove the old code.

Update sanity 413a 413b to support QoS mkdir of both plain and
striped directories.

Update lfs-setdirstripe man to reflect the changes.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I8f5f8e46faae68ffd9a49a4ac1d450e951e979c5
Reviewed-on: https://review.whamcloud.com/35825


Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

Like LOD, use lu_tgt_descs to manage tgts, so that they can
share tgt management code.

TODO: use the same tgt management code for LOV/LFSCK.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I843e85e0f8b05e21056567cebfea0a8fff1d4ef8
Reviewed-on: https://review.whamcloud.com/35218


Tested-by: jenkins <devops@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

PCC is a new framework which provides a group of local cache
on Lustre client side. No global namespace will be provided
by PCC. Each client uses its own local storage as a cache for
itself. Local file system is used to manage the data on local
caches. Cached I/O is directed to local filesystem while
normal I/O is directed to OSTs.

PCC uses HSM for data synchronization. It uses HSM copytool
to restore file from local caches to Lustre OSTs. Each PCC
has a copytool instance running with unique archive number.
Any remote access from another Lustre client would trigger
the data synchronization. If a client with PCC goes offline,
the cached data becomes inaccessible for other client
temporarilly. And after the PCC client reboots and the copytool
restarts, the data will be accessible again.

ToDo:
1) Make PCC exclusive with HSM.
2) Strong size consistence for PCC cached file among clients.
3) Support to cache partial content of a file.

Change-Id: I188ed36c48aae22...

Reuse the same object alloc QoS code as LOD, but the QoS code is
not moved to lower layer module, instead it's copied to LMV, because
it involves almost all LMV code, which is too big a change and should
be done separately in the future.

And for LMV round-robin object allocation, because we only need to
allocate one object, use the MDT index saved and update it to next
MDT.

Add sanity 413b.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I53c3d863dafda534eebb6b95da205b395071cd25
Reviewed-on: https://review.whamcloud.com/34657


Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>

If a plain directory default LMV hash type is "space", create
subdirs on all MDTs with balanced space usage:
* client mkdir allocate FID on MDT with balanced space usage
  (space QoS code is in next patch).
* MDT allows mkdir on different MDT with its parent if it has
  "space" hash type in default LMV, this is normally rejected
  because mkdir shouldn't create remote directory.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I284e21f334c07462211be4c8e38e965722d1e8a8
Reviewed-on: https://review.whamcloud.com/34360


Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

Sometimes the layout of striped directories may become broken:
* creation/unlink is partially executed on some MDT.
* disk failure or stopped MDS cause some stripe inaccessible.
* software bugs.

In this situation, this directory should still be accessible,
and specially be able to migrate to other active MDTs.

This patch add this support on both server and client: don't
imply stripe FID is sane, and when stripe doesn't exist, skip
it.

Add OBD_FAIL_MDS_STRIPE_FID to simulate insane stripe FID, and
OBD_FAIL_MDS_STRIPE_CREATE to simulate stripe creation failure.

Add sanity 60h.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I8a05a0e0cef8b051a935b3fa3d3e26c0b6ef3b4a
Reviewed-on: https://review.whamcloud.com/34750


Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Hongchao Zhang <hongchao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

This patch introduces a new striping/LMV format in order to
allow to specify an arbitrary external reference for a dir
in Lustre namespace.
The new LMV format is made of {newmagic, length, type, flags,
string[length]} to be as flexible as possible.
Foreign dir can be created by using the ioctl(LL_IOC_LMV_SETDIRSTRIPE)
operation and it can only be and remain an empty dir until removed.
A new API method llapi_dir_create_foreign() has been introduced
and "lfs {get,set}dirstripe" and "lfs find" modified to understand
new format.
The idea behind this is to provide Lustre namespace support and
striping prefetch/caching under lock protection, for user/external
usage.

This patch is the LMV/dirs complement of LOV/files previous change
(Change-Id: I5d9c0642fe8e7009c30918bfa946cac7c00c9af8) and has
been rebased on top of the latter along with some with obvious
mutualizations and simplifications.

Code has been added for lfsck to handle foreign dirs, and
a new sub-test has been a...

Security layer needs to fetch security context of files/dirs
upon metadata ops like lookup, getattr, open, truncate, and
layout, for its own purpose and control checks.
Retrieving the security context consists in a getxattr operation
at the file system level. The fact that the requested metadata
operation and the getxattr are not atomic can create a window
for a dead-lock situation where, based on some access patterns,
all MDT service threads can become stuck waiting for lookup lock
to be released and thus unable to serve getxattr for security context.
Another problem is that sending an additional getxattr request for
every metadata op hurts performance.

This patch introduces a way to get atomicity by having
the MDT return security context upon granted lock reply,
sparing the client an additional getxattr request.

Test-Parameters: serverbuildno=62488 serverjob=lustre-reviews testlist=sanity,sanity-selinux clientselinux
Test-Parameters: clientbui...

This is to remove an obsoleted code where it causes drastic
performance degradation. This code is written before PERM lock
is introduced, and it requests UPDATE lock at path walk for
remote directory, which will be cancelled at later file creation.

Tests result before and after this patch is applied:

Test case:
rm -rf /mnt/lustre_purple/testdir
lfs mkdir -i 0 /mnt/lustre_purple/testdir
lfs mkdir -i 2 /mnt/lustre_purple/testdir/dir2
./lustre-release/lustre/tests/createmany -o \
	/mnt/lustre_purple/testdir/dir2/f 10000

Before the patch is applied:
total: 10000 open/close in 12.82 seconds: 780.22 ops/second

After the patch is applied:
total: 10000 open/close in 4.89 seconds: 2044.75 ops/second

Signed-off-by: Jinshan Xiong <jinshan.xiong@gmail.com>
Change-Id: Ib474dc28d6edc7d15801b6821edc0e1d108bb4b6
Reviewed-on: https://review.whamcloud.com/34291


Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Teste...

Rather than using a macro, just access the required
field directly.

Linux-commit: 5b7cc4e4ce3cc15f67462ae75c55eecc7edc3a40

Change-Id: I325cac7458265d1cf6ad9f195a513f2612865906
Signed-off-by: NeilBrown <neilb@suse.com>
Reviewed-on: https://review.whamcloud.com/33984


Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Ben Evans <bevans@cray.com>

Migrating directory contains stripes of both old and new layout, and
its sub files may be located on either one. To avoid race between
access and new creations, there are 4 rules to access migrating
directory:
1. always create new file under new layout.
2. any operation that tries to create new file under old layout will
   be rejected, e.g., 'mv a <migrating_dir>/b', if b exists and is
   under old layout, this rename should fail with -EBUSY.
3. operations that access file by name should try old layout first,
   if file doesn't exist, then it will retry new layout, such
   operations include: lookup, getattr_name, unlink, open-by-name,
   link, rename.
4. according to rule 1, open(O_CREAT | O_EXCL) and create() will
   create new file under new layout, but they should check existing
   file in one transaction, however this can't be done for old
   layout, so check existing file under old layout on client side,
   then issue the open/create request to new layout.

Disable sanity 230d for ZFS backend because it will trigger lots of
sync, which may cause system hung.

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: Icd587eb17f4c6dc2234dcc3c7ba40759bc3e8d31
Reviewed-on: https://review.whamcloud.com/31504


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

Move all the remaining lustre headers shared between user land
and kernel space to the uapi directory. OBD_MAX_IOCTL_BUFFER
can be removed since the utilites don't even acknowledge it
value.

Change-Id: I6dcdf3f824dae6bc8a0786f0e61787ca68c10bd8
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/25246


Tested-by: Jenkins
Reviewed-by: Quentin Bouget <quentin.bouget@cea.fr>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ben Evans <bevans@cray.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Update copyright messages for files updated in 2016, excluding
trivial patches.

Add trivial patches to updatecw.sh script exclude list.  Improve
the output of patches causing the most changes to include the
patch summary instead of just the hash.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ifb098e8126f698ea975e10ba13f5f47c449ae4ad
Reviewed-on: https://review.whamcloud.com/24233


Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>

The link to the GPL license at Sun is no longer there.
Also Sun Inc is gone so lets remove all references to
it in our source code.

Test-Parameters: trivial

Change-Id: Ib47d07dcef49a1ef624d0d8a4a91b9731f486278
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/22800


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Frank Zago <fzago@cray.com>

mdt_intent_getxattr() can return EFAULT if a buffer cannot be found,
it is returned after lock_replace, where a new lock is installed into
lockp. An error forces ldlm_lock_enqueue() to destroy the original
lock, but ldlm_handle_enqueue0() drops the reference on the new lock.
xattr client code implied intent error is returned under a lock,
which is immediately cancelled. Check if a lock obtained and cancel
it properly for error cases. Note: we should support both cases for
interop needs, an intent error under a lock and with a lock abort.
Keep returning a lock with an intent error for interop purposes for
now, to be dropped later when client will get old enough.
make all intent ops to work through md_intent_lock: getxattr
and layout, which should extract the intent error.

Signed-off-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Change-Id: I7b628b50448c4bdb26a3a8758fc16a44212ad9ac
Seagate-bug-id: MRP-3072 MRP-3137
Reviewed-by: Andrew Perepechko <andrew.perepechko@seagate.com>
Reviewed-by: Andriy Skulysh <andriy.skulysh@seagate.com>
Tested-by: Elena V. Gryaznova <elena.gryaznova@seagate.com>
Reviewed-on: http://review.whamcloud.com/17220


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Assume a real connection in lmv_connect(). Mark OBD_CONNECT_REAL
obsolete. Remove the then unnecessary refcount and exp members of
struct lmv_obd. Remove calls to lmv_check_connect(). Disconnect the
export in the appropriate error path of lmv_connect().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I6d4f449506020964afd7f012983bfb2339429e0f
Reviewed-on: http://review.whamcloud.com/18018


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>

Change the __u64 *cookie parameter of md_set_lock_data() to const
struct lustre_handle *lockh.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ie6de252a10c532ee395e407ad2aff258773478a8
Reviewed-on: http://review.whamcloud.com/17072


Reviewed-by: Frank Zago <fzago@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Change the void *it_data member of struct lookup_intent to struct
ptlrpc_request *it_request.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: If3eb215620dfd376a18e4b15d7ab9f346468add0
Reviewed-on: http://review.whamcloud.com/17070


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>

Replace the union in struct lookup_intent with the members of struct
lustre_indent_data. Remove the then unused struct lustre_intent_data.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I47d815f7c8e51bcff33ea7dba6efbc08337ca509
Reviewed-on: http://review.whamcloud.com/17069


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Frank Zago <fzago@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Update copyright messages in files modified by Intel employees
in 2015 by non-trivial patches.  Exclude patches that are only
deleting code, renaming functions, or adding or removing whitespace.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I70fe6a346790e15d23606a3f380e7ef8fb8b84a0
Reviewed-on: http://review.whamcloud.com/16758


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Remove nlink < 2 check in lmv_revalidate_slaves, because
after nlink reaches to LDISKFS_LINK_MAX (65000), the inode
nlink will be set to 1.

Add test_300o in sanity to verify the case.

And also add version check for striped dir test.

Test-Parameters: envdefinitions=SLOW=yes testlist=sanity
Change-Id: I1a333ea7f68da9157679c1358df5f7a54aee8e51
Signed-off-by: di wang <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/16490


Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

If there are bad stripe during striped dir revalidation,
most likely due the race between close(unlink) and
getattr, then let's revalidate the dentry, instead of
return error, like normal directory.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I410940473d123796f75607d39153240b7d3f737b
Reviewed-on: http://review.whamcloud.com/15720


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Do not revalidate slave stripes while holding master lock.
Otherwise if the revalidating slaves are blocked, then the
master lock can not be released in time.

Remove some unnecesary merging in ll_revalidate_slave(), and
the attributes will be stored in each stripe, only
merging them if required.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I57c43236894e2bbbf9a20b1d90c5ab2a5dc62ef1
Reviewed-on: http://review.whamcloud.com/13432


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Update the copyright messages for changes made in 2014.

In some cases, changes made in 2013 did not generate an updated
copyright message, so these files are updated accordingly.  In some
cases there were no licenses on newly-added files, so add them (either
with new copyrights, or including copyrights from the source files).

Some types of changes that are not actually changing the code
(e.g. whitespace, mass changes to add or remote prefixes, patches
updating the copyright messages, patch reversions and the reverted
patch, patches that only delete code, etc.) have been excluded.

The updatecw.sh script was updated to fix a number of problems:
- it now handles comments in nroff (man page) files properly
- remove need to handle non-Intel copyrights specially
- allow subdirectory to be specified on command-line
- improve script debugging when running with "sh -vx"
- print patches changing the most files, to catch mass replacements

There are still a few c...

Some functions and variables are only used in their C file, so reduce
their scope. This reduces the code size, and fixes sparse warnings
such as:

  warning: symbol 'proc_lnet_routes' was not declared.
      Should it be static?
  warning: symbol 'proc_lnet_routers' was not declared.
      Should it be static?

Some prototypes were removed from C files and added to the proper
header.

Signed-off-by: Frank Zago <fzago@cray.com>
Change-Id: I8dcc5224c1da75cfb5ef7afb1fdb0f72422a3ac0
Reviewed-on: http://review.whamcloud.com/12222


Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Change type of client_obd->*_mds_*size from int to __u32 and
argumanets of related create/rename/setattr functions.
Change type of op_data->op_namelen to size_t.
Change type of argument size for all mdc_*_pack() to size_t.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I97f4fa6928c24fa416c334206c75f9885266b1ae
Reviewed-on: http://review.whamcloud.com/11379


Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Enhance LFSCK code framework for striped directory verification,
including:

1) New lfsck component APIs: lfsck_operations::lfsck_open_dir()
   and lfsck_operations::lfsck_close_dir().
1.1) The namespace LFSCK uses lfsck_open_dir() to initialize the
     LMV EA verification environment.
1.2) The namespace LFSCK uses lfsck_close_dir() to indicate the
     end of the scanning the striped directory.

2) When create orphan parent, the LMV EA should be considered.

3) Data structure definition and extention.

4) Other code cleanup.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I01e14645796c3fa3269c5f3cdcbb755ccb455d7e
Reviewed-on: http://review.whamcloud.com/11845


Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

ltd_idx is used as unsigned.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Iae119e03b4cb58dddca2ce230d963d380255a57a
Reviewed-on: http://review.whamcloud.com/11879


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Move lustre_intent.h from lustre/include/linux to lustre/include.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I8b3947bc558e6721e4a238526497124d3ad193d0
Reviewed-on: http://review.whamcloud.com/11499


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>

In lmv_revalidate_slaves() ensure that the request returned by
md_intent_lock() is properly released on all paths.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I35d63b248d0c80261ebc97f43722b0c547eb1aac
Reviewed-on: http://review.whamcloud.com/11326


Tested-by: Jenkins
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

In lmv_intent_lock() if we will return an error then first release any
locks referenced by the intent.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I43fa0d3869cf365175750b0cf944145c13f31006
Reviewed-on: http://review.whamcloud.com/11319


Tested-by: Jenkins
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Remove liblustre includes in lmv/. Always assume a kernel build.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ia0e61d11cb8238e362e934a7d424b9e768e713fe
Reviewed-on: http://review.whamcloud.com/10195


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

For unknown hash type, LMV should try all stripes to locate
the name entry. But it will only for lookup and unlink, i.e.
we can only list and unlink entries under striped dir with
unknown hash type.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ifeed7131c24e48277a6cc8fd4c09b7534e31079f
Reviewed-on: http://review.whamcloud.com/10041


Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

Currently client open-by-fid often packs name into the request,
but the name may be invalid, eg. NFS export, and even if it's
valid, it may cause inconsistency because this operation is done
on this fid, which is globally unique, but name not.

Since open-by-fid doesn't pack name, for striped dir we can't know
parent stripe fid on client, so we set parent fid the same as
child fid, and MDT has to find its parent fid from linkea (this is
already supported by MDT).

M_CHECK_STALE becomes obsolete.

Unset MDS_OPEN_FL_INTERNAL from open syscall flags, because these
flags are internally used, and should not be set from user space.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Id8d32a26497b2df944c7f9ef51a418ec136a6a9b
Reviewed-on: http://review.whamcloud.com/7476


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Rename each member of struct mdt_body, adding the prefix mbo_.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: If229b803a5ccdb27378f6fc44445c1c9bf120d8b
Reviewed-on: http://review.whamcloud.com/10202


Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

Remove the lmm and lmmsize parameters from both functions, storing
that data in md_op_data when needed. Remove the unused lookup_flags
parameter from md_intent_lock(), and the unused reqp parameter from
md_enqueue(). Add a union ldlm_policy_data * parameter to
md_enqueue(). Remove the unused function lmv_enqueue_remote().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ia914318a7c7cbeb59fe86d4a169559cd86f3ad57
Reviewed-on: http://review.whamcloud.com/10205


Tested-by: Jenkins
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Separate master stripe with master object, so
1. stripeEA only exists on master object.
2. sub-stripe object will be inserted into master object
as sub-directory, and it can get the master object by "..".

By this, it will remove those specilities for stripe0 in
LMV and LOD. And also simplify LFSCK, i.e. consistency check
would be easier.

And also after this separation, LOD will know whether iterating
the whole stripe or single stripe eaisly, i.e. for master_object,
it will iterate the whole stripes, for sub_stripe, it will only
iterate the single stripe.

This patch also fixes a few things in osp orphan iteration, to
make it work with remote dir entry iteration.

When then master object becomes an orphan, we should
mark all of its sub-stripes as dead object as well,
otherwise client might still be able to create files
under these stripes.

A few fixes for striped directory layout lock:

1. stripe 0 should be locked as EX, same as other stripes.
2. Acquire the layout for directory, when it is being unliked.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I6212fb97a2360664b48e0a75424a89c857da2043
Reviewed-on: http://review.whamcloud.com/9511


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>