GitLab

When we are trying to grant a lock and met an AST error, rerunning
the policy is pointless since it cannot grant a potentially now eligible
lock and our lock is already in all the queues, just be like all the other
handlers for ERESTART return and run a full resource reprocess instead.

Lustre-change: https://review.whamcloud.com/#/c/39598/
Lustre-commit: 24e3b539



Change-Id: I3edb37bf084b2e26ba03cf2079d3358779c84b6e
Signed-off-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40412

Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>

The search_itree and search_queue functions should both
return either a pointer to a found lock or NULL.

Currently, search_itree just returns the contents of
data->lmd_lock, whether or not a lock was found.

search_queue will do the same under certain cirumstances.

Zero lmd_lock in both search_* functions, and also stop
searching in search_itree once a lock is found.

cray-bug-id: LUS-6783
Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: Ie231166756e60c228370f8f1a019ccfe14dfda6a
Reviewed-on: https://review.whamcloud.com/33754


Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/40399

Tested-by: jenkins <devops@whamcloud.com>

When there are lock-ahead write locks on a file, the server
sends one glimpse AST RPC to each client having such (it
may have many) locks. This callback is sent to the lock
having the highest offset.

Client's glimpse callback goes up to the clio layers and
gets the global (not lock-specific) view of size.  The clio
layers are connected to the extent lock through the
l_ast_data (which points to the OSC object).

Speculative locks (AGL, lockahead) do not have l_ast_data
initialised until an IO happens under the lock. Thus, some
speculative locks may not have l_ast_data initialized.

It is possible for the client to do a write using one lock
(changing file size), but for the glimpse AST to be sent to
another lock without l_ast_data initialized.  Currently, a
lock with no l_ast_data set returns ELDLM_NO_LOCK_DATA to
the server.  In this case, this means we do not return the
updated size.

The solution is to search the granted lock tree for any lock
with initialized l_ast_data (it points to the OSC object
which is the same for all the extent locks) and to reach the
clio layers for the size through this lock instead.

Lustre-change: https://review.whamcloud.com/33660
Lustre-commit: b3461d11



cray-bug-id: LUS-6747
Signed-off-by: Patrick Farrell <pfarrell@whamcloud.com>
Change-Id: I6c60f4133154a3d6652315f155af24bbc5752dd2
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/36406

Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

Layout_change is a reint operation, and it should be handled the
same as other reint operations, so that resent and replay can
work correctly.

Also replace the lock passed in ldlm_handle_enqueue0() with the
lock taken in mdt_layout_change(). This avoids taking lock again
in ldlm_handle_enqueue0(), and also makes replay eaiser. Note,
before replacing, the mode is downgraded from EX to CR, because
client only needs this mode, as can avoid unnecessary lock cancel
later.

Add missing resent reconstructor for REINT_RESYNC.

Lustre-change: https://review.whamcloud.com/35465
Lustre-commit: e1bebbdf

Signed-off-by: Lai Siyao <lai.siyao@whamcloud.com>
Change-Id: I328044dacbf18d03232c9bbb51271f6202e9b939
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/36376

Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

setxattr takes inode lock and sends reint to MDS.
truncate takes MDS_INODELOCK_DOM lock and wants
to acquire inode lock.

MDS locks are for different bits
MDS_INODELOCK_UPDATE|MDS_INODELOCK_XATTR vs
MDS_INODELOCK_DOM but they blocks each other if
some blocking lock was present earlier.

If IBITS waiting lock has no conflicts with any lock in the
granted queue or any lock ahead in the waiting queue then
it can be granted.

Use separate waiting lists for each ibit to eliminate full
lr_waiting list scan.

Lustre-change: https://review.whamcloud.com/35057
Lustre-commit: 2250e072



Cray-bug-id: LUS-6970
Change-Id: I95b2ed0b1a0063b7ece5277a5ee06e2511d44e5f
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35937

Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Stephan Thiell <sthiell@stanford.edu>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

to be able to lookup by current thread where it's too
complicated to pass env by argument.

this version has stats to see slow/fast lookups. so, in sanity-benchmark
there were 172850 fast lookups (from per-cpu cache) and 27228 slow lookups
(from rhashtable). going to see the ration in autotest's reports.

Fixes: 2339e1b3 ("LU-11483 ldlm ofd_lvbo_init() and mdt_lvbo_fill() create env")
Fixes: e02cb407 ("LU-11164 ldlm: pass env to lvbo methods")

Lustre-change: https://review.whamcloud.com/34566
Lustre-commit: aa82cc83



Change-Id: Ia760e10fa5c68e7a18284e4726d215b330fc0eed
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35110


Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

This patch removes ll_md_blocking_ast() processing for
not granted locks. The reason is ll_invalidate_negative_children()
can slow down I/O significantly without a reason if there
are thousands or millions of files in the directory
cache.

Lustre-change: https://review.whamcloud.com/19665
Lustre-commit: 2c126c5a



Change-Id: Ic69c5f02f71c14db4b9609677d102dd2993f4feb
Seagate-bug-id: MRP-3409
Signed-off-by: Andrew Perepechko <c17827@cray.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/35001


Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

DOM locks matching tries to ignore locks with
LDLM_FL_KMS_IGNORE flag during ldlm_lock_match() but
checks that after ldlm_lock_match() call. Therefore if
there is any lock with such flag in queue then all other
locks after it are ignored and new lock is created causing
big amount of locks on single resource in some access
patterns.
Patch extends lock_matches() function to check flags to
exclude and adds ldlm_lock_match_with_skip()p to use that
when needed.
Corresponding test was added in sanity-dom.sh

Test-Parameters: testlist=sanity-dom

Lustre-change: https://review.whamcloud.com/34261
Lustre-commit: b915221b

Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: Ic45ca10f0e603e79a3a00e4fde13a5fae15ea5fc
Reviewed-by: Patrick Farrell <pfarrell@whamcloud.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/34504


Tested-by: Jenkins
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

The previous commit 954cc675 skips bl_ast for local lock but
there are cases on MDT when local lock can become a client lock,
see mdt_intent_lock_replace(). In that case the client should be
notified if lock is a blocking lock.

Patch reverts commit 954cc675

 and provides alternative solution.
During downgrade to COS the lock renews own blocking AST states
and start reprocessing. Any new lock conflict will cause new
blocking AST and related async commit as needed.

Test-Parameters: mdssizegb=20 testlist=racer,racer,racer
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I41adab5c805a59fdbeade8ae3556556b779dc3c0
Reviewed-on: https://review.whamcloud.com/33458

Reviewed-by: Vitaly Fertman <c17818@cray.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

Use INIT_LIST_HEAD_RCU to avoid compiler optimization too much
in some case.
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I66b340ac3147d2cb911a2b7d3e210c6847047dac
Reviewed-on: https://review.whamcloud.com/33317


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: John L. Hammond <jhammond@whamcloud.com>

Update l_blocking_lock under with locking to prevent race
between lock_handle_convert0() and ldlm_work_bl_ast() code.
Signed-off-by: Mikhail Pershin <mpershin@whamcloud.com>
Change-Id: I881a1daf6f3b09677abcd6a85f6891d409926cc8
Reviewed-on: https://review.whamcloud.com/33124


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

to save on env allocation.

Benchmarks made by Shuichi Ihara demonstrated 13% improvement
for small I/Os: 564k vs 639k IOPS. the details can be found
LU-11164.

Change-Id: I797e3d7e19ef408993004a2b872842d655240525
Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-on: https://review.whamcloud.com/32832


Tested-by: Jenkins
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

In enum ldlm_intent_flags, remove the obsolete constants IT_UNLINK,
IT_TRUNC, IT_EXEC, IT_PIN, IT_SETXATTR. Remove any handling code for
these opcodes.
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I66f20e4c881cb77a481805a148a33f1c2daa5f0c
Reviewed-on: https://review.whamcloud.com/32361

Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mpershin@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

When race happen between ldlm_server_blocking_ast() and
ldlm_request_cancel(), the at_measured() is called with wrong
value equal to current time. And even worse, ldlm_bl_timeout() can
return current_time*1.5.
Before a time functions was fixed by LU-9019(e920be68

) for 64bit,
this race leads to ETIMEDOUT at ptlrpc_import_delay_req() and
client eviction during bl ast sending. The wrong type conversion
take a place at pltrpc_send_limit_expired() at cfs_time_seconds().

We should not take cancels into accoount if the BLAST is not send,
just because the last_activity is not properly initialised - it
destroys the AT completely.
The patch devides l_last_activity to the client l_activity and
server l_blast_sent for better understanding. The l_blast_sent is
used for blocking ast only to measure time between BLAST and
cancel request.

For example:
 server cancels blocked lock after 1518731697s
 waiting_locks_callback()) ### lock callback timer expired after 0s:
 evicting client
Signed-off-by: Alexander Boyko <c17825@cray.com>
Change-Id: I44962d2b3675b77e09182bbe062bdd78d6cb0af5
Cray-bug-id: LUS-5736
Reviewed-on: https://review.whamcloud.com/32133


Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Keep the skipped locks in lru list will cause serious
contention for ns_lock. Since we have to travel them
every time in the ldlm_prepare_lru_list(). So we will
use a cursor to record position that last accessed
lock in lru list.

Change-Id: Ibda36a90e54076cb785a65910b34300639b3e140
Signed-off-by: Yang Sheng <yang.sheng@intel.com>
Signed-off-by: Sergey Cheremencev <c17829@cray.com>
Reviewed-on: https://review.whamcloud.com/26327

Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

For IBITS lock it is possible to drop just conflicting
bits and keep lock itself instead of cancelling it.
Lock convert is only bits downgrade on client and then
on server.
Patch implements lock convert during blocking AST.

Test-Parameters: mdssizegb=20 testlist=sanity-dom,dom-performance,racer,racer,racer
Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: Id073f14ec3283479a9a48fd80090ecd3b0967b6d
Reviewed-on: https://review.whamcloud.com/30202


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Policy processors can unlock resource to send BL AST,
so cached next list entry can become invalid.

Move sending BL ASTs to ldlm_reprocess_queue()
in case of LDLM_PROCESS_RECOVERY.

Cray-bug-id: LUS-5689
Change-Id: Ib9b757576461b2f74aaa916b4b62538a9abfa0dd
Signed-off-by: Andriy Skulysh <c17819@cray.com>
Reviewed-on: https://review.whamcloud.com/31726


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alexandr Boyko <c17825@cray.com>
Reviewed-by: Vitaly Fertman <c17818@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

This patch includes a couple of changes:
- add new wrapper function obd_import_nid2str
- use obd_import_nid2str and obd_export_nid2str to replace all
  libcfs_nid2str conversions for NID of export/import connection
Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Change-Id: I57d08e6ef902c6a34c705663de0ed73bb3dc76f2
Reviewed-on: https://review.whamcloud.com/12956

Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Amir Shehata <amir.shehata@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Patch removes lock mode convert mechanics from Lustre,
it is obsoleted and not functional at the moment. Also
there are no plans to restore it and use again.
Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I477caf24927768dfcdc15888e59a7d5e62d5b577
Reviewed-on: https://review.whamcloud.com/30491


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Update copyright messages for files updated in 2016, excluding
trivial patches.

Add trivial patches to updatecw.sh script exclude list.

Revert some changes that were incorrectly attributed to the
2016 (d10200a8) and
2015 (0754bc8f

) copyright
update patches themselves, since they were not in the exclude
list when the subsequent script was run.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I82f21c30c4dac75792bb49fc139bee2ca51f5545
Reviewed-on: https://review.whamcloud.com/30341


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

In that case, the assert for 'list_empty(&lock->l_bl_ast)' will fail
because the lock is already in a cancel list.

This patch checks if the lock is already being canceled in prior.
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I04b9df789cf3b3b5e6dd61a647376307f9975f4a
Reviewed-on: https://review.whamcloud.com/29080

Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

Replace the last cfs_time_current_sec() to avoid the overflow
issues in 2038 with ktime_get_real_seconds(). Reduce the jiffies
usage to the bare minimum which is useage for mod_timer() and
schedule_timeout(). This makes the ldlm totally 64 bit time
compliant.

Change-Id: Iaee92c17d51fdfc55bd26e9e813e30a6ce794856
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/29295


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

The DoM bit combining with other ibits opens way to several
optimization, like getting IO lock in advance along with OPEN,
size attribute caching at client and others.

Test-Parameters: mdssizegb=20 testlist=sanity-dom,dom-performance
Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I442f742b21efba84a0c61501f4877a36c7e9d572
Reviewed-on: https://review.whamcloud.com/28024


Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>

LVBO lvbo_update call was using ldlm_resource as parameter,
the ldlm_lock parameter is added to recognize DoM resource
in MDT LVBO functions and perform DoM specific actions.
Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: Id1cfee12bfc13893206c771f45acf7dd44da56ba
Reviewed-on: https://review.whamcloud.com/28016

Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>

The ldlm sysfs file handles lru_max_age in jiffies which is wrong
since jiffies are not consistent across machine since HZ is
configurable at compile time. Talking to most users they thought
lru_max_age was in seconds which is incorrect. The best way to
fix this is to move lru_max_age to millisecs since most systems
lustre deals with sets HZ to 1000. To make it clear it is in
milliseconds print out lru_max_age with "ms". Since users tend
to think in seconds allow passing in seconds besides milliseconds
and internally converting them to nanaseconds. Since we have to
support milliseconds move to ktime_t since we can't use time64_t.
Unfortunately, this makes a relatively large patch, but I could
not find a way to split it up some more without breaking atomicity
of the change.

Change-Id: I0b1814fd9d903767f62fe141d2c95845b75fb95a
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: https://review.whamcloud.com/28370

Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Ladvise lockahead is a new feature allowing userspace to
request extent locks in advance of the IO which will use
them. These locks are not expanded beyond the size
requested by userspace.  They are intended to make it
possible to address lock contention between multiple
clients resulting from lock expansion.  They should allow
optimizing various IO patterns, notably strided writing.
(Further information in LU-6179)

Asynchronous glimpse locks are a speculative version of
glimpse locks, and already implement the required behavior.
Lockahead requests share this behavior.

Additionally, lockahead creates extent locks in advance
of IO, and so breaks the assumption that the holder of the
highest lock knows the current file size.

So we also modify the ofd_intent_policy code to glimpse
PW locks until it finds one it knows to be in use, taking
care to send only one glimpse to each client.

The current patch allows asynchronous non-blocking lock
ahead requests and synch...

This is to avoid a case that IR lock revocation is going on when
the obd is being stopped, an extra ldlm_reprocess_recovery_done() is
required to make revocation process move forward.

Turn off 'set -e' in rundbench. Otherwise killing dbench process will
return an error to wait(1) in rundbench. Since test-framework has
turned on error on exit, it will set test result as failure, which
is actually a false alarm.

Test-Parameters: envdefinitions=SLOW=yes,ONLY=26 testlist=replay-dual,replay-dual,replay-dual,replay-dual
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I43ab1df9c8fe5aea15da6c90175fd08a0b099ea2
Reviewed-on: https://review.whamcloud.com/17853


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

There is code to get lock without blocking, e.g. 'try'
to get it. It enqueues lock with all wanted bits and
get it only when all bits are not conflicting.
The new code allows to get as much 'tried' bits as
possible. It also doesn't require extra locking attempt
for that.
Patch intoduces new inodebits field try_bits which contains
optinal bits to get if possible without blocking. Those
bits are checked along with the required bits and only
non-blocked are added to the new IBITS lock.
Signed-off-by: Mikhal Pershin <mike.pershin@intel.com>
Change-Id: I78d181243556e65d3bd702d332307d0723d5b665
Reviewed-on: https://review.whamcloud.com/25262


Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

REP-ACK hr may race with transaction commit, and the latter will
release saved locks, so in REP-ACK hr we need to get locks early to
convert them to COS mode safely.

But the locks got may be decrefed and canceled, in this case it
can't be converted to COS mode, remove an assert in
ldlm_lock_downgrade() for this.

Also protect mdt_steal_ack_locks() with rs_lock because it may also
race with REP-ACK hr. And move ldlm_lock_decref() outside of locks
because it may sleep.
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Ia9a3ba6a83689c0552ae8aaf2eb735c3f06b62e2
Reviewed-on: https://review.whamcloud.com/27207


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Lock's resource can change on a client.
Take a resource reference under spinlock
to print debug information.

Change-Id: Id8acb801ea549bf3c1ce1bcf6349db31578579f3
Xyratex-bug-id: MRP-2760
Signed-off-by: Andriy Skulysh <andriy.skulysh@seagate.com>
Reviewed-on: https://review.whamcloud.com/16139


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

If blocking AST wasn't received by client before recovery,
we need to scan the whole waiting lock list to send blocking
ASTs after lock replay done, otherwise, client could be
evicted unpurposely like following:

- cl1 has a granted lock;
- cl2 has a waiting lock, BL AST is sent but lost on a way;
- failover, locks are replayed and applied on the server in
  the correct order;
- waiting lock is just put to the resource, no new BL AST
  is re-sent, no timeout can happen for the granted lock on
  server, no timeout for the waiting lock on client;
- cl2 will be hanging for a long time until cl1 will cancel
  its aged lock; may lead to cl2 eviction.
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I2a3fecf3b7fa79f96874d5ae21c599725334d9a5
Reviewed-on: https://review.whamcloud.com/24716

Reviewed-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>

We need some mechanism to set write barrier on the MDT, with that,
all the subsequent modification will be blocked, then the barrier
sponsor can operate on the frozen system, such as make snapshot.

It is implemented via new MGS/MGC plain LDLM lock on the virtual
barrier config-log. The MGS sends the barrier request to the MDTs
via glimpse callback against such barrier plain ldlm lock. The MGC
returns is local barrier status via the glimpse reply.

This patch also enhances mgs lproc interface logic.
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I349249fc21e3a6394e658d7529cf87299c050f1a
Reviewed-on: https://review.whamcloud.com/24262

Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Update copyright messages for files updated in 2016, excluding
trivial patches.

Add trivial patches to updatecw.sh script exclude list.  Improve
the output of patches causing the most changes to include the
patch summary instead of just the hash.

Test-Parameters: trivial
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ifb098e8126f698ea975e10ba13f5f47c449ae4ad
Reviewed-on: https://review.whamcloud.com/24233

Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>

Postpone lock reprocess during lock replay stage.
Reprocess is needed during request replay stage
beacause local locks are still in use until
client ACK.

Change-Id: I250d22fee471db643f12a900fdfc51eacfa94aa2
Seagate-bug-id: MRP-3516
Signed-off-by: Andriy Skulysh <andriy.skulysh@seagate.com>
Reviewed-on: http://review.whamcloud.com/21059


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

The link to the GPL license at Sun is no longer there.
Also Sun Inc is gone so lets remove all references to
it in our source code.

Test-Parameters: trivial

Change-Id: Ib47d07dcef49a1ef624d0d8a4a91b9731f486278
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/22800


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Frank Zago <fzago@cray.com>

Originally when lustre code was built for userland we needed
a proper way to handle 32 bit and 64 bit platforms when
reporting unsigned longs. Now that this code is only built
for kernel space and the kernel has it own special string
handling functions we don't need this abstraction anymore.
Remove this abstraction from the client side code.

Change-Id: Ic0c55a413237bdf57d60031c12d5d9b62fa39cef
Signed-off-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-on: http://review.whamcloud.com/20590


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

These #if 0 blocks have been in place for years. Assume
they are not used and remove them

Cleanup functions that are no longer used:
sec_gss.c: gss_netobj_swabber
sec_gss.c: gss_swab_netobj
osd_iam_lvar.c: e_check
osd_iam.c: iam_leaf_check
osc_page.c: osc_page_is_dlocked
osc_page.c: osc_page_protected
mgs_llog.c: mgs_check_failnid

Test-Parameters: trivial
Signed-off-by: Ben Evans <bevans@cray.com>
Change-Id: Ib3d337f59a14c5cd7340be7e1aef4f2ca9663ac4
Reviewed-on: http://review.whamcloud.com/20416

Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Frank Zago <fzago@cray.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

This patch resolves IO vs eviction race.
After eviction failed export stayed at stale list,
a client had IO processing and reconnected during it.
A client sent brw rpc with last lock cookie and new connection.
The lock with failed export was found and assert was happened.
(ost_handler.c:1812:ost_prolong_lock_one())
	ASSERTION( lock->l_export == opd->opd_exp ) failed:

1. Skip the lock at ldlm_handle2lock if lock export failed.
2. Validation of lock for IO was added at hpreq_check(). The lock
   searching is based on granted interval tree. If server doesn`t
   have a valid lock, it reply to client with ESTALE.
3. A lock prolong was moved at ldlm layer.
Signed-off-by: Alexander Boyko <alexander.boyko@seagate.com>
Seagate-bug-id: MRP-2787
Change-Id: I2177736564c3b8164f1ad3e4cc02dca1704a0e6e
Reviewed-on: http://review.whamcloud.com/18120

Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

During ldlm_lock_cancel()->ldlm_cancel_callback(), the lock could
possibly be added back to waiting list with no resource lock holding.

Put back the second checking and removing from the waiting list.
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I8841422d6e2f6bf92efa24d542583936123ead66
Reviewed-on: http://review.whamcloud.com/20509


Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

It is a regression after LU-2177.
LCK_NL is used by obdecho also.

Perform flock sanity checks only for LDLM_FLOCK.

Change-Id: I83f87b41a13c4d666af36af389ccf1a127197294
Xyratex-bug-id: MRP-2336
Signed-off-by: Andriy Skulysh <andriy.skulysh@seagate.com>
Reviewed-on: http://es-gerrit.xyus.xyratex.com:8080/9406


Tested-by: Jenkins
Reviewed-by: Mirza Arshad Mirza Hussain <arshad.hussain@seagate.com>
Reviewed-by: Alexey Leonidovich Lyashkov <alexey.lyashkov@seagate.com>
Tested-by: Parinay Vijayprakash Kondekar <parinay.kondekar@seagate.com>
Reviewed-on: http://review.whamcloud.com/17791

Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>