b_cmobd_merge should not be used for any further development or testing!

cMD_PRE_CMD_SMASH are the relevant tags.

The pre-smashing contents of HEAD will be committed by yury into b_cmd as a
merge of the two branches.  These will be stabilized on b_cmd and eventually
landed onto HEAD.

b=3284, b=3285, b=3043, b=3236, revert liblustre group changes

b=2947, b=2543, b=1557, b=2009

- remove extN/ files

* bug fixes
 - Fix ldlm_lock_match on the MDS to avoid matching remote locks (592)
 - Fix fsfilt_extN_readpage() to read a full page of directory
   entries, or fake the remainder if PAGE_SIZE != blocksize (500)
 - Avoid extra mdc_getattr() in ll_intent_lock when possible (534, 604)
 - Fix imbalanced LOV object allocation and out-of-bound access (469)
 - Most intent operations were removed, in favour of a new RPC mode
   that does a single RPC to the server and bypasses most of the VFS
 - All LDLM resource ID arrays were removed in favour of ldlm_res_id
 - Aggressively cancel local locks on DLM servers
 - mds_reint_unlink sends EA to the client if it's the last nlink.
   client uses that EA to unlink OST objects.
 - mds_reint_{rename,unlink,link} were rewritten to take ordered locks
 - recursive symlinks were fixed (440)
 - fixed NULL deref in DEBUG_REQ
 - filter_update_lastobjid no longer calls sync, which annoyed extN
 - fixed multi-client small-writes to a single file problem (445)
 - fixed mtime updates during file writes (607)
 - fixed vector writes on obdfilter causing problems when ENOSPC (670)
 - fixed bug in obd_brw_read/write() (under guise of testing 367)
 - fixed Linux OST size reporting problem (444, 656)
 - OST now updates object mtime with writes or setattr (607, 619)
 - client verifies file size before zeroing page past EOF (445)
 - OST now writes last allocated objid to disk with allocation (108)
 - LOV on echo now works (409)
 * protocol changes
 - mds_reint_unlink sends a new buffer, with the EA included.  this
   buffer is only valid if body->valid & OBD_MD_FLEASIZE, which is only
   set if a regular file was being unlinked, and it was the last link
 - use PtlGet from the target for bulk writes (315)
 - OST now updates object mtime with writes or setattr (607, 619)
 - LDLM now has a grant-time callback to revalidate locked items, if
   necessary (604)
 - Many MDS operations were reorganized to combat race conditions
* other changes
 - Merge b_intel branch (updated lprocfs code) - now at /proc/fs/lustre
 - configure check to avoid gcc version 2.96 20000731-2.96-98 (606)

Fixes a _huge_ number of bugs:
  - Fully reactivate OST imports after reconnection (512, others)
  - Make sure client sees our -ENOTCONN from mds_handle (513 - partial)
  - More graceful error handling for truncating on dead OST (515)
  - Don't error out unless we're actually accessing dead stripes (474)
  - Fix garbage sizes when stripes are missing (410)
  - LRU counters were broken, causing constant lock purge (433, 432)
  - garbage on read from stripes with failed OSTs (441)
  - mark OSCs as active before reconnecting during recovery (438)
  - lov_enqueue and lov_cancel need to handle inactive OSTs (403)
  - lfind did not preserve OST order in output (443)
  - symlinks cause hung clients, incorrect data (439)
  - stop dereferencing request after dropping refcount (457)
  - don't LASSERT(spin_is_locked) on non-SMP (455)
  - fixes for many rename() bugs
  - fstat didn't correctly synchronize attributes (399)
  - server must handle lock cancellation during blocking AST prep (487)
  - bulk descriptors were free()d too soon (511)
  - fix paths in lconf, which would load incorrect modules (451, 507)
  - fix confusing lconf 'host not found' error message (386)
  - fix lock order deadlock on OST (O/R i_sem before journal ops, 478)
  - fix race condition in mdc_blocking_ast() for inode access (526)
  - fix lov_unpackmd() unpacking wrong number of stripes (537)
  - fix lov_set_osc_active() marking wrong OSC inactive (440)
  - fix bad lstripe lov_unpackmd() assertion (fix layering too) (527)
  - fix multiple writes of stripe MD to MDS (358, maybe 519)
  - fix lstripe in several ways (kernel side) (527)
  - fix request leak in ldlm_cli_enqueue (262)
  - incorrect OSC was marked inactive after OST failure
  - call mds_fs_cleanup before unmounting filesystem (524)
  - fix races between taking ns_lock and ldlm_lock_change_resource
  - fix races updating LOV export open file list
  - fix lov_enqueue error path, avoid decref-ing bad lock handle
  - fix recovery NULL deref in ldlm_cli_cancel_unused
  - fix some DLM races by using new hash table for lock handles (419)
  - permit the client to specify desired inodes, at replay
  - duplicate requests when we queue them for replay reintegration
  - fix last_rcvd offset calculation
  - sync after each recovered transaction, so we always make progress
  - never, not always, ERESTART requests without transnos
  - store the lov_desc in the MDS, so we don't depend on getlovinfo to set it
  - skip replay if the MDS says that the client is already connected
  - don't check for a recovery-enabled export to match lctl's UUID
  - don't INC_USE_COUNT for phantom exports
  - don't crash when cleaning up phantom exports (567)
  - don't double-finish or set replay data for errored mdc_open requests
  - abort requests when they time out, so we don't get old replies
  - send/receive replies for AST messages again
  - if the client says that it doesn't have the lock, cancel it on the server
  - if we timeout during I/O, don't try to cancel an in-use lock; instead
    mark it as destroyed, it will all work out when decref is called
  - fix module use counts (22, 581)
 * protocol changes
  - ASTs now expect a reply (server cancels lock on error reply)

- fixes of many metadata bugs:
  - multiple client file opens
  - missing/extra intent releases
  - symlink fixes
- OST/MDS server threads clobbering each other
  - dbench 2 works mostly
- many others too numerous to mention here (ChangeLog is a good record)

the basename of the config script, so you can do:

NAME=lov sh llmount.sh
NAME=lov sh llmountcleanup.sh

and it will work.

Also fix a bug in llmount.sh, which I think would have prevented the
XML config file from being regenerated properly.

- remove old xml files

- It's a little late to be fixing llmountcleanup, but I did it anyways

  this patch
- the mds has a new lovconfig command to tell it the UUID's and
  default striping pattern of the targets it needs to use (these are
  the UUID's of the OSC's typically).
- To make this scalable I changed some of the memory management in the
  class ioctl handling
- The LOV device has a trivial attach method and setup only tells it
  what MDC to use to get its information, by giving it the MDC-UUID.
- As discussed before, the MDS really provides the persistent storage
  for the LOV, what little it needs.  So during the obd_connect call
  for the object storage target (which is made from read_super) the
  storage target learns how it is striped and then connects to all the
  targets.
- We are in need of better configuration scripts for this stuff and
  tomorrow we will push the XML configurations a little further.
- Updated the documentation
- Began to cleanup the /proc/lustre/ stuff -- have some neat ideas
  about that and SNMP now.

- Fix mknod regression
- Fixed one connection leak

- More intent lock progress

- Considerable intent locking infrastructure, but it's not working perfectly
  yet.
- Updated documentation
- Many bugs.  Hold on.
- Test script changes to allow multiple mounts

I'll leave the branch around for a day or two in case we somehow missed
anything, then it will die.

Convert scripts over to new setup methods where possible to avoid them
becoming increasingly outdated.  Some scripts are already broken, and
I don't use them so I'm not sure whether to remove them or fix them.

Scripts updated are llmount.sh, llrmount.sh, llecho.sh, lldlm.sh,
llmount-client.sh and llmount-server.sh.  They use the default config
scripts net*.cfg, obd*.cfg, ldlm.cfg, mds.cfg as needed to do the same
thing they used to do.

- add ext2obd patch for 2.4.9
- change ha_assist2 to failover at LLNL
- fix exit code from llmountcleanup.sh to allow kimberlite to work.

features (newdev, name2dev FOO, and setup $FOO).  Also changed the "runtests"
script over to using the new configuration setup so that it is easier to run
with both ext2/ext3 MDS and obdext2/obdfilter OBDs.

For OST devices, use "OBDDEV" for all types (ext2obd,filterobd,echo) so that
you can clean them all up the same way.

 - add a fail function to common.sh to notify user that umount failed

- give all attaches a name.
- clean up llmountcleanup.sh with name2dev
- remove debugging printouts from obdctl

- ENTRY/EXIT/RETURN cleanups
- extraneous header file removal
- made forgetting the 'setup' arguments in MDS/OST less fatal
- added the connection and token to the CONNECT RPC, so that we can do the
  pointer/token connection trick now
- fixed the 60 byte/-60 byte "leak"
- added ptlrpc and ldlm setup to llmount.sh, llrmount.sh, -client, and -server

Update your user mode linux to the 2.4.18 um15 + extra patch
  take source from ftp.lustre.org:/pub/people/robert
[call_usermode_helper has a bug in older versions of UML]

- new infrastructure for MDS failover recovery
  - recovd replaces the ha daemon. Now located in RPC directory
  - this comes with a service daemon to answer connect calls
  - wrapped it in an obd, for management purposes, and to be able
    to run multiple daemons on one system eventually
- commit callback daemon added to the lustre_lite directory
- removed small amounts of dead code from the rpc directory
- this code has seen little testing
- added the configuration management section to the master document

a peer, so the peer is passed in and managed separately.

- All of the DLM calls are hooked up to the RPC system now
- Some unmaintained local processing code removed
- Added ldlm network-related bits to the test scripts

                "/mnt/obd" --> "/mnt/lustre"
- small changes to RPC code to handle request packet freeing later,
  separate connection initialization and connect.
- rename event handlers and queues, add separate request_out and
  reply_out queue

now use the functions in rpc/pack_generic.c, which are based around
struct lustre_msg.  That file is small and easily understood, see it for more
details.

Now that everyone uses the same functions, the code can probably be refactored
more sensibly.  This is left as an exercise for the reader.

Other, cleanups:
- we don't need to memset() after OBD_ALLOC
- the MDS and OST request and reply packets were so similar that they've been
merged, into struct mds_body and ost_body respectively.

- fix tiny bug in filter.c

- some of this is going to change dramatically as we feel we
  are handling too many resources now.