1. 07 Aug, 2020 1 commit
  2. 23 Jun, 2020 1 commit
  3. 31 Mar, 2020 1 commit
  4. 28 Oct, 2019 1 commit
    • Lai Siyao's avatar
      LU-12624 lod: alloc dir stripes by QoS · c1d0a355
      Lai Siyao authored
      
      
      Similar to file OST object allocation, introduce directory stripe
      allocation by space usage, but they don't share the same code because
      of the many differences between them: file has mirrors, PFL, object
      precreation; while for directory, the first stripe is always on the
      same MDT where its master object is on. The changes include:
      * add lod_mdt_alloc_qos() to allocate stripes by space/inode usage.
      * add lod_mdt_alloc_rr() to allocate stripes round-robin.
      * add lod_mdt_alloc_specific() to allocate stripes in the old way.
      * add sysfs support for lmv_desc field in LOD structure, and move
        those remain in procfs to sysfs.
      
      This patch also changes LMV QoS code:
      * mkdir by QoS if user mkdir by command 'lfs mkdir -i -1 ...', or the
        parent directory default LMV starting MDT index is -1.
      * with the above change, 'space' hash flag is useless, remove all
        related code.
      * previously 'lfs mkdir -i -1' QoS code is in lfs_setdirstripe(),
        but now it's done in LMV, remove the old code.
      
      Update sanity 413a 413b to support QoS mkdir of both plain and
      striped directories.
      
      Update lfs-setdirstripe man to reflect the changes.
      
      Signed-off-by: default avatarLai Siyao <lai.siyao@whamcloud.com>
      Change-Id: I8f5f8e46faae68ffd9a49a4ac1d450e951e979c5
      Reviewed-on: https://review.whamcloud.com/35825
      
      
      Tested-by: default avatarjenkins <devops@whamcloud.com>
      Tested-by: default avatarMaloo <maloo@whamcloud.com>
      Reviewed-by: default avatarAndreas Dilger <adilger@whamcloud.com>
      Reviewed-by: default avatarHongchao Zhang <hongchao@whamcloud.com>
      Reviewed-by: default avatarOleg Drokin <green@whamcloud.com>
      c1d0a355
  5. 20 Sep, 2019 1 commit
  6. 13 Jun, 2019 2 commits
    • Li Xi's avatar
      LU-10092 llite: Add persistent cache on client · f172b116
      Li Xi authored
      PCC is a new framework which provides a group of local cache
      on Lustre client side. No global namespace will be provided
      by PCC. Each client uses its own local storage as a cache for
      itself. Local file system is used to manage the data on local
      caches. Cached I/O is directed to local filesystem while
      normal I/O is directed to OSTs.
      
      PCC uses HSM for data synchronization. It uses HSM copytool
      to restore file from local caches to Lustre OSTs. Each PCC
      has a copytool instance running with unique archive number.
      Any remote access from another Lustre client would trigger
      the data synchronization. If a client with PCC goes offline,
      the cached data becomes inaccessible for other client
      temporarilly. And after the PCC client reboots and the copytool
      restarts, the data will be accessible again.
      
      ToDo:
      1) Make PCC exclusive with HSM.
      2) Strong size consistence for PCC cached file among clients.
      3) Support to cache partial content of a file.
      
      Change-Id: I188ed36c48aae22...
      f172b116
    • Lai Siyao's avatar
      LU-11213 lmv: reuse object alloc QoS code from LOD · b601eb35
      Lai Siyao authored
      
      
      Reuse the same object alloc QoS code as LOD, but the QoS code is
      not moved to lower layer module, instead it's copied to LMV, because
      it involves almost all LMV code, which is too big a change and should
      be done separately in the future.
      
      And for LMV round-robin object allocation, because we only need to
      allocate one object, use the MDT index saved and update it to next
      MDT.
      
      Add sanity 413b.
      
      Signed-off-by: default avatarLai Siyao <lai.siyao@whamcloud.com>
      Change-Id: I53c3d863dafda534eebb6b95da205b395071cd25
      Reviewed-on: https://review.whamcloud.com/34657
      
      
      Tested-by: Jenkins
      Tested-by: default avatarMaloo <maloo@whamcloud.com>
      Reviewed-by: default avatarHongchao Zhang <hongchao@whamcloud.com>
      Reviewed-by: default avatarAndreas Dilger <adilger@whamcloud.com>
      b601eb35
  7. 07 Jun, 2019 1 commit
  8. 29 May, 2019 1 commit
  9. 08 May, 2019 1 commit
    • Bruno Faccini's avatar
      LU-11376 lmv: new foreign LMV format · fdad3878
      Bruno Faccini authored
      This patch introduces a new striping/LMV format in order to
      allow to specify an arbitrary external reference for a dir
      in Lustre namespace.
      The new LMV format is made of {newmagic, length, type, flags,
      string[length]} to be as flexible as possible.
      Foreign dir can be created by using the ioctl(LL_IOC_LMV_SETDIRSTRIPE)
      operation and it can only be and remain an empty dir until removed.
      A new API method llapi_dir_create_foreign() has been introduced
      and "lfs {get,set}dirstripe" and "lfs find" modified to understand
      new format.
      The idea behind this is to provide Lustre namespace support and
      striping prefetch/caching under lock protection, for user/external
      usage.
      
      This patch is the LMV/dirs complement of LOV/files previous change
      (Change-Id: I5d9c0642fe8e7009c30918bfa946cac7c00c9af8) and has
      been rebased on top of the latter along with some with obvious
      mutualizations and simplifications.
      
      Code has been added for lfsck to handle foreign dirs, and
      a new sub-test has been a...
      fdad3878
  10. 01 Apr, 2019 1 commit
    • Bruno Faccini's avatar
      LU-9193 security: return security context for metadata ops · fca35f74
      Bruno Faccini authored
      Security layer needs to fetch security context of files/dirs
      upon metadata ops like lookup, getattr, open, truncate, and
      layout, for its own purpose and control checks.
      Retrieving the security context consists in a getxattr operation
      at the file system level. The fact that the requested metadata
      operation and the getxattr are not atomic can create a window
      for a dead-lock situation where, based on some access patterns,
      all MDT service threads can become stuck waiting for lookup lock
      to be released and thus unable to serve getxattr for security context.
      Another problem is that sending an additional getxattr request for
      every metadata op hurts performance.
      
      This patch introduces a way to get atomicity by having
      the MDT return security context upon granted lock reply,
      sparing the client an additional getxattr request.
      
      Test-Parameters: serverbuildno=62488 serverjob=lustre-reviews testlist=sanity,sanity-selinux clientselinux
      Test-Parameters: clientbui...
      fca35f74
  11. 21 Mar, 2019 1 commit
    • Jinshan Xiong's avatar
      LU-11999 dne: performance improvement for file creation · bfbd062e
      Jinshan Xiong authored
      
      
      This is to remove an obsoleted code where it causes drastic
      performance degradation. This code is written before PERM lock
      is introduced, and it requests UPDATE lock at path walk for
      remote directory, which will be cancelled at later file creation.
      
      Tests result before and after this patch is applied:
      
      Test case:
      rm -rf /mnt/lustre_purple/testdir
      lfs mkdir -i 0 /mnt/lustre_purple/testdir
      lfs mkdir -i 2 /mnt/lustre_purple/testdir/dir2
      ./lustre-release/lustre/tests/createmany -o \
      	/mnt/lustre_purple/testdir/dir2/f 10000
      
      Before the patch is applied:
      total: 10000 open/close in 12.82 seconds: 780.22 ops/second
      
      After the patch is applied:
      total: 10000 open/close in 4.89 seconds: 2044.75 ops/second
      
      Signed-off-by: default avatarJinshan Xiong <jinshan.xiong@gmail.com>
      Change-Id: Ib474dc28d6edc7d15801b6821edc0e1d108bb4b6
      Reviewed-on: https://review.whamcloud.com/34291
      
      
      Reviewed-by: default avatarLai Siyao <lai.siyao@whamcloud.com>
      Reviewed-by: default avatarAndrew Perepechko <c17827@cray.com>
      Teste...
      bfbd062e
  12. 16 Jan, 2019 1 commit
  13. 01 Oct, 2018 1 commit
    • Lai Siyao's avatar
      LU-4684 lmv: support accessing migrating directory · 976b609a
      Lai Siyao authored
      
      
      Migrating directory contains stripes of both old and new layout, and
      its sub files may be located on either one. To avoid race between
      access and new creations, there are 4 rules to access migrating
      directory:
      1. always create new file under new layout.
      2. any operation that tries to create new file under old layout will
         be rejected, e.g., 'mv a <migrating_dir>/b', if b exists and is
         under old layout, this rename should fail with -EBUSY.
      3. operations that access file by name should try old layout first,
         if file doesn't exist, then it will retry new layout, such
         operations include: lookup, getattr_name, unlink, open-by-name,
         link, rename.
      4. according to rule 1, open(O_CREAT | O_EXCL) and create() will
         create new file under new layout, but they should check existing
         file in one transaction, however this can't be done for old
         layout, so check existing file under old layout on client side,
         then issue the open/create request to new layout.
      
      Disable sanity 230d for ZFS backend because it will trigger lots of
      sync, which may cause system hung.
      
      Signed-off-by: default avatarLai Siyao <lai.siyao@whamcloud.com>
      Change-Id: Icd587eb17f4c6dc2234dcc3c7ba40759bc3e8d31
      Reviewed-on: https://review.whamcloud.com/31504
      
      
      Tested-by: Jenkins
      Tested-by: default avatarMaloo <hpdd-maloo@intel.com>
      Reviewed-by: default avatarFan Yong <fan.yong@intel.com>
      Reviewed-by: default avatarAndreas Dilger <adilger@whamcloud.com>
      Reviewed-by: default avatarOleg Drokin <green@whamcloud.com>
      976b609a
  14. 13 Aug, 2017 1 commit
  15. 17 Dec, 2016 1 commit
  16. 08 Oct, 2016 1 commit
  17. 10 Sep, 2016 1 commit
  18. 11 Aug, 2016 1 commit
  19. 22 Jun, 2016 1 commit
  20. 21 May, 2016 1 commit
  21. 02 May, 2016 1 commit
  22. 03 Nov, 2015 1 commit
  23. 02 Oct, 2015 1 commit
  24. 07 Aug, 2015 1 commit
  25. 23 Jan, 2015 1 commit
  26. 28 Dec, 2014 1 commit
    • Andreas Dilger's avatar
      LU-6068 misc: update Intel copyright messages 2014 · 2b294992
      Andreas Dilger authored
      Update the copyright messages for changes made in 2014.
      
      In some cases, changes made in 2013 did not generate an updated
      copyright message, so these files are updated accordingly.  In some
      cases there were no licenses on newly-added files, so add them (either
      with new copyrights, or including copyrights from the source files).
      
      Some types of changes that are not actually changing the code
      (e.g. whitespace, mass changes to add or remote prefixes, patches
      updating the copyright messages, patch reversions and the reverted
      patch, patches that only delete code, etc.) have been excluded.
      
      The updatecw.sh script was updated to fix a number of problems:
      - it now handles comments in nroff (man page) files properly
      - remove need to handle non-Intel copyrights specially
      - allow subdirectory to be specified on command-line
      - improve script debugging when running with "sh -vx"
      - print patches changing the most files, to catch mass replacements
      
      There are still a few c...
      2b294992
  27. 09 Dec, 2014 1 commit
  28. 04 Nov, 2014 1 commit
  29. 30 Oct, 2014 1 commit
  30. 25 Sep, 2014 1 commit
  31. 06 Sep, 2014 1 commit
  32. 27 Aug, 2014 1 commit
  33. 11 Aug, 2014 1 commit
  34. 16 Jul, 2014 1 commit
  35. 01 Jul, 2014 1 commit
  36. 20 Jun, 2014 1 commit
  37. 04 Jun, 2014 1 commit
  38. 14 May, 2014 1 commit
  39. 08 May, 2014 1 commit
    • wang di's avatar
      LU-4690 lod: separate master object with master stripe · 60e07b97
      wang di authored
      
      
      Separate master stripe with master object, so
      1. stripeEA only exists on master object.
      2. sub-stripe object will be inserted into master object
      as sub-directory, and it can get the master object by "..".
      
      By this, it will remove those specilities for stripe0 in
      LMV and LOD. And also simplify LFSCK, i.e. consistency check
      would be easier.
      
      And also after this separation, LOD will know whether iterating
      the whole stripe or single stripe eaisly, i.e. for master_object,
      it will iterate the whole stripes, for sub_stripe, it will only
      iterate the single stripe.
      
      This patch also fixes a few things in osp orphan iteration, to
      make it work with remote dir entry iteration.
      
      When then master object becomes an orphan, we should
      mark all of its sub-stripes as dead object as well,
      otherwise client might still be able to create files
      under these stripes.
      
      A few fixes for striped directory layout lock:
      
      1. stripe 0 should be locked as EX, same as other stripes.
      2. Acquire the layout for directory, when it is being unliked.
      
      Signed-off-by: default avatarwang di <di.wang@intel.com>
      Change-Id: I6212fb97a2360664b48e0a75424a89c857da2043
      Reviewed-on: http://review.whamcloud.com/9511
      
      
      Tested-by: Jenkins
      Tested-by: default avatarMaloo <hpdd-maloo@intel.com>
      Reviewed-by: default avatarAndreas Dilger <andreas.dilger@intel.com>
      Reviewed-by: default avatarJohn L. Hammond <john.hammond@intel.com>
      Reviewed-by: default avatarOleg Drokin <oleg.drokin@intel.com>
      60e07b97