• Lai Siyao's avatar
    LU-11025 dne: directory restripe and auto split · a336d7c7
    Lai Siyao authored
    A specific restriper thread is created for each MDT, it does three
    tasks in a loop:
    1. If there is directory whose total sub-files exceeds threshold
       (50000 by default, can be changed "lctl set_param
       mdt.*.dir_split_count=N"), split this directory by adding new
       stripes (4 stripes by default, which can be adjusted by
       "lctl set_param mdt.*.dir_split_delta=N").
    2. If a directory stripe LMV is marked 'MIGRATION', migrate sub file
       from current offset, and update offset to next file.
    3. If a directory master LMV is marked 'RESTRIPING', check whether
       all stripe LMV 'MIGRATION' flag is cleared, if so, clear
       'RESTRIPING' flag and update directory LMV.
    In last patch, the first part of manual directory stripe is
    implemented, and in this patch, sub file migrations and dir layout
    update is done. Directory auto-split is done in similar way, except
    that the first step is done by this thread too.
    Directory auto-split can be enabled/disabled by "lctl set_param
    mdt.*.enable_dir_auto_split=[0|1]", it's turned on by default.
    Auto split is triggered at the end of getattr(): since now the attr
    contains dirent count, check whether it exceeds threshold, if so,
    add this directory into mdr_auto_split list and wake up the dir
    restriper thread.
    Restripe migration is also triggered in getattr(): if the object is
    directory stripe, and LMV 'MIGRATION' flag set, add this object into
    mdr_restripe_migrate list and wake up the dir restriper thread.
    Directory layout update is similar: if current directory is striped,
    and LNV 'RESTRIPING' flag is set, add this directory into
    mdr_restripe_update list and wake up restriper thread.
    By default restripe migrate dirent only, and leave inode unchanged, it
    can be adjusted by "lctl set_param mdt.*.dir_restripe_nsonly=[0|1]".
    Currently DoM file inode migration is not supported, migrate dirent
    only for such files to avoid leaving dir migration/restripe
    Add sanity.sh 230o, 230p and 230q, adjust 230j since DoM files migrate
    Signed-off-by: default avatarLai Siyao <lai.siyao@whamcloud.com>
    Change-Id: I8c83b42e4acbaab067d0092d0b232de37f956588
    Reviewed-on: https://review.whamcloud.com/37284
    Tested-by: default avatarjenkins <devops@whamcloud.com>
    Reviewed-by: default avatarAndreas Dilger <adilger@whamcloud.com>
    Tested-by: default avatarMaloo <maloo@whamcloud.com>
    Reviewed-by: default avatarHongchao Zhang <hongchao@whamcloud.com>
    Reviewed-by: default avatarOleg Drokin <green@whamcloud.com>