LU-16502 python: improve support for virtual environments

Python virutal environments make it easy to install the
correct Python packages isolated from the rest of the
system.

 https://docs.python.org/3/library/venv.html



.venv is added to .gitignore and a simple virtual environment
example has been added to the README.

This patch collects all of the requirements for various
scripts in the Lustre tree and consolidates them in a
top level requirements.txt. lu_object.py spacing was
fixed due to parsing errors.

Test-Parameters: trivial
Signed-off-by: Timothy Day <timday@amazon.com>
Change-Id: I69d074e9ba50022817bd243fb82d004366ab6adf
Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/53209


Tested-by: jenkins <devops@whamcloud.com>
Tested-by: Maloo <maloo@whamcloud.com>
Reviewed-by: Arshad Hussain <arshad.hussain@aeoncomputing.com>
Reviewed-by: Shaun Tancheff <shaun.tancheff@hpe.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>

.gitignore

@@ -98,6 +98,10 @@ TAGS
 /stamp-h1
 /LUSTRE-VERSION-FILE
 
+# common python venv names
+/.venv
+/venv
+
 #
 # Top level generic files
 #

README

@@ -46,9 +46,9 @@ To report bugs, please visit:
 The official repository is hosted at:
     https://git.whamcloud.com/
 
-+----------------------+
-| Building and Testing |
-+----------------------+
++----------+
+| Building |
++----------+
 
 Detailed instructions for building, configuring and running Lustre:
     http://wiki.lustre.org/Compiling_Lustre
@@ -66,14 +66,35 @@ To see all available make targets:
 
     make help
 
++---------+
+| Testing |
++---------+
+
+Explanation of Lustre test suites:
+    https://wiki.lustre.org/Test_Descriptions
+and Lustre test environment variables:
+    https://wiki.whamcloud.com/display/PUB/Lustre+Test+Tools+Environment+Variables
+
+
 To play with a test Lustre filesystem:
 
     ./lustre/tests/llmount.sh
 
+To run a test on the filesystem:
+
+    ./lustre/tests/auster -v sanity --only 1
+
 To clean up:
 
     ./lustre/tests/llmountcleanup.sh
 
+To run LUTF (LNet Unit Test Framework) in a Python virtual environment:
+
+    python3 -m venv venv
+    source venv/bin/activate
+    pip3 install -r requirements.txt
+    ./lustre/tests/lutf.sh
+
 +--------------+
 | Contributing |
 +--------------+

contrib/debug_tools/epython_scripts/lu_object.py

@@ -40,12 +40,12 @@ def lov_print_raid0(obj, depth=0, ruler=RULER):
         magic = lsm.lsm_magic
         stripes = lsm.lsm_stripe_count
         layout_gen = lsm.lsm_layout_gen
-	pattern = lsm.lsm_pattern
+        pattern = lsm.lsm_pattern
     except Exception as e:
         magic = lsm.lsm_wire.lw_magic
         stripes = lsm.lsm_wire.lw_stripe_count
         layout_gen = lsm.lsm_wire.lw_layout_gen
-	pattern = lsm.lsm_wire.lw_pattern
+        pattern = lsm.lsm_wire.lw_pattern
     if magic==LOV_MAGIC or magic==LOV_MAGIC_V3:
         r0 = obj.u.raid0
     lli = readU32(Addr(obj) + member_offset('struct lov_object', 'lo_layout_invalid'))

requirements.txt

+# Packages required to use various Lustre and LNet scripts
+bcrypt
+cffi
+cryptography
+netifaces
+paramiko
+psutil
+pycparser
+PyNaCl
+PyYAML
+Requests
+six
+crash