running_user_centos:user# the user with root permissions on the three centos LAP machines. Can be root
certificate_parent_dir:/etc/datalake/
certificate_path:/etc/datalake/dlcert.cer
private_key_parent_dir:/etc/datalake/
private_key_path:/etc/datalake/dlkey.pem
certificate_parent_dir:/etc/datalake/# location of the host certificate directory
certificate_path:/etc/datalake/dlcert.cer# location of the host certificate
private_key_parent_dir:/etc/datalake/# location of the host private key directory
private_key_path:/etc/datalake/dlkey.pem# location of the host private key
enable_firewalld:false# enable firewalld if you need to secure the services. Be aware that this may affect other services
enable_self_signed:false# only enable if you do not have a cert ready for this service. Self-signed certs are not widely accepted and pose a security risk, but are enough to test the services
An ansible setup, that sets up a Local Access Point (LAP) of the data lake infrastructure currently in development at GSI. A LAP consists of two dynafed instances (Tier-1 and Tier-2) a squid proxy and a storage endpoint (a server that is able to speak http(s) and is able to store and deliver files).
A request sent to the squid proxy will be handled by an ecap adapter: this adapter is responsible for first finding the requested file and transferring it to the client. The ecap adapter first requests the file from a configured peer, which is the Tier-1 dynafed instance. This instance redirects the request to a relevant Tier-2 dynafed instance that claims to know the file location. This instance further redirects to the storage endpoint holding the data, which is then requested and transferred to the client via the squid proxy.
The proxies use is to a) cache the data and b) hide the internal elements of the datalake from the client.
An ansible setup, that sets up a Local Access Point (LAP) of the data lake infrastructure currently in development at GSI. A LAP consists of two dynafed instances (Tier-1 and Tier-2) a nginx proxy and a storage endpoint (a server that is able to speak http(s) and is able to store and deliver files).
A request will be sent to the nginx proxy. This proxy is configured as a reverse proxy to handle redirects internally. It will first contact the Tier-1 dynafed instance of its own LAP. This instance redirects the request to a relevant Tier-2 dynafed instance that claims to know the file location. This instance further redirects to the storage endpoint holding the data, which is then requested and transferred to the client via the nginx reverse proxy.

...
...
@@ -29,17 +28,14 @@ The proxies use is to a) cache the data and b) hide the internal elements of the
* your_http_storage_port
## Credentials.yml
* Place "template_credentials" in configs folder, rename and modify
* with the running_user_centos you want to run as
* root or user with root permissions
* create user with root permissions
* on centos: adduser USER && passwd USER && usermod *aG wheel USER
* Place "template_credentials" in configs folder, rename and modify. You need:
* a root or user with root permissions
* if necessary: create user with root permissions
* put user public key in /home/USER/.ssh/.authorized_keys
* mkdir /home/USER/.ssh && vi /home/USER/.ssh/authorized_keys
* http_proxy (if needed), set to none if no proxy needed.
* dynafed_json: The location of your dynafedInstances.json
* Xrootd is only relevant if you want to provision an xrootd dataserver as a http storage
*squid proxy ports for http and https: the port squid should listen for http/https
*nginx proxy ports for http and https: the port nginx should listen for http/https