Skip to content
Snippets Groups Projects
Commit 3341f32a authored by p.n.kramp's avatar p.n.kramp
Browse files

Updated readme and added info to credentials options

parent 2c913663
No related branches found
No related tags found
No related merge requests found
......@@ -5,10 +5,10 @@ http_port: 8080
#centos settings
running_user_centos: user # the user with root permissions on the three centos LAP machines. Can be root
certificate_parent_dir: /etc/datalake/
certificate_path: /etc/datalake/dlcert.cer
private_key_parent_dir: /etc/datalake/
private_key_path: /etc/datalake/dlkey.pem
certificate_parent_dir: /etc/datalake/ # location of the host certificate directory
certificate_path: /etc/datalake/dlcert.cer # location of the host certificate
private_key_parent_dir: /etc/datalake/ # location of the host private key directory
private_key_path: /etc/datalake/dlkey.pem # location of the host private key
enable_firewalld: false # enable firewalld if you need to secure the services. Be aware that this may affect other services
enable_self_signed: false # only enable if you do not have a cert ready for this service. Self-signed certs are not widely accepted and pose a security risk, but are enough to test the services
......@@ -19,6 +19,8 @@ dynafed_json: configs/GSI_dynafedInstances.json #the file that specifies details
#ONLY necessary if you also want to setup an xrootd dataserver
xrootd_interface: http
xrootd_port: 80
#debian settings only necessary if debian xrootd is used
running_user_debian: root
apt_http_proxy:
apt_http_port:
......
# Summary
An ansible setup, that sets up a Local Access Point (LAP) of the data lake infrastructure currently in development at GSI. A LAP consists of two dynafed instances (Tier-1 and Tier-2) a squid proxy and a storage endpoint (a server that is able to speak http(s) and is able to store and deliver files).
A request sent to the squid proxy will be handled by an ecap adapter: this adapter is responsible for first finding the requested file and transferring it to the client. The ecap adapter first requests the file from a configured peer, which is the Tier-1 dynafed instance. This instance redirects the request to a relevant Tier-2 dynafed instance that claims to know the file location. This instance further redirects to the storage endpoint holding the data, which is then requested and transferred to the client via the squid proxy.
The proxies use is to a) cache the data and b) hide the internal elements of the datalake from the client.
An ansible setup, that sets up a Local Access Point (LAP) of the data lake infrastructure currently in development at GSI. A LAP consists of two dynafed instances (Tier-1 and Tier-2) a nginx proxy and a storage endpoint (a server that is able to speak http(s) and is able to store and deliver files).
A request will be sent to the nginx proxy. This proxy is configured as a reverse proxy to handle redirects internally. It will first contact the Tier-1 dynafed instance of its own LAP. This instance redirects the request to a relevant Tier-2 dynafed instance that claims to know the file location. This instance further redirects to the storage endpoint holding the data, which is then requested and transferred to the client via the nginx reverse proxy.
![](docs/LAP_flow.png)
......@@ -29,17 +28,14 @@ The proxies use is to a) cache the data and b) hide the internal elements of the
* your_http_storage_port
## Credentials.yml
* Place "template_credentials" in configs folder, rename and modify
* with the running_user_centos you want to run as
* root or user with root permissions
* create user with root permissions
* on centos: adduser USER && passwd USER && usermod *aG wheel USER
* Place "template_credentials" in configs folder, rename and modify. You need:
* a root or user with root permissions
* if necessary: create user with root permissions
* put user public key in /home/USER/.ssh/.authorized_keys
* mkdir /home/USER/.ssh && vi /home/USER/.ssh/authorized_keys
* http_proxy (if needed), set to none if no proxy needed.
* dynafed_json: The location of your dynafedInstances.json
* Xrootd is only relevant if you want to provision an xrootd dataserver as a http storage
* squid proxy ports for http and https: the port squid should listen for http/https
* nginx proxy ports for http and https: the port nginx should listen for http/https
## Running the playbook
* ansible-playbook playbook.yml -e "credentials_file=credentials_YOUR" -i YOUR_HOSTS_FILE --private-key ~/.ssh/USER_PRIVATE_KEY
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment