-
Notifications
You must be signed in to change notification settings - Fork 1
Edit Storage System Definition
#####Previous Step - Generate Agave API Authorization Token
Agave supports about a dozen authentication protocols for accessing systems, including iRODS and Amazon S3. I will provide a couple simple examples that will allow you to register a system using either username/password or SSH keys; refer to this documentation for a more comprehensive set of examples.
Option 1 - Authenticate with Username and Password:
Create a text file on your system called my-system.json (or another name that you prefer) in an easily accessible directory.
my-system.json:
{
"id": "foobar-corral",
"name": "Foobar's Corral Storage System",
"type": "STORAGE",
"description": "My storage system using SFTP to store data",
"storage": {
"host": "corral.tacc.utexas.edu",
"port": 22,
"protocol": "SFTP",
"rootDir": "/",
"homeDir": "/home/amagill",
"auth": {
"username": "foobar",
"password": "foobar's password",
"type": "PASSWORD"
}
}
}
- id - Any globally-unique (each system on CyVerse must be named distinctly) name that you choose to identify your system. Note that if another CyVerse user has already named a system 'Corral', or 'My Laptop', you will need to choose another name.
- name - Arbitrary text, more of a short description than a name.
- type - 'Storage', indicates that we will not run jobs on this sytem.
- description - Arbitrary text.
- rootDir - This is the lowest level directory that you will be able to access when listing the directory contents on '/'.
-
homeDir - The path to use, relative to the rootDir, as you virtual home directory. You may want to SSH in to your storage system to find the full path of your home directory using the
pwd
command. On Corral, my home directory is/home/foobar
, however, on Wrangler, my home directory is/home/03738/foobar
. You can use your work directory, or you scratch directory, as your virtual home directory (e.g./home/foobar/scratch
). Any request that does not begin with/
will operate in your virtual home directory. - host - The hostname or IP address that you would use to SSH into the machine (e.g. corral.tacc.utexas.edu).
- port - The port through which Agave will communicate with your storage system, please leave as 22.
- protocol - The method through which Agave will communicate with your storage system, please leave as 'SFTP'.
- username - The username you use to log into this system.
- password - The password you use when logging into this system.
######Please see the Agave API Systems Tutorial for a full explanation of these fields.
Option 2 - Authenticate with SSH keys:
Similar to above, create a text file on your system called my-system.json (or another name that you prefer) in an easily accessible directory.
You will need a public/private key pair on your the system that you are registering with Agave.
Log in to remote system:
$ ssh foobar@corral.tacc.utexas.edu
Create a directory for your SSH keys, if one does not already exist:
$ mkdir ~/.ssh
Generate new key-pair (unless you already have one you would like to use). You will be prompted for a filename, then a passphrase; you may choose the default filename ('id_rsa'), leave the passphrase blank.
$ cd ~/.ssh
$ ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
Copy and paste your public and private keys into the system description below. You can print you keys to the screen using 'cat':
$ cat id_rsa
$ cat id_rsa.pub
When you paste into the JSON system description file, you may want to remove line feeds, in other words, turn this:
x9XxWDwpY1gJl7c8ppxraLXc7/i5NSsOFwWHi3WsGfah0fVN+/CixQKkgQDTISxP
4WJgoaJxfzm7rwqbD7CPrOuBQXN5OFikQX0y//TZ9df5OsX1xxM406Jhzaj8zwJV
into this:
x9XxWDwpY1gJl7c8ppxraLXc7/i5NSsOFwWHi3WsGfah0fVN+/CixQKkgQDTISxP4WJgoaJxfzm7rwqbD7CPrOuBQXN5OFikQX0y//TZ9df5OsX1xxM406Jhzaj8zwJV
my-system.json:
{
"id": "foobar-corral",
"name": "Foobar's Corral Storage System",
"type": "STORAGE",
"description": "My storage system using SFTP to store data",
"storage": {
"host": "corral.tacc.utexas.edu",
"port": 22,
"protocol": "SFTP",
"rootDir": "/",
"homeDir": "/home/foobar",
"auth": {
"username": "foobar",
"publicKey": "ssh-rsa AAAAB3NzaC ... RtUC7CY4SBw== foobar@tacc.utexas.edu",
"privateKey": "-----BEGIN RSA PRIVATE KEY-----\nM.oI+n8Y ... lqU0g==\n-----END RSA PRIVATE KEY-----",
"type": "SSHKEYS"
}
}
}
Save your system definition file, and continue to the next step.
#####Next Step - Upload System Definition File