How do I transfer data from the Internet to my storage once I am logged in to the DTN Node?

 

Data can be pulled to the user directories and pushed to the desired destination in DTN storage using two different ways

---

Method 1: Using Linux commands

 

CURL

• To download and save a file with file name as original, execute the first command as mentioned below. The second command is used to verify if the file is present in the location.

• To download and save the file with different name, execute the command as shown below: 

• To download multiple files

wget

• Execute the wget command as shown below. Then check the path if you received the file/dataset. 

rsync

• Transferring files within the same node from one directory to another.

• Use below command to transfer files from source directory to target directory within the same node (dtn.uncc.edu)

Source directory: catalogs   

Target directory: /tmp/test_directory/

• Note: If you wish to transfer data between two different nodes, then please make sure the required port access is available and SSH connectivity is set up between the nodes before using rsync

SFTP/FTP 

• If you have an external sftp account from which you have to pull or upload data, use the following command inside the DTN node in the required directory. 

Login to SFTP: sftp

Enter the password

To upload the files from DTN to SFTP:    put

To download files from DTN to SFTP:     get

---

Method2: Using UNC Charlotte's DTN End point

 

 

To start using Globus connect, please make sure to get the required access to our DTN endpoint “Charlotte-URC-DTN” by following the below steps. 

• Go to https://cilogon.org/ 
• Choose University of North Carolina at Charlotte as identity provider
• Login using your UNC Charlotte's account credentials
• Click on Certificate information
• Copy the details of certificate subject and create a ticket for access request with this information
• Log out

---

Creating a ticket for access : 

• Create a ticket with this detail, Visit: https://servicecatalog.uncc.edu/service/research-computing/high-performance-computing-research

 

---

Transferring files :

• When the access to DTN end point is available, use DTN endpoint in collections files inside the Globus connect to upload and download data.

Login to Globus connect using your UNC Charlotte account's credentials. You will see the below screen after login

 

 

• Click on the download Globus connect personal for windows button. For other operating systems please select the relevant options. An .exe file gets downloaded. Please execute and set up the Globus connect

 

 

 

• After the endpoint creation, check if the endpoint is displayed in the Globus connect by clicking on the endpoint-> administered by you

 

 

• When the endpoint is displayed, start to transfer the file between your laptop and the endpoint which is required. Here , UNC Charlotte account's endpoint is used as the destination. Click start on the source

 

 

• A successful file transfer notification will be displayed in the right corner. An email notification is also sent to your email account

 

 

Email notification

• Similarly, from any authorized source such as an endpoint from an external cloud hosting, different university’s end point, an external endpoint, etc.,  can be used to transfer files to UNC Charlotte endpoint and vice versa

 

---

To Delete a file 

• Select the file from the end point and click on “Delete Selected”

 

 

• A delete confirmation appears on the screen. Please confirm your decision to delete

 

 

• Delete notification will pop on the right corner of the screen and email notification will also be sent. The file which was deleted will be ruled out in the UI as shown below

 

 

Delete email notification

 

Different options such as share documents, get a link to the document and share them, etc., provided as part of Globus can also be used as per need.

More info: https://docs.globus.org/how-to/ 

---