User Tools

Site Tools


wiki:2016networking

CONNECTING TO REMOTE SERVERS


Before we can start to manipulate genomics datasets, we need to know how to obtain the data. Even if you are generating your own data, you will almost certainly need reference genome sequences and genome annotation information to guide the analyses of your data. A common challenge for biologists is obtaining reference data and manipulating it so that it can be used in their analyses. Linux tools are incredibly useful in this regard.

OUTLINE

  1. What is a server?
  2. How are servers accessed?

1. What is a server?

  • Desktop/laptop computers have limited ability to store and process data.
  • Computer networks involving servers can have tremendous capacity for storing and processing data.
  • Large datasets are often stored, processed, analyzed on remote servers.
  • Desktop and laptop computers are getting faster processors, more cores, more RAM, and more storage but datasets are getting bigger and demand more and more computing power to process and more space for storage.

Some Common Features of a Computer/Server

  • Hard drive/solid-state drive: non-volatile data storage and retrieval (space measured in GB or TB).
  • Processor (CPU): perform computational tasks (speed measured in GHz).
  • Core: an independent processor.
  • Memory/RAM: temporary volatile storage of data (space measured in GB).
  • CPU cache: faster memory for temporary storage of copies of frequently used data (usually measured in MB).

2. How are servers accessed?

  • Shell or terminal
  • GUI (Cyberduck or Filezilla; or web browser)


Accessing servers via the command line (ssh and sftp)

ssh….secure shell.

$ ssh username@server_address


A common method for downloading data from a public server (ftp)

ftp….file transfer protocol.

$ ftp ftp.someaddress.org	


Transferring data between computers/servers (scp)

scp….secure copy protocol.

$ scp /path/to/local/file username@hostname:/path/to/remote/file


Run multiple sessions in one window or keep processes running on a remote server without a job scheduler (GNU screen)

$ screen

*See UNIX cheat sheet and exercises for additional information.

wiki/2016networking.txt · Last modified: 2017/09/01 10:36 by tai