Netmap Project Installation

Netmap Project Installation

The NetMap project aims to measure bandwidth and map network connectivity between the nodes of H3ABioNet, as well as to other locations that might be relevant when considering transfers of genomic data across the Internet. To paint a clearer picture of the current bandwidth situation, and measure how this changes over time, a set of Python scripts have been developed that run these tests in an automated fashion, submitting the results to a central location for later analysis using the Amazon SQS message queuing system. Currently the tests performed are the two common network tools iperf and traceroute, which provide details on latency, path and actual bandwidth between locations. A trial was performed last year which prompted the design of a client which can be run unsupervised to gain data more regularly. This new client is being installed and tested for any potential bugs between Central, Wits, SANBI, IPT and UVRI before being rolled out to all nodes across the network.

  • Netmap Client: will be installed on all the nodes to gatehr date from all the nodes to the servers.
  • Netmap Server: will installed in soem nodes to administrate the transfer.

This tutorial will help teach how to install Natmap on user Linux server ans start it to act as a client by sending information to the server every 15 min.

Installation Steps

First we install iperf, traceroute, python, python-pip:

$ sudo yum install iperf traceroute python python-pip

Then we install boto:

$ sudo -E pip install boto

We create a local folder for the netmap client:

$ sudo mkdir -p /usr/local/share/netmap_client

We fetch the latest version on the netmap client. In our case we need to use this command to get it from train0:

$ sudo scp setup@train0.bi.up.ac.za:/home/setup/client.py /usr/local/share/netmap_client/ 

Use “setup2015” as a password

We copy the appropriate credentials.csv file (check which ID to use on the spreadsheet) and place it in /usr/local/share/netmap_client

$ sudo scp setup@train0.bi.up.ac.za:/home/setup/credentials-TECHXX.csv /usr/local/share/netmap_client/ 

If it doesn’t run you might need to change the permissions:

$ sudo chmod u+x /usr/local/share/netmap_client/client.py

We test our client by running as fallow

$ sudo cd /usr/local/share/netmap_client/
$ sudo mv credentials-TECHXX.csv credentials.csv 
$ sudo ./client.py 

Now, we install a cronjob by typing:

$ sudo crontab -e

Then we add the following line to the cronfile that gets displayed, to tell the cronjob to run every 15 minutes:

*/15 * * * * root /usr/local/share/netmap_client/client.py -c /usr/local/share/netmap_client/credentials.csv 

Then save and exit. Cron will take care of the job scheduling.

Leave a Reply

Your email address will not be published. Required fields are marked *