Running simulations in the cloud: Difference between revisions

From SESLINK Wiki
Jump to navigation Jump to search
← Older editNewer edit →
VisualWikitext
Ingo (talk | contribs)
106 edits
Ingo (talk | contribs)
106 edits
No edit summary
Line 29: Line 29:


== R in the clouds ==
== R in the clouds ==
Run a local script on a remote (fast) computer directly without RServer
Run a local script on a remote (fast) computer directly without RServer. Please see the [[R tips and tricks]] page or instructions [http://www.seslink.org/wiki/R]
 
'''Prerequsites:'''
* you can access the remote computer with 'ssh' (secure shell) from a terminal
* R (and necessary packages) is installed on the remote computer
* R has access to all the data that are analyzed (upload i.e. with 'rsync', 'sftp' or [https://filezilla-project.org/ filezilla])
* to see plots directly you a local X-server installed:
** MacOS: [https://www.xquartz.org/ XQuarz],
** Windows:
**:[http://mobaxterm.mobatek.net/ MobaXterm] - a powerful X-Server with many remote access features or
**:[https://sourceforge.net/projects/xming/ Xming], a minimalistic X server for Windows
** Linux: ... needs nothing ;-)
 
'''Procedure:'''
# Write your R script in a(ny) text editor (perferably one with R syntax highlight mode and bracket auto-close: e.g. Geany, Gedit, TextWrangler, Atom, etc.)
# Save it (Ctrl-s/Cmd-s). Do not close the editor!
# Open a terminal
# use commmand:
#:<br />
#:  ssh -X user@ssh-server 'R --slave' < ~/path-to/MyRscript.R
#:<br />
#: (exchange 'user', 'ssh-server' and 'MyScript.R' accordingly; mind the ' ' in the command)
# Look at your results,
# modify your script, save ('Ctrl-s/Cmd-s')
# re-run terminal command (use the 'up arrow' on your keyboard to get it back)
 
'''Repeat steps 6 and 7 as often as needed'''
 
''Comments:''
*The '--slave' flag tells R not to be interactive and show no terminal output. If you want output you need to store it locally with 'sink()' in your script
*The '-X' flag in ssh allows you to have a result plot shown directly as a pop-up window.
*The '~' is the sign for 'my Home directory'
*In Windows exchange path separators '/' for '\'
*Important: In Windows the X-Servers have to be running before conducting the R-script!
*In MacOS XQuarz sometimes does not start automatically or it takes some time; start it manually if nothing shows up
*In order to avoiding loggin in with password into the remote machine every time create a key pair with the commands:
*:ssh-keygen (creates a private and a public key pair)
*:ssh-copy-id user@ssh-server (exports the public key to the remote machine)

Revision as of 15:37, 30 January 2017

Amazon Web Services (AWS)

Amazon Web Services, or AWS for short, is a cloud computing platform created by Amazon. We use it to rent computational capacity when we need to run models or other computations in the cloud. There are two main ways of running computations on AWS; using an existing machine or using a spot instance. Both ways are outlined below.

AWS offers a whole host of different services, most of them named something incomprehensible. This handy glossary will help you decipher AWS's odd naming scheme.

Accessing AWS machines

TODO: Describe SSH procedure

Existing machines

TODO: AWS HOWTO

Spot instances

TODO: Scripts

Paying for time

Amazon requires that you add a credit card to the account that owns the virtual servers. This will then be charged per month for usage over the last calendar month. Note that the person who owns the account will need to do the reimbursement.

Cost reimbursement should be charged to the SES-LINK project. Use the following cost carriers in Primula: 

 Project 1483701 EU SES Link  
 Department 481
 Project 1483701
 Unit 48110
 Cost carrier 481002

The expense should be recorded in Primula as an expense in foreign currency. Add it as "Laboratory supplies" and add a note saying that it's computation time. Attach the invoice from Amazon, and if desired a printout from your bank account so you get the correct exchange rate. Send these in as any other expense, and CC Maja on the e-mail so she can verify the expense to the project.

NetLogo in the cloud

Run headless on an AWS machine. Please see the NetLogo page for instructions on how to run NetLogo headless.

R in the clouds

Run a local script on a remote (fast) computer directly without RServer. Please see the R tips and tricks page or instructions [1]

Retrieved from "https://seslink.org/wiki/index.php?title=Running_simulations_in_the_cloud&oldid=341"