R tips and tricks

From SESLINK Wiki
(Redirected from R)
Jump to navigation Jump to search

R in the clouds

Run a local script from your computer on a remote (fast) computer (for example Gunvor) directly over command line without 'RStudio/RServer' to be installed on the remote computer

Prerequsites:

  • you can access the remote computer with 'ssh' (secure shell) from a terminal. In Mac OSX/Linux 'ssh' is part of the OS. Windows users need to install putty or MobaXterm to get 'ssh'
  • R (and necessary packages) is installed on the remote computer
  • R has access to all the data that are analyzed (upload i.e. with 'rsync', 'sftp' or filezilla or sync R with you Dropbbox -> see below)
  • to see plots directly you a local X-server installed:
    • MacOS: XQuarz,
    • Windows:
      MobaXterm - a powerful X-Server with many remote access features or
      Xming, a minimalistic X server for Windows
    • Linux: ... needs nothing ;-)

Procedure:

  1. Write your R script in a(ny) text editor (perferably one with R syntax highlight mode and bracket auto-close: e.g. Geany, Gedit, TextWrangler, Atom, etc.)
  2. Save it (Ctrl-s or Cmd-s). Do not close the editor!
  3. Open a terminal
  4. use commmand:
    5. ssh -X user@ssh-server 'R --slave' < ~/path-to/MyRscript.R -> exchange 'user', 'ssh-server' and 'MyScript.R' accordingly; mind the ' ' in the command You can download a little try out script from MyRscript
  5. Look at your results
  6. modify your script, save again ('Ctrl-s or Cmd-s')
  7. re-run terminal command (use the 'up arrow' on your keyboard to get it back)

Repeat steps 6. and 7. as often as needed

Comments:

  • The '--slave' flag tells R not to be interactive and show no terminal output. If you want output you need to store it locally with 'sink()' in your script
  • The '-X' flag in ssh allows you to have a result plot shown directly as a pop-up window.
  • The '~' is the sign for 'my Home directory'
  • In Windows exchange path separators '/' for '\'
  • Important:
    • In Windows the X-Server ('Xmin' or 'MobiXTerm') has to be running before conducting the R-script!
    • In MacOS 'XQuarz' sometimes does not start automatically or it takes some time; start it manually if nothing shows up

Tip:

In order to avoid logging in with password into the remote machine every time you can create a key pair from the command line with the commands: 

 ssh-keygen                    #creates a private and a public key pair
 ssh-copy-id user@ssh-server   #exports the public key to the remote machine

'ssh' will now do a automatic login using your key pair.

Syncing R to your Dropbox

The orignal manual text and command description is provided from here

Prerequisites

  1. a Dropbox account
  2. installation of R package 'rdrop2'

This package provides programmatic access to Dropbox from R. The functions in this package provide access to a full suite of file operations, including dir/copy/move/delete operations, account information (including quotas) and the ability to upload and download files from any Dropbox account. This package replaces the old rDrop.

Installation

The latest stable version of 'rdrop2' can be installed from CRAN in R using the command: 

     install.packages('rdrop2')

Starting and Authentication

    library(rdrop2)
    drop_auth()

This will launch your browser and request access to your Dropbox account. If not copy link and paste it into your browser. You will be prompted to log in if you aren't already logged in. Once completed, close your browser window and return to R to complete authentication. The credentials are automatically cached (you can prevent this) for future use.

Retrieve Dropbox account information

   library(dplyr)
      drop_acc() %>% 
      select(uid, display_name, email_verified, quota_info.quota)

Dropbox directory listing

   drop_dir()

or specify a path 

   drop_dir('public/gifs')


Filter directory listing by filetype

(e.g. png files) 

   drop_dir() %>% 
   filter(mime_type == "image/png")

Create folders on Dropbox

   drop_create('drop_test')

or provide the full path where it needs to be created 

   drop_create('public/drop_test')

Upload a file into Dropbox

e.g. csv data files: 

   write.csv(mtcars, 'mtcars.csv')
   drop_upload('mtcars.csv')

or upload to a specific folder 

   drop_upload('mtcars.csv', dest = "drop_test")

You can also do this for any other file type and large files are supported regardless of your memory.

Download a file

   drop_get('mtcars.csv')

or add path if file is not in root 

   drop_get("test_folder/mtcars.csv")

Delete a file

   drop_delete('mtcars.csv')

Move files

   drop_create("new_folder")
   drop_move("mtcars.csv", "new_folder/mtcars.csv")

Copy files

  drop_create("new_folder2")
  drop_copy("new_folder/mtcars.csv", "new_folder2/mtcars.csv")

Search your Dropbox

  foo <- drop_search('gif')
  
  dim(foo)
  [1] 751  14
  
  tail(foo)
  Source: local data frame [6 x 14]
  rev thumb_exists path is_dir
  1  1b206e7f519 TRUE # /obscure_path/bgnoise.gif  FALSE
  2  1d906e7f519 TRUE # /obscure_path/images/ploslogo.gif  FALSE
  3  1da06e7f519 TRUE # /obscure_path/images/treebase_logo.gif  FALSE
  4  1db06e7f519 TRUE # /obscure_path/images/fishbaselogo.gif  FALSE
  Variables not shown: client_mtime (chr), icon (chr), read_only (lgl), bytes (# int), modified (chr), size (chr), root (chr), mime_type (chr), revision (int), parent_shared_folder_id (chr)

Search and download files

  x <- drop_search("rabbit")

  drop_get(x$path, local_file = '~/Desktop/bugs.gif')
  Response https://api-content.dropbox.com/1/files/auto//Public/gifs/duck_rabbit.gif
  Date: 2015-04-04 15:34
  Status: 200
  Content-Type: image/gif
  Size: 337 kB
  <ON DISK>  ~/Desktop/bugs.gif

  gifs <- drop_search("rabbit")
  drop_share(gifs$path)

  url = https://db.tt/PnNKg99G 
  expires = Tue, 01 Jan 2030 00:00:00 +0000 
  visibility = PUBLIC

The shared URL resolves here https://www.dropbox.com/s/aikiaug0x2013dp/duck_rabbit.gif?dl=0

Read csv files directly from Dropbox

   write.csv(iris, file = "iris.csv")
   drop_upload("iris.csv")

Now let's read this back into an R session

Note that there is a quiet download happening to your temp dir 

   new_iris <- drop_read_csv("iris.csv")

Accessing Dropbox on Shiny and remote servers

If you expect to access a Dropbox account via Shiny or on a remote cluster, AWS EC2, Digital Ocean etc, you can leave the cached oauth file in the same directory, or pass the token explicitly to drop_auth. You can also save the output of drop_auth into an R object, sink that to disk, and pass that as a token. If using on Travis or similar, you should consider encrypting the oauth cache file to prevent unauthorized access to your Dropbox account. If you have multiple tokens and accounts, it is also possible to override the environment token and explicitly pass a specific token for each drop_ function. 

   token <- drop_auth()
   saveRDS(token, "droptoken.rds")

Upload droptoken to your server

******** WARNING ********

Losing this file will give anyone complete control of your Dropbox account

You can then revoke the rdrop2 app from your dropbox account and start over.

Read it back with readRDS 

   token <- readRDS("droptoken.rds")

Then pass the token to each drop_ function 

    drop_acc(dtoken = token)

Bugs and known issues

Given that this package hasn't been around for very long there are likely some undiscovered issues. So please file any issues or problems as they arise.

Retrieved from "https://seslink.org/wiki/index.php?title=R_tips_and_tricks&oldid=483"