Sunday, March 13, 2016

Using AWS for Research


What is Amazon Web Services (AWS)? 


AWS is a cloud computing service provided by Amazon. Amazon EC2 is the service we'll go over in this post. It allows users to launch their own virtual instances from a variety of operating systems. Amazon provides these computational resources to the public and private sector at an extremely low price. I have listed below both the advantages and disadvantages of using cloud computing resources. For me the advantages definitely outweigh the disadvantages :) AWS is generally much more flexible than your universities local cluster in that you can choose the size, type and number of machines and have the rights to download and run whenever needed.

 Advantages 

  1. None of the maintenance of hosting your own server 
  2. Access to high computing machines 
  3. Access to high parallel computing machines (like those running hadoop) 
  4. The cost is extremely cheap 
  5. Sudo user rights (can install whatever you want without asking your sysadmin) 
  6. The computers available to you come in a variety shapes and sizes

 Disadvantages 

  1. There might not be enough machines for everyone and their maybe a wait time (This has yet happen) 
  2. It cost money (but amazon provides educational grants) 
  3. AWS is a blank slate therefore you need to install everything and copy files over (They make this easy with thing like s3 for storing files and allow you to create images of an instance so you can relaunch one with software pre-installed) 

 AWS Getting Started 

Using EC2 is easy once you've launched your instance you can ssh using your ssh keys:
 If your machine is ubuntu then  ssh -i location_to_pem_file.pem ubuntu@ec2-54-153-7-122.us-west-1.compute.amazonaws.com All inputs and outputs should be saved in your /mnt/ directory (this is where all of the system storage is).

By default you are not the owner of this directory so you need to change the permissions.

cd /mnt/
sudo chown ubuntu:ubuntu .
mkdir data

 If you need to mount more storage on your EC2 machine: 

  1.  Go to AWS Console  
  2. Create an EBS volume of the size needed 
  3. Attach this volume to your running EC2 machine 
  4. Then mount the volume to your EC2 machine using the following commands 

sudo mkfs -t ext4 /dev/xvdf 
sudo mount /dev/xvdf /mnt/data 
# /dev/xvdf might be diff depending on where volume was added 
cd /mnt/data/ 
mount 
# mount while in data dir 
sudo chown ubuntu:ubuntu . 
df 
# df to confirm that your volume was successfully added


 All data can be easily on saved on S3 and instances or machines with preinstalled packages can be saved and reused as images

15 comments:

  1. Very nice blog. The blog was quite informative & educational. For more information on office suite you can check buy microsoft office for mac UK.

    ReplyDelete
  2. Unavoidably, a few voices disagree with such hopeful conjectures. While it presently appears that AI has at long last left the shadows, not similar remains constant about responses to innovation versus human work discusses. machine learning course in pune

    ReplyDelete
  3. Thanks for sharing such an informative blog that explains the importance of why we need to invest in AWS is a cloud computing service. You can opt for Mobile Web Development Services for more information.

    ReplyDelete
  4. The development of artificial intelligence (AI) has propelled more programming architects, information scientists, and different experts to investigate the plausibility of a vocation in machine learning. Notwithstanding, a few newcomers will in general spotlight a lot on hypothesis and insufficient on commonsense application. IEEE final year projects on machine learning In case you will succeed, you have to begin building machine learning projects in the near future.

    Projects assist you with improving your applied ML skills rapidly while allowing you to investigate an intriguing point. Furthermore, you can include projects into your portfolio, making it simpler to get a vocation, discover cool profession openings, and Final Year Project Centers in Chennai even arrange a more significant compensation.


    Data analytics is the study of dissecting crude data so as to make decisions about that data. Data analytics advances and procedures are generally utilized in business ventures to empower associations to settle on progressively Python Training in Chennai educated business choices. In the present worldwide commercial center, it isn't sufficient to assemble data and do the math; you should realize how to apply that data to genuine situations such that will affect conduct. In the program you will initially gain proficiency with the specialized skills, including R and Python dialects most usually utilized in data analytics programming and usage; Python Training in Chennai at that point center around the commonsense application, in view of genuine business issues in a scope of industry segments, for example, wellbeing, promoting and account.

    ReplyDelete
  5. The information which you have provided in this blog is really useful to everyone. Thanks for sharing.
    AWS Training in Hyderabad

    ReplyDelete
  6. Greetings! Very useful recommendation within this article! It is the little modifications that produce the largest modifications. site Many thanks for sharing!

    ReplyDelete
  7. Creativity with artistic visualization can be an added advantage. Growing awareness of health, diet, and lifestyle has augmented preference towards this field. machine learning and artificial intelligence courses in hyderabad

    ReplyDelete
  8. How do I make money from playing games and earning
    These ventureberg.com/ are the three most popular forms of gambling, and communitykhabar are explained in worrione a very concise and 바카라 사이트 concise manner. The most common forms of gambling are: งานออนไลน์

    ReplyDelete
  9. V-ray for Sketchup 2022 License Key also offers features for visualization and designing of computer graphics for a variety of industries. VRay 6.00.05 Crack

    ReplyDelete
  10. Sylenth1 3.064 Crack Full Version is the most popular software with a variety of powerful tools. Furthermore, it enables sound recordists ...Sylenth1 License Code

    ReplyDelete