reference resources stay AWS Configure deep learning host on <https://zhuanlan.zhihu.com/p/25066187>

AWS file : <https://aws.amazon.com/cn/documentation/ec2/>


Because the local development environment is limited ( No high-end graphics card ), Therefore, deep learning operation can only use virtual machine of major factories ; After comparison , Discover Amazon's aws The most cost-effective way to build a deep learning virtual host , So I decided to use it aws.

As recommended , To solve the requirements of basic application , have access to aws On p2.xlarge Host instance ,p2.xlarge It is a very suitable cloud server for deep learning , It's equipped with 12G Prominently stored
Tesla K80 Video card ,4 nucleus CPU,60G Memory , as well as 500M Network speed of .

Register account and fill in work order

This part can refer to the original , I need one visa/Master Card/ Express /JCB
Credit Card , It should be noted that the registration process will be deducted 1 Advance charge in USD , To verify the validity of the credit card , this 1 Dollars will be returned to the credit card in a few days .
In addition, in addition to the common mail verification during the registration process , There will be phone verification , Just enter the verification code in the phone .

aws Instance settings

AMI choice

AMI
I.e. operating environment , You can create your own , You can also choose what has been built ; Because to use p2.xlarge host , So first of all, we should pay attention to which regions are provided by the host instance , At present, Virginia and Oregon in the United States have confirmed the availability of the host instance , Region selection is in the upper right corner of the login interface :

Registration complete , After successful login , get into aws Console , You can see :


Click the EC2, get into EC2 Console :



In the restrictions column on the left side of the console , You can see the current permissions of this account , If specific services are required , click “ Request to increase limit ”, Then submit the work order .

Click to start the instance Button , Start selection AMI:

Choose the right project here AMI, Usually pre installed tensorflow and conda Environmental Science , And installed Nvida Driven AMI, You can search in the search box , such as :


commonly AMI All with a short description , Choose the right one for your project , If there is a self built AMI, Or self built AMI, ad locum , I chose the second one AMI.

Select the appropriate instance in the instance type selection , such as p2.xlarge:

Bidding model can be selected in configuration instance , Bidding model recommended , You can set the maximum price manually , When the price is greater than the current available price , Instance will be created ; In the subnet option, you can select a specific instance subnet ;




Then next to configure security groups



In security group configuration , Inflow and outflow interfaces can be set , The simplest thing here is to set all ports , But there may be security issues ( Save the key file )

It can also be set as required :

Recommended or set to all traffic ;

Enter the audit page , After checking all parts of the information before , Click Start , Key will be required , You can choose to create a new one ( Download and save after new creation ) Or there is a key ;

Created successfully , Yes :




The original reference article is based on local Linux System connection method , stay windos10 The following versions of windos In the system , It's not self-contained ssh service , So you can only choose putty connect , Related methods can be found in aws Found in document :
use PuTTY from Windows connection to Linux example
<https://docs.aws.amazon.com/zh_cn/AWSEC2/latest/UserGuide/putty.html?icmpid=docs_ec2_console>
, And for windos10 system , The system has been integrated ssh tool , You can add enable in settings , Enable and Linux equally , Direct use ssh connect ;

There are key( secret key ) Folder of , use bash Terminal input :
ssh -i "a.pem" [email protected]
Of which key For your own key,xxxx The address should also be changed to the server you bought ip address . If it prompts you if you want to make sure you are connected , Click OK :
If it is not enabled in the folder where the key is stored ssh, For example, in the desktop root , Then change to :
ssh -i address/a.pem [email protected]
address For local storage key Relative address of ;

After connection , At the terminal :

At this time, the virtual host is connected ;
stay bash Command line experiment , display python edition :
python --version


start-up jupyter notebook, Open a remote connection support jupyter notebook:
jupyter notebook --ip=0.0.0.0
Will be displayed in the terminal jupyter Browser address for :
http://0.0.0.0:8888/?token=xxxxxxxx
Then enter in the browser The server IP:8888 You can enter jupyter notebook , If it was previously set jupyter
password , Password is required . For example, the virtual host assigns IP by 52.152.152.10, Enter in browser :
http://52.152.152.10:8888/?token=xxxxxxxx
So you can get in jupyter 了 :


Document upload and download

aws Virtual host of scp Upload and download documents , about windows system , have access to winscp This software :


After downloading and installing , open , Yes :


That is, fill in the host name aws host DNS domain name , User name is ubuntu( Different AMI User name may be different ), Click Advanced again —ssh— verification , fill .ppk Suffix key , Click to connect , To connect ;
Need attention :
aws The generated key file is a.pem, Need to use putty Turn it into a.ppk Document in form , The conversion process can be found in aws file : use PuTTY from Windows
connection to Linux example
<https://docs.aws.amazon.com/zh_cn/AWSEC2/latest/UserGuide/putty.html?icmpid=docs_ec2_console>
,
First use winscp Security warning will pop up when remote host is linked , click “ yes / determine ” that will do ;

Connection successful :

Local folder on the left , Remote virtual host folder on the right , Drag the file to finish the upload and download operation .
thus ,aws The basic operation requirements of the main engine have been fully met , You can use it to work .

PS: Create private AMI

When bidding instance is used , Once the instance is stopped ,AMI All documents and records in will be destroyed , If you want to save this development environment , Continue to use next time , You can put this AMI Environment preserved :


click “ Create Image ” To create a AMI image , Preservation process 3-5 About minutes , After saving successfully , Images that can be in the task list on the left –AMI
Found in , Next time you turn on the remote virtual host , You can select this directly AMI.


aws Provided for a period of one year 30G Free storage for , exceed 30G Or charge for more than one year , therefore , When the project is completed , Please put the unnecessary AMI Delete all .