My AI for Earth project is quite memory intensive so I have been learning about ways to take the data storage off the local disk and into the cloud, while still maintaining on the fly access to crucial files on my virtual or local machine. My classification problem started off requiring just a few GB … Continue reading AI Adventures in Azure: Blob storage
Tag: cloud
AI Adventures in Azure: Uploading data to the VM
There are many ways to transfer data from local storage to the virtual machine. Azure provides Blob storage for unstructured data managed through the user's storage account as well as specific storage options for files and tables. There is also the option to use Data Lakes. These are all useful for storing large datasets and … Continue reading AI Adventures in Azure: Uploading data to the VM
AI Adventures in Azure: Choosing VM Size
The main purpose of a VM is to accelerate scripts compared to running locally on a laptop or desktop by outsourcing the computation to a more powerful remote computer. There is an overwhelming number of options for Azure VM sizes, each of which is optimised for a particular purpose, so to get the best performance … Continue reading AI Adventures in Azure: Choosing VM Size
AI Adventures in Azure: Accessing the VM via terminal or remote desktop
Accessing the Data Science Virtual Machine Once the virtual machine is set up and started (by clicking “start” on the appropriate VM in the Azure portal) there are several ways to interface with it. The first is via the terminal (I am running Ubuntu 16.04 on both my local machine and the virtual machine). To … Continue reading AI Adventures in Azure: Accessing the VM via terminal or remote desktop
AI Adventures in Azure
A lot of my work at the moment requires quite computationally heavy geospatial analysis that stretches the processing capabilities of my laptop. I invested in a pretty powerful machine – i7-7700GHz processor, 32GB RAM – and sped things up by spreading the load across cores and threads, but it can still be locked up for … Continue reading AI Adventures in Azure
