Python: Arrays

Run and experiment with the code in this tutorial using the Jupyter Notebook Intro Abstraction is one of the fundamental concepts in object-oriented programming. It enables a user to implement an idea without having to grapple with its implementation, or put another way it allows access to a high level concept while hiding away lower-level … Continue reading Python: Arrays

AI Adventures in Azure: making use of blob i/o

As part of my AI for Earth software development, I am working hard on a system that simultaneously applies a supervised classification algorithm, and inverted radiative transfer model, an albedo calculation and an energy balance model pixel-wise over satellite imagery over the Greenland Ice Sheet. This is hugely computationally expensive and very memory hungry. I … Continue reading AI Adventures in Azure: making use of blob i/o

AI Adventures in Azure: hosting and hitting REST APIs using the Azure container registry

This week I had the pleasure of attending the AI for Earth summit at Microsoft HQ where the focus was on developing codes that run locally into APIs hosted on Azure that can be accessed from anywhere. I've got to admit I'm still developing my working knowledge of this process, but I am getting there, … Continue reading AI Adventures in Azure: hosting and hitting REST APIs using the Azure container registry

AI Adventures in Azure: Blob storage

My AI for Earth project is quite memory intensive so I have been learning about ways to take the data storage off the local disk and into the cloud, while still maintaining on the fly access to crucial files on my virtual or local machine. My classification problem started off requiring just a few GB … Continue reading AI Adventures in Azure: Blob storage

AI Adventures in Azure: Ice Surface Classifiers

For this post I will introduce what I am actually trying to achieve with the AI for Earth grant and how it will help us to understand glacier and ice sheet dynamics in a warming world. The Earth is heating up - that's a problem for the parts of it made of ice. Over a … Continue reading AI Adventures in Azure: Ice Surface Classifiers

AI Adventures in Azure: Ways to Program in Python on the DSVM

Having introduced the set up and configuration of a new virtual machine and the ways to interact with it, I will now show some ways to use it to start programming in Python. This post will assume that the VM is allocated and that the user is accessing the VM using a remote desktop client. … Continue reading AI Adventures in Azure: Ways to Program in Python on the DSVM

AI Adventures in Azure: Choosing VM Size

The main purpose of a VM is to accelerate scripts compared to running locally on a laptop or desktop by outsourcing the computation to a more powerful remote computer. There is an overwhelming number of options for Azure VM sizes, each of which is optimised for a particular purpose, so to get the best performance … Continue reading AI Adventures in Azure: Choosing VM Size

AI Adventures in Azure: Accessing the VM via terminal or remote desktop

Accessing the Data Science Virtual Machine Once the virtual machine is set up and started (by clicking “start” on the appropriate VM in the Azure portal) there are several ways to interface with it. The first is via the terminal (I am running Ubuntu 16.04 on both my local machine and the virtual machine). To … Continue reading AI Adventures in Azure: Accessing the VM via terminal or remote desktop

AI Adventures in Azure

A lot of my work at the moment requires quite computationally heavy geospatial analysis that stretches the processing capabilities of my laptop. I invested in a pretty powerful machine – i7-7700GHz processor, 32GB RAM – and sped things up by spreading the load across cores and threads, but it can still be locked up for … Continue reading AI Adventures in Azure

ASD spectra processing with Linux & Python

I’m sharing my workflow for processing and analysing spectra obtained using the ASD Field Spec Pro, partly as a resource and partly to see whether others have refinements or suggestions for improving the protocols. I'm specifically using Python rather than any proprietary software to keep it all open source, transparent and to keep control over … Continue reading ASD spectra processing with Linux & Python