Python Data Analysis Library There is a Jupyter Notebook accompanying this post HERE! In a previous session we explored NumPy in detail, learning about array structures and vectorisation. While NumPy is extremely powerful, it has some limitations. For example, data is organised into rows and columns (often in more than 2 dimensions) without labels other … Continue reading Python: Pandas
Tag: Python
Python: NumPy
There is a Jupyter Notebook accompanying this post HERE. NumPy is a Python package built around the concept of ndarrays (n-dimensional arrays) along with a suite of efficient functions for applying operations over those arrays. Many of the other important packages for data scientists are built on top of NumPy (e.g. Pandas, scikit-learn). In the … Continue reading Python: NumPy
Python: Arrays
Run and experiment with the code in this tutorial using the Jupyter Notebook Intro Abstraction is one of the fundamental concepts in object-oriented programming. It enables a user to implement an idea without having to grapple with its implementation, or put another way it allows access to a high level concept while hiding away lower-level … Continue reading Python: Arrays
AI Adventures in Azure: making use of blob i/o
As part of my AI for Earth software development, I am working hard on a system that simultaneously applies a supervised classification algorithm, and inverted radiative transfer model, an albedo calculation and an energy balance model pixel-wise over satellite imagery over the Greenland Ice Sheet. This is hugely computationally expensive and very memory hungry. I … Continue reading AI Adventures in Azure: making use of blob i/o
AI Adventures in Azure: hosting and hitting REST APIs using the Azure container registry
This week I had the pleasure of attending the AI for Earth summit at Microsoft HQ where the focus was on developing codes that run locally into APIs hosted on Azure that can be accessed from anywhere. I've got to admit I'm still developing my working knowledge of this process, but I am getting there, … Continue reading AI Adventures in Azure: hosting and hitting REST APIs using the Azure container registry
AI Adventures in Azure: Blob storage
My AI for Earth project is quite memory intensive so I have been learning about ways to take the data storage off the local disk and into the cloud, while still maintaining on the fly access to crucial files on my virtual or local machine. My classification problem started off requiring just a few GB … Continue reading AI Adventures in Azure: Blob storage
AI Adventures in Azure: Ice Surface Classifiers
For this post I will introduce what I am actually trying to achieve with the AI for Earth grant and how it will help us to understand glacier and ice sheet dynamics in a warming world. The Earth is heating up - that's a problem for the parts of it made of ice. Over a … Continue reading AI Adventures in Azure: Ice Surface Classifiers
AI Adventures in Azure: Ways to Program in Python on the DSVM
Having introduced the set up and configuration of a new virtual machine and the ways to interact with it, I will now show some ways to use it to start programming in Python. This post will assume that the VM is allocated and that the user is accessing the VM using a remote desktop client. … Continue reading AI Adventures in Azure: Ways to Program in Python on the DSVM
AI Adventures in Azure: Choosing VM Size
The main purpose of a VM is to accelerate scripts compared to running locally on a laptop or desktop by outsourcing the computation to a more powerful remote computer. There is an overwhelming number of options for Azure VM sizes, each of which is optimised for a particular purpose, so to get the best performance … Continue reading AI Adventures in Azure: Choosing VM Size
AI Adventures in Azure: Accessing the VM via terminal or remote desktop
Accessing the Data Science Virtual Machine Once the virtual machine is set up and started (by clicking “start” on the appropriate VM in the Azure portal) there are several ways to interface with it. The first is via the terminal (I am running Ubuntu 16.04 on both my local machine and the virtual machine). To … Continue reading AI Adventures in Azure: Accessing the VM via terminal or remote desktop
