flood insurance contract Testing

If you like this content consider tipping jmcook.eth

Code associated with this post available HERE

One of the more powerful aspects of blockchain technology is that transactions are immutable – once they are on chain they can’t be updated or removed. This is why local and testnet testing is so important.

In the previous posts the project code has just been tested by calling functions in brownie console and checking that the responses are consistent with expectations. That was OK for demonstration purposes, but a real project needs automated, repeatable, transparent tests with as close to comprehensive coverage as possible to ensure the contracts behave as intended and that any weird edge-cases, potential exploits or weak point that can be hacked are resolved as far as possible. It is also helpful to make the tests accessible to other developer and auditors that might want to check the strength and security of your project code. Automated, scripted testing is a good way to achieve this. This post will demonstrate how to build some simple testing scripts for the flood insurance project described in (see EOLink 0.1.3EOLink 0.1.5).

To achieve this, I will use pytest. This is a common testing framework used to write automated tests for Python code and it works very well with brownie.

Local vs testnet

It is often most convenient to test a project on a local blockchain (e.g. ganache) before using a testnet because a local chain is quick to spin up, quick to run, does not require funds to be retrieved from faucets, does not require connection to a public blockchain, helps keep the project clean of multiple deployments, and carries no risk of accidentally exposing sensitive information like private keys etc in pre-production code.

However, for projects that link to oracles, local testing presents a challenge because the oracles are smart contracts – usually written by a third party – that already live on a public blockchain, rather than their code being part of your project’s codebase. When a project is run on a local blockchain, it can’t connect to those oracle contracts which means critical aspects of the project can’t be tested.

There are two solutions to this problem: one is to build “mocks” – contracts held locally that mirror the functionality of the real oracle. The other option is to immediately test on a testnet where the oracle contracts already exist, such as Kovan or Rinkeby.

In this post, I will only test on the Kovan testnet, planning to come back to mocks in a later post. Testnets are public blockchains that use assets with no real-world value. They exist to provide a safe sandbox where smart contracts behave as they would on Ethereum mainnet (with testnet-specific caveats). Deploying a project to mainnet without first deploying and thoroughly testing on a testnet such as Kovan would be pretty bold.

OBTAINING KOVAN funds

To deploy and run the project on Kovan, the contract owner needs funds to pay for gas, just like on Ethereum mainnet. There are two types of gas to cover: a) transaction gas, which is paid in ETH and covers the cost of updating the state of the blockchain; b) oracle gas, which is paid in LINK and covers the cost of the oracle making a request to an external API. These are not the same assets that are bought and sold on public exchanges and they do not have any real-world value. They are available for free from “faucets”. These faucets freely dispense testnet ETH and testnet LINK to valid addresses. There are limits to the amount of each asset the faucet will dispense per unit time, but these are pretty high limits for most use-cases, and are certainly more than enough for our purposes here.

Kovan ETH

Kovan ETH is available from the Kovan Eth Faucet. This requires signing in through Github, and providing a valid wallet address. For me, this is the “main” account that I have saved in brownie accounts, and named “Kovan Development Account” in my MetaMask, as described here. Providing valid Github credentials leads to a page where the wallet private key can be provided. After a minute or two, 2 ETH will appear in the wallet, which can be verified by opening MetaMask or querying the account balance via brownie console.

Kovan LINK

Obtaining Kovan LINK requires an additional step because the wallet created in MetaMask does not display LINK by default, it must be added to the wallet using the “add token” button. The Kovan test-link is available at this adress: 0xa36085F69e2889c224210F603D836748e7dC0088 which should be provided in the “add token” menu in MetaMask. Once added, we can grab test LINK from the Kovan LINK faucet by providing the wallet address. 100 LINK will then be added to the account.

FLOOD Token

This project also uses an ERC20 token called “FLOOD” as its unit of exchange. We can also add this to MetaMask using the “add token” button and providing the FLOOD deployment address. This can be helpful just to see the balance changing as the tests execute without having to repeatedly query the balances in brownie.

Introduction to pytest

Now that the contract owner has plenty of ETH and LINK to pay gas on Kovan, the testing can begin. The testing framework, pytest, uses fixtures to configure a testing environment. These fixtures are functions that always execute when any testing occurs because they establish the conditions required to run specific unit tests. For example, in order to test a function belonging to a contract, the contract must be deployed, or if it is already deployed an instance of it needs to be created in the test session. This is always going to be true, so this can be defined as a fixture in pytest. Fixtures are generally collected together in a separate file called “conftest”, where pytest knows to look for fixtures.

To start, create a new file in the /tests directory and name it “conftest.py”. Then there are some imports required. First, pytest, then some key information imported from brownie, including the accounts and the network information.

import pytest
from brownie import accounts, network

The rest of the contents of conftest.py comprises functions wrapped in “@pytest.fixture()” which indicates to pytest that these functions should be run in every testing session. The first two items every testing session will need are the insurance contract and the contract for the FLOOD token. These were already deployed to Kovan so we can instantiate them here using “at” and the deployment addresses. I also added a condition that the active network must be Kovan because local testing is skipped for now and these contract addresses are only valid on Kovan.

@pytest.fixture(scope='module')
def get_token():
    """
    load deployed contract for FLOOD token
    """
    if network.show_active() == 'kovan':

        floodToken = FloodToken.at('0x63585C9f4968658cB36C48fa33e34BE513c5e4D9')
    
    else:
        pytest.fail('Please test on kovan network')

    return floodToken



@pytest.fixture(scope='module')
def get_contract():
    """
    load deployed insurance contract
    """
    
    if network.show_active() == 'kovan':
        contract = floodInsurance.at('0xE094A61c7e10b5ECbEE6006a1207239d515d1548') 

    else:
        pytest.fail('Please test on kovan network')

    return contract

There is a bit of artistry in deciding what information belongs in fixtures and what should be part of individual unit tests. I decided that the account addresses, oracle gas and payout amount should all be fixtures as these will be constant across all testing of these contracts because they were defined in the contract constructor when the contract was deployed, so any testing will fail if these values are incorrect. Setting them as fixtures therefore makes sense to me. These were included as follows:

@pytest.fixture(scope='session')
def load_account1():
    """
    load insurer/contract owner account
    """
    account1 = accounts.load('main')

    return account1


@pytest.fixture(scope='session')
def load_account2():
    """
    load customer account
    """
    account2 = accounts.load('account2')

    return account2


@pytest.fixture
def oracleGas():
    """
    define number of LINK required by oracle
    """
    return 10e16 # 0.1 LINK



@pytest.fixture(scope='session')
def set_payout_amount():
    """
    define amount of insurance payout
    NB must match what was defined in contract constructor at deployment
    """
    return 500000e18

This is all the fixtures needed for now, so we can close this file and start a new one for writing specific unit tests. The script that contains the unit tests must have a file name that begins with “test_” because this signifies to pytest that this is where the test functions live. I named my file “test_FloodInsurance.py’. Inside that file we need some imports again, then the remaining contents of the file are functions that define individual unit tests. Each of those functions require names that begin with “test_” to indicate to pytest that these functions are individual test cases.

I started with some simple tests relating to the FLOOD token. First, just check that the fixture in conftest has successfully instantiated the token contract. The “assert” keyword is critical, as it sets up a conditional where the statement following “assert” is either true, in which case the program continues to run, or it evaluates to false, in which case an exception is thrown. Notice that the argument representing the contract that is passed to the function IS the fixture that instantiated the contract rather than the contract object itself. The fixture returns the contract object, and that is what is passed to the unit test.

The second unit test is to test that the token contract can be used to successfully transact FLOOD tokens between the accounts associated with this project. Again, the fixtures relating to the instantiated contract and the loaded accounts are passed as arguments. This time, I wanted to test the premise that a token transfer between account1 and account2 would lead to a predictable change in balance in both accounts. If not, the token is useless as a unit of exchange. Therefore, I instantiated the contract, determined the initial balance, transferred a known amount, and then asserted that the new balance of each account should be their initial balance adjusted by the transfer amount. If everything works as expected, this assert statement evaluates to true and the test passes.

def test_token_deploy(get_token):
    

    floodToken = get_token
    assert floodToken is not None

    return


def test_token_transfer(get_token,load_account1, load_account2):

    floodToken = get_token

    initialBalance = floodToken.balanceOf(load_account2)
    floodToken.transfer(load_account2, 100e18,{'from':load_account1})

    assert floodToken.balanceOf(load_account2) == initialBalance+100e18

    floodToken.transfer(load_account1, 100e18, {'from':load_account2})

    assert floodToken.balanceOf(load_account2) == initialBalance

    return

The remaining tests are slightly more complicated, but all establish some expected outcome from a specific function associated with the insurance contract and use “assert” to test whether that outcome occurs. This includes ancillary functions such as checking that the contract accepts LINK and FLOOD tokens as expected, and also the core functions such as the oracle request and the payout to the customer/contract owner.


def test_sendLINK(get_contract,oracleGas,load_account1):
    
    contract = get_contract

    if interface.LinkTokenInterface(config["networks"][network.show_active()]["link_token"]).balanceOf(contract) >= oracleGas*5:
        pytest.skip("Contract already funded")
    
    else:
        interface.LinkTokenInterface(config["networks"][network.show_active()]["link_token"]).transfer(contract,oracleGas*5,{'from':load_account1})
        
        assert interface.LinkTokenInterface(config["networks"][network.show_active()]["link_token"]).balanceOf(contract) >= oracleGas*5
    
    return


def test_WarningLevel(get_contract, load_account1):

    contract = get_contract
    contract.requestWarningLevel({'from':load_account1})

    time.sleep(30)

    warningLevel = contract.warningLevel()

    assert warningLevel is not None

    assert warningLevel >= 0

    return warningLevel



def test_checkFund(get_contract):
    
    contract = get_contract
    Funded = contract.checkFund()

    assert Funded is not None

    return



def test_sendTokenToContract(get_contract, load_account1, load_account2, set_payout_amount, get_token):

    contract= get_contract
    token= get_token

    if token.balanceOf(contract) < set_payout_amount:
        token.transfer(contract, set_payout_amount-token.balanceOf(contract), {'from':load_account1})

    if token.balanceOf(load_account2) > 0:
        token.transfer(load_account1, token.balanceOf(load_account2), {'from':load_account2})
    
    assert token.balanceOf(contract) >= set_payout_amount
    assert token.balanceOf(load_account2) == 0

    return



def test_settleClaim(get_contract, get_token, set_payout_amount, load_account1, load_account2, oracleGas):
    
    network.gas_limit(6700000)

    contract = get_contract
    token = get_token

    gasBalance = interface.LinkTokenInterface(config["networks"][network.show_active()]["link_token"]).balanceOf(contract)
    
    if gasBalance < oracleGas*5:
        interface.LinkTokenInterface(config["networks"][network.show_active()]["link_token"]).transfer(contract,oracleGas*5,{'from':load_account1})

    if token.balanceOf(contract) < set_payout_amount:
        token.transfer(contract, set_payout_amount-token.balanceOf(contract), {'from':load_account1})

def test_sendLINK(get_contract,oracleGas,load_account1):
    
    contract = get_contract

    if interface.LinkTokenInterface(config["networks"][network.show_active()]["link_token"]).balanceOf(contract) >= oracleGas*5:
        pytest.skip("Contract already funded")
    
    else:
        interface.LinkTokenInterface(config["networks"][network.show_active()]["link_token"]).transfer(contract,oracleGas*5,{'from':load_account1})
        
        assert interface.LinkTokenInterface(config["networks"][network.show_active()]["link_token"]).balanceOf(contract) >= oracleGas*5
    
    return


def test_WarningLevel(get_contract, load_account1):

    contract = get_contract
    contract.requestWarningLevel({'from':load_account1})

    time.sleep(30)

    warningLevel = contract.warningLevel()

    assert warningLevel is not None

    assert warningLevel >= 0

    return warningLevel


def test_checkFund(get_contract):
    
    contract = get_contract
    Funded = contract.checkFund()

    assert Funded is not None

    return


def test_sendTokenToContract(get_contract, load_account1, load_account2, set_payout_amount, get_token):

    contract= get_contract
    token= get_token

    if token.balanceOf(contract) < set_payout_amount:
        token.transfer(contract, set_payout_amount-token.balanceOf(contract), {'from':load_account1})

    if token.balanceOf(load_account2) > 0:
        token.transfer(load_account1, token.balanceOf(load_account2), {'from':load_account2})
    
    assert token.balanceOf(contract) >= set_payout_amount
    assert token.balanceOf(load_account2) == 0

    return


def test_settleClaim(get_contract, get_token, set_payout_amount, load_account1, load_account2, oracleGas):
    
    network.gas_limit(6700000)

    contract = get_contract
    token = get_token

    gasBalance = interface.LinkTokenInterface(config["networks"][network.show_active()]["link_token"]).balanceOf(contract)
    
    if gasBalance < oracleGas*5:
        interface.LinkTokenInterface(config["networks"][network.show_active()]["link_token"]).transfer(contract,oracleGas*5,{'from':load_account1})

    if token.balanceOf(contract) < set_payout_amount:
        token.transfer(contract, set_payout_amount-token.balanceOf(contract), {'from':load_account1})

    assert token.balanceOf(contract) >= set_payout_amount
    assert gasBalance >= oracleGas


    contract.settleClaim({'from':load_account1})

    if contract.warningLevel() < 3:

        assert token.balanceOf(contract) < set_payout_amount
        assert token.balanceOf(load_account1) >= set_payout_amount

    if contract.warningLevel() >= 3:
        
        assert token.balanceOf(contract) < set_payout_amount
        assert token.balanceOf(load_account2) >= set_payout_amount

    return

    assert token.balanceOf(contract) >= set_payout_amount
    assert gasBalance >= oracleGas


    contract.settleClaim({'from':load_account1})

    if contract.warningLevel() < 3:

        assert token.balanceOf(contract) < set_payout_amount
        assert token.balanceOf(load_account1) >= set_payout_amount

    if contract.warningLevel() >= 3:
        
        assert token.balanceOf(contract) < set_payout_amount
        assert token.balanceOf(load_account2) >= set_payout_amount

    return

running the tests

Running these tests in brownie is a case of opening a terminal, navigating to the project folder and simply running:

$ brownie test --network kovan

Now, brownie will run all the fixtures in conftest then sequentially run each of the unit tests, reporting the status to the console. If all tests pass, the output looks something like:

Tests that do not pass are detailed in the console with detailed tracebacks. If the tests are set up correctly, these tracebacks should be assert errors, indicating that the test executed correctly but failed, indicating that the bug sits somewhere in the function inside the deployed contract. However, other errors are also possible that result from a poorly defined or incorrectly coded test. A process of test-debugging is often necessary!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s