EOLink 0.2.6: Aggregating data from multiple oracles

If you like this content consider tipping in ETH, LINK or an ERC20 stablecoin at this address: 0xf9f6849230cBe10200B43dB103511b778898e71C

Code associted with this post available HERE

So far the BrightLink project has gathered data from one single oracle to determine the distribution of funds. This is a big problem because it undermines the decentralization of the system. There is no point using a transparent, open, trustless blockchain network to manage the flow of information and funds if the trigger for that system is a single, centralized oracle that can be manipulated by a single actor. At the same time, a single oracle has no redundancy against server failure, data invalidity, bugs etc. For those reasons, in this post I update the system to aggregate data from several oracles.

This project is still in the protoyping stages, so in reality I still have centralized control of the data ingested into the contract because I have unique access to the API endpoints accessed by the oracles. In a production environment each of the data sources for the oracles would have distinct origins. For example, I envisage eventually having oracles calculating similar summary data for identical areas from multiple satellites (Landsat, Sentinel, MODIS?) calculated on several virtual machines from different providers (Planetary Computer, Google Earth Engine, Azure VM?) to give multiple sources and redundancy against any single point of failure. For now though, to demonstrate the mechanics of how multiple oracle requests can be incorporated into a single contract, the data sources will just be arbitrary json data hosted at individual urls.

In this post, I will assume that the three endpoints are providing the same metric but from three sources, for example the same spatial statistic gathered using the same analysis applied to three different satellite data sets. I will impose a requirement that at least two of the oracles must return valid data for the contract to execute, because if only one oracle returns valid data, the system is not at all decentralized. If no oracles return valid data then the contract will not execute correctly. Assuming multiple oracles return valid data, then the transaction is executed depending upon the weighted mean of each valid datum returned. The weights are, in this version, provided by the contract owner but could later come from a Curve-style gauge in a DAO, where users stake their governance tokens as a way to express how much trust the community has in each data source, and by extension how much weighting it should have in the final data aggregation.

The contract

To incorporate multiple data requests into the contract, first the oracle request function must be adapted to receive an array of urls and to populate an array of output data, which are then aggregated into one single value to trigger a payment. The various data points will be aggregated using a weighted average, with weights defined by the user. This means several new variables need to be defined. These will be:

APIadresses: this is an array that will contain multiple urls corresponding to API endpoints

oracleData: this will now be an array that will contain data returned from each API call

w1, w2, w3: these are integers used to weight each API in the aggregation

minResponses: an integer to determine the minimum number of valid API calls required

aggregateData: an integer that will contain the weighted average of the data returned by the oracles

badOracles: an integer that counts how many API calls fail

None are required at deployment time, so they can all be added to the main list of variables inside the contract definition but before the constructor function:

contract BrightLink_v01 is ChainlinkClient {

    // var definitions
    uint256 public depositedFunds;
    address private owner;
    uint16 private referral = 0;
    address public dai_address;
    address public adai_address;
    address public poolAddress;
    address public poolAddressProvider;
    address private customer;
    address private donor;
    uint16 private threshold;
    uint256 public value;
    address private oracle; // oracle address
    bytes32 private jobID; // oracle jobID
    uint256 private fee; // oracle fee
    string[3] public APIaddresses; 
    uint256[3] public oracleData;
    IERC20 public dai;
    IERC20 public adai;
    ILendingPoolV2 public lendingPool;
    ILendingPoolAddressesProviderV2 public provider;
    uint16 index = 0;
    uint256 aggregateData;
    uint16 w1;
    uint16 w2;
    uint16 w3;
    uint16 minResponses = 2;
    uint16 badOracles = 0;

Of these newly defined variables, minResponses and badOracles were instantiated with zeros already, some should be instantiated in the constructor function. Specifically, I decided to define the url addresses here, and give the oracleData dummy values of 0.

constructor(address _dai_address, address _adai_address, address _link, address _poolAddressProvider, address _oracle, address _customer, address _donor, string memory _jobID, uint256 _fee, uint16 _threshold) public{
        
    // var instantiations
    owner = msg.sender;
    customer = _customer;
    donor = _donor;
    depositedFunds = 0;
    dai_address = _dai_address;
    adai_address = _adai_address;
    poolAddressProvider = _poolAddressProvider;
    threshold = _threshold;
    dai = IERC20(dai_address);
    adai = IERC20(adai_address);
    provider = ILendingPoolAddressesProviderV2(poolAddressProvider); 
    poolAddress = provider.getLendingPool();
    lendingPool = ILendingPoolV2(poolAddress);
    oracle = _oracle;
    jobID = stringToBytes32(_jobID);
    fee = _fee;
    APIaddresses[0] = "https://raw.githubusercontent.com/jmcook1186/jmcook1186.github.io/main/Data/example.json";
    APIaddresses[1] = "https://raw.githubusercontent.com/jmcook1186/jmcook1186.github.io/main/Data/example2.json";
    APIaddresses[2] = "https://raw.githubusercontent.com/jmcook1186/jmcook1186.github.io/main/Data/example3.json";
    oracleData[0] = 0;
    oracleData[1] = 0;
    oracleData[2] = 0;

    // set link token address depending on network
    if (_link == address(0)) {
        setPublicChainlinkToken();
    } else {
        setChainlinkToken(_link);
    }
}

The three API endpoints happen to provide data in a uniform data type, so a single Chainlink oracle can be used to grab all three values. However, this still requires some recoding compared to the previous single-data-source example. Now instead of having the user call the oracleRequest function, three calls to oracleRequest are made by a higher-level function, requestDataFromAPI(), which is called by the user. As before, the fulfill() function is called from oracleRequest() and is never seen by the end-user. In this case, the three functions that control the API requests are:

function requestDataFromAPI() public onlyOwner{
    // calls the oracleRequest function with each URL
    // aggregation happens inside fulfill() function
    oracleRequest(APIaddresses[0]);
    oracleRequest(APIaddresses[1]);
    oracleRequest(APIaddresses[2]); 
    }

function oracleRequest(string memory url) internal returns (bytes32 requestId){
    // oracle request happens here. URL is passed as var url
    // args are jobID, callback address (this contract) and fulfill function from this
    Chainlink.Request memory request = buildChainlinkRequest(jobID, address(this),this.fulfill.selector);
        
    // Set the URL to perform the GET request on
    request.add("get", url);
    request.add("path", "data.0.number");
        
    // Sends the request
    return sendChainlinkRequestTo(oracle, request, fee);
    }

function fulfill(bytes32 _requestId, uint256 _value) public recordChainlinkFulfillment(_requestId){

    // assign data from oracle to position in oracleData array
    oracleData[index] = _value; 
    // iterate through array indexes
    index++;
    // calculate weighted mean of data in oracleData array
    aggregateData = ((w1*oracleData[0]/100)+(w2*oracleData[1]/100)+(w3*oracleData[2]/100))/3;
    }

The data aggregation occurs in the fulfill function by taking the mean of the returned values after multiplying by the respective weights. Each time the oracle fulfills a request, it costs 0.1 LINK, so make sure the contract is sufficiently funded with LINK tokens (0.3 LINK) otherwise the data gathering will fail. I also decided to add some simple data validation, simply by checking that none of the returned values were zeros. To do this I wrote a simple condition that increments badOracles by 1 for each zero element in oracleData. Later, transactions can only occur if the sum of oracleData is less than the threshold for valid oracle results.

function validateOracleData() public {    
    // ensures a sufficient number of oracles return valid data
    // avoids accidentally centralizing workflow by relying on 1 oracle

    for (uint16 i = 0; i<oracleData.length; i++) {  //for loop example
        if (oracleData[i] ==0){

            badOracles++;
        } 
    }
}

The viewOracleData and retrieveDAI functions then simply required updating to use the aggregateData variable instead of the original single-endpoint value.

function viewValueFromOracle() public view returns(uint256 viewValue){
    //  show aggregated orace data to user
    viewValue = aggregateData;
}


function retrieveDAI() public onlyOwner onlyForValidOracles{
        
    // send DAI from contract back to owner
    dai.approve(address(this), depositedFunds);
    dai.approve(customer, depositedFunds);
    dai.approve(donor, depositedFunds);

    if (aggregateData > threshold){
        
        require(dai.transfer(customer, depositedFunds));
          
        if (dai.balanceOf(address(this))>0){
            require(dai.transfer(owner,dai.balanceOf(address(this))));
        }
    }

    else{

        require(dai.transfer(donor, depositedFunds));
           
        if (dai.balanceOf(address(this))>0){

            require(dai.transfer(owner, dai.balanceOf(address(this))));
        }
    }

}

Finally, the condition that sufficient oracles returned good data can be formalised in a modifier used to restrict access to the retrieveDAI() function:

modifier onlyForValidOracles(){
    require(badOracles<minResponses,"INSUFFICIENT VALID ORACLE RESPONSES: WITHDRAWAL PROHIBITED");
   _;
}

This contract is now ready to transact according to aggregated data from three API endpoints, at a total cost of 0.3 LINK.

Running/testing

Scripts for running and thoroughly testing this contract are available here!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s