Parsing JSON in ink based Smart Contracts

Question

Currently I am developing a Fat Contract (Native to Phala Network) that also supports Ink Based Smart Contracts to be written within the Fat Contracts which is what my example is doing.

To describe the problem, I am able to send HTTP requests from my Fat Contract that will return a JSON format response. This is where I get stuck bc in the runtime, I can usually parse the response using serde, but serde is not able to be used when writing these contracts due to a float being used in the library that causes the contract to get rejected during the upload. Typically this piece of code would be able to parse this response:

{"id":"12196238-420ae612726839da09-🐤-DREAMLIKE_BIRD_007-00000007","forsale":0,"owner":"Gy2SSkZWXq3jHwmSEBSpqVpKN8cWEAECwKnvS7j3srV7sZ9","rootowner":"Gy2SSkZWXq3jHwmSEBSpqVpKN8cWEAECwKnvS7j3srV7sZ9","collectionId":"420ae612726839da09-🐤","metadata":"ipfs://ipfs/bafkreiduurl6jkqxqx3h5nhmegdkxq2smq7hbmbaryaudrz2xw563xtrza","sn":"00000007","updated_at":"2022-04-18T05:53:02.409768+00:00","image":"https://kanaria-img.rmrk.link/1650261182409/jpeg/12196238-420ae612726839da09-🐤-DREAMLIKE_BIRD_007-00000007.jpg"}

With this piece of code:

let response = http_get!(api_url);
if response.status_code != 200 {
    return Err(Error::TokenValidationFailed);
}

let body = response.body;
let json_body = json!(body);
let rmrk_nft: RmrkNft =
    serde_json::from_value(json_body).or(Err(Error::TokenValidationFailed))?;

Where RmrkNft struct is defined as:

/// RMRK NFT structure
#[derive(Deserialize, Debug, Eq, PartialEq)]
pub struct RmrkNft {
    id: String,
    metadata: String,
    image: String,
    rootowner: String,
}

My question is if there is an alternative to serde that will allow me to parse the JSON response from my HTTP request to be able to load into my RmrkNft struct?

Answer 1

but serde is not able to be used when writing these contracts due to a float being used in the library

Yes, this is intentional. Floating point numbers should not be used directly in a blockchain context ‒ the tl;dr is that floating point arithmetic is non-deterministic which means that different processors compute (slightly) different results for the same operation.

See the ink! FAQ on Why is it not possible to use floating point data types in ink!? for more details.

My question is if there is an alternative to serde that will allow me to parse the JSON response from my HTTP request to be able to load into my RmrkNft struct?

I know too little about Phala, but in general the idea for smart contracts is that parsing of any kind should not happen in a smart contract. Smart contracts charge users fees for their execution and hence a user would pay every time something needs to be parsed. It's better to put the parsing outside ‒ in the frontend, the Dapp, or in your case the runtime ‒ and submit just the result of this parsing to the smart contract.

Unfortunately this doesn't really answer your question how this can be done in a FAT contract in the Phala TEE, but maybe someone from Phala can answer this? I'll forward the question to some people as well.

Answer 2

We were able to figure this out by using the serde_json_core library. An example can be seen here subgraph-nouns Fat Contract.

In this example, we make an HTTPS requests to Subgraph Endpoint for NounsDAO and retrieve the current data on the top bid of a NounsDAO auction. Here is the HTTPS POST request using curl

curl -X POST 'https://api.thegraph.com/subgraphs/name/nounsdao/nouns-subgraph' --data '{"query":"query MyQuery {\n  auctions(orderBy: endTime, orderDirection: desc, first: 1) {\n    amount\n    id\n  settled\n}\n}\n","variables":null,"operationName":"MyQuery"}'

This then will return a result response with body data like this:

{
    "data": {
        "auctions": [{
            "amount": "81600000000000000000",
            "id": "335",
            "settled": false
        }]
    }
}

Next, we take this body of data and parse it into the following structs:

#[derive(Deserialize, Debug)]
pub struct Data<'a> {
    #[serde(borrow)]
    data: DataInfo<'a>,
}
#[derive(Deserialize, Debug)]
#[serde(bound(deserialize = "alloc::vec::Vec<NounsInfo<'a>>: Deserialize<'de>"))]
pub struct DataInfo<'a> {
    #[serde(borrow)]
    auctions: Vec<NounsInfo<'a>>,
}

#[derive(Deserialize, Encode, Clone, Debug, PartialEq)]
pub struct NounsInfo<'a> {
    id: &'a str,
    amount: &'a str,
    settled: bool,
}

This is shown in this snippet of code where we use the serde_json_core::from_slice(&body) to parse the data into the Data<'a> struct.

// Get the latest nouns info through http_post
let response = http_post!(NOUNS_HTTP_ENDPOINT, HTTP_POST_DATA.as_bytes().to_vec());
if response.status_code != 200 {
    return Err(Error::RequestFailed);
}
let body = response.body;
// Extract Nouns Info
let (data, _): (Data, usize) =
    serde_json_core::from_slice(&body).or(Err(Error::InvalidBody))?;
let nouns_info = data.data.auctions[0].clone();

If you want to see the boilerplate frontend demoing this functionality, you can check out this little video I created Fat Contracts Cross-Chain Use Case

Answer 3

It is such a bad idea in general to do things like JSON parsing in the runtime, not only are you bloating your contract size by including a huge library like serde into it, but you are also bloating your contract, fees, and the blockchain hosting your contract by including tons of dead data which is included in JSON.

JSON is not a blockchain friendly format, and is usually used when fields are unknown and should be variably parsed.

Contracts on the other hand are mostly constant, have well known and expected types, and publicly available APIs for the data they understand.

This is why formats like SCALE have no fields, because it is ultimately wasted data. Instead both the sender and the receiver (the contract) understand what exactly should be sent across the wire, and the contract spends the least amount of resources needed to parse that data.

So if you are using JSON in your smart contract, you are doing something wrong.