2

I was commissioned to create a C# RPC client and after successfully implementing SCALE (as verified by polkadot-java tests), I decided to start by creating a deserializer for the V11 metadata.

Initially I started creating the object model using the Rust documentation and then found the types-support package that seemed to contain both a reference hex payload as well as the decoded payload, ideal for creating a unit test for the deserializer.

In due process I found some discrepancies that I wasn't able to explain and I hope this question will shed some light. Specifically:

  • Some of the keys don't match. e.g. args (types-support) vs arguments (Rust)
  • Some types seem to have a stripped prefix. e.g. BlockNumber (types-support) vs T::BlockNumber in the decoded data.
  • Some other types seem to have a stripped suffix. e.g. Vec<UncleEntryItem> vs Vec<UncleEntryItem<T::BlockNumber, T::Hash, T::AccountId>>

I understand that V11 is not the latest version of the metadata (and for the life of me I can't remember why I went with V11 instead of V14 for the first test right now), but shouldn't all these match?

Full JSON can be found in this gist.

1 Answer 1

1

In the completely unmaintained v11, the types were all sync-ed manually between the runtime and SDKs. This means that for each and every Rust PR made, there needed to be eyes on it, and then all the types aligned and duplicated into the JS codebase. Some things like args were shortened since it aligned with the types used inside the JS API, i.e. less overhead from a JS maintenance pov.

In addition, all namespaces were stripped. This is for a specific reason since in the TS type generation we can't do export type T::Balance = .... The same applies to generics, where TS has an own generic type system.

So up to v13, all generics as well as trait prefixed were removed. The long and the short of it is that to support all the tooling around the JS API and the fact that everything is manual, there is no way these types and names could match.

For v14 all this type information comes from the metadata, so field names match in any struct. (There are a couple of renames remaining where the actual field names conflict with JS reserved words, but these are generally few.)

In the case of type-names for v14, they are generated based on the type paths specified in the metadata, so eg. a type called sp_runtime::TokenError becomes SpRuntimeTokenError. (There is some slight type de-dupes also at play here, e.g. pallet_staking::pallet::pallet::Event would become PalletStakingPalletEvent)

You really should not be developing on anything that is not metadata v14 at this point in the Substrate history.

5
  • Got it, thank you @Jaco. Is working against the Rust docs the best approach for my task?
    – georgiosd
    Apr 12, 2022 at 16:46
  • Start with metadata v14 - parse the metadata (ensure you have those structs right, this would be alignment with Rust) and extract the types from there. If you can setup all those types and encode/decode them, you have the bulk of everything supported (on a SCALE-level).
    – Jaco
    Apr 12, 2022 at 18:45
  • I've started on that but I'm seeing another discrepancy which I can't account for. It seems that typename string are encoded with a first byte that when = 0 denotes a missing value and otherwise signifies the beginning of a string, but I'm not seeing the length of it as a compact integer. Am I missing something? Working with raw.githubusercontent.com/polkadot-js/api/master/packages/…
    – georgiosd
    Apr 12, 2022 at 19:42
  • The JSON is information decoded from a SCALE input, it is not a SCALE display, SCALE is the raw hex byte stream. In JSON a string "xyz" has length 3, since we have delimiters in the format itself. In the SCALE input it would be [3 << 2, 0x78, 0x79, 0x7a].
    – Jaco
    Apr 13, 2022 at 5:42
  • Yeah, I've gotten that far but I'm losing it somewhere with the encoded types in v14. Must be a subtle bug somewhere, I'll keep looking. Thank you for your help.
    – georgiosd
    Apr 13, 2022 at 7:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.