How to handle XCM encoding and building in JavaScript

Question

Talisman and I are looking into building a simple JS module that helps developers build XCM for transactions and abstract common patterns like cross chain transfers into a simple JS function taking inputs like: fromParachain, toParachain, assetId, recipient etc. These functions would then return the XCM object and allow the developer to encode it into hex so that it can be submitted to the network via the submitExtrinsic call.

I am currently working on an example transfer which would allow a user to transfer an asset from one parachain to another. You can see it here: https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Fbasilisk-rpc.dwellir.com#/extrinsics/decode/0x34030208000400010200e12e0500000b00e057eb481b0e010004010100411f081300010200e12e050000070010a5d4e80107001c040a5d0d010004000101008eaf04151687736326c9fea17e25fc5287613693c912909cb226aa4794f26a48001c040a5d000000

However, I am running into the following difficulties:

• How does one encode the XCM into hex? I know that this is possible as it is done on the web interface but there is very little insight into how this is actually done in practice. I have tried looking through the web portal's code but it is so abstract. I want to be able to build and encode the XCM I have in the above link.
• XCM itself is not parsable in JavaScript, it looks like JSON but it isn’t. I am struggling to understand how we can formulate such a simple XCM message in JavaScript and then encode it into hex so it can be submitted as a transaction. I know that it is using something called SCALE but I am at a loss as to how to use it and import the types available in XCM v2.

I am also curious to hear from developers as to whether they believe such a module would be useful for them. I know that it is currently very difficult to make an XCM transaction in polkadot.js so I imagine this would be a very important piece of infrastructure.

Please help.

UPDATE:

I am trying to encode my XCM message by doing the following (using the WithdrawAsset section as an example):

1. Convert the XCM into JSON const data = {"WithdrawAsset":[{"id":{[...]},"fun":{[...]}}]}
2. Define the SCALE codec object const withdrawAssetCodec = scale.object(["WithdrawAsset", scale.array(idCodex, fungibleCodex)]);
3. Encode with data const encoded = withdrawAssetCodec.encode(data);
4. Grab the hex string representation of the data and submit it via api.rpc.author.submitAndWatchExtrinsic('0x' + Buffer.from(encoded).toString('hex'));

I have hacked something together here, with the steps above: https://github.com/James-Sangalli/xcm-module-sample/blob/master/index.js#L153

Further questions:

1. If we have to represent the XCM as JSON to use the scale codec library, why isn't XCM represented as JSON? Why use a totally new format? Are we safe in assuming that this XCM: interior: { X2: [ { Parachain: 3,000 } { GeneralIndex: 0 } ] } } can be represented as this: "interior": { "X2": [ { "Parachain": 3000 }, { "GeneralIndex": 0 } ] }, in JSON?
2. How do we encode the call function signature into this XCM call? In this instance it is polkadotXcm.execute(message, maxWeight) and I noticed that in my link above, the first two bytes 3403 are a call index which tells the polkadot portal I am using the polkadotXcm.execute function.

Many of the types used in my XCM sample are not defined in the codec library, but Rob has referenced capi and there are such types defined here: https://github.com/paritytech/capi/issues/241.

It would be great if someone could include an example of a transaction that shows how we get from an XCM message to an encoded transaction that can be submitted to the network.

Answer 1

XCM messages are simply SCALE-encoded according to their specification.

SCALE is a binary encoding format which is described here: https://docs.substrate.io/reference/scale-codec/

You can use the TypeScript scale library by ParityTech here: https://github.com/paritytech/parity-scale-codec-ts

There are some examples I'll copy from the repo on encoding and decoding:

import * as $ from "https://deno.land/x/scale/mod.ts";

const $person = $.object(
  ["name", $.str],
  ["nickName", $.str],
  ["superPower", $.option($.str)],
);

const valueToEncode = {
  name: "Magdalena",
  nickName: "Magz",
  superPower: "Hydrokinesis",
};

const encodedBytes: Uint8Array = $person.encode(valueToEncode);
const decodedValue: Person = $person.decode(encodedBytes);

assertEquals(decodedValue, valueToEncode);

Encoded objects have the type Uint8Array which can be converted into hex easily. There are a variety of answers to the question of converting byte arrays to hex strings here: https://stackoverflow.com/questions/39225161/convert-uint8array-into-hex-string-equivalent-in-node-js .