1

It is not clear how to use pagination in py-substrate-interface. In the sample code below, there are only 32 records so all the query should be complete, but result.loading_complete shows false.

from substrateinterface import SubstrateInterface

url = 'wss://shiden.api.onfinality.io/public-ws'
substrate = SubstrateInterface(url)

result = substrate.query_map('Contracts', 'CodeStorage', block_hash = None, page_size = 100, ignore_decoding_errors = True)
start_key = result.last_key
print(f"loading complete {result.loading_complete} | size {len(result.records)} | next start key {start_key}")

When I do want to query the next page, I am running the same query_map call with the added start_key parameter. Is that correct, because there is also a retrieve_next_page call in the result object, but I can't figure out how to use it.

result = substrate.query_map('Contracts', 'CodeStorage', block_hash = None, page_size = 100, start_key = start_key, ignore_decoding_errors = True)

Here are the relevant components of the results object: ..., 'block_hash', 'current_index', 'ignore_decoding_errors', 'last_key', 'loading_complete', 'max_results', 'module', 'page_size', 'params', 'records', 'retrieve_next_page', 'storage_function', 'substrate']

So unless I am misunderstanding something, loading_complete is not correctly reporting when it is done and retrieve_next_page doesn't seem to do what it implies. Also, the documentation could be improved by adding some examples, especially one that includes pagination. Nonetheless, py-substrate-interface is a very handy tool.

1

1 Answer 1

2

The easiest way to handle query_map is use it as an iterator instead of manual last-key tracking.
For example all accounts of System::Account for Kusama:

from substrateinterface import SubstrateInterface

substrate = SubstrateInterface(
    url="wss://kusama-rpc.polkadot.io",
)

query = substrate.query_map('System', 'Account', page_size=1000)

for (i, (account, account_info)) in enumerate(query):
    print(f"#{i} '{account.value}': {account_info.value['data']['free']}")

You can play around with the page size and see that it prints way more than 1000 accounts.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.