v0.38.0

[0.38.0] - 2024-10-24

This release doesn't introduce any substantial breaking changes and focuses primarily on incremental improvements, testing and bug fixes. A few of the highlights include:

• #1785: Support decoding V5 extrinsics in blocks (currently Subxt will still submit V4 extrinsics). This also unifies our extrinsic decoding logic into one place.
• #1802: Stabilizing the subxt::backend::unstable::UnstableBackend (it's now called subxt::backend::chain_head::ChainHeadBackend). This backend can be used to interact with the modern chainHead RPC methods exposed by Smoldot and compliant RPC nodes. See this example.
• #1803: Stabilizing the reconnecting-rpc-client. See this example.
• #1720: A nice little QoL improvement if you have the raw runtime WASM and would like to generate an interface directly from that (ie with #[subx(runtime_path = "path/to/runtime.wasm")]).
• #1661: Support loading keys directly from the PolkadotJS JSON to be used in Subxt.
• #1638: Improve support for Eth style chains by defining a 20-byte account ID type directly in subxt-core. See this example.

The notable changes in this release are as follows:

Added

• add reconnecting tests for unstable_backend (#1765)
• add support for generating metadata from runtime wasm files (#1720)
• support loading keys from Polkadot-JS accounts (#1661)
• allow tx payloads to be boxed (#1690)
• add hash method to ExtrinsicDetails (#1676)
• expose secret_key method for ecdsa::Keypair and eth::Keypair (#1628)
• add 20-byte account id to subxt_core (#1638)

Changed

• make it clearer which extrinsic failed to decode (#1835)
• chore(deps): bump frame-metadata from 16 to 17 (#1836)
• chore(deps): bump scale family crates, primitive-types and impl-serde (#1832)
• chore(deps): replace instant with web-time (#1830)
• deps: use polkadot-sdk umbrella crate (#1786)
• stabilize reconnecting-rpc-client (#1803)
• stabilize chainhead backend (#1802)
• derive serialize on more types (#1797)
• use frame-decode for core extrinsic decode logic (including v5 support) (#1785)
• reconn-rpc-client: parse URL before connecting (#1789)
• update proc_macro_error to proc_macro_error2 (#1767)
• chore(deps): update Smoldot to the latest version (#1400)
• remove unneeded ?Sized bound and replace never type with () (#1758)
• improve test coverage for legacy Backend impl (#1751)
• add integration tests for unstable-reconnecting-rpc-client (#1711)
• replace reconnecting-jsonrpsee-ws-client with subxt-reconnecting-rpc-client (#1705)
• allow PartialExtrinsic to be held across await points (#1658)
• chore(deps): bump jsonrpsee from 0.22.5 to 0.23.1 (#1656)

Fixed

• fix stripping metadata in the case where enums like RuntimeCall are handed back (#1774)
• fix: defalt-feature -> default-features Cargo.toml (#1828)
• avoid hang by notifying subscribers when the backend is closed (#1817)
• fix: error message on rpc errors (#1804)
• docs: fix typos (#1776)
• examples: fix reconnecting logging target (#1733)
• docs: fix spelling issues (#1699)
• chore: fix some comments (#1697)
• codegen: Fix decode error by adding #[codec(dumb_trait_bound)] (#1630)

v0.37.1

[0.37.1] - 2024-10-22

This patch release adds basic support for V5 extrinsics into subxt-core. We only bump subxt-core in this release; other crates should then take this patch automatically but you may need to cargo update to get the change.

v0.37.0

[0.37.0] - 2024-05-28

This release mainly adds support for the sign extension CheckMetadataHash and fixes a regression introduced in v0.36.0
where the type de-duplication was too aggressive and lots of the same type such as BoundedVec was duplicated to
plenty of different types such as BoundedVec1, BoundedVec2, .. BoundedVec.

Added

• Implemented sign_prehashed for ecdsa::Keypair and eth::Keypair (#1598)
• Add a basic version of the CheckMetadataHash signed extension (#1590)

Changed

• Remove derive_more (#1600)
• chore(deps): bump scale-typegen v0.8.0 (#1615)

Full Changelog: v0.36.0...v0.37.0

v0.36.1

[0.36.1] - 2024-05-28 YANKED

Yanked because the typegen changed, it's a breaking change.

v0.36.0

[0.36.0] - 2024-05-16

This release adds a few new features, which I'll go over below in more detail.

subxt-core

We now have a brand new subxt-core crate, which is #[no-std] compatible, and contains a lot of the core logic that is needed in Subxt. Using this crate, you can do things in a no-std environment like:

• blocks: decode and explore block bodies.
• constants: access and validate the constant addresses in some metadata.
• custom_values: access and validate the custom value addresses in some metadata.
• metadata: decode bytes into the metadata used throughout this library.
• storage: construct storage request payloads and decode the results you'd get back.
• tx: construct and sign transactions (extrinsics).
• runtime_api: construct runtime API request payloads and decode the results you'd get back.
• events: decode and explore events.

Check out the docs for more, including examples of each case.

A breaking change that comes from migrating a bunch of logic to this new crate is that the ExtrinsicParams trait is now handed &ClientState<T> rather than a Client. ClientState is just a concrete struct containing the state that one needs for things like signed extensions.

Support for reconnecting

We've baked in a bunch of support for automatically reconnecting after a connection loss into Subxt. This comes in three parts:

1. An RPC client that is capable of reconnecting. This is gated behind the unstable-reconnecting-rpc-client feature flag at the moment, and
2. Handling in the subxt Backends such that when the RPC client notifies it that it is reconnecting, the backend will transparently handle this behind the scenes, or else pass on a DisconnectedWillReconnect error to the user where it cannot. Note that the individual LegacyRpcMethods and UnstableRpcMethods are not automatically retried on reconnection. Which leads us to..
3. A couple of util helpers (subxt::backend::retry and subxt::backend::retry_stream) which can be used in conjunction with a reconnecting RPC client to make it easy to automatically retry RPC method calls where needed.

We'd love feedback on this reconnecting work! To try it out, enable the unstable-reconnecting-rpc-client feature flag and then you can make use of this like so:

use std::time::Duration;
use futures::StreamExt;
use subxt::backend::rpc::reconnecting_rpc_client::{Client, ExponentialBackoff};
use subxt::{OnlineClient, PolkadotConfig};

// Generate an interface that we can use from the node's metadata.
#[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_small.scale")]
pub mod polkadot {}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create a new client with with a reconnecting RPC client.
    let rpc = Client::builder()
        // We can configure the retry policy; here to an exponential backoff.
        // This API accepts an iterator of retry delays, and here we use `take`
        // to limit the number of retries.
        .retry_policy(
            ExponentialBackoff::from_millis(100)
                .max_delay(Duration::from_secs(10))
                .take(3),
        )
        .build("ws://localhost:9944".to_string())
        .await?;

    // Use this reconnecting client when instantiating a Subxt client:
    let api: OnlineClient<PolkadotConfig> = OnlineClient::from_rpc_client(rpc.clone()).await?;

Check out the full example here.

Better Ethereum support

We've added built-in support for Ethereum style chains (eg Frontier and Moonbeam) in subxt-signer, making it easier to sign transactions for these chains now.

Check out a full example here.

We plan to improve on this in the future, baking in better Ethereum support if possible so that it's as seamless to use AccountId20 as it is AccountId32.

Stabilizing the new V2 RPCs (#1540, #1539, #1538)

A bunch of the new RPCs are now stable in the spec, and have consequently been stabilized here, bringing the unstable-backend a step closer to being stabilized itself! We'll probably first remove the feature flag and next make it the default backend, in upcoming releases.

All of the notable changes in this release are as follows:

Added

• Add frontier/ethereum example (#1557)
• Rpc: add full support reconnecting rpc client (#1505)
• Signer: ethereum implementation (#1501)
• subxt-core crate (#1466)

Changed

• Bump scale-decode and related deps to latest (#1583)
• Update Artifacts (auto-generated) (#1577)
• Update deps to use scale-type-resolver 0.2 (#1565)
• Stabilize transactionBroadcast methods (#1540)
• Stabilize transactionWatch methods (#1539)
• Stabilize chainHead methods (#1538)
• Rename traits to remove T suffix (#1535)
• Add Debug/Clone/etc for common Configs for convenience (#1542)
• Unstable_rpc: Add transactionBroadcast and transactionStop (#1497)

Fixed

• metadata: Fix cargo clippy (#1574)
• Fixed import in subxt-signer::eth (#1553)
• chore: fix typos and link broken (#1541)
• Make subxt-core ready for publishing (#1508)
• Remove dupe storage item if we get one back, to be compatible with Smoldot + legacy RPCs (#1534)
• fix: substrate runner libp2p port (#1533)
• Swap BinaryHeap for Vec to avoid Ord constraint issue (#1523)
• storage_type: Strip key proper hash and entry bytes (32 instead of 16) (#1522)
• testing: Prepare light client testing with substrate binary and add subxt-test macro (#1507)

v0.35.3

[0.35.3] - 2024-04-11

Another bug-fix release that substitutes BinaryHeap for Vec in the same way that we do for BTreeMap/Set to avoid issues with the Ord constraint on the generic type (because this may be a generated type, and we don't automatically apply Ord to generated types).

v0.35.2

[0.35.2] - 2024-04-09

This is a small patch release that fixes the storage key decoding. Previously, we assumed the length of the hash of the storage prefix or entry name was 8 bytes, however it is 16.

v0.35.1

[0.35.1] - 2024-04-03

This is a small patch release that adds support for BinaryHeap in the codegen/subxt-cli
which wasn't supported by scale-typegen. This was discovered because the most recent metadata in polkadot
are now utilizing BinaryHeap.

v0.35.0

[0.35.0] - 2024-03-21

This release contains several fixes, adds no_std support to a couple of crates (subxt-signer and subxt-metadata) and introduces a few quality of life improvements, which I'll quickly cover:

Reworked light client (#1475)

This PR reworks the light client interface. The "basic" usage of connecting to a parachain now looks like this:

#[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_small.scale")]
pub mod polkadot {}

use subxt::lightclient::LightClient;

// Instantiate a light client with the Polkadot relay chain given its chain spec.
let (lightclient, polkadot_rpc) = LightClient::relay_chain(POLKADOT_SPEC)?;
// Connect the light client to some parachain by giving a chain spec for it.
let asset_hub_rpc = lightclient.parachain(ASSET_HUB_SPEC)?;

// Now, we can create Subxt clients from these Smoldot backed RPC clients:
let polkadot_api = OnlineClient::<PolkadotConfig>::from_rpc_client(polkadot_rpc).await?;
let asset_hub_api = OnlineClient::<PolkadotConfig>::from_rpc_client(asset_hub_rpc).await?;

This interface mirrors the requirement that we must connect to a relay chain before we can connect to a parachain. It also moves the light client specific logic into an RpcClientT implementation, rather than exposing it as a subxt::client::LightClient.

Typed Storage Keys (#1419)

This PR changes the storage interface so that, where possible, we now also decode the storage keys as well as the values when iterating over storage entries:

#[subxt::subxt(runtime_metadata_path = "../artifacts/polkadot_metadata_small.scale")]
pub mod polkadot {}

// Create a new API client, configured to talk to Polkadot nodes.
let api = OnlineClient::<PolkadotConfig>::new().await?;

// Build a storage query to iterate over account information.
let storage_query = polkadot::storage().system().account_iter();

// Get back an iterator of results (here, we are fetching 10 items at
// a time from the node, but we always iterate over one at a time).
let mut results = api.storage().at_latest().await?.iter(storage_query).await?;

while let Some(Ok(kv)) = results.next().await {
    // We used to get a tuple of key bytes + value. Now we get back a
    // `kv` struct containing the bytes and value as well as the actual
    // decoded keys:
    println!("Decoded key(s): {:?}", kv.keys);
    println!("Key bytes: 0x{}", hex::encode(&kv.key_bytes));
    println!("Value: {:?}", kv.value);
}

When using the static interface, keys come back as a tuple of values corresponding to the different hashers used in constructing the key. When using a dynamic interface, keys will be encoded/decoded from the type given so long as it implements subxt::storage::StorageKey, eg Vec<scale_value::Value>.

Extrinsic Params Refinement (#1439)

Prior to this PR, one could configure extrinsic signed extensions by providing some params like so:

// Configure the transaction parameters; we give a small tip and set the
// transaction to live for 32 blocks from the `latest_block` above:
let tx_params = Params::new()
    .tip(1_000)
    .mortal(latest_block.header(), 32)
    .build();

let hash = api.tx().sign_and_submit(&tx, &from, tx_params).await?;

If you want to customize the account nonce, you'd use a different call like create_signed_with_nonce instead.

One of the downsides of the above approach is that, if you don't provide any explicit params, transactions will be immortal by default (because the signed extensions didn't have the information to do any better).

Now, with the help of a RefineParams trait, transactions will default to being mortal and living for 32 blocks unless an explicit mortality is provided as above.

One notable change is that the offline-only create_signed_with_nonce and create_partial_signed_with_nonce functions have lost the _with_nonce suffix. Since we can't discover nonce/mortality settings offline, you should now provide Params and set an explicit nonce (and mortality, if you like) when using these calls, otherwise the nonce will be set to 0 and the mortality to Immortal.

For a full list of changes, please see the following:

Added

• Reworked light client (#1475)
• no_std compatibility for subxt-signer (#1477)
• Typed Storage Keys (#1419)
• Extrinsic Params Refinement (#1439)
• Make storage_page_size for the LegacyBackend configurable (#1458)
• no_std compatibility for subxt-metadata (#1401)
• Experimental reconnecting-rpc-client (#1396)

Changed

• scale-type-resolver integration (#1460)
• subxt: Derive std::cmp traits for subxt payloads and addresses (#1429)
• CLI: Return error on wrongly specified type paths (#1397)
• rpc v2: chainhead support multiple finalized block hashes in FollowEvent::Initialized (#1476)
• rpc v2: rename transaction to transactionWatch (#1399)

Fixed

• Avoid a panic in case we try decoding naff bytes (#1444)
• Fix error mapping to wrong transaction status (#1445)
• Update DispatchError to match latest in polkadot-sdk (#1442)
• Handle errors when fetching storage keys from Unstablebackend (#1440)
• Swap type aliases around to be semantically correct (#1441)

v0.34.0

[0.34.0] - 2024-01-23

This release introduces a bunch of features that make subxt easier to use. Let's look at a few of them.

Codegen - Integrating scale-typegen and adding type aliases (#1249)

We rewrote the code generation functionality of subxt and outsourced it to the new scale-typegen crate, which serves a more general purpose.

Since a lot of types used in substrate are rich with generics, this release introduces type aliases into the generated code.
A type alias is generated for the arguments/keys or each call, storage entry, and runtime API method (#1249).

Macro - Errors for misspecified type paths (#1339)

The subxt macro provides attributes to specify custom derives, attributes, and type substitutions on a per-type basis.
Previously we did not verify that the provided type paths are part of the metadata. This is now fixed:
If you provide an invalid type path, the macro will tell you so. It also suggests similar type paths, you might have meant instead.

#[subxt::subxt(
    runtime_metadata_path = "metadata.scale", 
    derive_for_type(path = "Junctions", derive = "Clone")
)]
pub mod polkadot {}

This gives you a compile-time error like this:

Type `Junctions` does not exist at path `Junctions`

A type with the same name is present at: 
xcm::v3::junctions::Junctions
xcm::v2::multilocation::Junctions

Macro - Recursive derives and attributes (#1379)

Previously adding derives on a type containing other types was also cumbersome, see this example:

#[subxt::subxt(
    runtime_metadata_path = "metadata.scale",
    derive_for_type(path = "xcm::v2::multilocation::MultiLocation", derive = "Clone"),
    derive_for_type(path = "xcm::v2::multilocation::Junctions", derive = "Clone"),
    derive_for_type(path = "xcm::v2::junction::Junction", derive = "Clone"),
    derive_for_type(path = "xcm::v2::NetworkId", derive = "Clone"),
    derive_for_type(path = "xcm::v2::BodyId", derive = "Clone"),
    derive_for_type(path = "xcm::v2::BodyPart", derive = "Clone"),
    derive_for_type(
        path = "bounded_collections::weak_bounded_vec::WeakBoundedVec",
        derive = "Clone"
    )
)]
pub mod polkadot {}

We introduced a recursive flag for custom derives and attributes that automatically inserts the specified derives on all child types:

#[subxt::subxt(
    runtime_metadata_path = "metadata.scale",
    derive_for_type(path = "xcm::v2::multilocation::MultiLocation", derive = "Clone", recursive),
)]
pub mod polkadot {}

Subxt CLI - New features and usability improvements (#1290, #1336, and #1379)

Our CLI tool now allows you to explore runtime APIs and events (#1290). We also fully integrated with scale-typegen-description, a crate that can describe types in a friendly way and provide type examples. The output is also color-coded to be easier on the eyes. Get started with these commands:

# Show details about a runtime API call:
subxt explore --url wss://westend-rpc.polkadot.io api StakingAPI nominations_quota
# Execute a runtime API call from the CLI:
subxt explore --url wss://westend-rpc.polkadot.io api core version -e 
# Discover what events a pallet can emit:
subxt explore --url wss://westend-rpc.polkadot.io pallet Balances events

All CLI commands that take some metadata via --file or --url, can now also read the metadata directly from stdin with --file - (#1336).
This allows you to pipe in metadata from other processes like in this command chain:

parachain-node export-metadata | subxt codegen --file - | rustfmt > main.rs

Similar to the macro, the subxt codegen command can now also use recursive flags:

subxt codegen --derive-for-type xcm::v2::multilocation::MultiLocation=Clone,recursive
subxt codegen --attributes-for-type "xcm::v2::multilocation::MultiLocation=#[myerror],recursive"

Minor changes and things to be aware of

• Using insecure connections is now an explicit opt-in in many places (#1309)
• When decoding extrinsics from a block into a static type, we now return it's details (e.g. signature, signed extensions, raw bytes) alongside the statically decoded extrinsic itself (#1376)

We also made a few fixes and improvements around the unstable backend and the lightclient, preparing them for more stable usage in the future.

Added

• Errors for misspecified type paths + suggestions (#1339)
• CLI: Recursive derives and attributes (#1379)
• CLI: Explore runtime APIs and events, colorized outputs, scale-typegen integration for examples (#1290)
• Add chainflip to real world usage section of README (#1351)
• CLI: Allow using --file - to read metadata from stdin (#1336)
• Codegen: Generate type aliases for better API ergonomics (#1249)

Changed

• Return Pending rather than loop around if no new finalized hash in submit_transaction (#1378)
• Return ExtrinsicDetails alongside decoded static extrinsics (#1376)
• Improve Signed Extension and Block Decoding Examples/Book (#1357)
• Use scale-typegen as a backend for the codegen (#1260)
• Using insecure connections is now opt-in (#1309)

Fixed

• Ensure lightclient chainSpec is at least one block old (#1372)
• Typo fix in docs (#1370)
• Don't unpin blocks that may show up again (#1368)
• Runtime upgrades in unstable backend (#1348)
• Generate docs for feature gated items (#1332)
• Backend: Remove only finalized blocks from the event window (#1356)
• Runtime updates: wait until upgrade on chain (#1321)
• Cache extrinsic events (#1327)