Skip to main content

Using the Archival Store and Service (Beta)

This guide provides practical examples using the Archival Store and Service. For core concepts, see the corresponding concepts page.

For developers, access an Archival Service hosted by any participating provider. The Sui Foundation offers an Archival Store and Service as a public good on a Bigtable-based store through the endpoints:

  • Testnet: archive.testnet.sui.io
  • Mainnet: archive.mainnet.sui.io

This service has strict rate limits.

Integration points​

  • gRPC APIs: Apps can query the Archival Service directly when gRPC on full nodes returns retention-related errors. This approach provides a graceful fallback without requiring the app or full node to store historical data itself. Access the Archival Service in the same way you access gRPC LedgerService APIs on a full node. Replace the full node URL with the Archival Service endpoint.

  • GraphQL RPC: GraphQL RPC delegates historical lookups to the Archival Service using the planned integration. It enables cost-efficient setup of a General-purpose Indexer database with limited retention.

Operating the Archival Store and Service​

For RPC and data providers, operate the Archival Store and Service yourself:

  • Deploy the Archival Service binary.
  • Maintain a store like Bigtable.
  • Populate the store with checkpoints using an ingestion mechanism.

Populating the Archival Store​

Run an indexer pipeline to keep the Archival Store updated. Sui Foundation provides an ingestion mechanism for Bigtable:

  1. Download and install the sui-kv-store tool.

  2. Run the following script to initialize the Bigtable cluster:

$ ./crates/sui-kvstore/src/bigtable/init.sh <name_of_bigtable_cluster>
  1. Run the ingestion daemon with:
$ cargo run --bin sui-kvstore <name_of_your_bigtable_cluster> ingestion <mainnet|testnet>

If you are using a different storage backend, build a compatible indexer using the custom indexing framework and ensure it emits data in a structure that is compatible with the gRPC API's LedgerService endpoints.

Running the Archival Service​

The Archival Service implements the gRPC API's LedgerService. Any implementation must support this interface. See the reference implementation for Bigtable: sui-kv-rpc, deploy the service independently, or deploy it colocated with other infrastructure.

•
Bigtable

Low-latency, Cassandra, and HBase-compatible NoSQL database service for machine learning, operational analytics, and user-facing applications.