Skip to main content

Archival Store and Service (Beta)

⚙️Early-Stage Feature

This content describes an alpha/beta feature or service. These early stage features and services are in active development, so details are likely to change.

This feature or service is currently available in

  • Testnet
  • Mainnet

The Archival Store and Service provide a scalable, consistent foundation for accessing historical data on Sui beyond what Full nodes or indexer databases typically retain. They power rich developer experiences through gRPC and GraphQL APIs while keeping storage and operations manageable.

note

GraphQL RPC will integrate with the Archival Service after the GraphQL RPC and General-purpose Indexer beta launch. Until then, the GraphQL RPC beta will only be available as a hosted service from the Sui Foundation, and connects directly to the Archival Store (Bigtable) operated by the Sui Foundation.

info

Refer to Access Sui Data for an overview of options to access Sui network data.

Overview​

  • Archival Store: A long-term storage system (e.g., Bigtable) that holds checkpoint-indexed Sui data.
  • Archival Service: A gRPC API (LedgerService) layer that exposes access to this store, enabling point lookups of historical data.

Together, they let developers and data providers access older transactions, object versions, and checkpoints that Full nodes no longer store.

When and why to use​

Though their goals and usage patterns differ, both developers and RPC / data providers benefit from using the Archival Store and Service.

Developers​

  • Access historical data for rendering UI features like transaction history, object lineage, and previous app states.
  • Rely on hosted archival services for long-term data to avoid infrastructure overhead.
  • Implement fallback mechanisms in apps such that they can query archival data if Full nodes or indexer databases return insufficient retention.

RPC and data providers​

  • Operate a differentiated infrastructure service by offering historical data access to developers.
  • Extend the retention horizon of GraphQL or gRPC-based APIs by using the Archival Service.
  • Maintain operational independence by running your own Archival Store and Service (using Bigtable or a custom backend).

Accessing the Archival Service​

Developers​

Access the Archival Service hosted by any participating provider. Sui Foundation offers a public good service on a Bigtable-based store. This service has strict rate limits:

  • Testnet: archive.testnet.sui.io
  • Mainnet: archive.mainnet.sui.io

RPC and data providers​

Operate the Archival Store and Service yourself:

  • Deploy the Archival Service binary.
  • Maintain a store like Bigtable.
  • Populate the store with checkpoints using an ingestion mechanism.

Refer to populating the Archival Store and running the Archival Service.

Populating the Archival Store​

Run an indexer pipeline to keep the Archival Store updated. Sui Foundation provides an ingestion mechanism for Bigtable:

  • Tool: sui-kv-store
  • Run the following script to initialize the Bigtable cluster:
$ ./crates/sui-kvstore/src/bigtable/init.sh <name_of_bigtable_cluster>
  • Run the ingestion daemon with:
$ cargo run --bin sui-kvstore <name_of_your_bigtable_cluster> ingestion <mainnet|testnet>

If you're using a different storage backend:

Running the Archival Service​

The Archival Service implements the gRPC API's LedgerService. Any implementation must support this interface.

  • Reference implementation for Bigtable: sui-kv-rpc.
  • Deploy the service independently or colocated with other infrastructure.

Integration points​

gRPC APIs​

  • Apps can query the Archival Service directly when gRPC on Full nodes returns retention-related errors. This approach provides a graceful fallback without requiring the app or Full node to store historical data itself.
  • Access the Archival Service in the same way you access gRPC LedgerService APIs on a Full node. Replace the Full node URL with the Archival Service endpoint.

GraphQL RPC​

  • GraphQL RPC will delegate historical lookups to the Archival Service using the planned integration after the upcoming beta release. It will enable cost-efficient setup of a General-purpose Indexer database with limited retention.