Skip to main content

Custom Indexing Framework

The custom indexing framework exposes specific interfaces that you implement to define your data processing logic. Some common APIs include:

  • process(): Transform raw checkpoint data (transactions, events, object changes) into your desired database rows. This is where you extract meaningful information, filter relevant data, and format it for storage.

  • commit(): Store your processed data to the database with proper transaction handling. The framework calls this with batches of processed data for efficient bulk operations.

  • prune(): Clean up old data based on your retention policies (optional). Useful for managing database size by removing outdated records while preserving recent data.

Sequential and concurrent pipeline types and their trade-offs are detailed in Pipeline Architecture.

Click to open

CheckpointData struct

crates/sui-types/src/full_checkpoint_content.rs
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct CheckpointData {
    pub checkpoint_summary: CertifiedCheckpointSummary,
    pub checkpoint_contents: CheckpointContents,
    pub transactions: Vec<CheckpointTransaction>,
}
•
Build a Custom Indexer

Build a custom indexer using the sui-indexer-alt-framework module. The example indexer demonstrates a sequential pipeline that extracts transaction digests from Sui checkpoints and stores them in a local PostgreSQL.

•
Custom Indexer and Walrus

Walrus is a content-addressable storage protocol, where data is retrieved using a unique identifier derived from the content itself, rather than from a file path or location. Integrating a custom Sui Indexer with Walrus can provide novel user experiences.

•
Bring Your Own Store (BYOS)

Implement a custom storage backend for the customer indexer framework.