Storage tiers

NOTE - Storage tiers are in early access. The tier names, storage methods, and technologies used are subject to change as this feature is further developed.

A single storage tier is a group of storage methods with similar access properties when retrieving seismic data from an ingested file.

The available storage methods within a single storage tier is pre-configured on a per tenant basis.

When a file is ingested, a storage tier can be specified as the target for that specific file. An appropriate storage method within that tier (currently predetermined based on tenant configuration) will be used for the ingestion of that file.

To move a dataset to another storage tier, order a new ingestion using the regular ingest API, specifying the required storage tier in the API call. File data will be temporarily unavailable until re-ingestion is complete.

Currently there are two available storage tiers:

Tier one

Tier one focuses on fast lookup speeds when retrieving seismic data from an ingested file. This is done by persisting the individual traces from a SEG-Y file into a clustered database.

This can be considered “hot storage”.

Tier one is the default storage tier when ingesting files.

Currently the only available storage method in tier one is BigTable.

Tier two

Tier two focuses on allowing for the retrieval of seismic data from the original SEG-Y file instead of persisting that data in a clustered database. This trades off trace retrieval performance for a reduction in storage costs.

This can be considered “cold storage”. Lookup times will be slower than tier one (approximately 30% lower throughput than tier one, varying by query type).

Currently the only available storage method in tier two is Google cloud storage.

When a file is ingested with tier two, it is assumed that that file will not be moved or renamed in its original bucket, otherwise re-ingestion is necessary. If the file is removed, all requests to access data from the file will fail until file location metadata have been updated and the file has been re-ingested.

Examples

# Ingest the file with tier one storage (default):
client.file.ingest(file_id="example.sgy")

# Ingest the file with tier two storage:
client.file.ingest(file_id="example.sgy", target_tier_name="tier2_cloudstorage")