Skip to main content

· 5 min read
Ovais Tariq

Tigris globally distributed object
storage [src: playground.com]

Eighteen years ago today, Amazon completely changed how developers work with data storage by giving us Simple Storage Service (S3).

S3 rewrote the rules of storage and propelled us into a new era of cloud computing. Traditional storage solutions were cumbersome and costly, and they shackled developers to the limitations of the hardware. With S3, Amazon introduced a shift towards Storage as a Service, liberating developers from the burdensome tasks of purchasing, provisioning, and managing physical storage. No longer were they bound by the precarious dance of capacity planning, where overestimating meant wasted resources and underestimating spelled disaster for uptime.

· 4 min read
Ovais Tariq

Hello, world! We're Tigris Data, and today we're announcing the public beta of Tigris. Tigris is a globally distributed object storage service that provides low latency anywhere in the world, enabling developers like you to store and access any amount of data using the S3 libraries you're already using in production. Today, we're launching our public beta on top of Fly.io.

Tigris globally distributed object
storage [Midjourney prompt: tiger face, illustrated in binary code, blue and white.]

· 6 min read
Himank Chaudhary

Tigris is a globally available, S3 compatible object storage service.

Tigris uses FoundationDB's transactional key-value interface as its underlying storage engine. In our blog post Skipping the boring parts of building a database using FoundationDB we went into the details of why we chose to build on FoundationDB. To recap, FoundationDB is an ordered, transactional, key-value store with native support for multi-key strictly serializable transactions across its entire keyspace. We leverage FoundationDB to handle the hard problems of durability, replication, sharding, transaction isolation, and load balancing so we can focus on higher-level concerns.

We are starting a series of blog posts that go into the details of how Tigris has been implemented. In the first post of the series, we will share the details of how we have built the metadata storage layer on top of FoundationDB. We will cover the topics of data layout, and schema management.

How we architected Tigris

· One min read
Garren

I did a presentation at the FoundationDB Monthly Meetup. The talk is about, how building secondary indexes for a database is always about balance. A balance between a system that scales and is easy to manage and an API that is intuitive and delightful for a developer to use. Recently at Tigris Data, we have been adding secondary indexes to our metadata store and have been working hard to achieve a good balance between scale and developer delight. Tigris is a storage platform that leverages FoundationDB as one of its core components. In this talk I cover four aspects we had to balance:

  1. Handling schema changes automatically in our secondary indexes.
  2. The trade-off between auto-indexing all fields and indexing select fields.
  3. Changes we made after performance testing.
  4. How we plan build indexes in the background with minimal conflicts.

· 4 min read
Jigar Joshi

CSE

Client-side encryption refers to the practice of encrypting data on the client side (user's device) before it is transmitted to a server or stored in a remote location. This approach enhances the security and privacy of user data by ensuring that sensitive information is encrypted locally, and only the encrypted form is transferred or stored on the server.

Here's how client-side encryption typically works:

CSE_Flow

· 9 min read
Peter Boros

We are running FoundationDB with the official kubernetes operator. FoundationDB supports logical backups (with backup_agent) and disaster recovery (with dr_agent) through copying the database/streaming changes logically. It also supports binary backups through disk snapshots.

In this blog post, we will describe how to make a backup of FoundationDB via backup_agent. The FoundationDB operator supports making logical backups via backup_agent, but it does not support running DR with dr_agent. We decided to run backup_agent as a separate deployment to allow a symmetric setup with dr_agent.

· 3 min read
Himank Chaudhary

Tigris is a globally available, S3 compatible object storage service. Tigris utilizes FoundationDB's transactional key-value interface for its underlying metadata storage. This blog delves into the topics of serializable transactions, the mechanics of transactions within Tigris, and concurrency control.

· 9 min read
Himank Chaudhary
Ovais Tariq

Building a new storage platform

The most complicated and time-consuming parts of building a new storage system are usually the edge cases and low-level details. Concurrency control, consistency, handling faults, load balancing, that kind of thing. Almost every mature storage system will have to grapple with all of these problems at one point or another. For example, at a high level, load balancing hot partitions across brokers in Kafka is not that different from load balancing hot shards in MongoDB, but each system ends up re-implementing a custom load-balancing solution instead of focusing on their differentiating value to end-developers.

This is one of the most confusing aspects of the modern data infrastructure industry, why does every new system have to completely rebuild (not even reinvent!) the wheel? Most of them decide to reimplement common processes and components without substantially increasing the value gained from reimplementing them. For instance, many database builders start from scratch when building their own storage and query systems, but often merely emulate existing solutions. These items usually take a massive undertaking just to get basic features working, let alone correct.