Skip to content
TephraTephra

Getting started

This page runs start to finish against a real server. Everything after “Run the server” uses the Rust tephra-client crate; the Go and JavaScript clients cover the same operations, so see Clients if you are working in one of those.

Run the server

Section titled “Run the server”

The server is a synchronous, thread-per-connection TCP server. There are four ways to start it, none needing a checkout of the workspace. Pick whichever fits how you already run things.

Terminal window
docker run -p 9000:9000 -v tephra-data:/data git.tqwewe.com/tephra/tephra:latest

Run through Nix, Cargo, or a prebuilt binary, the server binds 127.0.0.1:9000 and writes its log and index segments under tephra-data in the working directory. The Docker image publishes on 0.0.0.0:9000 and keeps its data in the /data volume, mapped above to a named tephra-data volume. Bind address, data directory, and everything else are configurable by flag, environment variable, or a TOML file: see Operations for the full surface. The defaults, including the 256 MiB segment size, are the server’s, not the library’s.

Add the client

Section titled “Add the client”

In the project that will talk to the server:

Terminal window
cargo add tephra-client

Connect

Section titled “Connect”

Client::connect takes anything that resolves to a socket address and opens one TCP connection. A Client is not shared between threads: give each thread its own.

let mut client = Client::connect(addr)?;

For an encrypted connection use Client::connect_tls, and for a server that requires a bearer token use the *_with variants; see transport security and authentication.

Append

Section titled “Append”

An event is a type, a set of tags, and an opaque payload. append takes an iterator of events and an optional condition; here the condition is None, so the write is unconditional. It returns the position range the batch landed at.

let event = Event::new(
    "CourseOpened",
    ["course:c1"],
    br#"{"course":"c1","seats":30}"#.to_vec(),
)?;
let result = client.append([event], None)?;
println!("recorded positions {} to {}", result.first, result.last);

The client’s Event is a friendly owned type (built from &str, an iterator of tag strings, and bytes). The engine has a second, packed Event used only when you embed the engine directly; the two do not interconvert, and the client type is the one you want here. See Embedded for the other one.

Read

Section titled “Read”

read and read_all take a query, an after position, and a limit. Position::ZERO means “from the start of the log”, and a limit of None reads the whole matching history. Results come back in ascending position order.

let query = Query::item(QueryItem::with_tags(Tags::new([Tag::new("course:c1")?])?));
// `None` reads the whole matching history; pass `Some(n)` to cap the number of events.
let (events, _watermark) = client.read_all(query, Position::ZERO, None)?;
for seq in &events {
    println!("{} {}", seq.position(), seq.event().event_type());
}

Because the writer publishes each commit before the append reply returns, a client sees its own writes immediately: the read above returns the event the append above just recorded.

Paginate

Section titled “Paginate”

Pass Some(n) as the limit to cap how many events a read returns. The cap is applied on the server during planning, so reading the first ten events of an entity does the work of ten, not of its whole history. Pair limit with after to page through a large result: each page resumes at the last position seen, with no gap and no duplicate at the seam. A page shorter than the limit is the last one.

// `limit` caps how many events a read returns. Pair it with `after` (an exclusive lower
// bound) to page through a large result: each page resumes at the last position seen, with
// no gap and no duplicate at the seam. A page shorter than the limit is the last one.
let query = Query::item(QueryItem::with_tags(Tags::new([Tag::new("course:c1")?])?));
let page_size = 2;
let mut after = Position::ZERO;
let mut seen = 0usize;
loop {
    let (page, _watermark) = client.read_all(query.clone(), after, Some(page_size))?;
    let short = (page.len() as u64) < page_size;
    if let Some(last) = page.last() {
        after = last.position();
    }
    seen += page.len();
    if short {
        break;
    }
}

Read newest-first

Section titled “Read newest-first”

read_back and read_all_back are the newest-first duals of read and read_all: they return matching events in descending position order. Where a forward read takes after (an exclusive lower bound), a backward read takes before (an exclusive upper bound), so Position::MAX starts at the tip. This is what an event explorer showing recent activity first wants.

// `read_back` / `read_all_back` are the newest-first duals of `read` / `read_all`: they
// return matching events in descending position order. `before` is an exclusive upper bound,
// so `Position::MAX` starts at the tip. Pair it with a `limit`, then pass the oldest position
// of a page back as the next `before`, to page an event explorer newest-first.
let query = Query::item(QueryItem::with_tags(Tags::new([Tag::new("course:c1")?])?));
let (newest, _watermark) = client.read_all_back(query, Position::MAX, Some(2))?;
let newest_positions: Vec<u64> = newest.iter().map(|seq| seq.position().get()).collect();

Pagination is symmetric: pass a limit for the page size, then set the next before to the oldest position the page returned. The cap is applied on the server during planning, so a page near the tip does the work of a page, not of the whole history.

Subscribe

Section titled “Subscribe”

A subscription catches up on history and then tails the live edge, as one loop. It yields events in order, then a CaughtUp marker each time it reaches the live edge. The marker is where a read model can report that it is current.

// Subscribe from the start: catch up on history, then a CaughtUp marker at the live edge.
let query = Query::item(QueryItem::with_tags(Tags::new([Tag::new("course:c1")?])?));
let (mut stream, cancel) = client.subscribe(query, Position::ZERO)?;
for item in &mut stream {
    match item? {
        SubEvent::Event(seq) => {
            println!("live {} {}", seq.position(), seq.event().event_type());
        }
        // Reached the live edge. A long-running consumer keeps going; this example stops.
        SubEvent::CaughtUp(_) => break,
    }
}
cancel.cancel();

A subscription does not end on its own: drop the stream, or call cancel on the paired SubscribeCancel, to close it.

Check server stats

Section titled “Check server stats”

stats returns a point-in-time snapshot of the server: the durable event count, the on-disk size, uptime, and the live connection and subscription counts. It is the same data the --healthcheck probe reads.

// A point-in-time snapshot of the server: the durable event count (also the tip position),
// the on-disk size, uptime, and the live connection and subscription counts.
let stats = client.stats()?;
println!(
    "{} events across {} segments, {} bytes on disk",
    stats.event_count, stats.segment_count, stats.disk_bytes
);

Next

Section titled “Next”

Core concepts is the precise definition of events, tags, positions, and query semantics. Guides builds a real decision model and a read model on top of what is here.