Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

TXN 708: Async Commit and 1PC

TXN 404 introduced the classic two-phase commit path. TiDB first prewrites every key, then commits the primary key before it can report that the transaction has succeeded.

This chapter covers two faster commit paths:

  • Async Commit lets TiDB return success once all prewrites complete, because their persisted state makes one outcome recoverable.
  • One-phase commit (1PC) commits the transaction during prewrite when it involves only one Region.

Async Commit

Classic 2PC uses the primary key as the source of truth for the transaction's outcome. Before TiDB reports success, it must send a commit request that turns the primary lock into a committed write CF record.

Async Commit makes the outcome recoverable from the persisted prewrite state instead. A prewrite records a transaction lock and any value data required by each mutation it applies. The locks carry two additional pieces of information:

  • Each lock records a minimum commit timestamp, or min_commit_ts.
  • The primary lock additionally records the secondary keys that recovery must check.

The min_commit_ts is a lower bound, not yet the transaction's final commit_ts.

Suppose one transaction writes three keys in different Regions:

primary:   account/alice
secondary: account/bob
secondary: audit/9001

TiDB sends the prewrites, and each TiKV Region returns the minimum commit timestamp required by its part of the transaction:

account/alice -> min_commit_ts 24
account/bob   -> min_commit_ts 26
audit/9001    -> min_commit_ts 25

After every prewrite succeeds, TiDB chooses the largest value as the transaction's commit_ts:

commit_ts = max(24, 26, 25) = 26

At this point, the transaction is considered committed. TiDB can report success without waiting for a separate request to commit the primary key. It sends the commit requests in the background instead.

The foreground step is safe because the persisted locks contain enough information to recover one final outcome. If the original TiDB disappears, a later lock resolver reads the secondary-key list from the primary and checks those locks.

If every required lock remains, the resolver commits them using the largest min_commit_ts. Otherwise, it checks the missing key's write CF record. A commit record supplies the transaction's commit_ts; a rollback record, or no lock and no commit record, means the transaction must roll back. The resolver gives every remaining lock that same outcome.

The important change is not merely that the commit RPC runs later. The prewrite state now contains the evidence needed to finish the transaction without the original TiDB coordinator.

One-Phase Commit

Async Commit still prewrites locks because its keys may span several Regions. In contrast, 1PC writes no persistent transaction locks. TiDB attempts it only when all mutations fit in one prewrite batch for one Region, where TiKV can write the committed MVCC records directly.

1PC prewrite request
        |
        v
one Region checks every mutation
        |
        v
one Raft command writes:
  default CF <- values, when needed
  write CF   <- committed records
        |
        v
return commit_ts

Despite the API still being called prewrite, a successful 1PC request has already committed the transaction, and TiDB sends no separate commit request.

If TiDB must split the mutations into several batches, it uses classic 2PC instead. Even after receiving a 1PC request, TiKV can fall back to ordinary 2PC locks when it cannot choose a valid 1PC commit timestamp. TiDB then continues with classic 2PC.

The Concurrency Manager

Both protocols calculate timestamp constraints during prewrite. For Async Commit, each prewrite batch returns a lower bound and TiDB chooses the largest as the final commit_ts. For 1PC, TiKV chooses the final commit_ts in that prewrite. A concurrent read may begin while that work is still moving through Raftstore. During this interval, TiKV may have started the write while neither its lock nor its committed record is visible in RocksDB.

TiKV must prevent this result:

a write will use commit_ts 20
read starts at read_ts 25
read takes a RocksDB snapshot before the transaction reaches RocksDB
read misses a version that should be visible at timestamp 25

The concurrency manager closes this gap with two pieces of local, in-memory state:

  • max_ts records the largest timestamp observed by this TiKV store.
  • An in-memory lock table records transaction locks that are being written but are not yet safely visible in RocksDB.

Before a read takes its RocksDB snapshot, it updates max_ts with its read_ts and checks the in-memory lock table.

If the read arrives first, a later prewrite sees the updated max_ts and chooses a min_commit_ts greater than it:

read                              prewrite
----                              --------
update max_ts to 25
                                  choose min_commit_ts > 25
take RocksDB snapshot

max_ts itself exists only in memory. The chosen min_commit_ts is copied into the lock created by the prewrite: first into the in-memory lock table, then into the persisted lock CF when Raft applies the prewrite. TiKV returns the prewrite result only after that apply. A later resolver reads those persisted lower bounds and does not depend on the old max_ts value.

The transaction will become visible after the read's timestamp, so the read can safely return the older version.

If the prewrite arrives first, it places its lock in the in-memory table before the RocksDB write completes:

prewrite                          read
--------                          ----
install in-memory lock
                                  update max_ts to 25
                                  find the in-memory lock
write the state to RocksDB
remove the in-memory lock

The read checks the pending transaction instead of silently reading past it. If the transaction could be visible at the read's timestamp, the lock conflicts with the read; if its min_commit_ts is later, the read can safely use the older version. Once RocksDB contains the lock or committed record, TiKV removes the in-memory entry and the normal MVCC read path takes over.

The three commit paths reach the same durable result through different foreground work:

Classic 2PC   prewrite locks -> commit primary -> return success
Async Commit prewrite locks and recovery metadata -> return success
1PC          write committed records in one Region -> return success

Every successfully committed write transaction is eventually represented by committed records in the write CF. The difference is how soon TiDB can know that the result is final, and what persisted information makes that conclusion safe.