@siglo/postgres Adapter Guide
@siglo/postgres provides an enterprise ACID-compliant persistence, event auditing, distributed locking, and idempotency coordination layer for Siglo workflows using PostgreSQL. It is engineered for mission-critical transactional workflows requiring zero state loss across crashes, network partitions, and multi-node worker clusters.
1. Overview & Architecture
When workflows coordinate financial, compliance, or enterprise operations, persistence must survive node failures and process crashes.@siglo/postgres implements two primary classes:
PostgresStorageAdapter: Implements the SigloStorageAdapterinterface for durable execution checkpoints with Optimistic Concurrency Control (OCC), immutable context mutation audit logging, durable asynchronous wait/continuation records, and distributed session-level advisory workflow locking.PostgresIdempotencyStore: Implements the SigloIdempotencyStoreinterface for atomic, distributed single-flight execution claims, cached result retrieval, and automated lock re-claiming upon TTL expiration.
2. Installation & Requirements
Install@siglo/postgres alongside @siglo/core and the pg driver:
[!WARNING] Connection Pooling & Session Advisory Locks (PgBouncer):PostgresStorageAdapteruses PostgreSQL session-level advisory locks (pg_try_advisory_lock) to coordinate single-flight workflow execution across distributed worker nodes. These locks are tied to the physical PostgreSQL backend connection. If deploying behind connection proxies such as PgBouncer, configurepool_mode = session. Transaction-level pooling (pool_mode = transaction) causes advisory locks to be associated with random connections, breaking mutual exclusion and triggering premature lock releases.
Peer Dependency Matrix
3. Database Schema & Automated DDL
BothPostgresStorageAdapter and PostgresIdempotencyStore provide automated schema synchronization via ensureTables() and ensureTable(). These methods execute idempotent DDL statements (CREATE TABLE IF NOT EXISTS, ALTER TABLE ... ADD COLUMN IF NOT EXISTS, CREATE INDEX IF NOT EXISTS).
DDL Definitions
SQL Injection Protection
Custom table names passed toPostgresStorageAdapter or PostgresIdempotencyStore are validated against strict SQL identifier constraints:
- Must match
/^[a-zA-Z_][a-zA-Z0-9_]*$/. - Must not exceed 63 characters (PostgreSQL standard identifier length limit).
- Throws an descriptive
Errorif invalid characters or injection attempts are detected.
4. PostgresStorageAdapter API Reference
Implements the Siglo StorageAdapter interface (name = 'postgres').
Constructor Options (PostgresStorageAdapterOptions)
Checkpointing & Optimistic Concurrency Control (OCC)
ensureTables(): Promise<void>
Initializes checkpointsTable, eventsTable, and waitRecordsTable along with indexes and columns if they do not exist.
saveCheckpoint(checkpoint: SerializedExecutionStack, expectedVersion?: number): Promise<void>
Persists or updates an execution stack.
- OCC Enforcement: If
expectedVersionis specified (or exists oncheckpoint.occVersion), executes an atomic version-guarded update: - Conflict Handling: If
rowCount === 0, it fetches the current database version and throws anOptimisticLockException(stackId, currentVersion, expectedVersion). - Cancellation Preservation: If a workflow was previously cancelled, updating the checkpoint preserves the
'cancelled'status atomically. - Version Tracking: On success, assigns the new incremented version to
checkpoint.occVersionandcheckpoint.revision. - Write Tracking: Automatically serialized under
withWorkflowLockscopes to drain all pending writes before lock release.
getCheckpoint(stackId: string): Promise<SerializedExecutionStack | undefined>
Retrieves the serialized execution stack by stackId.
- Automatically assigns
occVersionandrevisionfrom the databaseversioncolumn. - Returns
undefinedifstackIdis empty or not found.
cleanExpiredCheckpoints(): Promise<number>
Deletes checkpoints where (state->>'expiresAt')::bigint <= (EXTRACT(EPOCH FROM NOW()) * 1000)::bigint. Returns the count of deleted rows.
Context Mutation Audit Logging
onContextChange(entry): Promise<void>
Records step mutation audit events into eventsTable.
- Parameter
entry: - Converts
timestamp(in milliseconds) usingto_timestamp(timestamp / 1000.0). - Operates as a non-blocking stream: internal errors are caught to avoid disrupting primary workflow execution.
Distributed Workflow Locking
withWorkflowLock<T>(stackId: string, action: () => Promise<T>): Promise<T>
Executes action inside a PostgreSQL session-level advisory lock using pg_try_advisory_lock:
- Distributed Mutex: Serializes step continuations, event resumptions, and compensations across multi-instance worker fleets.
- Auto-Release on Process Failure: Because PostgreSQL advisory locks are bound to the database session/connection, if a worker node crashes or disconnects, PostgreSQL automatically releases the lock immediately.
- Re-entrancy: Safely allows nested invocations for the same
stackIdwithin the same execution context. - Pool Awareness: Detects pooled clients (
'totalCount' in pool) and callspool.connect(), ensuring a dedicated connection is held for the lock duration and released back to the pool in afinallyblock. - Non-Pooled / Direct Client Guard: Guards direct
Clientinstances against concurrent re-use viadirectClientBusy. - Write Queue Draining: Uses
AsyncLocalStorageand internal write tracking (tailpromise) to ensure all pending checkpoint and wait record writes are completely flushed to PostgreSQL before the advisory lock is unlocked. - Throws:
DuplicateEventException(stackId)if the lock cannot be acquired or if the direct client is busy.
Durable Wait Records & Resumption
saveWaitRecord(record: any): Promise<void>
Persists or updates a durable wait record in waitRecordsTable and embeds the record into the checkpoint metadata (checkpoint.metadata.waitRecord).
- Key properties on
record:correlationKey,stackId,namespace,eventName,status,continuationStepIndex,version,data,expiresAt.
getWaitRecord(correlationKeyOrStackId: string): Promise<any | undefined>
Retrieves a wait record from waitRecordsTable matching either correlation_key or stack_id. Falls back to querying siglo_checkpoints.state->'metadata'->'waitRecord' if not found in the wait records table.
claimWaitRecord(correlationKey: string, claimantId?: string, options?: { allowExpired?: boolean; eventPayload?: any }): Promise<any>
Atomically claims a wait record for resumption via a single Compare-And-Swap (CAS) update:
- Expiration Safety: If the record has expired and
allowExpiredis not set, throwsExpiredWaitException(correlationKey, expiresAt, eventName). - Duplicate Safety: If the record is already claimed or resumed, throws
DuplicateEventException(correlationKey). - Returns: The claimed wait record with
status: 'RESUMING',lockedBy, and parsed JSONresumePayload.
completeWaitRecord(correlationKey: string, claimantId: string): Promise<void>
Marks the claimed wait record as 'RESUMED', setting resumed_by and resumed_at = NOW(). Drains pending async writes before completing.
cancelWaitRecord(correlationKey: string): Promise<void>
Transitions a wait record status to 'CANCELLED'.
cancelWorkflow(stackId: string): Promise<void>
Atomically cancels both active wait records and the execution checkpoint for stackId:
- Cancels all wait records:
UPDATE siglo_wait_records SET status = 'CANCELLED' WHERE stack_id = $1. - Updates the checkpoint state:
UPDATE siglo_checkpoints SET state = jsonb_set(state, '{status}', '"cancelled"'), version = version + 1 WHERE stack_id = $1.
Retention & Cleanup
5. PostgresIdempotencyStore API Reference
Implements the Siglo IdempotencyStore interface.
Constructor Options (PostgresIdempotencyOptions)
Methods
ensureTable(): Promise<void>
Creates tableName and the index on expires_at if they do not exist.
claim(key: string, ttlMs?: number): Promise<'claimed' | 'completed' | 'in-progress'>
Atomically attempts to claim the key:
- Initial Insert: Executes
INSERT ... VALUES ($1, 'in-progress', $2, NOW()) ON CONFLICT (key) DO NOTHING. Returns'claimed'if row inserted. - Expired Lock Reclamation: If the key exists but
expires_at <= NOW(), updates status back to'in-progress', updatesexpires_at, clearsresult, and returns'claimed'. - Completed Result: If
status === 'completed', returns'completed'. - Active Claim: If actively running and unexpired, returns
'in-progress'.
complete(key: string, result: unknown, ttlMs?: number): Promise<void>
Transitions key status to 'completed', serializes result into result JSONB, and sets expires_at = NOW() + ttlMs.
fail(key: string, _error?: unknown): Promise<void>
Deletes in-progress claims (WHERE key = $1 AND status = 'in-progress') so subsequent retries can proceed immediately.
getResult(key: string): Promise<unknown | undefined>
Retrieves and deserializes the stored result JSONB for completed keys. Returns undefined if key is not completed or missing.
get(key: string): Promise<unknown | undefined>
Convenience alias for getResult(key).
getRecord(key: string): Promise<any | undefined>
Inspects the full raw idempotency record:
cleanupExpired(): Promise<number>
Deletes expired idempotency records (DELETE FROM siglo_idempotency WHERE expires_at < NOW()). Returns the count of deleted rows.
6. Concurrency & Locking Architecture
Siglo workflows employ a defense-in-depth concurrency model combining PostgreSQL advisory session locks with table-level Optimistic Concurrency Control (OCC):Advisory Locks vs. Optimistic Concurrency Control (OCC)
withWorkflowLock(PostgreSQL Advisory Locks): Used around step executions, timer/event resumptions, and compensations. Guarantees that only one worker executes steps for a givenstackIdat any time. If the worker crashes, the PostgreSQL session lock automatically drops.- OCC (
versioncolumn): Used bysaveCheckpoint. Guards against stale writes or late-arriving updates across worker partitions. If a write conflicts with an unexpected version, anOptimisticLockExceptionis thrown.
7. Core Exception Types
All exceptions extendSigloError and can be imported from @siglo/core:
1. OptimisticLockException
Thrown by saveCheckpoint when an update fails because the version in the database differs from expectedVersion:
err.stackId: The workflow execution stack ID.err.currentVersion: The version currently in PostgreSQL.err.expectedVersion: The version expected by the writer.
2. DuplicateEventException
Thrown by withWorkflowLock when another worker currently holds the workflow lock, or by claimWaitRecord when a wait record is already claimed or resumed:
err.correlationKey/err.stackId: The resource key that could not be locked.
3. ExpiredWaitException
Thrown by claimWaitRecord when attempting to claim a wait record whose expires_at timestamp has passed and allowExpired is not set:
err.correlationKey: The wait record correlation key.err.expiresAt: Expiration timestamp in milliseconds.err.eventName: Name of the expected event.
