@siglo/ai Step & Agent Guide
@siglo/ai provides resilient, enterprise-grade, multi-provider LLM inference, native structured outputs, and agentic step execution for Siglo workflows. It replaces fragile ad-hoc AI SDK scripts with coordinated rate-limit fallback cascades, automated self-healing schema repair loops, Step-as-Tool agent loops backed by parent saga compensations, AsyncLocalStorage (ASL) network perimeter enforcement, deterministic offline replay testing, and OpenTelemetry GenAI observability.
1. Executive Summary & Problem Space
Integrating Large Language Models (LLMs) and generative agents directly into production workflows introduces non-deterministic outputs, transient rate limits (429), latency spikes, and severe security challenges. When developers call LLM SDKs directly inside workflow steps, pipelines suffer from systemic failure modes:
2. Architecture & Step Lifecycle Integration
Siglo executes steps through a formal 5-phase lifecycle:BEFORE → VALIDATION → ACTION → AFTER → COMPENSATE. @siglo/ai hooks natively into every phase:
3. Installation & Setup
Install@siglo/ai and @siglo/http alongside @siglo/core:
@siglo/core:^1.0.0@siglo/http:^1.0.0(for network perimeter and Standard Schema integration)@opentelemetry/api:^1.0.0(optional, for distributed trace context propagation)
4. Deep Technical Capabilities
4.1 Multi-Provider Abstraction
@siglo/ai provides a unified interface across leading generative AI providers and local runtimes:
Credential Isolation & Environment Variables
Credentials resolve independently for each provider. There is no implicit credential sharing across primary and fallback models:- Static or dynamic
apiKeyprovided on the specific model config: - Standard environment variable matching the provider:
OPENAI_API_KEYforopenaiANTHROPIC_API_KEYforanthropicGEMINI_API_KEYforgemini
- Custom and Ollama endpoints require an explicit
baseUrland receive no implicit environment keys.
4.2 Zero-Trust ASL Network Perimeter
@siglo/ai integrates with Siglo’s AsyncLocalStorage (ASL) network egress perimeter:
- Preflight Enforcement: Every configured primary and fallback endpoint hostname is registered in the step’s
.allowNetwork()policy during construction and verified during theBEFOREphase before any request or cache lookup. - Pre-Dispatch Verification: Right before
fetchexecution,assertNetworkPolicy()re-verifies the exact destination URL and hostname. - SSRF & Localhost Defense: Localhost endpoints (such as
http://localhost:11434for Ollama) require explicitallowLocalhost: trueon both the step and enclosing flow when network policies are active. - Strict Redirect Defense: HTTP redirects (
301,302,307,308) are rejected withredirect: 'error'to prevent credential forwarding or egress perimeter escalation.
4.3 Native Structured Outputs & Standard Schema V1
@siglo/ai enforces strict type safety using the Standard Schema V1 (~standard) specification supported by Zod, Valibot, and ArkType.
TypeScript Type Inference
Outputs are statically inferred into the workflow execution context:Object Enveloping for Primitives & Arrays
OpenAI and Anthropic native structured outputs require an object root. When your schema is a primitive (z.string()) or array (z.array(...)), @siglo/ai transparently envelopes the request into { data: schema } for the provider, while unwrapping the validated value so your workflow receives the exact primitive or array.
Local Validation vs Native Enforcement
If using models or providers that do not support native JSON schema constraints, disable native enforcement while preserving local runtime schema validation and self-healing repair:4.4 Self-Healing Prompt Repair Fallback Loop
If an LLM produces malformed JSON or hallucinates fields that violate the schema,@siglo/ai automatically intercepts the error and executes an automated repair turn:
- Bounded Budget: Controlled by
maxRepairAttempts(default: 2, set to 0 to disable). - Diagnostics Preservation: If repair turns are exhausted, throws
AiOutputValidationErrorcontaining the structured diagnostic issues for debugging. - Accurate Accounting: Every repair attempt is tracked in metrics (
metrics.repairAttempts) and token usage sums across all repair turns.
4.5 Model Fallback Cascades & Transient Error Classification
Network hiccups, sudden traffic surges (429 Rate Limit), and transient cloud outages (500, 502, 503, 504) are handled automatically without crashing your workflow:
Error Classification Rules
4.6 Step-as-Tool Execution Bridge & Parent Saga Integration
@siglo/ai allows standard Siglo workflow steps to be passed directly as tools to the model using defineAiTool:
Tool Execution Governance
- Pre-Validation: The entire batch of tool calls emitted by the model is validated against each tool’s
inputSchemabefore the first side-effecting step runs. - Sequential Lifecycle Execution: Tools run via
Step.startwithin the active workflow stack (stackId), inheriting parent cancellation signals and network policies. - Parent Saga Integration: Tool steps register compensations onto the parent saga stack. If subsequent inference fails or downstream steps fail, tool compensations roll back in reverse order (LIFO).
- Safety Restrictions: Tool-bearing AI steps strictly forbid step-level caching, failure recovery/retries, and idempotent replay to prevent accidental duplicate side effects.
- Turn Bounding: Bounded by
maxToolTurns(default: 5) to prevent infinite agentic ping-pong loops.
4.7 Deterministic Caching for AI Steps
For schema-validated or text inference steps without tools,@siglo/ai integrates with Siglo’s .cache():
- Deterministic Hash: The cache key is computed as a SHA-256 digest covering:
cacheVersion(bump when prompt wording or validation rules change)- Resolved prompt string or messages
- Model cascade configurations and credential scopes
- Native JSON schema and validator code
- Sampling parameters (
temperature,maxTokens) - Workflow input payload
- Stampede Protection: In-flight stampede sharing defaults to
falseso that aborting one request never aborts concurrent workflow callers. - Cache Hits: Cache hits return original metrics without making external network calls or emitting duplicate
ai:completeevents.
4.8 OpenTelemetry GenAI Observability & Cost Accounting
@siglo/ai instruments every request according to the OpenTelemetry GenAI Semantic Conventions:
- Span Attributes:
gen_ai.system: e.g.'openai','anthropic','gemini'gen_ai.provider.name: e.g.'openai','anthropic','gcp.gemini'gen_ai.operation.name:'chat'gen_ai.request.model: e.g.'gpt-4o'gen_ai.response.model: e.g.'gpt-4o'gen_ai.usage.input_tokens: total prompt tokensgen_ai.usage.output_tokens: total completion tokens
- W3C Distributed Tracing: Injects
traceparentandtracestateheaders onto outgoing provider HTTP requests. - Live Monitoring Events: Emits typed
ai:completemonitoring events containing token counts, latency, model used, and repair/fallback statistics (prompts and generated text are excluded to prevent secret/PII leaks). - Cost Estimation: Automatically computes
estimatedCostUsdusing built-in baseline rates or custom per-model pricing overrides (pricing: { inputPerMillion, outputPerMillion }).
4.9 Offline Testing & Deterministic Replay (MockAiProvider)
@siglo/ai/testing provides MockAiProvider for zero-network, 100% deterministic testing:
Record & Replay
Recordings can be exported and replayed for regression verification:5. Production-Ready Scenarios
Scenario A: Financial Loan Underwriting Pipeline with Model Fallbacks
Scenario B: Agentic Portfolio Advisor with Compensating Tools
6. API Reference
aiStep(name, options)
Instantiates a typed AiStep execution unit extending Siglo’s core Step.
Configuration Options (AiStepOptions)
Output Result Structure (AiStepResult<T>)
Every AI step produces a wrapped result structure:
Metrics Structure (AiMetrics)
7. Initial Release Boundaries & Guidelines
- Excluded from v1: Streaming (
ReadableStream) and mid-agent loop durable suspensions are outside the initial release scope. - Tool Restrictions: Tool-bearing AI steps strictly reject step-level caching, failure recovery/retries, and idempotent replay.
- Schema Compatibility: Provider schemas must be valid JSON Schema objects. Primitive and array outputs use an internal object envelope for OpenAI and Anthropic.
- Offline CI: All continuous integration tests run offline without network egress or live API keys using
MockAiProvider.
