Each phase serves a distinct purpose in document processing:

1. INITIALIZE: Parse document AST and populate database with specifications, spec_objects, floats, relations, views, and attributes
2. ANALYZE: Resolve relations between objects (link target resolution, type inference)
3. TRANSFORM: Pre-compute views, render external content (PlantUML, charts), prepare for output
4. VERIFY: Run proof views to validate data integrity, type constraints, cardinality rules
5. EMIT: Assemble final documents and write to output formats (docx, html5, markdown, json)

Handlers register via register_handler(handler) with required fields:

• name: Unique string identifier for the handler
• prerequisites: Array of handler names that must execute before this handler

Handlers declare participation in phases via hook methods (on_initialize, on_analyze, on_verify, on_transform, on_emit). Duplicate handler names cause registration error.

For each phase, the pipeline:

1. Identifies handlers participating in the phase (those with on_{phase} hooks)
2. Builds dependency graph from prerequisites (only for participating handlers)
3. Executes Kahn’s algorithm to produce execution order
4. Sorts alphabetically at each level for deterministic output
5. Detects and reports circular dependencies with error listing remaining nodes

All handlers implement on_{phase}(data, contexts, diagnostics) hooks that receive the full contexts array. The pipeline orchestrator calls each handler’s hook once per phase via run_phase(), passing all document contexts. Handlers are responsible for iterating over contexts internally.

This enables cross-document optimizations, transaction batching, and parallel processing within any phase.

The execute(docs) method creates context objects for each input document with:

• doc: Pandoc document AST (via DocumentWalker)
• spec_id: Specification identifier derived from filename
• config: Preset configuration (styles, captions, validation)
• build_dir: Output directory path
• output_format: Target format (docx, html5, etc.)
• template: Template name for model loading
• reference_doc: Path to reference.docx for styling
• docx, html5: Format-specific configuration
• outputs: Array of {format, path} for multi-format output
• bibliography, csl: Citation configuration
• project_root: Root directory for resolving relative paths

Context flows through all phases, enriched by handlers (e.g., verification results in VERIFY phase).

The specifications table stores metadata for each specification document parsed during INITIALIZE Phase phase:

• identifier: Unique specification ID derived from filename (e.g., “srs-main”)
• root_path: Source file path for the specification
• long_name: Human-readable title extracted from L1 header
• type_ref: Specification type (validated against spec_specification_types)
• pid: Optional PID from @PID syntax in L1 header

L1 headers register as specifications. Type validation checks spec_specification_types table. Invalid types fall back to default or emit warning.

The spec_objects table stores structured specification items extracted from L2+ headers:

• identifier: SHA1 hash of source path + line + title (content-addressable)
• specification_ref: Foreign key to parent specification
• type_ref: Object type (validated against spec_object_types)
• from_file: Source file path
• file_seq: Document order sequence number
• pid: Project ID from @PID syntax (e.g., “REQ-001”)
• title_text: Header text without type prefix or PID
• label: Unified label for cross-referencing (format: {type_lower}:{title_slug})
• level: Header level (2-6)
• start_line, end_line: Source line range
• ast: Serialized Pandoc AST (JSON) for section content

Type resolution order: explicit TYPE: prefix, implicit alias lookup, default type fallback.

The spec_floats table stores content blocks that receive sequential numbering:

• identifier: Short format “float-{8-char-sha1}” for DOCX compatibility
• specification_ref: Foreign key to parent specification
• type_ref: Float type resolved from aliases (e.g., “csv” -> “TABLE”, “puml” -> “FIGURE”)
• from_file: Source file path
• file_seq: Document order for numbering
• label: User-provided label for cross-referencing
• number: Sequential number within counter_group (assigned in TRANSFORM Phase)
• caption: Caption text from attributes
• raw_content: Original code block text
• raw_ast: Serialized Pandoc CodeBlock (JSON)
• parent_object_ref: Foreign key to containing spec_object
• attributes: JSON-serialized attributes (caption, source, language)
• syntax_key: Original class syntax for backend matching

Counter Group share numbering (e.g., FIGURE, CHART, PLANTUML all increment “FIGURE” counter).

The spec_views table stores view definitions from code blocks and inline syntax:

• identifier: SHA1 hash of specification + sequence + content
• specification_ref: Foreign key to parent specification
• view_type_ref: Uppercase view type (e.g., “SELECT”, “SYMBOL”, “MATH”, “ABBREV”)
• from_file: Source file path
• file_seq: Document order sequence number
• raw_ast: View definition content (SQL query, symbol path, expression)

View types with needs_external_render = 1 in spec_view_types are delegated to specialized renderers. Inline views use type: content syntax (e.g., symbol: Class.method).

The spec_relations table stores inter-element references:

• identifier: SHA1 hash of specification + target + type + parent
• specification_ref: Foreign key to parent specification
• source_ref: Foreign key to source spec_object
• target_text: Raw link target from syntax (e.g., “REQ-001”, “fig:diagram”)
• target_ref: Resolved target identifier (populated in ANALYZE Phase phase)
• type_ref: Relation type from spec_relation_types (e.g., “TRACES”, “XREF_FIGURE”)
• from_file: Source file path

Link syntax: [PID](@) for PID references, [type:label](#) for float references, [@citation] for bibliographic citations. Default relation types are determined by is_default and link_selector columns in spec_relation_types.

The spec_attributes table stores typed attribute values extracted from blockquote syntax:

• identifier: SHA1 hash of specification + owner + name + value
• specification_ref: Foreign key to parent specification
• owner_ref: Foreign key to owning spec_object
• name: Attribute name (field name without colon)
• raw_value: Original string value
• string_value, int_value, real_value, bool_value, date_value: Type-specific columns
• enum_ref: Foreign key to enum_values for ENUM types
• ast: JSON-serialized Pandoc AST for rich content (XHTML type)
• datatype: Resolved datatype from spec_attribute_types

Attribute syntax: > name: value in blockquotes following headers. Datatypes include STRING, INTEGER, REAL, BOOLEAN, DATE, ENUM, XHTML. Multi-line attributes use continuation blocks.

Proof views are SQL queries registered in the VERIFY Phase phase that check for constraint violations:

• Specification-level (missing required attributes, invalid types)
• Object-level (missing required, cardinality, cast failures, invalid enum/date, bounds)
• Float-level (orphans, duplicate labels, render failures, invalid types)
• Relation-level (unresolved, dangling, ambiguous)
• View-level (materialization failures, query errors)

The validation policy (configurable in project.yaml) determines severity: error, warn, or ignore.

Type handlers control how specification content is rendered during the TRANSFORM Phase phase. The loading mechanism supports:

• Object types: Loaded from models/{model}/types/objects/{type}.lua
• Specification types: Loaded from models/{model}/types/specifications/{type}.lua
• Float types: Loaded from models/{model}/types/floats/{type}.lua
• View types: Loaded from models/{model}/types/views/{type}.lua

Module loading uses require() with path models.{model}.types.{category}.{type}. Type names are converted to lowercase for file lookup (e.g., “HLR” -> “hlr.lua”).

Each model follows this structure:

models/{model_name}/
  types/
    objects/       -- Spec object type handlers (HLR, LLR, VC, etc.)
    specifications/-- Specification type handlers (SRS, SDD, SVC)
    floats/        -- Float type handlers (TABLE, PLANTUML, CHART)
    views/         -- View type handlers (ABBREV, SYMBOL, MATH)
    relations/     -- Relation type definitions (TRACES_TO, etc.)
  data_views/      -- Data view generators for chart data injection
  filters/         -- Pandoc filters (docx.lua, html.lua, markdown.lua)
  postprocessors/  -- Format-specific postprocessors
  styles/          -- Style presets and templates

Model names are referenced via project configuration template field or context model_name. The “default” model provides base implementations with fallback behavior.

Each handler category defines specific interfaces:

Object Type Handlers export:

• M.object: Type schema with id, long_name, description, attributes
• M.handler.on_render_SpecObject(obj, ctx): Render function returning Pandoc blocks

Specification Type Handlers export:

• M.specification: Type schema with id, long_name, attributes
• M.handler.on_render_Specification(ctx, pandoc, data): Render document title

Float Type Handlers export:

• M.float: Type schema with id, caption_format, counter_group, aliases, needs_external_render
• M.transform(raw_content, type_ref, log): For internal transforms (TABLE, CSV)
• external_render.register_renderer(type_ref, callbacks): For external renders (PLANTUML, CHART)

View Type Handlers export:

• M.view: Type schema with id, inline_prefix, aliases
• M.handler.on_render_Code(code, ctx): Inline code rendering

Type schemas provide metadata stored in registry tables:

Object Types (spec_object_types):

M.object = {
    id = "HLR",                    -- Unique identifier (uppercase)
    long_name = "High-Level Requirement",
    description = "A top-level system requirement",
    extends = "TRACEABLE",         -- Base type for inheritance
    header_unnumbered = true,      -- Exclude from section numbering
    header_style_id = "Heading2",  -- Custom-style for headers
    body_style_id = "Normal",      -- Custom-style for body
    attributes = {                 -- Attribute definitions
        { name = "status", type = "ENUM", values = {...}, min_occurs = 1 },
        { name = "rationale", type = "XHTML" },
        { name = "created", type = "DATE" },
    }
}

Float Types (spec_float_types):

M.float = {
    id = "CHART",
    caption_format = "Figure",     -- Caption prefix
    counter_group = "FIGURE",      -- Counter sharing (FIGURE, CHART, PLANTUML)
    aliases = { "echarts" },       -- Alternative syntax identifiers
    needs_external_render = true,  -- Requires external tool
}

View Types (spec_view_types):

M.view = {
    id = "ABBREV",
    inline_prefix = "abbrev",      -- Syntax: `abbrev: content`
    aliases = { "sigla", "acronym" },
    needs_external_render = false,
}

Model path resolution follows this order:

1. Check SPECCOMPILER_HOME environment variable: $SPECCOMPILER_HOME/models/{model}
2. Fall back to current working directory: ./models/{model}

For type modules not found in the specified model, the system falls back to the “default” model:

-- Try model-specific path first
local module = require("models." .. model_name .. ".types.floats.table")
-- Fallback to default model
local module = require("models.default.types.floats.table")

This enables partial model customization where models only override specific types.

Float types requiring external tools (PlantUML, ECharts, etc.) register with the external render handler:

external_render.register_renderer("PLANTUML", {
    prepare_task = function(float, build_dir, log, data, model_name)
        -- Return task descriptor with cmd, args, output_path, context
    end,
    handle_result = function(task, success, stdout, stderr, data, log)
        -- Update resolved_ast in database
    end
})

The core orchestrates: query items -> prepare tasks -> cache filter -> batch spawn -> dispatch results. This enables parallel execution across all external renders.

Data views are Lua modules that generate data for charts:

-- models/{model}/data_views/{view_name}.lua
local M = {}

function M.generate(params, data)
    -- params: user parameters from code block attributes
    -- data: DataManager instance for SQL queries
    return { source = { {"x", "y"}, {1, 10}, {2, 20} } }
end

return M

Views are loaded via data_loader.load_view(view_name, model_name, data, params). Resolution tries the specified model first, then falls back to default.

Usage in code blocks:

```chart:gaussian{view="gaussian" sigma=2.0}
{...echarts config...}

Each handler loader maintains a cache keyed by {model}:{type_ref}:

local type_handlers = {}

local function load_type_handler(type_ref, model_name)
    local cache_key = model_name .. ":" .. type_ref
    if type_handlers[cache_key] ~= nil then
        return type_handlers[cache_key]
    end

    -- Load module via require()
    local ok, module = pcall(require, module_path)
    if ok and module then
        type_handlers[cache_key] = module.handler
        return module.handler
    end

    type_handlers[cache_key] = false  -- Cache negative result
    return nil
end

Cache stores false for failed lookups to avoid repeated require() calls for non-existent modules.

Purpose: Resolves cross-references between spec objects and infers missing type information.

Position: Second phase after INITIALIZE, before TRANSFORM.

Purpose: Stores content hashes to detect which documents have changed since last build.

Implementation: Compares current file hash against cached hash to skip unchanged files.

Purpose: Groups related float types to share sequential numbering.

Example: FIG and DIAGRAM types may share a counter, producing Figure 1, Figure 2, etc.

Purpose: Groups software units into higher-level structural components for design allocation.

Examples: src/core, src/db, src/infra.

Purpose: Captures file-level implementation units allocated to functional descriptions.

Examples: src/core/pipeline.lua, src/db/manager.lua.

Purpose: Produces structured data that can be injected into chart floats.

Implementation: Lua scripts that query database and return chart-compatible data structures.

Purpose: Flexible schema for storing typed attributes on spec objects.

Structure: Entity (spec object), Attribute (key name), Value (typed content).

Purpose: Assembles transformed content and writes final output documents.

Position: Final phase after VERIFY.

Purpose: Delegates rendering to external tools via subprocess execution.

Examples: PlantUML JAR for diagrams, chart libraries for data visualization.

Purpose: Encapsulates processing logic for a content type across all pipeline phases.

Structure: Implements phase methods (initialize, analyze, verify, transform, emit) for its content type.

Purpose: Parses markdown AST and populates intermediate representation containers.

Position: First phase, entry point for document processing.

Purpose: Bundles related type definitions, handlers, and styling for specific documentation domains.

Examples: SRS model for software requirements, HRS model for hardware requirements.

Purpose: Tracks when outputs were last generated to enable incremental builds.

Implementation: Compares source modification time against cached output timestamp.

Purpose: Separates document processing into well-defined sequential stages.

Phases: INITIALIZE, ANALYZE, TRANSFORM, VERIFY, EMIT.

Purpose: Orchestrates document processing through sequential phases.

Flow: Each phase completes for all handlers before the next phase begins.

Purpose: Declares which handlers must complete before a given handler can execute.

Usage: Handlers declare prerequisites to ensure data dependencies are satisfied.

Purpose: Determines valid execution order for handlers based on dependencies.

Algorithm: Uses Kahn’s algorithm to produce a topologically sorted handler sequence.

Purpose: Materializes database views into content and applies content transformations.

Position: Third phase after ANALYZE, before VERIFY.

Purpose: Provides shorthand or alternative names for types.

Example: csv is an alias for the TABLE type in float definitions.

Purpose: Dynamically discovers and instantiates type handlers from model definitions.

Implementation: Scans model directories for handler definitions and registers them.

Purpose: Stores type definitions including attributes, aliases, and validation rules.

Tables: spec_object_types, spec_float_types, spec_attribute_types, etc.

Purpose: Validates document content using proof views and constraint checking.

Position: Fourth phase after TRANSFORM, before EMIT.

Purpose: Represents document structure as a hierarchical tree of elements.

Source: Pandoc parses Markdown and produces JSON AST.

Usage: Handlers walk the AST to extract spec objects, floats, and relations.

Purpose: Indexes specification text for fast full-text search queries.

Implementation: SQLite FTS5 virtual tables populated during EMIT phase.

Usage: Web application uses FTS for search functionality.

Purpose: Defines system-level requirements that guide design and implementation.

Traceability: HLRs trace to verification cases (VC) and are realized by functional descriptions (FD).

Purpose: Stores parsed specification content in queryable form.

Storage: SQLite database with spec_objects, spec_floats, spec_relations tables.

Lifecycle: Populated during INITIALIZE, queried and modified through remaining phases.

Purpose: Provides unique, human-readable identifiers for traceability and cross-referencing.

Syntax: Written as @PID in header text (e.g., ## HLR: Requirement Title @REQ-001).

Auto-generation: PIDs can be auto-generated from type prefix and sequence number.

Purpose: Defines validation rules as SQL queries that detect specification errors.

Execution: Run during the VERIFY phase; violations are reported as diagnostics.

Examples: Missing required attributes, unresolved relations, cardinality violations.

Purpose: Provides persistent, portable storage for the intermediate representation.

Benefits: Single-file storage, ACID transactions, SQL query capability.

Usage: All pipeline phases read/write to SQLite via the database manager.

Purpose: Base type for objects that can be linked via traceability relations.

Types: Any spec object type registered in the model (e.g., HLR, LLR, SECTION).

Relations: Model-defined relation types (e.g., XREF_FIGURE, XREF_CITATION) inferred by specificity matching.

Purpose: Defines the schema, validation rules, and rendering behavior for a category of elements.

Categories: Object types (HLR, SECTION), float types (FIGURE, TABLE), relation types (TRACES_TO), view types (TOC, LOF).

Registration: Types are loaded from model directories and stored in the type registry.

Purpose: Defines how requirements are verified through test procedures and expected results.

Traceability: VCs trace to HLRs via traceability attribute links.

Naming: VC PIDs follow the pattern VC-{category}-{seq} (e.g., VC-PIPE-001).