SpecCompiler Requirements

1 Scope

This document defines the high-level requirements for SpecCompiler , a document processing pipeline for structured specifications.

The document is organized in two parts. The first part defines the SpecIR data model, the six core types that the <Pipeline> operates on. The second part specifies the functional requirements, grouped into System Features (Pipeline Execution through Audit and Integrity), each decomposed into High-Level Requirements.

1.1 SpecCompiler Core Data Dictionary

1.1.1 SpecIR Types

SpecIR (see <SpecIR>) is the data model that SpecCompiler builds from source Markdown during the <INITIALIZE Phase> phase. The core task of parsing is to lower Markdown annotations into a set of typed content tables that the <Pipeline> can analyze, transform, verify, and emit. The entries below define each of these six content tables as a formal tuple specifying the Markdown syntax that produces it.

SpecIR-01: Specification

A Specification is the root document container created from an H1 header. It represents a complete document like an SRS, SDD, or SVC. Each specification has a type, optional PID, and contains attributes and all spec objects within that document.

DESCRIPTION:

Formal definition: S=(τ,n,pid,𝒜,𝒪)S = \left( \tau,n,\text{pid},\mathcal{A},\mathcal{O} \right) — a tuple of type, title, project identifier, attributes, and child objects.

Syntax: # [TypeRef:] Text [@PID]

Full specification: See the CommonSpec Language Specification for the complete grammar, type inference rules, and examples.

SpecIR-02: Spec Object

A Spec Object represents a traceable element in a specification document, created from H2-H6 headers. Objects can be requirements (HLR, LLR), verification cases (VC), design elements (FD, CSC, CSU), or structural sections (SECTION). Each object has a type, PID and can contain attributes, body content, floats, relations, views, and child objects.

DESCRIPTION:

Formal definition: O=(τ,title,pid,β,𝒜,,,𝒱,𝒪)O = \left( \tau,\text{title},\text{pid},\beta,\mathcal{A},\mathcal{F},\mathcal{R},\mathcal{V},\mathcal{O} \right) — a tuple of type, title, project identifier, body content, attributes, floats, relations, views, and child objects.

Syntax: ##...###### [TypeRef:] Title [@PID]

Full specification: See the CommonSpec Language Specification for the complete grammar, type inference, PID auto-generation, and examples.

SpecIR-03: Spec Float

A Spec Float represents a floating element like a figure, table, or diagram. Floats are created from fenced code blocks with a TypeRef:Label pattern. They are automatically numbered within their counter group and can be cross-referenced by their label. Some floats require external rendering (e.g., PlantUML diagrams).

DESCRIPTION:

Formal definition: F=(τ,label,𝓀𝓋,content)F = \left( \tau,\text{label},\mathcal{\text{kv}},\text{content} \right) — a tuple of type, label, key-value metadata, and raw content.

Syntax: ```TypeRef:Label[{Key=Value, ...}] Content ```

Full specification: See the CommonSpec Language Specification for float types, aliases, counter groups, and examples.

SpecIR-04: Attribute

An Attribute stores metadata for specifications and spec objects using an Entity-Attribute-Value (EAV) pattern. Attributes are defined in blockquotes following headers and support multiple datatypes including strings, integers, dates, enums, and rich XHTML content (Pandoc AST). Relations are extracted from the AST. Attribute definitions constrain which attributes each object type can have.

DESCRIPTION:

Formal definition: A=(τ,β,)A = \left( \tau,\beta,\mathcal{R} \right) — a triple of attribute type, blockquote content, and child relations.

Syntax: > TypeRef: value

Datatypes: STRING, INTEGER, REAL, BOOLEAN, DATE, ENUM, XHTML.

Full specification: See the CommonSpec Language Specification for the complete datatype semantics, attribute constraints, and examples.

SpecIR-05: Spec Relation

A Spec Relation represents a traceability link between specification elements. Relations are created from Markdown links where the link target (URL) acts as a selector that drives type inference. The relation type is not authored explicitly — it is inferred by constraint matching: each relation type defines optional constraints on selector, source attribute, source type, and target type. The most specific match (most non-NULL constraints) wins.

DESCRIPTION:

Formal definition: R=(s,t,σ,α)R = (s,t,\sigma,\alpha) — a 4-tuple of source object, target element, link selector, and source attribute.

Type inference: ρ=infer(σ,α,τs,τt)\rho = \text{infer}\left( \sigma,\alpha,\tau_{s},\tau_{t} \right) — the relation type is inferred by constraint matching with most-specific-wins across selector, attribute, source type, and target type.

Syntax: [Target](selector) where the URL starts with @ or #. Selectors are not hardcoded — they are defined by relation types in the model (e.g., PID_REF defines @, LABEL_REF defines #, XREF_CITATION defines @cite,@citep). Models can register any @... or #... selector.

Full specification: See the CommonSpec Language Specification for the complete inference algorithm and examples.

SpecIR-06: Spec View

A Spec View represents a dynamic query or generated content block. Views are materialized during the TRANSFORM phase and can generate tables of contents (TOC), lists of figures (LOF), or custom queries, abbreviations, and inline math. Views enable dynamic document assembly based on specification data.

DESCRIPTION:

Formal definition: V=(τ,ω)V = (\tau,\omega) — a pair of view type and parameter string.

Syntax: `TypeRef:[ViewParam]`

Full specification: See the CommonSpec Language Specification for view types, rendering, and examples.

2 Functional Requirements

With the data model established, the following sections define the functional requirements for SpecCompiler Core. Requirements are organized into System Features (SF), each covering a distinct functional domain. Every SF is decomposed into <High-Level Requirement>s that state what the system shall do.

2.1 Pipeline Requirements

SF-001: Pipeline Execution

Five-phase document processing lifecycle with <Handler> orchestration and <Topological Sort> ordering.

DESCRIPTION:
Groups requirements for the core <Pipeline> that drives document processing through <INITIALIZE Phase> , <RESOLVE Phase> , <TRANSFORM Phase> , <ANALYZE Phase> , <EMIT Phase> phases with declarative handler dependencies.
RATIONALE:
A structured processing pipeline enables separation of concerns, validation gates, and deterministic handler ordering.

HLR-PIPE-001: Five-Phase Lifecycle

The pipeline shall execute handlers in a five-phase lifecycle: INITIALIZE, RESOLVE, TRANSFORM, ANALYZE, EMIT.

DESCRIPTION:

Each phase serves a distinct purpose in document processing:

  1. INITIALIZE: Parse document AST and populate database with specifications, spec_objects, floats, relations, views, and attributes
  2. RESOLVE: Resolve relations between objects (link target resolution, type inference)
  3. TRANSFORM: Pre-compute views, render external content (PlantUML, charts), prepare for output
  4. ANALYZE: Run analyze queries to validate data integrity, type constraints, cardinality rules
  5. EMIT: Assemble final documents and write to output formats (docx, html5, markdown, json)
RATIONALE:
Separation of concerns enables validation between phases, allows early abort on errors, and supports format-agnostic processing until the final output stage.
STATUS:
Approved

HLR-PIPE-002: Handler Registration and Prerequisites

The pipeline shall support handler registration with declarative <Prerequisites> for dependency ordering.

DESCRIPTION:

Handlers register via register_handler(handler) with required fields:

  • name: Unique string identifier for the handler
  • prerequisites: Array of handler names that must execute before this handler

Handlers declare participation in phases via hook methods (on_initialize, on_analyze, on_verify, on_transform, on_emit). Duplicate handler names cause registration error.

RATIONALE:
Declarative prerequisites decouple handler ordering from registration order, enabling modular handler development and preventing implicit ordering dependencies.
STATUS:
Approved

HLR-PIPE-003: Topological Ordering via Kahn’s Algorithm

The pipeline shall order handlers within each phase using topological sort with Kahn’s algorithm.

DESCRIPTION:

For each phase, the pipeline:

  1. Identifies handlers participating in the phase (those with on_{phase} hooks)
  2. Builds dependency graph from prerequisites (only for participating handlers)
  3. Executes Kahn’s algorithm to produce execution order
  4. Sorts alphabetically at each level for deterministic output
  5. Detects and reports circular dependencies with error listing remaining nodes
RATIONALE:
Kahn’s algorithm provides O(V+E) complexity, clear cycle detection, and deterministic ordering through alphabetic tie-breaking.
STATUS:
Approved

HLR-PIPE-004: Phase Abort on ANALYZE Errors

The pipeline shall abort execution after ANALYZE phase if any errors are recorded.

DESCRIPTION:
After running ANALYZE phase, the pipeline checks diagnostics:has_errors(). If true, execution halts before EMIT phase, with TRANSFORM already completed. Error message is logged with error count. This prevents generating invalid output from documents with specification violations.
RATIONALE:
Early abort on verification failures saves computation and prevents distribution of invalid specification documents. Errors in ANALYZE indicate data integrity issues that would produce incorrect outputs.
STATUS:
Approved

HLR-PIPE-005: Batch Dispatch for All Phases

The pipeline shall use a single batch dispatch model for all phases where handlers receive all contexts at once.

DESCRIPTION:

All handlers implement on_{phase}(data, contexts, diagnostics) hooks that receive the full contexts array. The pipeline orchestrator calls each handler’s hook once per phase via run_phase(), passing all document contexts. Handlers are responsible for iterating over contexts internally.

This enables cross-document optimizations, transaction batching, and parallel processing within any phase.

RATIONALE:
A uniform dispatch model simplifies the pipeline engine, eliminates the dual-path batch/per-doc dispatch, and allows handlers in any phase to optimize across all documents (e.g., wrapping DB operations in a single transaction, parallel output generation in EMIT).
STATUS:
Approved

HLR-PIPE-006: Context Creation and Propagation

The pipeline shall create and propagate context objects containing document metadata and configuration through all phases.

DESCRIPTION:

The execute(docs) method creates context objects for each input document with:

  • doc: Pandoc document AST (via DocumentWalker)
  • spec_id: Specification identifier derived from filename
  • config: Preset configuration (styles, captions, validation)
  • build_dir: Output directory path
  • output_format: Target format (docx, html5, etc.)
  • template: Template name for model loading
  • reference_doc: Path to reference.docx for styling
  • docx, html5: Format-specific configuration
  • outputs: Array of {format, path} for multi-format output
  • bibliography, csl: Citation configuration
  • project_root: Root directory for resolving relative paths

Context flows through all phases, enriched by handlers (e.g., verification results in ANALYZE phase).

RATIONALE:
Unified context object provides handlers with consistent access to document metadata and build configuration without global state, enabling testable and isolated handler implementations.
STATUS:
Approved

HLR-PIPE-007: CommonSpec Input Parsing

The system shall parse <CommonSpec> documents during the <INITIALIZE Phase> phase, lowering Markdown annotations into <Intermediate Representation> content tables.

DESCRIPTION:

The INITIALIZE phase <Handler>s parse the Pandoc <Abstract Syntax Tree> and populate the six IR content tables according to these rules:

  1. H1 headers register as <Specification> records with optional TYPE: prefix and @PID suffix
  2. H2-H6 headers register as <Spec Object> records with type inference (explicit TYPE: prefix, implicit alias lookup, or default type fallback)
  3. Blockquote lines (> key: value) register as <Attribute> records attached to the enclosing spec object
  4. Fenced code blocks with TypeRef:Label class register as <Spec Float> records
  5. Markdown links with (@) or (#) targets register as <Spec Relation> records
  6. Inline code with TypeRef: content syntax registers as <Spec View> records

Each handler populates its content table with content-addressable identifiers (SHA1) and preserves document ordering via file_seq.

RATIONALE:
Formal parsing rules ensure deterministic lowering from CommonSpec to SpecIR, enabling round-trip fidelity and predictable behavior across document structures.
STATUS:
Approved

HLR-PIPE-008: Include File Expansion

When a document contains .include code blocks, the system shall expand them by embedding the referenced file content before <Pipeline> processing.

DESCRIPTION:

Include expansion runs before the five-phase pipeline on each source document:

  1. Fenced code blocks with class .include are identified
  2. Each line in the block is treated as a relative file path
  3. Paths are resolved relative to the including file’s directory
  4. Referenced files are read, parsed, and recursively expanded
  5. The include block is replaced with the parsed content blocks

Cycle detection prevents infinite recursion. Maximum include depth is bounded. Source position tracking attributes are injected for diagnostic reporting.

RATIONALE:
Include expansion enables modular document authoring where specifications are composed from reusable fragments. Pre-pipeline expansion ensures all downstream handlers see a complete, flattened document.
STATUS:
Approved

HLR-PIPE-009: PID Auto-Generation

When a spec object does not have an explicit @PID, the system shall auto-generate a <Project Identifier> during the <RESOLVE Phase> phase based on the object’s type definition.

DESCRIPTION:

The PID generator runs in the RESOLVE phase before relation resolution:

  1. Non-<Composite Object Type> objects: PIDs are generated using the type’s pid_prefix and pid_format (e.g., HLR-%03d produces “HLR-001”), starting from the next available sequence number
  2. <Composite Object Type> objects: Hierarchical PIDs are qualified by the specification PID (e.g., “SRS-sec1.2.3”)

Auto-generated PIDs never overwrite explicit @PID annotations. Collision detection ensures global uniqueness across all specifications.

RATIONALE:
Auto-generation reduces authoring burden while maintaining stable identifiers for traceability.
STATUS:
Approved

HLR-PIPE-010: Relation Type Inference

The system shall infer relation types during the <RESOLVE Phase> phase using constraint-based matching with <Specificity Scoring> scoring.

DESCRIPTION:

For each unresolved relation, the relation analyzer:

  1. Filter: Identifies candidate relation types whose constraints are compatible with the relation’s <Relation Selector>, source attribute, source type, and target type
  2. Resolve: Calls the resolver (determined by the type’s extends chain root) to find the target object
  3. Score: Counts matching non-NULL constraints across all four dimensions; NULL constraints act as wildcards (match anything but do not increase specificity)
  4. Pick: The highest specificity match wins; ties mark the relation as ambiguous

Same-specification targets are preferred over cross-specification targets. The relation’s target_ref and type_ref are updated in the database.

RATIONALE:
Type inference from constraints eliminates explicit relation type annotation in source documents, reducing authoring burden. Specificity scoring ensures the most specific matching type is selected, enabling both generic and specialized relation types to coexist.
STATUS:
Approved

HLR-PIPE-011: Prerequisite-Not-Found Diagnostic

The system shall distinguish a handler prerequisite that resolves to another phase from one that resolves to no registered handler at all, emitting a diagnostic only for the latter.

DESCRIPTION:
The per-phase topological sort deliberately drops a prerequisite that does not participate in the current phase (handlers cannot be ordered across phases). Pipeline:validate_prerequisites(), run once when all handlers are registered, distinguishes the two cases the filter conflates: a prerequisite registered in ANOTHER phase (legitimate, silent) versus one registered in NO phase (a typo/bug). It emits a prerequisite_not_found diagnostic only for the latter, and only for phase handlers (those declaring an on_<phase> hook) – a decorated per-item callback’s prerequisites field is inert and not flagged. The cross-phase filter itself is unchanged.
RATIONALE:
An unresolved same-phase prerequisite can change handler order and corrupt output. The pipeline reports this error without rejecting valid cross-phase prerequisites.
STATUS:
Approved

HLR-PIPE-012: Section Scope Termination

A <Spec Object>’s scope shall be closed either automatically by the next header of equal-or-shallower level, or manually by a ---- thematic break, and a heading shall not be empty.

DESCRIPTION:

During INITIALIZE, the first ---- thematic break ends the active section scope. The marker does not appear in rendered output. Later content keeps its document position and belongs to the parent section.

An empty heading (a header whose title is blank) is rejected: it would otherwise render as an empty numbered heading and corrupt downstream numbering. The object_broken_hierarchy analyze query reports it as an error and directs the author to use ---- instead.

RATIONALE:
The marker closes a section without adding a rendered heading. Rejection of empty headings prevents invalid numbered sections.
STATUS:
Approved

2.2 Storage Requirements

SF-002: Specification Persistence

<SQLite Database>-based storage with incremental build support and output caching.

DESCRIPTION:
Groups requirements for the persistence layer including ACID-compliant storage, <EAV Model> attribute model, <Build Cache> , <Output Cache> , and incremental rebuild support.
RATIONALE:
Reliable persistence with change detection enables efficient rebuilds for large specification projects.

HLR-STOR-001: SQLite Persistence

The system shall persist all specification data to SQLite database with ACID guarantees.

DESCRIPTION:
All specifications, spec_objects, floats, relations, views, and attribute values stored in SQLite database. Database operations wrapped in transactions to ensure atomicity, consistency, isolation, and durability.
RATIONALE:
SQLite provides a reliable, single-file persistence layer suitable for specification documents. ACID guarantees prevent data corruption during concurrent access or system failures.
STATUS:
Approved

HLR-STOR-002: EAV Attribute Model

The system shall store spec object attributes using Entity-Attribute-Value pattern.

DESCRIPTION:
Attribute values stored in attribute_values table with polymorphic typed columns (string_value, int_value, real_value, bool_value, date_value, enum_ref). Each attribute record links to owner object via owner_ref and stores datatype for proper retrieval.
RATIONALE:
EAV pattern enables flexible attribute schemas without database migrations. Different spec object types (HLR, LLR, VC) have different attributes that can evolve independently.
STATUS:
Approved

HLR-STOR-003: Build Cache

The system shall maintain a build cache for document hash tracking.

DESCRIPTION:
Source file hashes stored as build_graph nodes (root_path, node_path, node_sha1), where the root document is recorded as its own node. Build cache module provides is_document_dirty() to check if document content has changed since last build. Hash comparison enables change detection.
RATIONALE:
Hash-based change detection allows the build system to skip unchanged documents, reducing rebuild times for large specification sets.
STATUS:
Approved

HLR-STOR-004: Output Cache

The system shall cache output generation state with P-IR hash and timestamps.

DESCRIPTION:
Output cache stored in output_cache table (spec_id, output_path, pir_hash, generated_at). P-IR (Processed Intermediate Representation) hash captures complete specification state. is_output_current() checks if output file exists and P-IR hash matches cached value.
RATIONALE:
Output caching avoids regenerating unchanged outputs (docx, html5). P-IR hash ensures output is regenerated when any upstream data changes, not just source file changes.
STATUS:
Approved

HLR-STOR-006: EAV Pivot Views for External Queries

The system shall generate per-object-type SQL views that pivot the EAV attribute model into typed columns for external BI queries.

DESCRIPTION:
For each non-composite spec_object_type, a view named view_{type_lower}_objects is dynamically generated (e.g., view_hlr_objects, view_vc_objects). Each view flattens the EAV join into one row per object with typed attribute columns, enabling queries like SELECT * FROM view_hlr_objects WHERE status = 'approved'. These views are NOT used by internal pipeline queries — all internal code queries the raw EAV tables directly because it needs access to raw_value, datatype, ast, enum_ref, and other columns that the pivot views abstract away. Internal queries also frequently operate cross-type or need COUNT/EXISTS checks that the MAX()-based pivot cannot provide.
RATIONALE:
External BI tools, ad-hoc SQL queries, and custom model scripts benefit from a flat relational interface over the EAV model. Generating views at runtime from the type system ensures the columns always match the current model configuration without manual maintenance.
STATUS:
Approved

HLR-STOR-005: Incremental Rebuild Support

The system shall support incremental rebuilds via build graph tracking.

DESCRIPTION:
Build graph stored in build_graph table (root_path, node_path, node_sha1). Records the root document and its include file dependencies as nodes. is_document_dirty() checks the root document and all includes in one node walk. update_build_graph() refreshes the file set after successful build.
RATIONALE:
Specification documents often include sub-files. Incremental builds must detect changes in any included file to trigger rebuild of parent document. Build graph captures this dependency structure.
STATUS:
Approved

2.3 Types Domain Requirements

SF-003: Type System

Dynamic type system providing typed containers for <Specification>, <Spec Object>, <Spec Float>, <Spec View>, <Spec Relation>, and <Attribute>.

DESCRIPTION:
Groups requirements for the six core containers that store parsed specification data plus the verification-view validation framework.
RATIONALE:
A typed container model enables schema validation, type-specific rendering, and data integrity checking through SQL analyze queries.

HLR-TYPE-001: Specifications Container

The type system shall provide a specifications container for registering document-level specification records.

DESCRIPTION:

The specifications table stores metadata for each specification document parsed during <INITIALIZE Phase> phase:

  • identifier: Unique specification ID derived from filename (e.g., “srs-main”)
  • root_path: Source file path for the specification
  • long_name: Human-readable title extracted from L1 header
  • type_ref: Specification type (validated against spec_specification_types)
  • pid: Optional PID from @PID syntax in L1 header

L1 headers register as specifications. Type validation checks spec_specification_types table. Invalid types fall back to default or emit warning.

RATIONALE:
Specifications represent the top-level organizational unit for document hierarchies. Storing specification metadata enables cross-document linking and multi-document project support.
STATUS:
Approved

HLR-TYPE-002: Spec Objects Container

The type system shall provide a spec_objects container for hierarchical specification objects.

DESCRIPTION:

The spec_objects table stores structured specification items extracted from L2+ headers:

  • identifier: SHA1 hash of source path + line + title (content-addressable)
  • specification_ref: Foreign key to parent specification
  • type_ref: Object type (validated against spec_object_types)
  • from_file: Source file path
  • file_seq: Document order sequence number
  • pid: Project ID from @PID syntax (e.g., “REQ-001”)
  • title_text: Header text without type prefix or PID
  • label: Unified label for cross-referencing (format: {type_lower}:{title_slug})
  • level: Header level (2-6)
  • start_line, end_line: Source line range
  • ast: Serialized Pandoc AST (JSON) for section content

Type resolution order: explicit TYPE: prefix, implicit alias lookup, default type fallback.

RATIONALE:
Content-addressable identifiers enable change detection for incremental builds. PID-based anchors provide stable cross-references independent of title changes.
STATUS:
Approved

HLR-TYPE-003: Spec Floats Container

The type system shall provide a spec_floats container for numbered floating content (figures, tables, listings).

DESCRIPTION:

The spec_floats table stores content blocks that receive sequential numbering:

  • identifier: Short format “float-{8-char-sha1}” for DOCX compatibility
  • specification_ref: Foreign key to parent specification
  • type_ref: Float type resolved from aliases (e.g., “csv” -> “TABLE”, “puml” -> “FIGURE”)
  • from_file: Source file path
  • file_seq: Document order for numbering
  • label: User-provided label for cross-referencing
  • number: Sequential number within counter_group (assigned in <TRANSFORM Phase>)
  • caption: Caption text from attributes
  • raw_content: Original code block text
  • raw_ast: Serialized Pandoc CodeBlock (JSON)
  • parent_object_ref: Foreign key to containing spec_object
  • attributes: JSON-serialized attributes (caption, source, language)
  • syntax_key: Original class syntax for backend matching

<Counter Group> share numbering (e.g., FIGURE, CHART, PLANTUML all increment “FIGURE” counter).

RATIONALE:
Type aliasing supports user-friendly syntax (e.g., csv:data instead of TABLE:data). Counter groups enable semantic grouping of related float types under a single numbering sequence.
STATUS:
Approved

HLR-TYPE-004: Spec Views Container

The type system shall provide a spec_views container for data-driven view definitions.

DESCRIPTION:

The spec_views table stores view definitions from inline view syntax (views are inline-only; a view alone in its own paragraph is transparently promoted to block output at EMIT):

  • identifier: SHA1 hash of specification + sequence + content
  • specification_ref: Foreign key to parent specification
  • view_type_ref: Uppercase view type (e.g., “TOC”, “SYMBOL”, “MATH”, “ABBREV”)
  • from_file: Source file path
  • file_seq: Document order sequence number
  • raw_ast: View definition content (symbol path, expression, parameters)

View types with needs_external_render = 1 in spec_view_types are delegated to specialized renderers. Inline views use prefix: content syntax (e.g., symbol: Class.method). The content may carry key=value parameters (e.g., [TOC], allocation_matrix: status=complete); the EMIT dispatcher parses the prefix, content, and parameters once and passes them to the view’s render hooks (ctx.subject.prefix/content/params).

RATIONALE:
Separating view definitions from rendering enables format-agnostic processing. External render delegation supports complex transformations (PlantUML, charts) without core handler changes.
STATUS:
Approved

HLR-TYPE-005: Spec Relations Container

The type system shall provide a spec_relations container for tracking links between specification elements.

DESCRIPTION:

The spec_relations table stores inter-element references:

  • identifier: SHA1 hash of specification + target + type + parent
  • specification_ref: Foreign key to parent specification
  • source_ref: Foreign key to source spec_object
  • target_text: Raw link target from syntax (e.g., “REQ-001”, “fig:diagram”)
  • target_ref: Resolved target identifier (populated in <RESOLVE Phase> phase)
  • type_ref: Relation type from spec_relation_types (e.g., “TRACES”, “XREF_FIGURE”)
  • from_file: Source file path

Link syntax: [PID](@) for PID references, [type:label](#) for float references, [@citation] for bibliographic citations. Default relation types are determined by is_default and link_selector columns in spec_relation_types.

RATIONALE:
Deferred resolution (target_text -> target_ref) enables forward references and cross-document linking. Type inference from source/target context reduces explicit markup requirements.
STATUS:
Approved

HLR-TYPE-006: Spec Attributes Container

The type system shall provide a spec_attributes container for structured metadata on specification objects.

DESCRIPTION:

The spec_attributes table stores typed attribute values extracted from blockquote syntax:

  • identifier: SHA1 hash of specification + owner + name + value
  • specification_ref: Foreign key to parent specification
  • owner_ref: Foreign key to owning spec_object
  • name: Attribute name (field name without colon)
  • raw_value: Original string value
  • string_value, int_value, real_value, bool_value, date_value: Type-specific columns
  • enum_ref: Foreign key to enum_values for ENUM types
  • ast: JSON-serialized Pandoc AST for rich content (XHTML type)
  • datatype: Resolved datatype from spec_attribute_types

Attribute syntax: > name: value in blockquotes following headers. Datatypes include STRING, INTEGER, REAL, BOOLEAN, DATE, ENUM, XHTML. Multi-line attributes use continuation blocks.

RATIONALE:
Multi-column typed storage enables SQL queries with type-appropriate comparisons. Storing original AST preserves formatting for XHTML attributes with links, emphasis, or lists.
STATUS:
Approved

HLR-TYPE-007: Type Validation

The type system shall provide analyze queries that detect data integrity violations across all specification containers.

The type system described above is not fixed at compile time. The following section defines how dic:model directories extend it with custom object types, float renderers, dic:data-view generators, and style presets.

DESCRIPTION:

Verification views are SQL queries registered in the <ANALYZE Phase> phase that check for constraint violations:

  • Specification-level (missing required attributes, invalid types)
  • Object-level (missing required, cardinality, cast failures, invalid enum/date, bounds)
  • Float-level (orphans, duplicate labels, render failures, invalid types)
  • Relation-level (unresolved, dangling, ambiguous)

The validation policy (configurable in project.yaml) determines severity: error, warn, or ignore.

RATIONALE:
Automated validation enables early detection of specification errors before document generation. Configurable severity allows projects to gradually enforce stricter quality standards.
STATUS:
Approved

2.4 Extension Requirements

SF-005: Extension Framework

The extension framework shall let models define types, rendering behavior, data views, verification queries, and output processing.

DESCRIPTION:
A model contains descriptor modules and optional format-specific components. The host loads each model without type-specific control flow.
RATIONALE:
Model extensions support domain-specific documents without changes to the core pipeline.

HLR-EXT-001: Type Descriptor Loading

The system shall load type descriptors from model directories.

DESCRIPTION:

The host scans these directories under models/{model}/types/:

  • objects/ for kind = "object"
  • specifications/ for kind = "specification"
  • floats/ for kind = "float"
  • views/ for kind = "view"
  • relations/ for kind = "relation"

Each Lua module returns one descriptor table. The host uses schema.id as the type identifier. The file name does not define the identifier.

RATIONALE:
One loading contract gives all type categories the same validation and registration behavior.
STATUS:
Approved

HLR-EXT-002: Model Directory Structure

The system shall use a standard model directory structure.

DESCRIPTION:

A model can contain the following paths:

models/{model}/
  model.yaml
  types/
    objects/
    specifications/
    floats/
    views/
    relations/
  analyze_queries/
  filters/
  postprocessors/
  styles/
  tools/

Only the directories required by the model must exist. A type directory can contain a Lua file or a subdirectory with init.lua.

RATIONALE:
A standard structure permits deterministic discovery and keeps model-owned components together.
STATUS:
Approved

HLR-EXT-003: Descriptor Registration

Each extension module shall return one descriptor with kind, schema, and optional hooks fields.

DESCRIPTION:
The host shall reject an unknown kind, a missing schema.id, an invalid hook, and behavior outside hooks. The host shall register schema data and index each behavior hook.
RATIONALE:
A uniform descriptor removes category-specific registration interfaces.
STATUS:
Approved

HLR-EXT-004: Type Schema

Each descriptor shall declare the schema fields required by its kind.

DESCRIPTION:
The host shall store schema data in the applicable SpecIR type tables. Object and float schemas can declare attribute definitions. A schema can use extends to inherit attributes and hooks.
RATIONALE:
Declarative schemas support registration, validation, and inheritance without procedural setup.
STATUS:
Approved

HLR-EXT-005: Model Resolution and Overlay

The system shall resolve and load models as ordered overlays.

DESCRIPTION:
The host shall search $SPECCOMPILER_HOME/models/{model} first. It shall then search models/{model} under the working directory. The host shall load default before the selected model. A later descriptor with the same kind and schema.id shall replace the earlier descriptor. A missing selected or required model shall stop the build.
RATIONALE:
Ordered overlays let a model replace selected definitions and inherit all other default definitions.
STATUS:
Approved

HLR-EXT-006: External Renderer Hooks

An externally rendered float shall declare task preparation and result handling in its descriptor.

DESCRIPTION:
The descriptor shall set schema.needs_external_render = true. Its hooks table shall provide prepare_task and handle_result. The core shall prepare tasks, apply the render cache, run tasks, and dispatch results. A float shall not declare both render and external-render hooks.
RATIONALE:
The hook pair separates type-specific rendering from process scheduling and cache control.
STATUS:
Approved

HLR-EXT-007: Data View Hooks

The system shall obtain generated data from hooks on view descriptors.

DESCRIPTION:
A view can declare a dataset hook for chart data. A TABLE_VIEW subtype shall provide an inherited or local build_block hook. The host shall map inline_prefix and aliases to view identifiers.
RATIONALE:
Data hooks keep query logic in the model that defines the view.
STATUS:
Approved

HLR-EXT-008: Hook Index

The host shall index registered behavior hooks by kind, type identifier, and hook name.

DESCRIPTION:
Consumers shall resolve hooks through the host index. Inherited lookup shall follow the schema.extends chain. Phase hooks shall use pipeline registration and shall not use the behavior-hook index.
RATIONALE:
One hook index provides deterministic dispatch for all model types.
STATUS:
Approved

HLR-EXT-009: Canonical Hook Context

The host shall pass each behavior hook one frozen context table.

DESCRIPTION:
The context shall contain the fields for its tier. The subject field shall contain the hook-specific input. The capability field shall identify the hook. The ctx:require(field) method shall stop execution when a required field is nil.
RATIONALE:
One context argument prevents positional-argument drift and makes required data explicit.
STATUS:
Approved

HLR-EXT-010: Model Manifest

The host shall read model dependencies from model.yaml.

DESCRIPTION:
The optional requires field shall contain model names. The host shall load each required model before the requesting model. It shall load each model at most once. A missing manifest or requires field shall define no dependencies.
RATIONALE:
Explicit dependencies produce a deterministic overlay order.
STATUS:
Approved

HLR-EXT-011: Hook Validation and Phase Registration

The host shall validate hooks and register phase participation from the descriptor.

DESCRIPTION:
A hook name shall be valid for the descriptor kind. A hook shall return the value type defined by its contract. A phase hook shall use the name on_<phase> in hooks. The host shall register phase hooks under <lowercase schema.id>_handler. The optional schema.phase_prerequisites field shall define handler ordering.
RATIONALE:
Load-time and dispatch-time checks detect invalid extensions at their source.
STATUS:
Approved

HLR-EXT-012: Analyze Query Descriptor

The system shall register each analysis query as a kind = "analyze" descriptor.

DESCRIPTION:
The schema shall contain id, policy_key, view, sql, and optional disabled. The optional message hook shall format a diagnostic for one query row. A later descriptor with the same policy_key shall replace the earlier descriptor in place. A descriptor with disabled = true shall remove that policy key.
RATIONALE:
Analyze queries use the same descriptor validation and model overlay rules as other extensions.
STATUS:
Approved

HLR-CFG-001: Manifest Configuration

The system shall read build configuration from project.yaml.

DESCRIPTION:
Environment variables can locate the installed toolchain and support terminal detection. Model behavior and output configuration shall use the frozen project configuration.
RATIONALE:
One configuration source makes builds reproducible.
STATUS:
Approved

2.5 Output Requirements

SF-004: Multi-Format Publication

Assembles transformed content and publishes DOCX/HTML5 outputs with cache-aware emission.

DESCRIPTION:
Single-source, multi-target publication. Groups requirements for document assembly, float resolution/numbering, and format-specific output generation.
RATIONALE:
Technical documentation must be publishable in multiple formats from a single Markdown source.

HLR-OUT-001: Document Assembly

The system shall reconstruct a complete Pandoc document from <Intermediate Representation> database content for each specification, preserving document order and embedding all resolved content.

DESCRIPTION:

During the <EMIT Phase> phase, the assembler queries spec_objects ordered by file_seq, decodes stored JSON <Abstract Syntax Tree> fragments back to Pandoc blocks, adjusts header levels for cross-file includes, and embeds:

  1. Specification title: From specifications.header_ast, wrapped in a title Div
  2. Spec objects: All objects belonging to the specification, in file_seq order, with their rendered body AST
  3. Spec floats: Placeholder CodeBlocks for floats at their document positions (resolved later by the float emitter)
  4. Spec views: Placeholder CodeBlocks for views at their document positions (expanded later by the view emitter)

The assembled document includes Pandoc metadata built from specification attributes (title, author, date). The result is a valid pandoc.Pandoc document suitable for format-specific output.

Header-level adjustment rescales the shallowest header level to the top output level uniformly; it therefore assumes the source header levels form a well-formed tree (contiguous nesting, a single root depth). That precondition is enforced declaratively by the object_broken_hierarchy analyze query, which rejects skipped levels and orphaned roots before emission rather than letting a malformed hierarchy mis-render silently.

RATIONALE:
Decoupling parsing from rendering enables format-agnostic processing through the pipeline. Database-backed assembly allows cross-document operations (shared numbering, cross-references) that sequential file processing cannot achieve.
STATUS:
Approved

HLR-OUT-002: Float Resolution

The system shall replace <Float> placeholder blocks in the assembled document with their rendered content, using results from the <TRANSFORM Phase> phase.

DESCRIPTION:

After document assembly, the float emitter walks all blocks and replaces CodeBlock placeholders (identified by float labels) with rendered Div elements containing:

  1. Rendered content: The resolved_ast from spec_floats (SVG images for PlantUML, parsed tables for CSV, chart images for ECharts)
  2. Caption: A formatted caption with type prefix and sequential number (e.g., “Figure 1 – Diagram Title”)
  3. Semantic classes: CSS classes (speccompiler-float, speccompiler-caption, type-specific class) for format-specific styling
  4. Bookmark anchor: An identifier anchor for cross-reference navigation

Floats whose resolved_ast is NULL (failed external renders) are preserved as error placeholders.

RATIONALE:
Separating float resolution from assembly enables parallel external rendering during TRANSFORM while maintaining correct document insertion order during EMIT.
STATUS:
Approved

HLR-OUT-003: Float Numbering

The system shall assign sequential numbers to <Float>s within their <Counter Group>, producing a single numbering sequence across all documents in the project.

DESCRIPTION:

During the <TRANSFORM Phase> phase, the float numberer:

  1. Queries all floats across all specifications, ordered by file_seq
  2. Groups floats by their counter_group (e.g., FIGURE, TABLE, LISTING, EQUATION)
  3. Assigns monotonically increasing numbers within each group (starting at 1)
  4. Float types sharing a counter_group share the same sequence (e.g., FIGURE, CHART, and PLANTUML all increment the “FIGURE” counter)

The assigned numbers are stored in spec_floats.number and used for caption formatting and cross-reference display text.

RATIONALE:
Consistent cross-document numbering prevents duplicate figure/table numbers and enables stable cross-references. Counter group sharing allows semantically related types (all visual content) to form natural sequences.
STATUS:
Approved

HLR-OUT-004: Multi-Format Output

The system shall generate output documents in all formats specified by the project configuration, skipping outputs whose <Processed Intermediate Representation> hash matches the cached value.

DESCRIPTION:

The emitter orchestrator iterates over config.outputs (an array of {format, path} pairs) and for each specification:

  1. Serializes the assembled Pandoc document to an intermediate JSON file
  2. Checks the output cache (is_output_current()) and skips generation when the P-IR hash matches
  3. Applies format-specific Pandoc Lua filters (e.g., docx.lua, html.lua from the model’s filters/ directory)
  4. Invokes Pandoc for format conversion
  5. Runs format-specific postprocessors after Pandoc generation completes
  6. Cleans up intermediate JSON files

Supported output formats: DOCX, HTML5, Markdown, JSON. Multiple formats can be generated from a single pipeline execution.

RATIONALE:
Single-source multi-target publication eliminates content duplication. Cache-aware skipping avoids regenerating unchanged outputs, reducing build times for large specification projects.
STATUS:
Approved

HLR-OUT-005: DOCX Generation

The system shall generate DOCX output with style customization via preset-based reference document generation and OOXML post-processing.

DESCRIPTION:

DOCX output generation follows this sequence:

  1. Preset loading: Loads style preset definitions from models/{model}/styles/presets/ with extends-chain merging and circular dependency detection
  2. Reference document generation: Generates a reference.docx from the resolved preset containing custom Word styles (headings, captions, code blocks, table styles)
  3. Pandoc conversion: Invokes Pandoc with --reference-doc pointing to the generated reference and format-specific Lua filters
  4. OOXML post-processing: Modifies the DOCX archive to apply style fixups, numbering definitions, and structural corrections that Pandoc cannot produce natively

The reference document is cached and regenerated only when the preset hash changes.

RATIONALE:
Preset-based styling enables consistent corporate branding without manual Word template editing. OOXML post-processing addresses Pandoc limitations for advanced Word formatting requirements.
STATUS:
Approved

HLR-OUT-006: HTML5 Generation

The system shall generate standalone HTML5 output with table of contents, section numbering, and embedded resources when configured.

DESCRIPTION:

HTML5 output generation follows this sequence:

  1. Pandoc conversion: Invokes Pandoc with HTML5-specific options from project configuration (number_sections, table_of_contents, toc_depth, standalone, embed_resources)
  2. Resource embedding: When embed_resources is enabled, all CSS, JavaScript, and image assets are embedded inline for single-file distribution
  3. Search index: When <Full-Text Search> tables are populated, the HTML5 postprocessor bundles the search index for client-side full-text search
  4. Internal links: Cross-reference (@) links resolve to #anchor URLs for in-page navigation

Configuration is specified in the html5: section of project.yaml.

RATIONALE:
Standalone HTML5 with embedded resources enables documentation distribution without web server infrastructure. FTS integration provides search capability for large specification sets.
STATUS:
Approved

HLR-OUT-007: Full-Text Search Indexing

The system shall populate dic:full-text-search virtual tables during the dic:emit-phase phase to enable full-text search across specification content.

Once documents are assembled and published, the system must also guarantee that its builds are reproducible and its processing is auditable. The following section addresses these integrity concerns.

DESCRIPTION:

The FTS indexer creates and populates three FTS5 virtual tables with Porter stemming:

  1. fts_objects: Indexes spec object titles and body text, keyed by identifier and spec_id
  2. fts_attributes: Indexes attribute names and string values, keyed by owner_ref and spec_id
  3. fts_floats: Indexes float captions and raw source content, keyed by identifier and spec_id

<Abstract Syntax Tree> content is converted to plain text before indexing.

RATIONALE:
Full-text search enables users to find specification content across large document sets. FTS5 with Porter stemming provides standard information retrieval capabilities suitable for technical documentation.
STATUS:
Approved

2.6 Audit & Integrity Requirements

SF-006: Audit and Integrity

Deterministic compilation, reproducible builds, and audit trail integrity.

DESCRIPTION:
Encompasses content-addressed hashing for incremental build detection, structured <Newline-Delimited JSON> logging for audit trails, include dependency tracking for proper cache invalidation, and structured diagnostic reporting for error traceability.
RATIONALE:
Certification environments require reproducible builds and auditable processing trails for traceability evidence.

HLR-AUDIT-001: Content-Addressed Hashing

The system shall compute SHA1 content hashes for all source documents and include files to enable change detection.

DESCRIPTION:

The build engine computes SHA1 hashes for each document. Hashes are compared against the cached values recorded as build_graph nodes (the root document is its own node, alongside its includes):

  1. Every node of the document’s build graph is compared against its current file hash
  2. If all hashes match, the document is skipped (cached <Intermediate Representation> state is reused)
  3. If any hash differs, a node file is missing, or no root node is recorded, the document is rebuilt from source
  4. After successful rebuild (no <ANALYZE Phase> errors), the build_graph nodes are rewritten with current hashes

This provides O(1) change detection without parsing unchanged documents.

RATIONALE:
Content-addressed hashing provides reliable change detection independent of filesystem timestamps. Deferred hash updates prevent cache poisoning from partial or failed builds.
STATUS:
Approved

HLR-AUDIT-002: Include Dependency Tracking

When a document contains include directives, the system shall track all included file dependencies in a build graph and detect circular includes.

DESCRIPTION:

Before <Pipeline> execution, the include handler expands .include code blocks by:

  1. Resolving include paths relative to the source file directory
  2. Detecting circular includes via a processed-file set (raises error on cycle)
  3. Recursively expanding nested includes up to a bounded maximum depth
  4. Recording all include paths and their SHA1 hashes in the build_graph table (root_path, node_path, node_sha1)

Subsequent builds use this graph to check if any included file has changed, triggering a rebuild of the root document when needed.

RATIONALE:
Include dependency tracking ensures that changes to sub-files correctly invalidate parent documents. Cycle detection prevents infinite recursion in include hierarchies.
STATUS:
Approved

HLR-AUDIT-003: Structured Diagnostic Reporting

The system shall collect and report processing errors and warnings with source location information throughout all pipeline phases.

DESCRIPTION:

The diagnostics collector provides structured error/warning reporting:

  1. Collection: <Handler>s report issues via diagnostics:error(file, line, code, msg) and diagnostics:warn(file, line, code, msg)
  2. Structured data: Each diagnostic record contains file path, line number, diagnostic key, and human-readable message
  3. Severity control: The has_errors() method enables the <Pipeline> to determine abort conditions after the <ANALYZE Phase> phase
  4. Output integration: Diagnostics are emitted through the structured logger with file and line context

Diagnostic keys are stable identifiers (e.g., invalid_enum, missing_required, dangling_relation) suitable for filtering and CI policy.

RATIONALE:
Structured diagnostics with source locations enable IDE integration, CI/CD reporting, and user-friendly error resolution. Machine-parseable diagnostic keys enable automated error classification.
STATUS:
Approved

HLR-AUDIT-004: Structured Logging

The system shall provide <Newline-Delimited JSON> structured logging with TTY-aware output formatting.

DESCRIPTION:

The logging subsystem supports two output modes selected automatically or via configuration:

  1. NDJSON mode (non-TTY): Outputs one JSON object per line with fields: level, message, timestamp, and optional context. Suitable for CI/CD log aggregation and jq filtering
  2. Console mode (TTY): Outputs human-readable formatted messages with ANSI color coding (respects NO_COLOR environment variable). Includes timestamp and level indicator

Log levels: DEBUG, INFO, WARN, ERROR. Configured via config.logging.level with environment override SPECCOMPILER_LOG_LEVEL.

RATIONALE:
NDJSON enables machine-parseable audit trails for certification environments. TTY-aware formatting provides developer ergonomics during interactive use. The NO_COLOR standard ensures accessibility compliance.
STATUS:
Approved

HLR-AUDIT-005: Build Reproducibility

Given identical source files, project configuration, and tool versions, the system shall produce identical outputs.

The glossary below defines the domain vocabulary used throughout this specification. Each term corresponds to a cross-reference encountered in the requirements above and is defined here with its purpose, scope, and usage context.

DESCRIPTION:

Build reproducibility is ensured through:

  1. Deterministic parsing: Content-addressable SHA1 identifiers ensure consistent object identity
  2. Deterministic ordering: <Topological Sort> with alphabetic tie-breaking produces the same <Handler> execution order
  3. Deterministic numbering: Float numbers assigned by file_seq ordering, which is stable across builds
  4. Deterministic output: Pandoc invoked with the same arguments and reference documents produces identical output

Cache invalidation is based solely on content hashes, not filesystem timestamps or system state.

RATIONALE:
Reproducible builds are a fundamental certification requirement in aerospace and safety-critical domains. Content-addressed processing ensures that build artifacts can be independently verified.
STATUS:
Approved

2.7 System Concepts

TERM-COMMONSPEC: CommonSpec

A structured Markdown language for authoring typed, traceable specifications.

DESCRIPTION:

Purpose: Defines the input language for SpecCompiler. Extends standard Markdown (CommonMark) with six constructs: specifications, spec objects, spec floats, attributes, spec relations, and spec views.

Architecture: CommonSpec (language) compiles into SpecIR (intermediate representation) via SpecCompiler (compiler).

Specification: See the CommonSpec Language Specification (docs/commonspec/) for the formal definition.

DOMAIN:
Core
TERM:
CommonSpec

TERM-SPECIR: SpecIR

A typed relational intermediate representation for specifications, stored in SQLite.

ACRONYM:
SpecIR
DESCRIPTION:

Purpose: Provides a portable, queryable storage format for specification data. Compilation target for CommonSpec and interchange format for other tools (ReqIF, DOORS CSV, SQL).

Schema: Two core layers: Type System tables (metamodel) and Content tables (data). Build infrastructure and FTS tables are not part of the SpecIR standard.

Specification: See the SpecIR Schema Specification (docs/specir/) for the formal definition.

DOMAIN:
Core
TERM:
Specification Intermediate Representation

TERM-20: RESOLVE Phase

The second phase in the pipeline that resolves references and infers types.

DESCRIPTION:

Purpose: Resolves cross-references between spec objects and infers missing type information.

Position: Second phase after INITIALIZE, before TRANSFORM.

TERM-30: Build Cache

SHA1 hashes for detecting document changes.

DESCRIPTION:

Purpose: Stores content hashes to detect which documents have changed since last build.

Implementation: Compares current file hash against cached hash to skip unchanged files.

TERM-28: Counter Group

Float types sharing a numbering sequence.

DESCRIPTION:

Purpose: Groups related float types to share sequential numbering.

Example: FIG and DIAGRAM types may share a counter, producing Figure 1, Figure 2, etc.

TERM-36: CSC (Computer Software Component)

A MIL-STD-498 architectural decomposition element representing a subsystem, layer, package, or service.

DESCRIPTION:

Purpose: Groups software units into higher-level structural components for design allocation.

Examples: src/core, src/db, src/infra.

TERM-37: CSU (Computer Software Unit)

A MIL-STD-498 implementation decomposition element representing a source file or code unit.

DESCRIPTION:

Purpose: Captures file-level implementation units allocated to functional descriptions.

Examples: src/core/pipeline.lua, src/db/manager.lua.

TERM-35: Data View

A view descriptor whose dataset data hook generates data for chart injection.

DESCRIPTION:

Purpose: Produces structured data that can be injected into chart floats.

Implementation: A dataset DATA hook receives a frozen data context (reading subject.params) and returns a { source / data / links } dataset. The descriptor is resolved from models/{requested}/types/views/{view} with fallback to models/default/types/views/{view}.

TERM-EAV: EAV Model

Entity-Attribute-Value pattern for typed attribute storage.

DESCRIPTION:

Purpose: Flexible schema for storing typed attributes on spec objects.

Structure: Entity (spec object), Attribute (key name), Value (typed content).

TERM-23: EMIT Phase

The final phase in the pipeline that assembles and outputs documents.

DESCRIPTION:

Purpose: Assembles transformed content and writes final output documents.

Position: Final phase after ANALYZE.

TERM-04: Float

A numbered element (table, figure, diagram) with caption and cross-reference. See <Spec Float> for full definition.

TERM-34: External Renderer

Subprocess-based rendering for types like PLANTUML, CHART.

DESCRIPTION:

Purpose: Delegates rendering to external tools via subprocess execution.

Examples: PlantUML JAR for diagrams, chart libraries for data visualization.

TERM-16: Handler

A named hook that a type descriptor contributes to the pipeline, indexed by the host engine and dispatched by capability rather than declared on a M.handler surface.

DESCRIPTION:

Purpose: Encapsulates processing logic for a content type. Behaviour lives only under a descriptor’s hooks table; the host classifies each hook by name and indexes it into the (kind, id) -> hook map.

Structure: Two kinds of contributions, both keyed by hook name:

  • Phase participationon_<phase> functions (on_initialize, on_analyze, on_transform, on_verify, on_emit) declared in hooks beside the behavior hooks; the host synthesizes them into pipeline:register_handler under the derived name <lower(id)>_handler, ordered via schema.phase_prerequisites. Each runs once per phase with full pipeline context.
  • Per-item hooks — RENDER hooks (render, render_block, render_link, message) and DATA hooks (dataset, build_block, transform, resolve, prepare_task, handle_result). The hook NAME selects one of two context tiers (a frozen render ctx or a frozen data ctx) and the return type the hook owes. Consumers resolve them via get_hook_inherited, which walks the extends chain.

TERM-19: INITIALIZE Phase

The first phase in the pipeline that parses AST and populates IR containers.

DESCRIPTION:

Purpose: Parses markdown AST and populates intermediate representation containers.

Position: First phase, entry point for document processing.

TERM-33: Model

A collection of type descriptors and styles for a domain, overlaid onto the default model by the host engine.

DESCRIPTION:

Purpose: Bundles related type descriptors (each carrying its own hooks) and styling for specific documentation domains. The host overlays default then each model later-wins-by-id, so a model overrides only the type ids it redefines.

Examples: SRS model for software requirements, HRS model for hardware requirements.

TERM-31: Output Cache

Timestamps for incremental output generation.

DESCRIPTION:

Purpose: Tracks when outputs were last generated to enable incremental builds.

Implementation: Compares source modification time against cached output timestamp.

TERM-17: Phase

A distinct stage in document processing with specific responsibilities.

DESCRIPTION:

Purpose: Separates document processing into well-defined sequential stages.

Phases: INITIALIZE, RESOLVE, TRANSFORM, ANALYZE, EMIT.

TERM-15: Pipeline

The 5-phase processing system (INITIALIZE -> RESOLVE -> TRANSFORM -> ANALYZE -> EMIT).

DESCRIPTION:

Purpose: Orchestrates document processing through sequential phases.

Flow: Each phase completes for all handlers before the next phase begins.

TERM-24: Prerequisites

Handler dependencies that determine execution order.

DESCRIPTION:

Purpose: Declares which handlers must complete before a given handler can execute.

Usage: Handlers declare prerequisites to ensure data dependencies are satisfied.

TERM-25: Topological Sort

Kahn’s algorithm for ordering handlers by prerequisites.

DESCRIPTION:

Purpose: Determines valid execution order for handlers based on dependencies.

Algorithm: Uses Kahn’s algorithm to produce a topologically sorted handler sequence.

TERM-22: TRANSFORM Phase

The third phase in the pipeline that materializes views and rewrites content.

DESCRIPTION:

Purpose: Materializes database views into content and applies content transformations.

Position: Third phase after RESOLVE, before ANALYZE.

TERM-27: Type Alias

Alternative syntax identifier for a type (e.g., “csv” -> “TABLE”).

DESCRIPTION:

Purpose: Provides shorthand or alternative names for types.

Example: csv is an alias for the TABLE type in float definitions.

TERM-38: Type Loader

The host engine (src/contract/registry.lua) that overlays models and registers their type descriptors.

DESCRIPTION:

Purpose: Discovers each model’s type descriptors and registers them with the SpecIR type tables, the data manager, and the pipeline.

Implementation: Overlays the default model then each requested model (later-wins-by-id; resolved repo-bundled under SPECCOMPILER_HOME/models/{model} then cwd, no out-of-tree path). For each models/{model}/types/{category}/ file it loads the single returned descriptor { kind, schema, [hooks] }, validates it (known kind, present schema.id, each hook valid for the kind, no behaviour on top-level keys), emits the type row, and eager-indexes each hook into a (kind, id) -> hook map read via get_hook / get_hook_inherited (which walks the extends chain). host:finalize() then propagates inherited attributes, creates the verification SQL views, and asserts required hooks.

TERM-26: Type Registry

Database tables (spec_*_types) storing type definitions.

DESCRIPTION:

Purpose: Stores type definitions including attributes, aliases, and validation rules.

Tables: spec_object_types, spec_float_types, spec_attribute_types, etc.

TERM-21: ANALYZE Phase

The fourth phase in the pipeline that validates content via analyze queries.

DESCRIPTION:

Purpose: Validates document content using analyze queries and constraint checking.

Position: Fourth phase after TRANSFORM, before EMIT.

TERM-AST: Abstract Syntax Tree

The tree representation of document structure produced by Pandoc.

ACRONYM:
AST
DESCRIPTION:

Purpose: Represents document structure as a hierarchical tree of elements.

Source: Pandoc parses Markdown and produces JSON AST.

Usage: Handlers walk the AST to extract spec objects, floats, and relations.

DOMAIN:
Core
TERM:
Abstract Syntax Tree

TERM-FTS: Full-Text Search

FTS5 virtual tables enabling search across specification content.

ACRONYM:
FTS
DESCRIPTION:

Purpose: Indexes specification text for fast full-text search queries.

Implementation: SQLite FTS5 virtual tables populated during EMIT phase.

Usage: Web application uses FTS for search functionality.

DOMAIN:
Database
TERM:
Full-Text Search

TERM-HLR: High-Level Requirement

A top-level functional or non-functional requirement that captures what the system must do or satisfy.

ACRONYM:
HLR
DESCRIPTION:

Purpose: Defines system-level requirements that guide design and implementation.

Traceability: HLRs trace to verification cases (VC) and are realized by functional descriptions (FD).

DOMAIN:
Core
TERM:
High-Level Requirement

TERM-IR: Intermediate Representation

The database-backed representation of parsed document content.

ACRONYM:
IR
DESCRIPTION:

Purpose: Stores parsed specification content in queryable form.

Storage: SQLite database with spec_objects, spec_floats, spec_relations tables.

Lifecycle: Populated during INITIALIZE, queried and modified through remaining phases.

DOMAIN:
Core
TERM:
Intermediate Representation

TERM-PID: Project Identifier

A unique identifier assigned to spec objects for cross-referencing (e.g., @REQ-001).

ACRONYM:
PID
DESCRIPTION:

Purpose: Provides unique, human-readable identifiers for traceability and cross-referencing.

Syntax: Written as @PID in header text (e.g., ## HLR: Requirement Title @REQ-001).

Auto-generation: PIDs can be auto-generated from type prefix and sequence number.

DOMAIN:
Core
TERM:
Project Identifier

TERM-VERIFICATIONVIEW: Analyze Query

A SQL query that validates data integrity constraints during the ANALYZE phase.

ACRONYM:
-
DESCRIPTION:

Purpose: Defines validation rules as SQL queries that detect specification errors.

Execution: Run during the ANALYZE phase; violations are reported as diagnostics.

Examples: Missing required attributes, unresolved relations, cardinality violations.

DOMAIN:
Core
TERM:
Analyze Query

TERM-SQLITE: SQLite Database

The embedded database engine storing the IR and build cache.

ACRONYM:
-
DESCRIPTION:

Purpose: Provides persistent, portable storage for the intermediate representation.

Benefits: Single-file storage, ACID transactions, SQL query capability.

Usage: All pipeline phases read/write to SQLite via the database manager.

DOMAIN:
Database
TERM:
SQLite Database

TERM-TRACEABLE: Traceable Object

A specification object that participates in traceability relationships.

ACRONYM:
-
DESCRIPTION:

Purpose: Base type for objects that can be linked via traceability relations.

Types: Any spec object type registered in the model (e.g., HLR, LLR, SECTION).

Relations: Model-defined relation types (e.g., XREF_FIGURE, XREF_CITATION) inferred by specificity matching.

DOMAIN:
Core
TERM:
Traceable Object

TERM-TYPE: Type

A category definition that governs behavior for objects, floats, relations, or views.

ACRONYM:
-
DESCRIPTION:

Purpose: Defines the schema, validation rules, and hooks behaviour for a category of elements.

Categories: Object types (HLR, SECTION), float types (FIGURE, TABLE), relation types (TRACES_TO), view types (TOC, LOF).

Registration: Each type is one descriptor ({ kind, schema, hooks }) loaded from a model’s category directory, validated by the host engine, and emitted into the type registry.

DOMAIN:
Core
TERM:
Type

TERM-VC: Verification Case

A test specification that verifies a requirement or set of requirements.

ACRONYM:
VC
DESCRIPTION:

Purpose: Defines how requirements are verified through test procedures and expected results.

Traceability: VCs trace to HLRs via traceability attribute links.

Naming: VC PIDs follow the pattern VC-{category}-{seq} (e.g., VC-PIPE-001).

DOMAIN:
Core
TERM:
Verification Case

TERM-COMPOSITE: Composite Object Type

A spec object type whose instances receive hierarchical PIDs qualified by the parent specification PID.

DESCRIPTION:

Purpose: Distinguishes object types that represent document structure (e.g., SECTION) from traceable types that receive standalone PIDs (e.g., HLR, VC).

PID behavior: Composite objects get hierarchical PIDs derived from the specification PID (e.g., SRS-sec1.2.3). Non-composite objects get independent PIDs from their pid_prefix and pid_format (e.g., HLR-001).

Configuration: Set via is_composite = true in the descriptor’s schema.

DOMAIN:
Core
TERM:
Composite Object Type

TERM-SELECTOR: Relation Selector

The URL scheme portion of a Markdown link that drives relation type inference.

DESCRIPTION:

Purpose: Identifies what kind of relation a Markdown link represents, enabling type-driven inference.

Selectors: @ (PID reference, e.g., [HLR-001](@)), # (label reference, e.g., [fig:diagram](#)), @cite (bibliographic citation).

Configuration: Each relation type declares a link_selector value in spec_relation_types. Selectors are model-defined, not hardcoded.

DOMAIN:
Core
TERM:
Relation Selector

TERM-SPECIFICITY: Specificity Scoring

The constraint-matching score used to select the best relation type during type inference.

DESCRIPTION:

Purpose: Resolves ambiguity when multiple relation types match a given link by selecting the most specific type.

Algorithm: Each non-NULL constraint match across four dimensions (selector, source_attribute, source_type, target_type) adds one point. The highest total score wins. Ties mark the relation as ambiguous.

Example: A relation type with constraints on selector + source_type + target_type (score 3) wins over one with only selector (score 1).

DOMAIN:
Core
TERM:
Specificity Scoring

TERM-PIR: Processed Intermediate Representation

The complete specification state after all pipeline phases have executed, captured as a hash for output cache invalidation.

ACRONYM:
P-IR
DESCRIPTION:

Purpose: Provides a single hash that represents the fully processed state of a specification, including all resolved relations, materialized views, and transformed content.

Usage: The output cache stores the P-IR hash alongside each generated output file. When rebuilding, the system compares the current P-IR hash to the cached hash to determine if output regeneration is needed.

Distinction: Unlike the build cache (which tracks source file hashes), the P-IR hash captures the post-processing state, detecting changes from cross-document operations.

DOMAIN:
Core
TERM:
Processed Intermediate Representation

TERM-NDJSON: Newline-Delimited JSON

A text format where each line is a valid JSON object, used for structured log output.

ACRONYM:
NDJSON
DESCRIPTION:

Purpose: Provides machine-parseable structured logging suitable for CI/CD log aggregation and filtering with tools like jq.

Format: One JSON object per line with fields: level, message, timestamp, and optional context fields.

Usage: The logger emits NDJSON when output is not connected to a TTY (e.g., piped to a file or running in CI).

DOMAIN:
Infrastructure
TERM:
Newline-Delimited JSON

TERM-VALIDATIONPOLICY: Validation Policy

A configuration mapping from analyze query policy_key to severity level, controlling which violations are reported and at what severity.

DESCRIPTION:

Purpose: Allows projects to control validation strictness by mapping each analyze query to a severity level.

Severity levels: error (blocks output generation), warn (reported but build continues), ignore (suppressed).

Configuration: Set in the validation: section of project.yaml (e.g., traceability_hlr_to_vc: warn).

Default: When a policy_key is not configured, the system applies its built-in default severity.

DOMAIN:
Core
TERM:
Validation Policy

TERM-DIAGNOSTIC: Diagnostic Record

A structured error or warning record emitted by Handlers during Pipeline processing.

DESCRIPTION:

Purpose: Provides machine-parseable and human-readable feedback on specification errors and warnings throughout all pipeline phases.

Fields: Each record contains file (source path), line (source line number), code (stable diagnostic key, usually the analyze query policy_key, e.g., dangling_relation), and msg (human-readable description).

Severity: Errors trigger abort after ANALYZE Phase phase; warnings are reported but do not block output generation.

DOMAIN:
Core
TERM:
Diagnostic Record

TERM-BUILDGRAPH: Build Graph

A dependency tracking structure recording include file hierarchies for incremental rebuild support.

DESCRIPTION:

Purpose: Records the full file set of each document build — the root document itself and every included file — enabling change detection across include hierarchies with a single node walk.

Storage: Stored in the build_graph table with columns root_path (the document being built), node_path (a file that build read: the root itself or an include), and node_sha1 (content hash at build time).

Usage: Queried by Build Cache is_document_dirty() to determine if the root or any included file has changed since the last successful build.

DOMAIN:
Database
TERM:
Build Graph

TERM-PLACEHOLDERBLOCK: Placeholder Block

A CodeBlock marker inserted during document assembly for deferred Spec Float and Spec View resolution.

DESCRIPTION:

Purpose: Marks positions in the assembled Pandoc document where floats and views will be substituted during the EMIT Phase phase.

Mechanism: During assembly, sdd: Document Assembler inserts CodeBlock elements at the correct file_seq positions. Downstream handlers (sdd: Float Emitter for floats, sdd: View Emitter for views) match these placeholders by label and replace them with rendered content.

Lifecycle: Created during assembly, consumed during float/view emission, never present in final output.

DOMAIN:
Pipeline
TERM:
Placeholder Block