SpecCompiler Requirements
This document defines the high-level requirements for SpecCompiler , a document processing pipeline for structured specifications.
The document is organized in two parts. The first part defines the SpecIR data model, the six core types that the <Pipeline> operates on. The second part specifies the functional requirements, grouped into System Features (Pipeline Execution through Audit and Integrity), each decomposed into High-Level Requirements.
SpecIR (see <SpecIR>) is the data model that SpecCompiler builds from source Markdown during the <INITIALIZE Phase> phase. The core task of parsing is to lower Markdown annotations into a set of typed content tables that the <Pipeline> can analyze, transform, verify, and emit. The entries below define each of these six content tables as a formal tuple specifying the Markdown syntax that produces it.
A Specification is the root document container created from an H1 header. It represents a complete document like an SRS, SDD, or SVC. Each specification has a type, optional PID, and contains attributes and all spec objects within that document.
Formal definition: — a tuple of type, title, project identifier, attributes, and child objects.
Syntax: # [TypeRef:] Text [@PID]
Full specification: See the CommonSpec Language Specification for the complete grammar, type inference rules, and examples.
A Spec Object represents a traceable element in a specification document, created from H2-H6 headers. Objects can be requirements (HLR, LLR), verification cases (VC), design elements (FD, CSC, CSU), or structural sections (SECTION). Each object has a type, PID and can contain attributes, body content, floats, relations, views, and child objects.
Formal definition: — a tuple of type, title, project identifier, body content, attributes, floats, relations, views, and child objects.
Syntax: ##...###### [TypeRef:] Title [@PID]
Full specification: See the CommonSpec Language Specification for the complete grammar, type inference, PID auto-generation, and examples.
A Spec Float represents a floating element like a
figure, table, or diagram. Floats are created from fenced code blocks
with a TypeRef:Label pattern. They
are automatically numbered within their counter group and can be
cross-referenced by their label. Some floats require external rendering
(e.g., PlantUML diagrams).
Formal definition: — a tuple of type, label, key-value metadata, and raw content.
Syntax: ```TypeRef:Label[{Key=Value, ...}] Content ```
Full specification: See the CommonSpec Language Specification for float types, aliases, counter groups, and examples.
An Attribute stores metadata for specifications and spec objects using an Entity-Attribute-Value (EAV) pattern. Attributes are defined in blockquotes following headers and support multiple datatypes including strings, integers, dates, enums, and rich XHTML content (Pandoc AST). Relations are extracted from the AST. Attribute definitions constrain which attributes each object type can have.
Formal definition: — a triple of attribute type, blockquote content, and child relations.
Syntax: > TypeRef: value
Datatypes: STRING, INTEGER, REAL, BOOLEAN, DATE, ENUM, XHTML.
Full specification: See the CommonSpec Language Specification for the complete datatype semantics, attribute constraints, and examples.
A Spec Relation represents a traceability link between specification elements. Relations are created from Markdown links where the link target (URL) acts as a selector that drives type inference. The relation type is not authored explicitly — it is inferred by constraint matching: each relation type defines optional constraints on selector, source attribute, source type, and target type. The most specific match (most non-NULL constraints) wins.
Formal definition: — a 4-tuple of source object, target element, link selector, and source attribute.
Type inference: — the relation type is inferred by constraint matching with most-specific-wins across selector, attribute, source type, and target type.
Syntax: [Target](selector) where the URL starts
with @ or #. Selectors are not hardcoded — they are
defined by relation types in the model (e.g., PID_REF defines @, LABEL_REF defines #, XREF_CITATION defines @cite,@citep). Models can register any
@... or #... selector.
Full specification: See the CommonSpec Language Specification for the complete inference algorithm and examples.
A Spec View represents a dynamic query or generated content block. Views are materialized during the TRANSFORM phase and can generate tables of contents (TOC), lists of figures (LOF), or custom queries, abbreviations, and inline math. Views enable dynamic document assembly based on specification data.
Formal definition: — a pair of view type and parameter string.
Syntax: `TypeRef:[ViewParam]`
Full specification: See the CommonSpec Language Specification for view types, rendering, and examples.
With the data model established, the following sections define the functional requirements for SpecCompiler Core. Requirements are organized into System Features (SF), each covering a distinct functional domain. Every SF is decomposed into <High-Level Requirement>s that state what the system shall do.
Five-phase document processing lifecycle with <Handler> orchestration and <Topological Sort> ordering.
The pipeline shall execute handlers in a five-phase lifecycle: INITIALIZE, RESOLVE, TRANSFORM, ANALYZE, EMIT.
Each phase serves a distinct purpose in document processing:
The pipeline shall support handler registration with declarative <Prerequisites> for dependency ordering.
Handlers register via register_handler(handler) with required
fields:
name: Unique string identifier for
the handler
prerequisites: Array of handler names
that must execute before this handler
Handlers declare participation in phases via hook methods (on_initialize, on_analyze, on_verify, on_transform, on_emit). Duplicate handler names cause
registration error.
The pipeline shall order handlers within each phase using topological sort with Kahn’s algorithm.
For each phase, the pipeline:
on_{phase} hooks)
The pipeline shall abort execution after ANALYZE phase if any errors are recorded.
diagnostics:has_errors(). If true,
execution halts before EMIT phase, with TRANSFORM already completed.
Error message is logged with error count. This prevents generating
invalid output from documents with specification violations.
The pipeline shall use a single batch dispatch model for all phases where handlers receive all contexts at once.
All handlers implement on_{phase}(data, contexts, diagnostics)
hooks that receive the full contexts array. The pipeline orchestrator
calls each handler’s hook once per phase via run_phase(), passing all document
contexts. Handlers are responsible for iterating over contexts
internally.
This enables cross-document optimizations, transaction batching, and parallel processing within any phase.
The pipeline shall create and propagate context objects containing document metadata and configuration through all phases.
The execute(docs) method creates
context objects for each input document with:
doc: Pandoc document AST (via
DocumentWalker)
spec_id: Specification identifier
derived from filename
config: Preset configuration (styles,
captions, validation)
build_dir: Output directory path
output_format: Target format (docx,
html5, etc.)
template: Template name for model
loading
reference_doc: Path to reference.docx
for styling
docx, html5: Format-specific configuration
outputs: Array of {format, path}
for multi-format output
bibliography, csl: Citation configuration
project_root: Root directory for
resolving relative paths
Context flows through all phases, enriched by handlers (e.g., verification results in ANALYZE phase).
The system shall parse <CommonSpec> documents during the <INITIALIZE Phase> phase, lowering Markdown annotations into <Intermediate Representation> content tables.
The INITIALIZE phase <Handler>s parse the Pandoc <Abstract Syntax Tree> and populate the six IR content tables according to these rules:
TYPE: prefix,
implicit alias lookup, or default type fallback)
> key: value) register as <Attribute> records
attached to the enclosing spec object
TypeRef:Label class register as <Spec Float>
records
TypeRef: content syntax registers as <Spec View> records
Each handler populates its content table with content-addressable
identifiers (SHA1) and preserves document ordering via file_seq.
When a document contains .include code blocks, the system shall
expand them by embedding the referenced file content before <Pipeline>
processing.
Include expansion runs before the five-phase pipeline on each source document:
.include are identified
Cycle detection prevents infinite recursion. Maximum include depth is bounded. Source position tracking attributes are injected for diagnostic reporting.
When a spec object does not have an explicit @PID, the system shall auto-generate a
<Project
Identifier> during the <RESOLVE Phase> phase based on the
object’s type definition.
The PID generator runs in the RESOLVE phase before relation resolution:
pid_prefix and pid_format (e.g., HLR-%03d produces “HLR-001”), starting
from the next available sequence number
Auto-generated PIDs never overwrite explicit @PID annotations. Collision detection
ensures global uniqueness across all specifications.
The system shall infer relation types during the <RESOLVE Phase> phase using constraint-based matching with <Specificity Scoring> scoring.
For each unresolved relation, the relation analyzer:
Same-specification targets are preferred over cross-specification
targets. The relation’s target_ref and type_ref are updated in the
database.
The system shall distinguish a handler prerequisite that resolves to another phase from one that resolves to no registered handler at all, emitting a diagnostic only for the latter.
Pipeline:validate_prerequisites(), run
once when all handlers are registered, distinguishes the two cases the
filter conflates: a prerequisite registered in ANOTHER phase
(legitimate, silent) versus one registered in NO phase (a typo/bug). It
emits a prerequisite_not_found
diagnostic only for the latter, and only for phase handlers (those
declaring an on_<phase>
hook) – a decorated per-item callback’s prerequisites field is inert and not
flagged. The cross-phase filter itself is unchanged.
A <Spec
Object>’s scope shall be closed either automatically by the next
header of equal-or-shallower level, or manually by a ---- thematic break, and a heading
shall not be empty.
During INITIALIZE, the first ---- thematic break ends the active
section scope. The marker does not appear in rendered output. Later
content keeps its document position and belongs to the parent
section.
An empty heading (a header whose title is blank) is rejected: it
would otherwise render as an empty numbered heading and corrupt
downstream numbering. The object_broken_hierarchy analyze query
reports it as an error and directs the author to use ---- instead.
<SQLite Database>-based storage with incremental build support and output caching.
The system shall persist all specification data to SQLite database with ACID guarantees.
The system shall store spec object attributes using Entity-Attribute-Value pattern.
The system shall maintain a build cache for document hash tracking.
The system shall cache output generation state with P-IR hash and timestamps.
The system shall generate per-object-type SQL views that pivot the EAV attribute model into typed columns for external BI queries.
view_{type_lower}_objects is dynamically
generated (e.g., view_hlr_objects,
view_vc_objects). Each view flattens
the EAV join into one row per object with typed attribute columns,
enabling queries like SELECT * FROM view_hlr_objects WHERE status = 'approved'.
These views are NOT used by internal pipeline queries — all internal
code queries the raw EAV tables directly because it needs access to
raw_value, datatype, ast, enum_ref, and other columns that the pivot
views abstract away. Internal queries also frequently operate cross-type
or need COUNT/EXISTS checks that the MAX()-based pivot cannot provide.
The system shall support incremental rebuilds via build graph tracking.
Dynamic type system providing typed containers for <Specification>, <Spec Object>, <Spec Float>, <Spec View>, <Spec Relation>, and <Attribute>.
The type system shall provide a specifications container for registering document-level specification records.
The specifications table stores
metadata for each specification document parsed during <INITIALIZE Phase>
phase:
identifier: Unique specification ID
derived from filename (e.g., “srs-main”)
root_path: Source file path for the
specification
long_name: Human-readable title
extracted from L1 header
type_ref: Specification type
(validated against spec_specification_types)
pid: Optional PID from @PID syntax in
L1 header
L1 headers register as specifications. Type validation checks spec_specification_types table. Invalid
types fall back to default or emit warning.
The type system shall provide a spec_objects container for hierarchical specification objects.
The spec_objects table stores
structured specification items extracted from L2+ headers:
identifier: SHA1 hash of source path
+ line + title (content-addressable)
specification_ref: Foreign key to
parent specification
type_ref: Object type (validated
against spec_object_types)
from_file: Source file path
file_seq: Document order sequence
number
pid: Project ID from @PID syntax
(e.g., “REQ-001”)
title_text: Header text without type
prefix or PID
label: Unified label for
cross-referencing (format: {type_lower}:{title_slug})
level: Header level (2-6)
start_line, end_line: Source line range
ast: Serialized Pandoc AST (JSON) for
section content
Type resolution order: explicit TYPE: prefix, implicit alias lookup, default type fallback.
The type system shall provide a spec_floats container for numbered floating content (figures, tables, listings).
The spec_floats table stores
content blocks that receive sequential numbering:
identifier: Short format
“float-{8-char-sha1}” for DOCX compatibility
specification_ref: Foreign key to
parent specification
type_ref: Float type resolved from
aliases (e.g., “csv” -> “TABLE”, “puml” -> “FIGURE”)
from_file: Source file path
file_seq: Document order for
numbering
label: User-provided label for
cross-referencing
number: Sequential number within
counter_group (assigned in <TRANSFORM Phase>)
caption: Caption text from attributes
raw_content: Original code block text
raw_ast: Serialized Pandoc CodeBlock
(JSON)
parent_object_ref: Foreign key to
containing spec_object
attributes: JSON-serialized
attributes (caption, source, language)
syntax_key: Original class syntax for
backend matching
<Counter Group> share numbering (e.g., FIGURE, CHART, PLANTUML all increment “FIGURE” counter).
csv:data instead of TABLE:data). Counter groups enable
semantic grouping of related float types under a single numbering
sequence.
The type system shall provide a spec_views container for data-driven view definitions.
The spec_views table stores view
definitions from inline view syntax (views are inline-only; a view alone
in its own paragraph is transparently promoted to block output at
EMIT):
identifier: SHA1 hash of
specification + sequence + content
specification_ref: Foreign key to
parent specification
view_type_ref: Uppercase view type
(e.g., “TOC”, “SYMBOL”, “MATH”, “ABBREV”)
from_file: Source file path
file_seq: Document order sequence
number
raw_ast: View definition content
(symbol path, expression, parameters)
View types with needs_external_render = 1 in spec_view_types are delegated to
specialized renderers. Inline views use prefix: content syntax (e.g., symbol: Class.method). The content may
carry key=value parameters (e.g.,
[TOC], allocation_matrix: status=complete); the
EMIT dispatcher parses the prefix, content, and parameters once and
passes them to the view’s render hooks (ctx.subject.prefix/content/params).
The type system shall provide a spec_relations container for tracking links between specification elements.
The spec_relations table stores
inter-element references:
identifier: SHA1 hash of
specification + target + type + parent
specification_ref: Foreign key to
parent specification
source_ref: Foreign key to source
spec_object
target_text: Raw link target from
syntax (e.g., “REQ-001”, “fig:diagram”)
target_ref: Resolved target
identifier (populated in <RESOLVE Phase> phase)
type_ref: Relation type from spec_relation_types (e.g., “TRACES”,
“XREF_FIGURE”)
from_file: Source file path
Link syntax: [PID](@) for PID
references, [type:label](#) for
float references, [@citation] for
bibliographic citations. Default relation types are determined by is_default and link_selector columns in spec_relation_types.
The type system shall provide a spec_attributes container for structured metadata on specification objects.
The spec_attributes table
stores typed attribute values extracted from blockquote syntax:
identifier: SHA1 hash of
specification + owner + name + value
specification_ref: Foreign key to
parent specification
owner_ref: Foreign key to owning
spec_object
name: Attribute name (field name
without colon)
raw_value: Original string value
string_value, int_value, real_value, bool_value, date_value: Type-specific columns
enum_ref: Foreign key to enum_values for ENUM types
ast: JSON-serialized Pandoc AST for
rich content (XHTML type)
datatype: Resolved datatype from
spec_attribute_types
Attribute syntax: > name: value in blockquotes
following headers. Datatypes include STRING, INTEGER, REAL, BOOLEAN,
DATE, ENUM, XHTML. Multi-line attributes use continuation blocks.
The type system shall provide analyze queries that detect data integrity violations across all specification containers.
The type system described above is not fixed at compile time. The following section defines how dic:model directories extend it with custom object types, float renderers, dic:data-view generators, and style presets.
Verification views are SQL queries registered in the <ANALYZE Phase> phase that check for constraint violations:
The validation policy (configurable in project.yaml) determines severity: error, warn, or ignore.
The extension framework shall let models define types, rendering behavior, data views, verification queries, and output processing.
The system shall load type descriptors from model directories.
The host scans these directories under models/{model}/types/:
objects/ for kind = "object"
specifications/ for kind = "specification"
floats/ for kind = "float"
views/ for kind = "view"
relations/ for kind = "relation"
Each Lua module returns one descriptor table. The host uses schema.id as the type identifier. The file
name does not define the identifier.
The system shall use a standard model directory structure.
A model can contain the following paths:
models/{model}/
model.yaml
types/
objects/
specifications/
floats/
views/
relations/
analyze_queries/
filters/
postprocessors/
styles/
tools/
Only the directories required by the model must exist. A type
directory can contain a Lua file or a subdirectory with init.lua.
Each extension module shall return one descriptor with kind, schema, and optional hooks fields.
schema.id, an invalid hook, and behavior
outside hooks. The host shall
register schema data and index each behavior hook.
Each descriptor shall declare the schema fields required by its kind.
extends to inherit attributes
and hooks.
The system shall resolve and load models as ordered overlays.
$SPECCOMPILER_HOME/models/{model} first.
It shall then search models/{model} under the working
directory. The host shall load default before the selected model. A
later descriptor with the same kind and schema.id shall replace the earlier
descriptor. A missing selected or required model shall stop the build.
An externally rendered float shall declare task preparation and result handling in its descriptor.
schema.needs_external_render = true. Its
hooks table shall provide prepare_task and handle_result. The core shall prepare
tasks, apply the render cache, run tasks, and dispatch results. A float
shall not declare both render and
external-render hooks.
The system shall obtain generated data from hooks on view descriptors.
dataset hook
for chart data. A TABLE_VIEW
subtype shall provide an inherited or local build_block hook. The host shall map
inline_prefix and aliases to view identifiers.
The host shall index registered behavior hooks by kind, type identifier, and hook name.
schema.extends
chain. Phase hooks shall use pipeline registration and shall not use the
behavior-hook index.
The host shall pass each behavior hook one frozen context table.
subject field shall contain the
hook-specific input. The capability field shall identify the
hook. The ctx:require(field)
method shall stop execution when a required field is nil.
The host shall read model dependencies from model.yaml.
requires field shall
contain model names. The host shall load each required model before the
requesting model. It shall load each model at most once. A missing
manifest or requires field shall
define no dependencies.
The host shall validate hooks and register phase participation from the descriptor.
on_<phase> in hooks. The host shall register phase
hooks under <lowercase schema.id>_handler.
The optional schema.phase_prerequisites field shall
define handler ordering.
The system shall register each analysis query as a kind = "analyze" descriptor.
id, policy_key, view, sql, and optional disabled. The optional message hook shall format a diagnostic
for one query row. A later descriptor with the same policy_key shall replace the earlier
descriptor in place. A descriptor with disabled = true shall remove that
policy key.
The system shall read build configuration from project.yaml.
Assembles transformed content and publishes DOCX/HTML5 outputs with cache-aware emission.
The system shall reconstruct a complete Pandoc document from <Intermediate Representation> database content for each specification, preserving document order and embedding all resolved content.
During the <EMIT
Phase> phase, the assembler queries spec_objects ordered by file_seq, decodes stored JSON <Abstract Syntax
Tree> fragments back to Pandoc blocks, adjusts header levels for
cross-file includes, and embeds:
specifications.header_ast, wrapped in a
title Div
file_seq order,
with their rendered body AST
The assembled document includes Pandoc metadata built from
specification attributes (title, author, date). The result is a valid
pandoc.Pandoc document suitable
for format-specific output.
Header-level adjustment rescales the shallowest header level to the
top output level uniformly; it therefore assumes the source header
levels form a well-formed tree (contiguous nesting, a single root
depth). That precondition is enforced declaratively by the object_broken_hierarchy analyze query,
which rejects skipped levels and orphaned roots before emission rather
than letting a malformed hierarchy mis-render silently.
The system shall replace <Float> placeholder blocks in the assembled document with their rendered content, using results from the <TRANSFORM Phase> phase.
After document assembly, the float emitter walks all blocks and replaces CodeBlock placeholders (identified by float labels) with rendered Div elements containing:
resolved_ast from spec_floats (SVG images for PlantUML,
parsed tables for CSV, chart images for ECharts)
speccompiler-float, speccompiler-caption, type-specific class)
for format-specific styling
Floats whose resolved_ast is NULL
(failed external renders) are preserved as error placeholders.
The system shall assign sequential numbers to <Float>s within their <Counter Group>, producing a single numbering sequence across all documents in the project.
During the <TRANSFORM Phase> phase, the float numberer:
file_seq
counter_group
(e.g., FIGURE, TABLE, LISTING, EQUATION)
The assigned numbers are stored in spec_floats.number and used for caption
formatting and cross-reference display text.
The system shall generate output documents in all formats specified by the project configuration, skipping outputs whose <Processed Intermediate Representation> hash matches the cached value.
The emitter orchestrator iterates over config.outputs (an array of {format, path} pairs) and for each
specification:
is_output_current()) and skips generation
when the P-IR hash matches
docx.lua, html.lua from the model’s filters/ directory)
Supported output formats: DOCX, HTML5, Markdown, JSON. Multiple formats can be generated from a single pipeline execution.
The system shall generate DOCX output with style customization via preset-based reference document generation and OOXML post-processing.
DOCX output generation follows this sequence:
models/{model}/styles/presets/ with
extends-chain merging and circular dependency detection
reference.docx from the resolved preset
containing custom Word styles (headings, captions, code blocks, table
styles)
--reference-doc pointing to the generated
reference and format-specific Lua filters
The reference document is cached and regenerated only when the preset hash changes.
The system shall generate standalone HTML5 output with table of contents, section numbering, and embedded resources when configured.
HTML5 output generation follows this sequence:
number_sections, table_of_contents, toc_depth, standalone, embed_resources)
embed_resources is enabled, all CSS,
JavaScript, and image assets are embedded inline for single-file
distribution
(@) links resolve to #anchor URLs for in-page navigation
Configuration is specified in the html5: section of project.yaml.
The system shall populate dic:full-text-search virtual tables during the dic:emit-phase phase to enable full-text search across specification content.
Once documents are assembled and published, the system must also guarantee that its builds are reproducible and its processing is auditable. The following section addresses these integrity concerns.
The FTS indexer creates and populates three FTS5 virtual tables with Porter stemming:
fts_objects:
Indexes spec object titles and body text, keyed by identifier and
spec_id
fts_attributes:
Indexes attribute names and string values, keyed by owner_ref and
spec_id
fts_floats:
Indexes float captions and raw source content, keyed by identifier and
spec_id
<Abstract Syntax Tree> content is converted to plain text before indexing.
Deterministic compilation, reproducible builds, and audit trail integrity.
The system shall compute SHA1 content hashes for all source documents and include files to enable change detection.
The build engine computes SHA1 hashes for each document. Hashes are
compared against the cached values recorded as build_graph nodes (the root document is
its own node, alongside its includes):
build_graph nodes are rewritten with
current hashes
This provides O(1) change detection without parsing unchanged documents.
When a document contains include directives, the system shall track all included file dependencies in a build graph and detect circular includes.
Before <Pipeline>
execution, the include handler expands .include code blocks by:
build_graph table (root_path, node_path, node_sha1)
Subsequent builds use this graph to check if any included file has changed, triggering a rebuild of the root document when needed.
The system shall collect and report processing errors and warnings with source location information throughout all pipeline phases.
The diagnostics collector provides structured error/warning reporting:
diagnostics:error(file, line, code, msg)
and diagnostics:warn(file, line, code, msg)
has_errors() method enables the <Pipeline> to determine
abort conditions after the <ANALYZE Phase> phase
Diagnostic keys are stable identifiers (e.g., invalid_enum, missing_required, dangling_relation) suitable for filtering
and CI policy.
The system shall provide <Newline-Delimited JSON> structured logging with TTY-aware output formatting.
The logging subsystem supports two output modes selected automatically or via configuration:
level, message, timestamp, and optional context. Suitable
for CI/CD log aggregation and jq
filtering
NO_COLOR environment variable). Includes
timestamp and level indicator
Log levels: DEBUG, INFO, WARN, ERROR. Configured via config.logging.level with environment
override SPECCOMPILER_LOG_LEVEL.
Given identical source files, project configuration, and tool versions, the system shall produce identical outputs.
The glossary below defines the domain vocabulary used throughout this specification. Each term corresponds to a cross-reference encountered in the requirements above and is defined here with its purpose, scope, and usage context.
Build reproducibility is ensured through:
file_seq ordering, which is stable
across builds
Cache invalidation is based solely on content hashes, not filesystem timestamps or system state.
A structured Markdown language for authoring typed, traceable specifications.
Purpose: Defines the input language for SpecCompiler. Extends standard Markdown (CommonMark) with six constructs: specifications, spec objects, spec floats, attributes, spec relations, and spec views.
Architecture: CommonSpec (language) compiles into SpecIR (intermediate representation) via SpecCompiler (compiler).
Specification: See the CommonSpec Language
Specification (docs/commonspec/) for
the formal definition.
A typed relational intermediate representation for specifications, stored in SQLite.
Purpose: Provides a portable, queryable storage format for specification data. Compilation target for CommonSpec and interchange format for other tools (ReqIF, DOORS CSV, SQL).
Schema: Two core layers: Type System tables (metamodel) and Content tables (data). Build infrastructure and FTS tables are not part of the SpecIR standard.
Specification: See the SpecIR Schema Specification
(docs/specir/) for the formal
definition.
The second phase in the pipeline that resolves references and infers types.
Purpose: Resolves cross-references between spec objects and infers missing type information.
Position: Second phase after INITIALIZE, before TRANSFORM.
SHA1 hashes for detecting document changes.
Purpose: Stores content hashes to detect which documents have changed since last build.
Implementation: Compares current file hash against cached hash to skip unchanged files.
Float types sharing a numbering sequence.
Purpose: Groups related float types to share sequential numbering.
Example: FIG and DIAGRAM types may share a counter, producing Figure 1, Figure 2, etc.
A MIL-STD-498 architectural decomposition element representing a subsystem, layer, package, or service.
Purpose: Groups software units into higher-level structural components for design allocation.
Examples: src/core, src/db, src/infra.
A MIL-STD-498 implementation decomposition element representing a source file or code unit.
Purpose: Captures file-level implementation units allocated to functional descriptions.
Examples: src/core/pipeline.lua, src/db/manager.lua.
A view descriptor whose dataset data hook generates data for chart
injection.
Purpose: Produces structured data that can be injected into chart floats.
Implementation: A dataset DATA hook receives a frozen data
context (reading subject.params)
and returns a { source / data / links } dataset. The
descriptor is resolved from models/{requested}/types/views/{view}
with fallback to models/default/types/views/{view}.
Entity-Attribute-Value pattern for typed attribute storage.
Purpose: Flexible schema for storing typed attributes on spec objects.
Structure: Entity (spec object), Attribute (key name), Value (typed content).
The final phase in the pipeline that assembles and outputs documents.
Purpose: Assembles transformed content and writes final output documents.
Position: Final phase after ANALYZE.
A numbered element (table, figure, diagram) with caption and cross-reference. See <Spec Float> for full definition.
Subprocess-based rendering for types like PLANTUML, CHART.
Purpose: Delegates rendering to external tools via subprocess execution.
Examples: PlantUML JAR for diagrams, chart libraries for data visualization.
A named hook that a type descriptor contributes to
the pipeline, indexed by the host engine and dispatched by capability
rather than declared on a M.handler surface.
Purpose: Encapsulates processing logic for a content
type. Behaviour lives only under a descriptor’s hooks table; the host classifies each
hook by name and indexes it into the (kind, id) -> hook map.
Structure: Two kinds of contributions, both keyed by hook name:
on_<phase> functions (on_initialize, on_analyze, on_transform, on_verify, on_emit) declared in hooks beside the behavior hooks; the
host synthesizes them into pipeline:register_handler under the
derived name <lower(id)>_handler, ordered via
schema.phase_prerequisites. Each
runs once per phase with full pipeline context.
render, render_block, render_link, message) and DATA hooks (dataset, build_block, transform, resolve, prepare_task, handle_result). The hook NAME selects
one of two context tiers (a frozen render ctx or a frozen data ctx) and
the return type the hook owes. Consumers resolve them via get_hook_inherited, which walks the
extends chain.
The first phase in the pipeline that parses AST and populates IR containers.
Purpose: Parses markdown AST and populates intermediate representation containers.
Position: First phase, entry point for document processing.
A collection of type descriptors and styles for a
domain, overlaid onto the default
model by the host engine.
Purpose: Bundles related type descriptors (each
carrying its own hooks) and
styling for specific documentation domains. The host overlays default then each model
later-wins-by-id, so a model overrides only the type ids it
redefines.
Examples: SRS model for software requirements, HRS model for hardware requirements.
Timestamps for incremental output generation.
Purpose: Tracks when outputs were last generated to enable incremental builds.
Implementation: Compares source modification time against cached output timestamp.
A distinct stage in document processing with specific responsibilities.
Purpose: Separates document processing into well-defined sequential stages.
Phases: INITIALIZE, RESOLVE, TRANSFORM, ANALYZE, EMIT.
The 5-phase processing system (INITIALIZE -> RESOLVE -> TRANSFORM -> ANALYZE -> EMIT).
Purpose: Orchestrates document processing through sequential phases.
Flow: Each phase completes for all handlers before the next phase begins.
Handler dependencies that determine execution order.
Purpose: Declares which handlers must complete before a given handler can execute.
Usage: Handlers declare prerequisites to ensure data dependencies are satisfied.
Kahn’s algorithm for ordering handlers by prerequisites.
Purpose: Determines valid execution order for handlers based on dependencies.
Algorithm: Uses Kahn’s algorithm to produce a topologically sorted handler sequence.
The third phase in the pipeline that materializes views and rewrites content.
Purpose: Materializes database views into content and applies content transformations.
Position: Third phase after RESOLVE, before ANALYZE.
Alternative syntax identifier for a type (e.g., “csv” -> “TABLE”).
Purpose: Provides shorthand or alternative names for types.
Example: csv
is an alias for the TABLE type in float definitions.
The host engine (src/contract/registry.lua) that overlays
models and registers their type descriptors.
Purpose: Discovers each model’s type descriptors and registers them with the SpecIR type tables, the data manager, and the pipeline.
Implementation: Overlays the default model then each requested model
(later-wins-by-id; resolved repo-bundled under SPECCOMPILER_HOME/models/{model} then
cwd, no out-of-tree path). For each models/{model}/types/{category}/ file
it loads the single returned descriptor { kind, schema, [hooks] }, validates
it (known kind, present schema.id, each hook valid for the
kind, no behaviour on top-level keys), emits the type row, and
eager-indexes each hook into a (kind, id) -> hook map read via
get_hook / get_hook_inherited (which walks the
extends chain). host:finalize() then propagates
inherited attributes, creates the verification SQL views, and asserts
required hooks.
Database tables (spec_*_types) storing type definitions.
Purpose: Stores type definitions including attributes, aliases, and validation rules.
Tables: spec_object_types, spec_float_types, spec_attribute_types, etc.
The fourth phase in the pipeline that validates content via analyze queries.
Purpose: Validates document content using analyze queries and constraint checking.
Position: Fourth phase after TRANSFORM, before EMIT.
The tree representation of document structure produced by Pandoc.
Purpose: Represents document structure as a hierarchical tree of elements.
Source: Pandoc parses Markdown and produces JSON AST.
Usage: Handlers walk the AST to extract spec objects, floats, and relations.
FTS5 virtual tables enabling search across specification content.
Purpose: Indexes specification text for fast full-text search queries.
Implementation: SQLite FTS5 virtual tables populated during EMIT phase.
Usage: Web application uses FTS for search functionality.
A top-level functional or non-functional requirement that captures what the system must do or satisfy.
Purpose: Defines system-level requirements that guide design and implementation.
Traceability: HLRs trace to verification cases (VC) and are realized by functional descriptions (FD).
The database-backed representation of parsed document content.
Purpose: Stores parsed specification content in queryable form.
Storage: SQLite database with spec_objects, spec_floats, spec_relations tables.
Lifecycle: Populated during INITIALIZE, queried and modified through remaining phases.
A unique identifier assigned to spec objects for
cross-referencing (e.g., @REQ-001).
Purpose: Provides unique, human-readable identifiers for traceability and cross-referencing.
Syntax: Written as @PID in header text (e.g., ## HLR: Requirement Title @REQ-001).
Auto-generation: PIDs can be auto-generated from type prefix and sequence number.
A SQL query that validates data integrity constraints during the ANALYZE phase.
Purpose: Defines validation rules as SQL queries that detect specification errors.
Execution: Run during the ANALYZE phase; violations are reported as diagnostics.
Examples: Missing required attributes, unresolved relations, cardinality violations.
The embedded database engine storing the IR and build cache.
Purpose: Provides persistent, portable storage for the intermediate representation.
Benefits: Single-file storage, ACID transactions, SQL query capability.
Usage: All pipeline phases read/write to SQLite via the database manager.
A specification object that participates in traceability relationships.
Purpose: Base type for objects that can be linked via traceability relations.
Types: Any spec object type registered in the model (e.g., HLR, LLR, SECTION).
Relations: Model-defined relation types (e.g., XREF_FIGURE, XREF_CITATION) inferred by specificity matching.
A category definition that governs behavior for objects, floats, relations, or views.
Purpose: Defines the schema, validation rules, and
hooks behaviour for a category of
elements.
Categories: Object types (HLR, SECTION), float types (FIGURE, TABLE), relation types (TRACES_TO), view types (TOC, LOF).
Registration: Each type is one descriptor ({ kind, schema, hooks }) loaded from a
model’s category directory, validated by the host engine, and emitted
into the type registry.
A test specification that verifies a requirement or set of requirements.
Purpose: Defines how requirements are verified through test procedures and expected results.
Traceability: VCs trace to HLRs via traceability attribute links.
Naming: VC PIDs follow the pattern VC-{category}-{seq} (e.g., VC-PIPE-001).
A spec object type whose instances receive hierarchical PIDs qualified by the parent specification PID.
Purpose: Distinguishes object types that represent document structure (e.g., SECTION) from traceable types that receive standalone PIDs (e.g., HLR, VC).
PID behavior: Composite objects get hierarchical
PIDs derived from the specification PID (e.g., SRS-sec1.2.3). Non-composite objects
get independent PIDs from their pid_prefix and pid_format (e.g., HLR-001).
Configuration: Set via is_composite = true in the descriptor’s
schema.
The URL scheme portion of a Markdown link that drives relation type inference.
Purpose: Identifies what kind of relation a Markdown link represents, enabling type-driven inference.
Selectors: @
(PID reference, e.g., [HLR-001](@)), # (label reference, e.g., [fig:diagram](#)), @cite (bibliographic citation).
Configuration: Each relation type declares a link_selector value in spec_relation_types. Selectors are
model-defined, not hardcoded.
The constraint-matching score used to select the best relation type during type inference.
Purpose: Resolves ambiguity when multiple relation types match a given link by selecting the most specific type.
Algorithm: Each non-NULL constraint match across four dimensions (selector, source_attribute, source_type, target_type) adds one point. The highest total score wins. Ties mark the relation as ambiguous.
Example: A relation type with constraints on selector + source_type + target_type (score 3) wins over one with only selector (score 1).
The complete specification state after all pipeline phases have executed, captured as a hash for output cache invalidation.
Purpose: Provides a single hash that represents the fully processed state of a specification, including all resolved relations, materialized views, and transformed content.
Usage: The output cache stores the P-IR hash alongside each generated output file. When rebuilding, the system compares the current P-IR hash to the cached hash to determine if output regeneration is needed.
Distinction: Unlike the build cache (which tracks source file hashes), the P-IR hash captures the post-processing state, detecting changes from cross-document operations.
A text format where each line is a valid JSON object, used for structured log output.
Purpose: Provides machine-parseable structured
logging suitable for CI/CD log aggregation and filtering with tools like
jq.
Format: One JSON object per line with fields: level, message, timestamp, and optional context
fields.
Usage: The logger emits NDJSON when output is not connected to a TTY (e.g., piped to a file or running in CI).
A configuration mapping from analyze query policy_key to severity level,
controlling which violations are reported and at what severity.
Purpose: Allows projects to control validation strictness by mapping each analyze query to a severity level.
Severity levels: error (blocks output generation), warn (reported but build continues),
ignore (suppressed).
Configuration: Set in the validation: section of project.yaml (e.g., traceability_hlr_to_vc: warn).
Default: When a policy_key is not configured, the system applies its built-in default severity.
Purpose: Provides machine-parseable and human-readable feedback on specification errors and warnings throughout all pipeline phases.
Fields: Each record contains file (source path), line (source line number), code (stable diagnostic key, usually the
analyze query policy_key, e.g.,
dangling_relation), and msg (human-readable description).
Severity: Errors trigger abort after ANALYZE Phase phase; warnings are reported but do not block output generation.
A dependency tracking structure recording include file hierarchies for incremental rebuild support.
Purpose: Records the full file set of each document build — the root document itself and every included file — enabling change detection across include hierarchies with a single node walk.
Storage: Stored in the build_graph table with columns root_path (the document being built),
node_path (a file that build
read: the root itself or an include), and node_sha1 (content hash at build
time).
Usage: Queried by Build Cache is_document_dirty() to determine if the
root or any included file has changed since the last successful
build.
A CodeBlock marker inserted during document assembly for deferred Spec Float and Spec View resolution.
Purpose: Marks positions in the assembled Pandoc document where floats and views will be substituted during the EMIT Phase phase.
Mechanism: During assembly, sdd: Document
Assembler inserts CodeBlock elements at the correct file_seq positions. Downstream handlers
(sdd: Float
Emitter for floats, sdd: View Emitter for views) match these
placeholders by label and replace them with rendered content.
Lifecycle: Created during assembly, consumed during float/view emission, never present in final output.