XProc Pipelines for XML Processing

Why XML pipelines matter

A production XML workflow rarely performs only one operation. It may parse input, validate against XSD, apply Schematron, transform with XSLT and store several outputs. XProc describes that sequence as a pipeline with named steps and explicit connections instead of hiding it in shell scripts or application glue.

Steps, ports and options

A step consumes documents through input ports, produces documents through output ports and accepts options or parameters. Connections define data flow. This makes intermediate documents and dependencies visible, which improves testing and allows a processor to manage execution consistently.

Validation as a pipeline stage

Grammar validation and rule validation should remain distinguishable stages. An XSD step can reject structural errors before a Schematron step evaluates business rules. The pipeline can preserve separate reports so users know which layer failed and why.

Side effects and reproducibility

File reads, HTTP requests and writes introduce environment dependencies. Keep them explicit, control base URIs and resolve schemas through a predictable catalog. A reproducible pipeline should produce the same logical result from the same declared inputs without relying on an operator’s working directory.

Testing a pipeline

Test individual custom steps and complete flows. Include failure paths, empty sequences, alternative branches and documents with several namespaces. Assertions should cover both primary results and reports, because a pipeline can return the expected document while silently omitting a required diagnostic.

Implementation notes

Model the pipeline as named inputs, explicit steps and observable outputs before adding branches. Parsing, grammar validation, Schematron validation, transformation and serialization should remain distinguishable stages. Preserve diagnostic reports as first-class outputs so an operator can tell whether a document failed structural validation, a business rule or a later delivery step.

Custom steps need contracts as precise as public APIs. Declare accepted content types, ports, options, cardinality and error behavior. Avoid reading implicit files from the working directory. Pass resources through ports or options and resolve stable schema identifiers through a controlled catalog. This makes the pipeline movable across processors and deployment environments.

Failure modes

A pipeline can appear deterministic while depending on current time, remote HTTP responses, filesystem ordering or processor defaults. Capture those dependencies and make caching rules visible. When parallel branches write to shared locations, define collision and completion behavior rather than assuming document order implies execution order.

Error handling should not convert every failure into an empty document. Preserve the step name, original error code and relevant document location, then decide at the orchestration boundary whether to stop or continue. Test invalid XML, unavailable resources, a failed secondary output and a branch that produces an empty sequence.

Review checklist

• Ports and options documented
• Validation stages separated
• Resource resolution controlled
• Reports preserved
• Failure branches tested
• Processor version pinned

Questions for a design review

Use this reference to make a review decision, not merely to recognize terminology. Record the concrete document, schema, processor or consumer being discussed; the language and processor versions; and the behavior that must remain compatible. A useful review produces fixtures and an owner for every unresolved assumption.

• What executable example or test demonstrates the intended behavior for why xml pipelines matter?
• What executable example or test demonstrates the intended behavior for steps, ports and options?
• What executable example or test demonstrates the intended behavior for validation as a pipeline stage?
• What executable example or test demonstrates the intended behavior for side effects and reproducibility?
• What executable example or test demonstrates the intended behavior for testing a pipeline?

Include at least one ordinary case, one boundary case and one deliberately invalid or unsupported case. Check the result in the actual production toolchain, because parsers, validators, code generators and reasoners do not all implement the same optional features. Store the selected contract version with the test result, then repeat the review when a dependency, namespace, profile or public declaration changes.

Continue with a working reference

Schematron tutorial · Practical XSLT