derive-resilience-preserves-progress-on-error
IN derived (depth 1)
The derive pipeline is resilient to partial failures: partial results are persisted via JSON reports after each round, and error states are signaled through return codes (-1 for error, 0 for saturation, positive for progress) rather than exceptions, enabling callers to recover and resume.
Summary
The derive pipeline is designed to handle failures gracefully without losing work. Partial results get saved to disk after each processing round, so if something goes wrong, you can pick up where you left off. Errors are communicated through return codes rather than crashing, which means the calling code can decide how to respond — retry, stop, or continue with what it has.
Justifications
SL — Two independently-established resilience properties (durable partial output + structured error signaling) combine to ensure no derivation work is lost on failure
Antecedents (all must be IN):
- derive-reports-survive-partial-runs — `cmd_derive` and `cmd_review_beliefs` write partial JSON reports after each round/batch via `_write_derive_report`, so crash recovery is possible from the last completed step.
- derive-returns-negative-one-on-error — `_derive_one_round` returns -1 on error, 0 on saturation, and a positive int for the number of beliefs added, which `cmd_derive` uses to control the exhaust loop.
Dependents
These beliefs depend on this one:
- derive-pipeline-achieves-end-to-end-fault-tolerance — The derive pipeline achieves end-to-end fault tolerance through three independent layers: proactive defense (fail-soft validation, Jaccard retraction guards, environment isolation), reactive resilience (partial results persisted via JSON reports after each round, error states signaled through return codes), and prompt reproducibility (deterministic sampling with fixed seeds enables consistent re-runs after failures).