persistence-backends

79 beliefs (64 IN, 15 OUT)

The persistence-backends topic covers how the reasons system stores and retrieves its belief network across two interchangeable storage backends: SQLite and PostgreSQL. This is a critical concern because the system's epistemic guarantees — atomic mutations, safe hypothetical reasoning, truth value preservation, and referential integrity — must hold regardless of which backend is in use. The core claim is that both backends are interchangeable (dual-storage-backends-are-interchangeable), providing equivalent safety through backend-appropriate mechanisms: SQLite uses context-managed load/save with write-flag gating, while PostgreSQL uses per-method transactions with composite-key multi-tenancy (atomicity-is-backend-independent, storage-layer-is-backend-agnostic-and-safe). This interchangeability extends to hypothetical reasoning, where SQLite discards speculative changes via write-flag gating and PgApi rolls back transactions (both-backends-support-safe-hypothetical-reasoning, pg-what-if-uses-transaction-rollback). The system is considered fully production-grade across both backends (storage-is-fully-production-grade-across-backends), with SQLite optimized via WAL mode and derived FTS5 indexes (storage-uses-wal-mode, storage-fts-is-derived-index) and PostgreSQL providing multi-tenant isolation via composite primary keys on every table (pg-uses-project-id-for-multi-tenancy, pgapi-multi-tenant-composite-keys).

The PostgreSQL backend, PgApi, is a SQL-native implementation that operates directly against the database with no in-memory Network object constructed (pgapi-no-in-memory-network, pgapi-is-sql-native-multi-tenant). It reimplements BFS propagation in application-level Python using JSONB containment queries against GIN indexes (pgapi-bfs-propagation-in-python, pg-uses-jsonb-with-gin-for-dependent-queries), and each public method constitutes a single committed transaction (pgapi-one-transaction-per-method). A notable design tension exists around referential integrity: antecedent and outlist references are stored as JSONB arrays without database-level foreign key constraints (pg-antecedent-refs-have-no-fk-constraints), so PgApi compensates with application-level validation that checks all referenced nodes exist before inserting justifications (pgapi-validates-refs-before-justification-insert, pgapi-enforces-referential-integrity-bidirectionally). The belief that database-level foreign keys would complete this picture is retracted (pgapi-referential-integrity-is-database-enforced), as is the claim of full defense-in-depth (pg-data-integrity-achieves-defense-in-depth), suggesting the system acknowledges this as an accepted gap rather than a resolved one. PostgreSQL routing uses a function-level early-return pattern where each API function checks for pg_conninfo and short-circuits to PgApi (pg-dispatch-is-function-level-early-return, all-api-functions-support-pg-dispatch), with psycopg imported lazily so the test suite works without Postgres dependencies (pg-psycopg-is-optional-dependency, pg-import-is-deferred).

The SQLite backend uses a full-snapshot persistence model: save replaces all rows and load constructs the complete network without incremental propagation (storage-save-is-full-replace, storage-load-bypasses-propagation, bootstrap-bypasses-incremental-propagation). Stored truth values are trusted on load without re-running propagation (storage-trusts-stored-truth-values), and justification ordering is preserved via AUTOINCREMENT rowid (justification-order-preserved-via-rowid, storage-justification-order-preserved). Lists like antecedents and outlist are stored as JSON text columns rather than normalized join tables (storage-uses-json-columns-for-lists), which is acceptable because SQLite always loads the full network. Schema evolution is handled pragmatically through try/except blocks that tolerate missing tables from older database versions (storage-handles-schema-evolution-via-try-except, storage-old-schema-compat), and the nogood counter is derived from existing data when the network_meta table is absent (storage-derives-counter-from-existing-nogoods). Each API function operates independently with no shared state or connection pooling (transaction-per-function).

Several retracted beliefs reveal the evolution of the system's self-understanding. The claim that PgApi is a complete SQL reimplementation achieving full behavioral parity is OUT (pgapi-is-complete-sql-reimplementation, pgapi-achieves-implementation-parity), as is the earlier claim of only partial API coverage (pgapi-partial-api-coverage), suggesting the reality lies between these extremes — substantial but not total parity. Similarly, broader claims that backend-independent guarantees extend to self-correction pipelines (backend-independent-self-correction), operational assurance (backend-agnostic-operational-assurance), and verified correctness (verified-correctness-extends-to-all-backends) are all retracted, indicating that while the core storage operations are backend-agnostic, higher-level system properties may still have backend-specific gaps. The retraction of persistence-is-snapshot-not-incremental and persistence-round-trip-is-lossless likely reflects refinement rather than fundamental disagreement, as their constituent claims remain active under more specific belief nodes. The testing infrastructure validates backend parity through a dedicated parity test suite (pg-test-suite-is-backend-parity) with per-test isolation via UUID-based project namespaces (pg-fixture-uses-uuid-isolation, pg-fixture-provides-per-test-isolation).