# SEP 10 — Low-rank Raman kernels along density paths | | | |---|---| | **SEP** | 10 | | **Title** | Low-rank Raman kernels along density paths | | **Author** | John Travers | | **Drafting agent** | Opus 5.5; GPT-6 (revision, 2026-10-22) | | **Status** | Draft | | **Review status** | In review | | **Reviewers** | GPT-6 (AI) | | **Type** | Feature | | **Created** | 2026-09-29 | | **Resolution** | — | | **Requires** | [SEP 3](archive/sep-0003-optmat-materials.md); [SEP 1](sep-0001-raman-response.md), the common execution contract and exact shifted-line reference from phase 1 | | **Supersedes** | — | | **Superseded-by** | — | | **Implementation** | Not started | | **Manual pages** | — | ## Part I — Proposal ### 1. Abstract Add an explicitly requested low-rank spectral representation of the sampled causal Raman kernel along a run's density path. At a stage, reconstruct the kernel from fixed complex rows and real coefficients, avoiding repeated line sampling, damping and kernel transforms. Extend SEP 1's common execution contract without replacing its exact sampled or analytic representations. Keep exact FFT as the default; positive tolerance selects compression. Qualify kernel construction, coefficient paths, source error and observable convergence separately, with bounded construction cost and NumPy, Numba and Kokkos CPU parity. Constant kernels bypass SVD. ### 2. Motivation and scope **Draft; implementation and production qualification have not started.** The current review uses solprop `20a2bafa` and optmat 0.8.0. SEP 3 is Final; its material migration is no longer prospective. The original prototype rank/timing observations below describe optmat's sampled experiment, not the cost or accuracy of this implementation. With per-line widths, rebuilding an effective kernel can dominate its convolution on a density gradient. N2 has 106 lines / 43 damping groups, dry air 170 / 82 at the documented reference state. H2 additionally shifts its Q lines with density. SEP 1 supplies the exact stage-dependent sampled kernel; this SEP compresses that family. Compression is a performance/accuracy trade-off, not a way to manufacture missing material data or a prerequisite for representing a shifted line exactly. Optmat's [response chapter][optmat-response] reports these exploratory ranks, using 64 densities and maximum spectral error relative to the sampled family's maximum: | Gas | Density (amagat) | Window and spacing | Rank at $10^{-4}$ | Rank at $10^{-6}$ | |---|---|---|---|---| | N2 | 0.5–2 | 131 ps, 4 fs | 4 | 5 | | N2 | 0.1–10 | 131 ps, 4 fs | 6 | 9 | | O2, dry air | 0.1–10 | 131 ps, 4 fs | 6 | 8 | | H2 | 1–30 | 66 ps, 2 fs | 5 | 6 | | D2 | 1–30 | 66 ps, 2 fs | 4 | 5 | These ranges are numerical compression experiments, not extensions of the independent Lorentzian line model's physical validity. Optmat reports N2 Q-branch line overlap at densities well below 10 amagat. A good low-rank reconstruction does not qualify the independent-line physics there. Translation preserves all material warnings, statuses and range records. Included are fixed-temperature, fixed-composition density paths; ordinary scalar and declared tensor responses; and constant regional silica kernels through their exact static representation. Supplied models use their declared sampled/dephasing laws; they need not expose analytic poles. Excluded are local thermal populations, arbitrary varying composition, new collision laws, line mixing, pole fitting, optical-frequency coupling, population dynamics and GPU qualification. ### 3. Context and compatibility See the [Raman data](../../guide/media/raman-data.md), [numerics](../../guide/media/raman-numerics.md), [kernel contract](../kernel-contract.md) and SEP 1 §§3–6. Current `RamanCoefficients` carry undamped sampled rows, rate paths, fraction, density, drive form and tensor weights. Compatible contributions are combined before the drive transforms. Regional silica has a distinct table and prefactor, not a gas-density leaf. SEP 1 makes the model and execution representation explicit: `sampled`, `analytic` and this SEP's `spectral` extension. Host analytic data remain available, and the backend receives poles when executing an analytic payload. Low-rank rows cannot be inverted to recover microscopic poles. Exact FFT and recurrence remain supported representations of the selected response; this SEP does not delete them. [SEP 11](sep-0011-finite-density-gas-index.md) changes how a consistent state is obtained from pressure, temperature and composition. Compression consumes its number-density path, without assuming an ideal or partial-pressure EOS. The same path supplies linear optics, Raman amplitude, collision widths and line shifts. Changing an EOS or material selection invalidates translation/cache identity. For optmat mixtures, strengths already contain mole fractions and the response uses the summed density of its labels. Do not multiply fractions twice. Supplied component amplitudes and collision-density paths retain their separate meanings. [SEP 6](archive/sep-0006-dispersive-nonlinearity.md) owns electronic dispersion. Its filters never act on Raman's drive or emitted source. The new [Maxwell–Bloch SEP](sep-0032-raman-maxwell-bloch.md) owns evolving molecular states; its nonlinear material solver is not compressed by this linear kernel representation. ### 4. Theory and construction #### One finite, sampled operator Let $N=\mathtt{n\_to}$, $L=2N$ and $K=N+1$. At material state $n$ (number density, m$^{-3}$), let $h_\kappa[j;n]$, $0\leq j`. - **Phase 1 — NumPy.** Spectral variant, schema/consumer checks, bounded construction, static/zero bypass, error records, explicit selectors, exact shifted-H2 comparisons, manual and independent local/source tests. Report resource-limit adequacy. - **Phase 2 — Numba CPU.** Preallocated assembly and stage invalidation, parity, interpreted-body coverage and measured stage costs. - **Phase 3 — Kokkos CPU and closure.** Native parity and ledger, final observable convergence, combined accuracy/cost/provenance evidence and documentation. During development slices run the repository's fast checks and affected tests. At the end of the SEP, run the full gate, slow lanes and coverage lane once on the final tree, or take them from CI. Rerun failing modules after failures. Documentation-only revisions use the documentation gate. Each phase reports actual results under Part II; GPU qualification stays separate and unsupported device use is refused. ### 12. Decisions and remaining questions The earlier open choices are resolved for this Draft: exact is the default, positive tolerance is explicit, vacuum-aware piecewise density sampling is selected, and constant supplied models bypass SVD. Maintainer scope directions are recorded below. | Question | Disposition | |---|---| | Coverage and resource limits for difficult supplied models | Phase 1 measures the declared finite construction policy; failure is explicit. No universal interval certificate is promised. | | Which ranks/tolerances give useful cost at achieved accuracy? | Phases 1–3 measure per case; no automatic policy or universal recommended tolerance is required. | | GPU assembly/transfer costs | Separate device qualification after CPU closure. | Human review and explicit acceptance remain outstanding. No pending physical-model choice is required to implement the selected fixed-state sampled-kernel approximation. ### 13. References 1. optmat 0.8.0, [Raman response functions][optmat-response] and [line parameters](https://github.com/LupoLab/optmat/blob/v0.8.0/docs/raman.md). The rank/window table is a sampled numerical experiment, not a physical-range qualification or a verified continuous error bound. 2. C. Eckart and G. Young, "The approximation of one matrix by another of lower rank," *Psychometrika* **1**, 211–218 (1936). [doi:10.1007/BF02288367](https://doi.org/10.1007/BF02288367). [optmat-response]: https://github.com/LupoLab/optmat/blob/v0.8.0/docs/raman_response.md ## Part II — Implementation record No implementation phases have started. ## Part III — History - 2026-09-29: Drafted from the low-rank design split out of SEP 3. - 2026-10-17: Restated reproducibility for SEP 25, retaining exact input-based identities. - 2026-10-22: Reviewed with SEP 1 and revised to use its execution contract, exact default, explicit approximation and current material/state semantics. ### Maintainer directions — 2026-10-22 John Travers requested a joint acceptance-readiness review, retained the current lag taper, chose explicit approximation tolerance with exact FFT default, adopted the mixture-EOS direction for SEP 11, and kept broadband coupling separate. The completed AI review leaves this Draft **In review**, pending his review and acceptance. ### AI review — 2026-10-22 — GPT-6 Review of the entire original Part I against SEP 1 and the current implementation. 1. **R1 (P1), §§3, 6 — Incompatible seam replacement.** Replacing all leaves with spectra while keeping poles only on the host prevents a backend recurrence. Extend a shared tagged contract and retain exact sampled/analytic evaluation. 2. **R2 (P1), §§4–5 — Full rank is not an exact continuous family.** A finite training span and interpolated coefficients need not reproduce unseen density. Zero tolerance must choose exact FFT. Part III's counterexample establishes this. 3. **R3 (P1), §4 — The proposed density floor is stale.** Current optmat widths are finite at vacuum and have capped-law transitions; logarithmic sampling and a fabricated floor omit the state the run may reach. Include zero and resolve the actual piecewise law. 4. **R4 (P1), §§4, 7 — Accuracy contracts are incomplete.** Clarify real-stacked projection, scale, combined compression/interpolation budget, support/endpoints and operator/source error. Sampled validation is not an interval proof and a kernel tolerance is not global propagation error or relative weak-line gain. 5. **R5 (P1), §§4–6 — SVD columns are not reproducible identities.** A sign rule cannot fix degenerate-subspace rotations. Compare kernels/projectors where defined, and bound construction resources with explicit failure. 6. **R6 (P2), §§4, 6 — Static/zero and memory claims need correction.** A zero family has rank zero, constant pressure need not imply constant density, and a complex half-spectrum row costs approximately the same as a real doubled row. 7. **R7 (P1), §§2–3, 7 — Material and composition boundaries are missing.** Exact shifted H2 needs a direct reference, mixture strengths contain fractions already, optmat's rank experiments cross some physical validity limits, and SEP 6 filters must not contaminate Raman. Record SEP 11 state compatibility and limits. 8. **R8 (P2), §§5, 11–12 — Defaults and completion policy conflict.** The proposed automatic $10^{-4}$ changes source accuracy without an observable budget; per-phase full-lane reruns conflict with current repository policy. Apply the agreed exact default and current testing policy. ### Response to review — 2026-10-22 - **R1:** §§3 and 6 extend SEP 1's contract; phase 1 is an explicit prerequisite. - **R2:** §§4–5 specify exact FFT for zero tolerance; the counterexample is retained. - **R3:** §4 samples density pieces with vacuum and linewidth switches, without a floor. - **R4:** §4 defines the real projection, final-row validation, separate budgets and DFT operator/source bound. §7 adds exact shifted-H2 and observable refinement. - **R5:** §§5–6 use input identities, reconstruction/subspace comparisons and bounded construction; rank threshold variations are recorded rather than hidden. - **R6:** §§4 and 6 add static/zero paths and correct byte accounting. - **R7:** §§2–3 and 7 distinguish material validity and mixture ownership from numerical compression, with SEP 6/11 coexistence and independent exact references. - **R8:** §§5 and 11–12 implement the maintainer's explicit-tolerance direction and slice/final testing policy. No automatic/default policy remains to settle. ### Review probes and source audit — 2026-10-22 For $j=0,\ldots,255$, take the dimensionless causal family $$ h_j(n)=e^{-0.01nj}\sin[(0.2+0.07n)j]. $$ A two-column time-domain matrix at $n=0.2,0.8$, retaining both thin-SVD columns, reconstructs $h(0.5)$ with a maximum error **0.731160** relative to its own maximum. The two training columns themselves are retained at roundoff. This is a counterexample to the old exactness claim, not an estimate of physical gas compression error. At $N=256$, float64 storage for $2N$ real samples is 4096 bytes; complex128 storage for $N+1$ bins is 4112 bytes. This checks the corrected row memory accounting. The current optmat response/width/shift APIs, material ranges and prototype table were inspected; the old prototype timings were not rerun and no production SVD or propagation benchmark is claimed. The SVD projection and DFT norm bound above are derived explicitly. The bibliography of [2] is retained; its original text was not newly inspected for coefficient/data transcription, of which there is none. Documentation checks passed: `git diff --check`, codespell on all seven changed documentation files, `uv run pytest tests/test_seps.py` (231 passed, two expected process-SEP skips), and `uv run poe docs` with warnings treated as errors. A missing roadmap subsection anchor from the first build was corrected before the successful build. Full numerical gates are not required for this documentation-only revision. All R1–R8 findings are addressed above. The joint review found no remaining design contradiction with SEP 1; the finite construction limits and useful ranks remain implementation measurements, not assumed successes. Human review and acceptance remain outstanding. No production compression, backend parity or propagation-error qualification is claimed by these review probes.