clophfit.fitting.model_validation#

Reusable model-validation helpers for ClopHfit fitting workflows.

These utilities are designed to live in clophfit.fitting and be reused by both package tests and manuscript-analysis scripts. They intentionally avoid any manuscript-specific paths, file formats, or plate names.

Classes#

`ResidualAnalysis`	Statistical and structural residual analyses over a residual table.
`ResidualDiagnostics`	Single fluent home for residual analysis over a canonical residual table.
`ResidualComparison`	Bundle residual diagnostics, well summaries, and trace parameter summaries.
`OutlierCriterion`	What counts as an outlier, expressed as data rather than as a function.
`ResidualTail`	Flag only the excess points in the calibrated tail of the residuals.
`RobustZMad`	Flag points by a robust z-score built on each group's median and MAD.
`OutlierProbability`	Flag points by posterior outlier probability from a mixture fit.

Functions#

`residual_normal_scores`(likelihood_residual, *[, ...])	Map likelihood-scale residuals onto a Normal diagnostic scale.
`robust_residual_outlier_mask`(likelihood_residual, *[, ...])	Flag observations by calibrated Normal-score residual magnitude.
`excess_tail_outlier_mask`(likelihood_residual, *[, ...])	Mask only residual outliers beyond an allowed tail fraction.
`mark_outliers`(residuals[, criterion, exclude_col])	Annotate a residual table with a single exclusion column.
`apply_exclusions`(results, residuals, *[, exclude_col, ...])	Mask out the rows `mark_outliers()` flagged, worst first.
`posterior_dataset`(trace)	Return the posterior xarray Dataset from ArviZ InferenceData or DataTree.
`sample_stats_dataset`(trace)	Return sample_stats Dataset from InferenceData or DataTree.
`trace_parameter_summary`(results)	Summarize scalar per-label PyMC noise parameters from traces.
`robust_likelihood_from_trace`(trace)	Infer the observation-likelihood family from a trace's posterior variables.
`robust_settings_from_trace`(trace)	Infer `(apply_student_t_transform, student_t_nu)` from a trace.
`x_axis_sanity`(trace)	Check pH/x-axis invariants for per-well `x_true` traces.
`trace_diagnostics`(trace, *[, compute_loo, ...])	Collect basic MCMC and optional LOO diagnostics from a PyMC trace.
`pareto_k_table`(multi_or_trace[, results])	Return pointwise PSIS-LOO Pareto-k values annotated by well, label, and step.
`pareto_k_summary`(pareto_k)	Summarize pointwise Pareto-k diagnostics.
`merge_log_likelihoods`(trace)	Merge multiple pointwise log-likelihood variables for ArviZ LOO/compare.
`residuals_from_multifit`(multi, trace_id, ...[, ...])	Build a long calibrated-residual table from a MultiFitResult.
`residuals_from_fit_results`(results, trace_id, ...[, ...])	Build a long calibrated residual table from classical FitResult objects.
`model_residual_score_table`(res_df)	Return model-level and detailed residual summary tables.

Module Contents#

class clophfit.fitting.model_validation.ResidualAnalysis#

Statistical and structural residual analyses over a residual table.

The single home for frame-returning residual analyses: per-(trace, label) distribution stats (distribution_summary()), residual-vs-x correlation (x_correlation()), lag-1 autocorrelation (lag1_autocorrelation()), covariance and correlation across x-positions (covariance(), correlation()), systematic label bias (label_bias()), and boolean quality-control checks (validate()). Reach it from ResidualDiagnostics.analysis, or construct it directly from a canonical residual table.

Parameters:: residuals (pd.DataFrame) – Canonical residual table (Schema B) carrying well/label/x/ std_res columns.

distribution_summary()#

Per-(trace, label) residual distribution stats.

See residual_distribution_summary(). Supersedes clophfit.fitting.residuals.residual_statistics().

Return type:: pandas.DataFrame

x_correlation()#

Pearson/Spearman correlation of residuals against x.

See residual_x_correlation(). Relates to clophfit.fitting.residuals.estimate_x_shift_statistics(); residual_x_trend_summary() gives the by-step trend.

Return type:: pandas.DataFrame

lag1_autocorrelation()#

Lag-1 residual autocorrelation per well and its per-label summary.

See residual_lag1_autocorrelation(). Supersedes clophfit.fitting.residuals.detect_adjacent_correlation().

Return type:: tuple[pandas.DataFrame, pandas.DataFrame]

covariance(*, value_col='std_res', by=None)#

Per-label covariance of residuals across titration points.

Wells are aligned on a shared point index (by) rather than the raw x, because in real titrations each well carries its own jittered x-values (e.g. pH), which would leave no two wells sharing a column.

Parameters:

value_col (str) – Residual column to use (default "std_res").
by (str | None) – Column that indexes the shared titration point across wells. None auto-selects the first available of "step", "raw_i", "x".

Returns:

One point-by-point covariance matrix per label, over the wells that share a complete set of points. Axes are labelled by the mean x at each point when available. Labels with fewer than two complete wells map to an empty frame.

Return type:

dict[str, pd.DataFrame]

correlation(*, value_col='std_res', by=None)#

Per-label correlation matrices derived from covariance().

Parameters:

value_col (str) – Residual column to use (default "std_res").
by (str | None) – Shared point index forwarded to covariance().

Returns:

One point-by-point correlation matrix per label (empty where the covariance is empty).

Return type:

dict[str, pd.DataFrame]

label_bias(*, n_bins=5)#

Detect systematic residual bias by label and x-range bin.

Residuals are globally z-scored before aggregation so the summaries are comparable across labels.

Parameters:: n_bins (int) – Number of equal-width x bins for the per-(label, bin) summary.
Returns:: (bias_by_label_and_bin, bias_by_label).
Return type:: tuple[pd.DataFrame, pd.DataFrame]

validate(*, verbose=False)#

Boolean residual-quality checks over the table.

Runs three checks: per-label systematic bias (t-test against zero), the overall outlier rate (beyond ±2 sigma), and serial correlation within each label (Durbin-Watson statistic on x-sorted residuals).

Parameters:: verbose (bool) – Print a warning for each failed check.
Returns:: {"bias_ok", "outliers_ok", "correlation_ok"}. An empty table passes every check.
Return type:: dict[str, bool]

class clophfit.fitting.model_validation.ResidualDiagnostics#

Single fluent home for residual analysis over a canonical residual table.

Build one from a fit (ResidualDiagnostics.from_fit_results(...)) or wrap an existing table (ResidualDiagnostics(fr.residuals)), then chain transforms and read summaries. This is the place to reach for residual analysis; the module-level residual_* functions are its building blocks.

Transforms (return a new ResidualDiagnostics): annotate(), step_centered(), label_scaled(), well_scaled(), with_relative_residuals().

Summaries (return frames): well_summary(), normality(), step_summary(), position_summary(), tail_rows().

Statistical and structural analyses (distribution, x-correlation, lag-1 autocorrelation, covariance, correlation, label bias, QA checks) live under analysis (a ResidualAnalysis). Their module-level building blocks are residual_distribution_summary / residual_x_correlation / residual_lag1_autocorrelation / residual_x_trend_summary / residual_cross_label_correlation.

Plots: plot_hist_qq(), plot_step(), plot_role(), plot_col(), plot_well_summary().

classmethod from_fit_results(results, trace_id, binding_function, *, robust=False, value_col='std_res')#

Create diagnostics from a mapping of well IDs to FitResult objects.

Parameters:

results (dict[str, Any])
trace_id (str)
binding_function (Callable[..., ArrayLike])
robust (bool)
value_col (str)

Return type:

ResidualDiagnostics

annotate(*, fit_df=None, ctrl_wells=(), extra_ctrl_wells=())#

Return diagnostics annotated with fit parameters, role, row, and column.

Parameters:

fit_df (pandas.DataFrame | None)
ctrl_wells (Iterable[str])
extra_ctrl_wells (Iterable[str])

Return type:

ResidualDiagnostics

step_centered(*, column=None)#

Return diagnostics with residuals centered within label and step.

Parameters:: column (str | None)
Return type:: ResidualDiagnostics

label_scaled(*, column=None)#

Return diagnostics with residuals divided by label-wise standard deviation.

Parameters:: column (str | None)
Return type:: ResidualDiagnostics

with_relative_residuals(*, denominator='yhat', raw_col='raw_res', eps=1e-12)#

Return diagnostics with raw and relative residual columns added.

Parameters:

denominator (str)
raw_col (str)
eps (float)

Return type:

ResidualDiagnostics

well_scaled(*, column=None, min_count=3)#

Return diagnostics with residuals divided by well-and-label SD.

Parameters:

column (str | None)
min_count (int)

Return type:

ResidualDiagnostics

well_summary(*, column=None)#

Summarize residual and signal behavior per well and label.

Parameters:: column (str | None)
Return type:: pandas.DataFrame

normality(*, column=None)#

Return Shapiro, D’Agostino, and Anderson normality diagnostics.

Parameters:: column (str | None)
Return type:: pandas.DataFrame

step_summary(*, column=None)#

Summarize residuals by label and titration step.

Parameters:: column (str | None)
Return type:: pandas.DataFrame

position_summary(*, column=None)#

Summarize residuals by row, column, edge column, and role when present.

Parameters:: column (str | None)
Return type:: dict[str, pandas.DataFrame]

property analysis: ResidualAnalysis#

Structural analyses (covariance, correlation, bias, QA) over the table.

Returns:: Accessor exposing covariance(), correlation(), label_bias(), and validate().
Return type:: ResidualAnalysis

tail_rows(n=30, *, column=None)#

Return rows with largest absolute residual values.

Parameters:

n (int)
column (str | None)

Return type:

pandas.DataFrame

plot_hist_qq(*, column=None)#

Plot histogram and Q-Q panels by label plus pooled.

Parameters:: column (str | None)
Return type:: Any

plot_step(*, column=None)#

Plot residual distributions by label and titration step.

Parameters:: column (str | None)
Return type:: Any

plot_role(*, column=None)#

Plot residual distributions by role, if diagnostics are annotated.

Parameters:: column (str | None)
Return type:: Any

plot_col(*, column=None)#

Plot residual distributions by plate column.

Parameters:: column (str | None)
Return type:: Any

plot_well_summary(*, x='yhat_mean', y='std_res_sd', hue='role')#

Plot per-well residual spread against signal or fitted parameters.

Parameters:

x (str)
y (str)
hue (str)

Return type:

Any

class clophfit.fitting.model_validation.ResidualComparison#

Bundle residual diagnostics, well summaries, and trace parameter summaries.

classmethod from_fit_results(results, trace_id, binding_function, *, fit_df=None, ctrl_wells=(), extra_ctrl_wells=(), robust=False, value_col='std_res')#

Create a compact residual-comparison object from fit results.

Parameters:

results (dict[str, Any])
trace_id (str)
binding_function (Callable[..., ArrayLike])
fit_df (pandas.DataFrame | None)
ctrl_wells (Iterable[str])
extra_ctrl_wells (Iterable[str])
robust (bool)
value_col (str)

Return type:

ResidualComparison

with_value(column)#

Return a copy using a different residual column as the active value.

Parameters:: column (str)
Return type:: ResidualComparison

clophfit.fitting.model_validation.residual_normal_scores(likelihood_residual, *, robust=False, student_t_nu=STUDENT_T_NU)#

Map likelihood-scale residuals onto a Normal diagnostic scale.

For Normal likelihoods this is the identity. For Student-t likelihoods, (y - mu) / sigma follows a t distribution, so Normal QQ plots and abs(residual) > 2 style diagnostics should use the probability integral transform to an equivalent standard-Normal score.

Parameters:

likelihood_residual (ArrayLike)
robust (bool)
student_t_nu (float)

Return type:

numpy.ndarray

clophfit.fitting.model_validation.robust_residual_outlier_mask(likelihood_residual, *, threshold=3.0, robust=False, student_t_nu=STUDENT_T_NU)#

Flag observations by calibrated Normal-score residual magnitude.

Parameters:

likelihood_residual (ArrayLike)
threshold (float)
robust (bool)
student_t_nu (float)

Return type:

numpy.ndarray

clophfit.fitting.model_validation.excess_tail_outlier_mask(likelihood_residual, *, threshold=3.0, allowed_tail_fraction=0.01, min_allowed_tail_count=1, robust=False, student_t_nu=STUDENT_T_NU)#

Mask only residual outliers beyond an allowed tail fraction.

The residuals are first mapped to the calibrated Normal diagnostic scale. Observations with abs(z) <= threshold are never removed. If more than allowed_tail_fraction of finite observations exceed the threshold, only the largest excess observations are marked for removal.

Parameters:

likelihood_residual (ArrayLike)
threshold (float)
allowed_tail_fraction (float)
min_allowed_tail_count (int)
robust (bool)
student_t_nu (float)

Return type:

numpy.ndarray

class clophfit.fitting.model_validation.OutlierCriterion#

Bases: Protocol

What counts as an outlier, expressed as data rather than as a function.

Each criterion owns its own knobs, so adding one does not widen the signature of mark_outliers().

evaluate(residuals)#

Score every row and say which rows are outliers.

Parameters:: residuals (pd.DataFrame) – Canonical residual table.
Returns:: The per-row severity score, and the boolean exclusion flag.
Return type:: tuple[pd.Series, pd.Series]

class clophfit.fitting.model_validation.ResidualTail#

Flag only the excess points in the calibrated tail of the residuals.

A Normal sample of size n is expected to put a small fraction beyond any threshold; this keeps that expected tail and flags only the surplus, per group.

Parameters:

residual_col (str) – Column holding the standardized residual.
group_cols (tuple[str, ...]) – Columns defining the groups scored independently. Missing columns are ignored; if none are present the whole table is one group.
threshold (float) – Standardized-residual cutoff defining the tail.
allowed_tail_fraction (float) – Fraction of each group tolerated beyond threshold before any point is flagged.
min_allowed_tail_count (int) – Floor on the tolerated count, regardless of group size.

evaluate(residuals)#

Score by |residual| and flag the surplus tail of each group.

Parameters:: residuals (pd.DataFrame) – Canonical residual table.
Returns:: Absolute residuals, and the excess-tail exclusion flag.
Return type:: tuple[pd.Series, pd.Series]

class clophfit.fitting.model_validation.RobustZMad#

Flag points by a robust z-score built on each group’s median and MAD.

The scale is estimated from the residuals themselves rather than taken from y_err or from the model’s standardized residual, so an error model whose scale is wrong cannot hide an outlier behind it. Unlike a mean/SD z-score, which the outlier itself inflates and which therefore cannot exceed sqrt(n - 1) - 2.449 over the seven points of a label-1 titration - this score has no ceiling.

It is deliberately model-free about shape: no expected tail is tolerated, unlike ResidualTail. Combine it with a min_keep in apply_exclusions() rather than with a tail allowance here.

Parameters:

residual_col (str) – Column holding the raw (unweighted) residual. Raw rather than standardized, so the scale comes from the data.
group_cols (tuple[str, ...]) – Columns defining the groups scored independently. Missing columns are ignored; if none are present the whole table is one group. Channel scales differ by up to an order of magnitude, so keep "label".
threshold (float) – Robust z beyond which a point is flagged.

evaluate(residuals)#

Score every row by its group’s robust z and flag those above the cutoff.

Parameters:: residuals (pd.DataFrame) – Canonical residual table.
Returns:: Robust z-scores, and the exclusion flag.
Return type:: tuple[pd.Series, pd.Series]

class clophfit.fitting.model_validation.OutlierProbability#

Flag points by posterior outlier probability from a mixture fit.

Parameters:

probability_col (str) – Column of per-point posterior outlier probabilities.
threshold (float) – Probability above which a point is flagged.
residual_threshold (float | None) – When set, a point must also exceed this absolute residual to be flagged. None applies the probability criterion alone.
residual_col (str) – Column compared against residual_threshold.

evaluate(residuals)#

Score by posterior probability and flag rows above the cutoff.

Parameters:: residuals (pd.DataFrame) – Canonical residual table.
Returns:: Posterior outlier probabilities, and the exclusion flag.
Return type:: tuple[pd.Series, pd.Series]

clophfit.fitting.model_validation.mark_outliers(residuals, criterion=None, *, exclude_col=EXCLUDE_COL)#

Annotate a residual table with a single exclusion column.

Parameters:

residuals (pd.DataFrame) – Canonical residual table, e.g. from residuals_from_multifit() or residuals_from_fit_results().
criterion (OutlierCriterion | None) – What counts as an outlier. Defaults to ResidualTail.
exclude_col (str) – Name of the boolean column to write.

Returns:

Copy of residuals with exclude_col and SCORE_COL added.

Return type:

pd.DataFrame

Raises:

TypeError – If residuals is not a pandas DataFrame.

clophfit.fitting.model_validation.apply_exclusions(results, residuals, *, exclude_col=EXCLUDE_COL, min_keep=3)#

Mask out the rows mark_outliers() flagged, worst first.

Intended for the second pass of a refit: fit once, compute residuals, mark, apply, refit. Points are dropped in decreasing SCORE_COL order, so when min_keep binds it is the mildest points that survive.

Parameters:

results (_t.Mapping[str, _t.Any]) – Well identifiers mapped to datasets, or to objects carrying one.
residuals (pd.DataFrame) – Residual table already annotated by mark_outliers().
exclude_col (str) – Boolean column naming the rows to mask.
min_keep (int) – Minimum unmasked points retained per label.

Returns:

Deep-copied datasets with the flagged rows masked out.

Return type:

dict[str, _t.Any]

Raises:

TypeError – If residuals is not a pandas DataFrame.

clophfit.fitting.model_validation.posterior_dataset(trace)#

Return the posterior xarray Dataset from ArviZ InferenceData or DataTree.

PyMC/ArviZ versions differ in whether returned objects are InferenceData-like or xarray DataTree-like. This helper hides that difference for validation code.

Parameters:: trace (Any)
Return type:: Any

clophfit.fitting.model_validation.sample_stats_dataset(trace)#

Return sample_stats Dataset from InferenceData or DataTree.

Parameters:: trace (Any)
Return type:: Any

clophfit.fitting.model_validation.trace_parameter_summary(results)#

Summarize scalar per-label PyMC noise parameters from traces.

Parameters:: results (_t.Mapping[str, _t.Any]) – Mapping from well identifiers to fit-result-like objects with mini or trace attributes and optional datasets.
Returns:: Wide table indexed by well and label with posterior means and standard deviations for recognized scalar noise parameters. An empty table with well and label columns is returned when no recognized trace parameters are available.
Return type:: pd.DataFrame

clophfit.fitting.model_validation.robust_likelihood_from_trace(trace)#

Infer the observation-likelihood family from a trace’s posterior variables.

Detects a contamination mixture (pi_outlier_* / outlier_inflate) or an inferred/fixed Student-t (a student_t_nu deterministic); anything else is a plain Normal. Used to label the residual_likelihood column.

Parameters:: trace (_t.Any) – A PyMC trace (xr.DataTree) or None for classical fits.
Returns:: The observation-likelihood family: "normal", "student_t", or "mixture".
Return type:: str

clophfit.fitting.model_validation.robust_settings_from_trace(trace)#

Infer (apply_student_t_transform, student_t_nu) from a trace.

robust here means specifically “standardize std_res via the Student-t probability-integral transform”, which is correct only for a Student-t likelihood. A contamination mixture uses Normal components, so its residuals are standardized as Normal (robust=False) and its outlier structure is reported through p_outlier instead. Used by FitResult.residuals / MultiFitResult.residuals so the residual table is standardized correctly without the caller re-supplying fit settings.

Parameters:: trace (_t.Any) – A PyMC trace (xr.DataTree) or None for classical fits.
Returns:: Whether to apply the Student-t transform, and the nu to use.
Return type:: tuple[bool, float]

Notes

A fixed-nu Student-t is detected via the student_t_nu deterministic recorded by clophfit.fitting.bayes._student_t_nu_value(). Pass robust= to residual_table to override. See robust_likelihood_from_trace() for the likelihood-family label.

clophfit.fitting.model_validation.x_axis_sanity(trace)#

Check pH/x-axis invariants for per-well x_true traces.

For per-well x models with a shared start pH, all wells at step 0 should be identical within each draw. x_step0_max_abs_spread should therefore be close to zero. A global 1-D x_true (deterministic x-error) has no well dimension and is skipped.

Parameters:: trace (Any)
Return type:: dict[str, Any]

clophfit.fitting.model_validation.trace_diagnostics(trace, *, compute_loo=False, summary_var_names=None)#

Collect basic MCMC and optional LOO diagnostics from a PyMC trace.

Parameters:

trace (Any)
compute_loo (bool)
summary_var_names (list[str] | None)

Return type:

dict[str, Any]

clophfit.fitting.model_validation.pareto_k_table(multi_or_trace, results=None)#

Return pointwise PSIS-LOO Pareto-k values annotated by well, label, and step.

Parameters:

multi_or_trace (_t.Any) – A MultiFitResult-like object with trace and results attributes, or a raw PyMC/ArviZ trace.
results (_t.Mapping[str, _t.Any] | None) – Fit results keyed by well. Required when multi_or_trace is a raw trace.

Returns:

One row per likelihood observation with pareto_k and observation metadata where it can be recovered from the fitted datasets.

Return type:

pd.DataFrame

Raises:

ValueError – If no fit-result mapping is available, if the trace lacks pointwise log-likelihood data, or if ArviZ’s pointwise output cannot be aligned to the fitted datasets.

clophfit.fitting.model_validation.pareto_k_summary(pareto_k)#

Summarize pointwise Pareto-k diagnostics.

Parameters:: pareto_k (pd.DataFrame) – Pointwise Pareto-k table, usually returned by pointwise_pareto_k().
Returns:: Summary table grouped by label and well when those columns are present, otherwise grouped by likelihood variable. The table includes count, maximum, mean, and warning fraction. Empty input returns an empty table.
Return type:: pd.DataFrame

clophfit.fitting.model_validation.merge_log_likelihoods(trace)#

Merge multiple pointwise log-likelihood variables for ArviZ LOO/compare.

Parameters:: trace (Any)
Return type:: Any

clophfit.fitting.model_validation.residuals_from_multifit(multi, trace_id, binding_function, *, include_fit_params=False, robust=False, student_t_nu=STUDENT_T_NU, outlier_threshold=3.0, residual_likelihood=None)#

Build a long calibrated-residual table from a MultiFitResult.

The returned table has one row per active observation and always includes trace, well, label, step, observed, predicted, sigma, raw residual, likelihood-scaled residual, Normal-score residual, likelihood family, and residual-outlier metadata columns.

likelihood_res is always (observed - predicted) / sigma. For Student-t robust fits, std_res is the equivalent standard-Normal score from the t CDF, suitable for Normal QQ plots and z-style outlier flags. residual_likelihood labels the residual_likelihood column; when None it defaults to "student_t" if robust else "normal".

Parameters:

multi (Any)
trace_id (str)
binding_function (Callable[..., ArrayLike])
include_fit_params (bool)
robust (bool)
student_t_nu (float)
outlier_threshold (float)
residual_likelihood (str | None)

Return type:

pandas.DataFrame

clophfit.fitting.model_validation.residuals_from_fit_results(results, trace_id, binding_function, *, include_fit_params=False, robust=False, student_t_nu=STUDENT_T_NU, outlier_threshold=3.0, trace=None, residual_likelihood=None)#

Build a long calibrated residual table from classical FitResult objects.

Parameters:

results (dict[str, _t.Any]) – Mapping of well identifier to FitResult-like object.
trace_id (str) – Value placed in the trace_id column.
binding_function (_t.Callable[..., ArrayLike]) – Model evaluated to obtain the prediction yhat.
include_fit_params (bool) – Append the K/S0_*/S1_* point estimates as extra columns.
robust (bool) – Standardize std_res via the Student-t probability-integral transform.
student_t_nu (float) – Degrees of freedom used for the robust standardization.
outlier_threshold (float) – Threshold on |std_res| for the is_residual_outlier flag.
trace (_t.Any) – Optional PyMC trace. When it carries outlier_probability_{label} deterministics (contamination-mixture fits), the per-point posterior outlier probability is extracted into p_outlier; otherwise that column is NaN. Mirrors residuals_from_multifit() so single-well and multi-well tables share the full schema.
residual_likelihood (str | None) – Label for the residual_likelihood column. When None it defaults to "student_t" if robust else "normal".

Returns:

The canonical residual table with columns RESIDUAL_TABLE_COLUMNS.

Return type:

pd.DataFrame

clophfit.fitting.model_validation.model_residual_score_table(res_df)#

Return model-level and detailed residual summary tables.

Parameters:: res_df (pandas.DataFrame)
Return type:: tuple[pandas.DataFrame, pandas.DataFrame, pandas.DataFrame, pandas.DataFrame, pandas.DataFrame]