clophfit.fitting.residuals#

Residual extraction and analysis utilities for fit results.

This module provides tools to extract, analyze, and validate residuals from fitting procedures. Useful for diagnostics, model validation, and comparing different fitting methods.

Classes#

ResidualPoint

Single residual data point with metadata.

Functions#

`extract_residual_points`(fr)	Extract residual points from a fit result.
`residual_dataframe`(fr)	Convert fit result residuals to a DataFrame.
`qq_fit`(z)	Slope and intercept of the ordered residuals against Normal quantiles.
`residual_statistics`(df)	Compute residual statistics by label.
`detect_adjacent_correlation`(all_res)	Detect correlation between adjacent (lag-1) residuals within wells.
`estimate_x_shift_statistics`(all_res[, fit_results])	Estimate potential systematic x-shifts per well (heuristics).
`plot_residual_vs_predicted`(all_res[, title])	Plot \|standardized residual\| vs predicted signal per label.
`plot_residual_distribution`(all_res[, title])	Histogram and Q-Q of the standardised residuals, per label.
`plot_residual_vs_yerr`(all_res[, title])	Plot raw residual² vs y_err² per label (error calibration check).

Module Contents#

class clophfit.fitting.residuals.ResidualPoint#

Single residual data point with metadata.

label#

Dataset label (e.g., ‘1’, ‘2’ for multi-label fits)

Type:: str

x#

X-value (pH or ligand concentration)

Type:: float

y#

Observed signal value.

Type:: float

yhat#

Model-predicted signal value (y - raw_res).

Type:: float

sigma#

Measurement uncertainty used during fitting.

Type:: float

raw_res#

Raw residual: y - yhat.

Type:: float

likelihood_res#

Likelihood-scale residual: (y - yhat) / sigma.

Type:: float

std_res#

Normal-scale standardized residual (equal to likelihood_res for a Normal likelihood, as produced by LMFit/ODR fits).

Type:: float

raw_i#

Index into the original (unmasked) arrays for this label (DataArray.xc/yc).

Type:: int

Notes

Field names follow the canonical residual-table schema shared with clophfit.fitting.model_validation.RESIDUAL_TABLE_COLUMNS.

clophfit.fitting.residuals.extract_residual_points(fr)#

Extract residual points from a fit result.

Parameters:: fr (FitResult) – Fit result containing residuals and dataset
Returns:: List of residual points with metadata for each observation
Return type:: list[ResidualPoint]
Raises:: ValueError – If residual length doesn’t match dataset sizes

Examples

>>> from clophfit.fitting.core import fit_binding_glob
>>> from clophfit.fitting.data_structures import Dataset, DataArray
>>> import numpy as np
>>> # Create test data
>>> x = np.array([9.0, 8.0, 7.0, 6.0, 5.0])
>>> y = 500 + 500 * 10 ** (7.0 - x) / (1 + 10 ** (7.0 - x))
>>> da = DataArray(xc=x, yc=y, y_errc=np.ones_like(y) * 10)
>>> dataset = Dataset({"1": da}, is_ph=True)
>>> fr = fit_binding_glob(dataset)
>>> residuals = extract_residual_points(fr)
>>> len(residuals) > 0
True
>>> residuals[0].label
'1'

clophfit.fitting.residuals.residual_dataframe(fr)#

Convert fit result residuals to a DataFrame.

Parameters:: fr (FitResult) – Fit result to extract residuals from
Returns:: DataFrame with the canonical residual columns: label, x, y, yhat, sigma, raw_res, likelihood_res, std_res, raw_i.
Return type:: pd.DataFrame

Examples

>>> from clophfit.fitting.core import fit_binding_glob
>>> from clophfit.fitting.data_structures import Dataset, DataArray
>>> import numpy as np
>>> x = np.array([9.0, 8.0, 7.0, 6.0, 5.0])
>>> y = 500 + 500 * 10 ** (7.0 - x) / (1 + 10 ** (7.0 - x))
>>> da = DataArray(xc=x, yc=y, y_errc=np.ones_like(y) * 10)
>>> dataset = Dataset({"1": da}, is_ph=True)
>>> fr = fit_binding_glob(dataset)
>>> df = residual_dataframe(fr)
>>> "label" in df.columns and "x" in df.columns
True

clophfit.fitting.residuals.qq_fit(z)#

Slope and intercept of the ordered residuals against Normal quantiles.

The slope is the scale the standardised residuals actually have: 1 when the model’s sigma is right, below 1 when the residuals are compressed (sigma too large, or a fitted scale that has absorbed its own tails), above 1 when sigma is too small. It is robust to a few outliers in a way the plain SD is not, since a straight line through the bulk barely moves for one wild point. Read from the same probability plot the figure draws, so the number and the picture agree.

Parameters:: z (ArrayF) – Standardised residuals; non-finite values are dropped.
Returns:: (slope, intercept), or (nan, nan) with fewer than three points.
Return type:: tuple[float, float]

Examples

>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> slope, _ = qq_fit(0.5 * rng.standard_normal(2000))
>>> round(slope, 1)
0.5

clophfit.fitting.residuals.residual_statistics(df)#

Compute residual statistics by label.

Parameters:

df (pd.DataFrame) – Residual DataFrame (from residual_dataframe or residuals_from_fit_results)

Returns:

Statistics by label: mean, std, median, mad, outlier_count, robust_outlier_count, n_points, outlier_rate, robust_outlier_rate, qq_slope, qq_intercept.

outlier_count thresholds the model-standardized std_res (> 2); a few points can hide by inflating the fitted scale. robust_outlier_count uses each label’s own median/MAD scale (modified z-score > ROBUST_Z_THRESHOLD), so masked points still surface. qq_slope is the scale of the residuals read off the probability plot (see qq_fit()): 1 means sigma is calibrated.

Return type:

pd.DataFrame

Examples

>>> from clophfit.fitting.core import fit_binding_glob
>>> from clophfit.fitting.data_structures import Dataset, DataArray
>>> import numpy as np
>>> x = np.array([9.0, 8.0, 7.0, 6.0, 5.0])
>>> y = 500 + 500 * 10 ** (7.0 - x) / (1 + 10 ** (7.0 - x))
>>> da = DataArray(xc=x, yc=y, y_errc=np.ones_like(y) * 10)
>>> dataset = Dataset({"1": da}, is_ph=True)
>>> fr = fit_binding_glob(dataset)
>>> all_res = residual_dataframe(fr)
>>> stats = residual_statistics(all_res)
>>> "mean" in stats.columns
True

clophfit.fitting.residuals.detect_adjacent_correlation(all_res)#

Detect correlation between adjacent (lag-1) residuals within wells.

Tests whether adjacent points show systematic patterns (e.g. positive then negative), which can indicate x-value errors or model misspecification.

Parameters:

all_res (pd.DataFrame) – Residual table with label, well, x and std_res columns.

Returns:

correlation_stats (pd.DataFrame) – Lag-1 correlation statistics per (label, well).
correlations_by_label (dict[str, ArrayF]) – Array of lag-1 correlations for each label.

Return type:

tuple[pandas.DataFrame, dict[str, clophfit.clophfit_types.ArrayF]]

clophfit.fitting.residuals.estimate_x_shift_statistics(all_res, fit_results=None)#

Estimate potential systematic x-shifts per well (heuristics).

Analyzes residual-vs-x patterns to flag wells whose x-values (e.g. pH) may be systematically off: a linear trend of residuals against x, and asymmetry between positive and negative residuals.

Parameters:

all_res (pd.DataFrame) – Residual table with label, well, x and std_res columns.
fit_results (dict[str, Any] | None) – Per-well fit results, reserved for future x-shift fitting; currently unused.

Returns:

Per-well shift indicators: residual_slope, residual_intercept, trend_strength, asymmetry and n_points.

Return type:

pd.DataFrame

clophfit.fitting.residuals.plot_residual_vs_predicted(all_res, title='')#

Plot |standardized residual| vs predicted signal per label.

A flat trend at ~0.80 (expected |N(0,1)|) confirms the error model is correctly calibrated. A rising trend indicates under-estimated errors at high signals (multiplicative noise).

Parameters:

all_res (pd.DataFrame) – Residual DataFrame from residuals_from_fit_results. Must contain columns label, yhat, and std_res.
title (str) – Figure suptitle suffix.

Returns:

Matplotlib figure (one panel per label).

Return type:

Figure

clophfit.fitting.residuals.plot_residual_distribution(all_res, title='')#

Histogram and Q-Q of the standardised residuals, per label.

Summary statistics cannot tell a heavy tail from a bimodal spread from a shifted centre, and those call for different fixes: a heavy tail wants a robust likelihood, a shift wants the mean model looked at, a bimodal spread usually means two populations of wells. The histogram against the reference normal shows centre and width; the Q-Q plot shows the tails, which is where an error model is normally wrong.

Parameters:

all_res (pd.DataFrame) – Canonical residual table with label and std_res columns.
title (str) – Figure title.

Returns:

Two rows - histogram and Q-Q - by one column per label.

Return type:

Figure

clophfit.fitting.residuals.plot_residual_vs_yerr(all_res, title='')#

Plot raw residual² vs y_err² per label (error calibration check).

Points should scatter around the y=x line if the assigned uncertainties match the actual scatter. A slope < 1 means errors are over-estimated; slope > 1 means under-estimated.

Parameters:

all_res (pd.DataFrame) – Residual DataFrame from residuals_from_fit_results. Must contain columns label, sigma, and raw_res.
title (str) – Figure suptitle suffix.

Returns:

Matplotlib figure (one panel per label).

Return type:

Figure