clophfit.fitting.gain_calibration ================================= .. py:module:: clophfit.fitting.gain_calibration .. autoapi-nested-parse:: Calibrate a gain-only noise model from a fit's own residuals, then refit. Feasible generalised least squares with ``sigma^2 = floor^2 + gain * yhat`` and alpha held at 0: fit every well under the current weights, pool the residuals of all wells, estimate the gain (and optionally the floor), rebuild the weights, refit, and repeat until the noise terms stop moving. The pool is one plate or several; with several, one gain per label is shared by every plate and each plate keeps its own floor. Two fitters plug into the same loop: the per-well least-squares fit (:func:`~clophfit.fitting.core.fit_binding_glob`, ``lm`` or ``huber``) and the plate-wide :func:`~clophfit.fitting.plate_lm.fit_plate_lm`. Classes ------- .. autoapisummary:: clophfit.fitting.gain_calibration.GainCalibration Functions --------- .. autoapisummary:: clophfit.fitting.gain_calibration.single_well_residuals clophfit.fitting.gain_calibration.plate_lm_residuals clophfit.fitting.gain_calibration.calibrate_gain clophfit.fitting.gain_calibration.calibrate_single_well clophfit.fitting.gain_calibration.calibrate_plate_lm Module Contents --------------- .. py:class:: GainCalibration[T] What the calibration loop converged to, and the fits made under it. .. attribute:: fits Plate to its fit under ``noise``, the weights the last pass used. :type: dict[str, T] .. attribute:: noise Plate to label to the floor and gain in force for ``fits``; alpha is 0. :type: NoiseByPlate .. attribute:: history One row per pass, plate and label: ``iteration``, ``plate``, ``label``, ``gain`` and ``sigma_floor`` in force for that pass, and ``n``, the residuals that pass contributed. :type: pd.DataFrame .. attribute:: converged Whether the last adopted step moved every term by less than the tolerance. Damping halves a term's step whenever it reverses, so a loop that oscillates settles between its two answers and still counts as converged; ``proposal_moved`` says whether that is a fixed point. :type: bool .. attribute:: proposal_moved Largest relative change the last *undamped* estimate asked for. Below the tolerance, the noise is a fixed point of fit -> estimate -> refit; above it, the loop settled by damping alone. :type: float .. attribute:: n_iter Fitting passes made. :type: int .. py:function:: single_well_residuals(results) Raw residuals of per-well fits, with each well's degrees-of-freedom factor. :param results: Well to its fit; wells without a result are skipped. :type results: Mapping[str, FitResult] :returns: ``well``, ``label``, ``yhat``, ``raw_res`` and ``dof_scale`` (one value per well, from all its labels' points and varied parameters). :rtype: pd.DataFrame .. py:function:: plate_lm_residuals(result) Raw residuals of a plate-wide fit, with the plate's degrees-of-freedom factor. :param result: A plate fit. :type result: PlateLMResult :returns: ``well``, ``label``, ``yhat``, ``raw_res`` and ``dof_scale``. :rtype: pd.DataFrame .. py:function:: calibrate_gain[T](fit, floors, *, fit_floor = False, gain_amp = None, max_iter = 20, tol = 0.001) Iterate fit -> pooled residuals -> floor and gain -> refit, to a fixed point. The first pass weights each plate by its floor alone. Each later pass weights by the floor and gain the previous pass's residuals gave, pooled over every well of every plate in *floors*. A term whose update reverses direction takes half the step from then on, which settles a loop that would otherwise cycle. The loop stops when no floor or gain moves by more than *tol* (relative), and returns the fits made under the weights that no longer move. :param fit: Fits one plate under one noise model, returning the fit and its raw residual table (``label``, ``yhat``, ``raw_res``, ``dof_scale``), as :func:`single_well_residuals` and :func:`plate_lm_residuals` build. :type fit: Callable[[str, dict[str, NoiseModelParams]], tuple[T, pd.DataFrame]] :param floors: Plate to label to its read-noise floor, held unless *fit_floor*, and the only weights of the first pass either way. :type floors: Mapping[str, Mapping[str, float]] :param fit_floor: Estimate each plate's floor along with the shared gain. :type fit_floor: bool :param gain_amp: Plate to label to the amplification its gain scales with; the shared gain is quoted at 1. ``None`` shares the gain unscaled. :type gain_amp: Mapping[str, Mapping[str, float]] | None :param max_iter: Most fitting passes. :type max_iter: int :param tol: Relative change below which the noise terms have converged. :type tol: float :returns: The last pass's fits and the noise they were made under. :rtype: GainCalibration[T] :raises ValueError: If a pass leaves no residuals to calibrate from. .. py:function:: calibrate_single_well(plates, floors, *, method = 'lm', fit_floor = False, gain_amp = None, max_iter = 20, tol = 0.001, **fit_kwargs) Fit every well on its own, calibrating floor and gain across all of them. :param plates: Plate to well to dataset. One plate calibrates that plate; several share one gain per label. :type plates: Mapping[str, Mapping[str, Dataset]] :param floors: Plate to label to read-noise floor. :type floors: Mapping[str, Mapping[str, float]] :param method: The per-well loss, passed to :func:`~clophfit.fitting.core.fit_binding_glob`. :type method: Literal["lm", "huber"] :param fit_floor: Estimate each plate's floor as well as the gain. :type fit_floor: bool :param gain_amp: See :func:`calibrate_gain`. :type gain_amp: Mapping[str, Mapping[str, float]] | None :param max_iter: Most fitting passes. :type max_iter: int :param tol: Relative convergence tolerance on floor and gain. :type tol: float :param \*\*fit_kwargs: Forwarded to ``fit_binding_glob`` (e.g. ``remove_outliers``). :type \*\*fit_kwargs: Any :returns: Per plate, well to its fit under the calibrated weights. :rtype: GainCalibration[dict[str, FitResult]] .. py:function:: calibrate_plate_lm(plates, groups, floors, *, loss = 'linear', fit_floor = False, gain_amp = None, max_iter = 20, tol = 0.001) Fit each plate jointly, calibrating floor and gain across all of them. Each pass is an ordinary :func:`~clophfit.fitting.plate_lm.fit_plate_lm` with the noise model fixed; the calibration happens between passes, so the fitter's own ``calibrate_noise`` (gain and alpha, uncorrected for degrees of freedom) is not involved. The profiled ``ye_mag`` still rescales each label, and with a well-calibrated gain it should come out near 1. :param plates: Plate to well to dataset. :type plates: Mapping[str, Mapping[str, Dataset]] :param groups: Plate to its control groups (group name to member wells), which pool one K; an empty mapping fits every well its own K. :type groups: Mapping[str, Mapping[str, Sequence[str]]] :param floors: Plate to label to read-noise floor. :type floors: Mapping[str, Mapping[str, float]] :param loss: The plate fit's loss. :type loss: Literal["linear", "huber", "soft_l1", "cauchy"] :param fit_floor: Estimate each plate's floor as well as the gain. :type fit_floor: bool :param gain_amp: See :func:`calibrate_gain`. :type gain_amp: Mapping[str, Mapping[str, float]] | None :param max_iter: Most fitting passes. :type max_iter: int :param tol: Relative convergence tolerance on floor and gain. :type tol: float :returns: Per plate, the plate fit under the calibrated weights. :rtype: GainCalibration[PlateLMResult]