ntqr.raxioms¶

The invariant relations obeyed by all classification tests can be represented in different spaces. This module implements them in the label response spaces, non-negative integer spaces for the label response variables.

Earlier version of the NTQR package implemented these in label accuracy spaces that are rational numbers (ratios of integers).

@author: Andrés Corrada-Emmanuel.

Classes¶

`SimplexAxioms`	Class for generating the simplex axioms for a set of classifiers.
`MarginalizationAxioms`	Class for generating the marginalization axioms for a set of classifiers.
`ObservableAxioms`	Class for generating the observable axioms for a set of classifiers.
`MAxiomsIdeal`	Class for generating the axioms related to M-sized subsets

Module Contents¶

class ntqr.raxioms.SimplexAxioms(labels: ntqr.Labels, classifiers: Sequence[str])¶

Class for generating the simplex axioms for a set of classifiers.

The count of question-aligned decision events given true label must sum to the count of a true label in the answer key. Therefore, there are R of these axioms, one for each true label. This class constructs them.

labels¶

classifiers¶

axioms¶

label_simplex_equation(qVars, rVars, label)¶

__repr__()¶

class ntqr.raxioms.MarginalizationAxioms(labels: ntqr.Labels, classifiers: Sequence[str])¶

Class for generating the marginalization axioms for a set of classifiers.

Given m classifiers, their R^m events given true label must each marginalize correctly to the m events possible by marginalizing one of the classifiers out – m*R^m of them.

labels¶

classifiers¶

_contributors¶

vars¶

axioms¶

_initialize_contributors()¶

_initialize_axioms()¶

__repr__()¶

class ntqr.raxioms.ObservableAxioms(labels: ntqr.Labels, classifiers: Sequence[str])¶

Class for generating the observable axioms for a set of classifiers.

Given m classifiers, the R^m ways they can agree and disagree must have a count equal to a sum of the same events conditioned by true label. There are R^m of these axioms.

labels¶

classifiers¶

axioms = []¶

__repr__()¶

class ntqr.raxioms.MAxiomsIdeal(labels: ntqr.Labels, classifiers: Sequence[str], m: int = 1)¶

Class for generating the axioms related to M-sized subsets of the classifiers.

Each subset of the classifiers has a set of axioms, of size R, involving the marginalized decisions of that subset. For each value of the size of a subset, M, we can establish universal relations between counts of observed M-sized decision tuples and the counts of those tuples and smaller ones given true labels.

The bottom of the M axioms ladder is M=1, the individual axioms. These carve out the evaluations for a given test taker given their marginalized response counts. This is a subset of the R-simplex, for an individual test taker, for any test of size Q. Thus, each test-taker in a group of N gets their own M=1 set of axioms.

The M=2 axioms correspond to all possible pairs in a group of N test takers. These involve all the variables in the M=1 axioms but now define a new set of axioms involving the pair response variables.

In R-space, the space of integer response counts, all these ideals are linear equations. Thus, it may seem that calling them ‘ideals’, while strictly true, is overkill. However, in P-space, these ideals are polynomials of degree 2 or higher.

Deprecated since version 0.7.6: Use *Axioms instead. The expressions computed here are correct but make solving for the possible and consistent sets harder than they need to be.

labels¶

classifiers¶

m = 1¶

qvars¶

mvars¶

all_agree_subs_dict¶

axiomatic_ideals¶

m_one_ideal_agreement(classifier: tuple[str]) → Mapping[str, sympy.UnevaluatedExpr]¶

The M=1 axioms ideal with agreement label response variables.

The axioms in label response space are easiest to understand and prove when we use ‘agreement’ variables. Those are variables where all the classifiers are agreeing in their responses on the true label.

Parameters:: classifier (tuple[str]) – Classifier to use for variable subscripts.
Returns:: m1_axioms_ideal – A mapping from label to its corresponding m=1 axiom for the given classifier.
Return type:: Mapping[str, sympy.UnevaluatedExpr]

_m_one_ideal(labels: tuple[str], m_subset: tuple[str]) → Mapping[str, sympy.UnevaluatedExpr]¶

Compute M=1 ideal expressed solely in terms of disagreement variables.

This may need to be deprecated. The original rationale for this function is that all variables belong to some simplex and therefore we can always get rid of the variable corresponding to all of the test takers agreeing on the correct answer so as to save some computational load.

The problem is that this form is more complex than the expression returned by self.m_one_ideal_agreement. This also made it more bug prone. Future development of NTQR will be based on the “natural” representation of the axioms indexed by the all-correct response variable. These are easier to write down from the top of one’s head and easier to write mechanical proofs for them using more expressive algebraic systems like the Wolfram language.

This function will be deprecated as it does the same computation as self.m_one_ideal_agreement.

Parameters:

labels (Sequence[str]) – Labels.
m_subset (tuple[str]) – M=m sequence of classifiers.

Returns:

axioms_by_label – M=m axioms indexed by label.

Return type:

Mapping[str->sympy.UnevaluatedExpr]

_m_two_ideal(pair: tuple[Any, Any]) → dict[str, sympy.UnevaluatedExpr]¶

Construct the m=2 ideal.

This is the ‘errors’ variables version. The one used for internal computation of the varieties. It has the drawback that it is hard to check or to write code for it.

Deprecated since version 0.7.6: Use SimplexAxioms|.ObservableAxioms instead. The ideal is correct but is not suited to the easiest way to compute their varieties. Using this representation of the axioms entangles the solution of response variables with their marginalized versions.

Parameters:: pair (Sequence[Any, Any]) – Pair of classifiers.
Returns:: m2_axioms_ideal – A mapping from label to its corresponding r-axiom.
Return type:: Mapping[label, sympy.UnevaluatedExpr]

m_two_ideal_agreement(pair: tuple[Any, Any]) → Mapping[str, sympy.UnevaluatedExpr]¶

Construct the M=2 algebraic ideal.

Starting with M=2 this will be the only way the axioms will be encoded from now on. The representation with ‘errors’ variables only, the one needed for generalizable code, will be created with the straightforward transformation that gives the all agree on the true label as the number of questions of that label minus all the other possible responses.

Parameters:: pair (tuple[Any, Any]) – The classifier pair.
Returns:: m2_axioms_ideal – Mapping from labels to its corresponding M=2 axiom.
Return type:: Mapping[str, sympy.UnevaluatedExpr]

m_three_ideal_agreement_representation(trio: tuple[str | int, str | int, str | int]) → Mapping[str, sympy.UnevaluatedExpr]¶

Construct the M=3 algebraic ideal with agreement variables.

Parameters:: trio (tuple[str | int, str | int]) – Any three classifiers.
Returns:: m2_axioms_ideal – Dictionary from label to its corresponding axiom.
Return type:: Mapping[str, sympy.UnevaluatedExpr]

initialize_all_agree_subs_dict() → None¶

Initialize the substitution dictionary for all correct responses.

Computations of the label response simplex variables is easiest when we represent all agree on the correct label in terms of the disagreeing label responses variables. This function initializes the substitution dictionary that can be used to turn any expression in terms of label response variables into one that only uses disagreement ones.

The initial justification for this operation was to eliminate the simplex axioms from the algebra by rewriting all correct variables in a simplex in terms of the other variables, where at least one classifier has made a classification error.

This is currently not being used since there was no significant speed ups observed in calculating the consistent evaluation set. But it does make it harder to code and debug when you eliminate the all-correct label response variables. Being correct and understandable is necessary before v1.0 can be released so the current practice is to not use this substitution dict moving forward.

Returns:: The self.all_correct_subs_dict is initialized.
Return type:: None