ntqr.statistics =============== .. py:module:: ntqr.statistics .. autoapi-nested-parse:: Responses to finite tests with finite choices can be described by integer counts of decision events. This module contains classes to construct these finite sample statistical variables. In unsupervised settings, we get to observe the counts of decision events. These are the 'response' variables. We do not know the counts of true labels in the answer key to the test (the 'Q' variables) nor the counts of decision events given true label (the 'label response' variables). NTQR uses the SymPy package to carry out its symbolic computations. The classes in this module build the variables of the form, 1. Q_label: for the q var associated with a label. 2. R_{label_i, label_j, ...} for the response variables of an ensemble. 3. R_{label_1, label_j, ..., label_true} for the label response variables. The Q variables are constructed by `AnswerKeyVariables`. The R variables are properties of `ResponseVariables`, `.responses` and `.label_responses` respectively. Classes ------- .. autoapisummary:: ntqr.statistics.AnswerKeyVariables ntqr.statistics.ResponseVariables ntqr.statistics.MClassifiersVariables ntqr.statistics.SingleClassifierVariables Module Contents --------------- .. py:class:: AnswerKeyVariables(labels: ntqr.Labels) Variables associated with the count of labels in the answer key. .. attribute:: qs The Q_{l_i} variables. There are R of them, one for each label. :type: Mapping[Label,simpy.Symbol] .. py:attribute:: _labels .. py:attribute:: _qs .. py:method:: __eq__(other) .. py:property:: labels .. py:property:: qs .. py:method:: __repr__() .. py:method:: __str__() .. py:class:: ResponseVariables(labels: ntqr.Labels, classifiers: Sequence[str]) Variables for the counts of decisions events given true label. .. attribute:: responses Mapping[Sequence[Label], sympy.Symbol]] Variables for the observed counts of decision events. :type: Mapping[Sequence[Label], responses_by_label : Mapping[Label, Mapping[Sequence[Label], Mapping[...]]] Variables for the counts of a decision event given true label. correct: Mapping[Label, sympy.Symbol] All correct variable given true label. errors: Mapping[Label, Mapping[Sequence[Label], sympy.Symbol]] .. py:attribute:: labels .. py:attribute:: classifiers .. py:attribute:: _responses .. py:attribute:: _label_responses .. py:attribute:: _correct .. py:attribute:: _errors .. py:method:: __eq__(other) .. py:property:: responses .. py:property:: label_responses .. py:property:: correct .. py:property:: errors .. py:method:: observables_dict(counts: Mapping[Sequence, int]) -> Mapping[sympy.Symbol, int] :param counts: Mapping from observable question-aligned decision events by the classifiers to their observed counts. :type counts: Mapping[Sequence, int] :returns: Mapping from observable response variable to its observed count. :rtype: dict .. py:method:: label_response_to_observable() -> Mapping[sympy.Symbol, sympy.Symbol] Constructs a mapping from a label response variable to its corresponding observable response variable. That is, R_{event, true_label} -> R_{event}. :returns: Mapping[sympy.Symbol, sympy.Symbol]. :rtype: dict .. py:method:: label_response_to_q() -> Mapping[sympy.Symbol, sympy.Symbol] Creates mapping from label response variable to Q_label. :returns: **lr_to_q** -- Mapping from a label response variable to its corresponding Q_label variable. :rtype: Mapping[sympy.Symbol, sympy.Symbol] .. py:method:: _response_variables(labels, classifiers) Constructs observable response variables given 'labels' and 'classifiers'. :param labels: Labels to use. :type labels: List :param classifiers: Labels to use for the classifiers. The label should support being stringified. :type classifiers: Sequence[str] :returns: * *Dictionary of response count variables. indexed by decisions* * *tuples.* .. py:method:: _label_response_variables(labels, classifiers) Constructs variables associated with correct and wrong response counts given true label. :param labels: Labels to use. :type labels: Sequence[Label] :param classifier: Index of classifier. :type classifier: Sequence[str] :returns: * *Dictionary of by-label response counts, indexed by true label.* * *In addition, each label contains a 'correct' key* * *that points to the variable associated with correct responses.* * *An 'errors' dictionary is indexed by possible wrong* * *label assignments.* .. py:method:: _label_correct() .. py:method:: _label_errors() .. py:method:: seq_str(decisions, classifiers) -> str :param decisions: A sequence of N labels. :type decisions: Sequence[labels] :param clsfr_strs: The N classifier labels. :type clsfr_strs: TYPE :rtype: Comma separated string of the forms l_c. .. py:method:: label_r_var_symbol(decisions, classifiers, label) .. py:method:: __repr__() .. py:class:: MClassifiersVariables(labels: ntqr.Labels, classifiers: Sequence[str]) Statistical responses variables for M classifiers. .. attribute:: qs The Q_l_i variables. There are R of them. :type: List[simpy.Symbol..] .. attribute:: responses :type: Mapping[m_subset, Mapping[Sequence[Label], simpy.Symbol]] responses_by_label : Mapping[m_subset, Mapping[Label, Mapping[...]]] .. deprecated :: 0.7.6 Use :class:`.ResponseVariables` instead. .. py:attribute:: labels .. py:attribute:: classifiers .. py:attribute:: qs .. py:attribute:: _responses .. py:attribute:: responses .. py:attribute:: _label_responses .. py:attribute:: _label_correct .. py:attribute:: _label_errors .. py:attribute:: label_responses .. py:attribute:: label_correct .. py:attribute:: label_errors .. py:method:: _response_variables(labels, classifiers) Constructs observable response variables given 'labels' and 'classifiers'. :param labels: Labels to use. :type labels: List :param classifiers: Labels to use for the classifiers. The label should support being stringified. :type classifiers: Sequence[str] :rtype: Dictionary of by-label response counts, one per label. .. py:method:: _label_response_variables(labels, classifiers) Constructs variables associated with correct and wrong response counts given true label. :param labels: Labels to use. :type labels: Sequence[Label] :param classifier: Index of classifier. :type classifier: Sequence[str] :returns: * *Dictionary of by-label response counts, indexed by true label.* * *In addition, each label contains a 'correct' key* * *that points to the variable associated with correct responses.* * *An 'errors' dictionary is indexed by possible wrong* * *label assignments.* .. py:method:: _label_correct_variables(labels, classifiers) .. py:method:: _label_error_variables(labels, classifiers) .. py:method:: seq_str(decisions, classifiers) -> str :param decisions: A sequence of N labels. :type decisions: Sequence[labels] :param clsfr_strs: The N classifier labels. :type clsfr_strs: TYPE :rtype: Comma separated string of the forms l_c. .. py:method:: label_r_var_symbol(decisions, classifiers, label) .. py:method:: r_var_symbol(decisions, classifiers) .. py:method:: pair_correlations(pair: tuple[str, str], decisions: tuple[str, str], l_true: str) -> sympy.UnevaluatedExpr :param pair: Pair of classifiers. :type pair: tuple[str,str] :param decisions: The decisions by the pair. :type decisions: tuple[str,str] :returns: * *Mapping from label to an expression for the pair correlation* * *for the decisions given true label.* .. py:method:: all_agree_subs_dict() -> Mapping[sympy.Symbol, sympy.UnevaluatedExpr] .. py:method:: __repr__() .. py:class:: SingleClassifierVariables(labels, classifier) Statistical variables associated with a single classifier. .. attribute:: question_numbers The variables for the count of correct questions for each true label. :type: List[simpy.Symbol..] .. attribute:: responses The variables for the observed interger count of labels in the test. Generally, of the form R_{l_ i} : Number of 'l' label responses by classifier 'i' :type: Mapping[label, simpy.Symbol] .. attribute:: responses_by_label The variables associated with correct and wrong responses given the true label. Variables are of the form, R_{l_i, l_true} : Number of l responses by classifier 'i' given true label, l_true. Given true label, responses_by_label[label] returns a dictionary with keys: 'correct': the variable for correct responses given true label. 'errors': dictionary, indexed by wrong label, to incorrect responses. 'l_1', 'l_2': dictionary, indexed by response label, given true_label. :type: Mapping[label, simpy.Symbol] .. attribute:: .. deprecated :: 0.7.6 Use :class:`.ResponseVariables` instead. .. py:attribute:: questions_number .. py:attribute:: responses .. py:attribute:: responses_by_label .. py:method:: response_variables(labels, classifier) Constructs observable response variables given 'labels' and 'classifier'. :param labels: Labels to use. :type labels: List :param classifier: Index of classifier. :type classifier: int :rtype: Dictionary of by-label response counts, one per label. .. py:method:: label_response_variables(labels, classifier) Constructs variables associated with correct and wrong response counts given true label. :param labels: Labels to use. :type labels: List :param classifier: Index of classifier. :type classifier: int :returns: * *Dictionary of by-label response counts, three per label.* * *In addition, each label contains a 'correct' key* * *that points to the variable associated with correct responses.* * *An 'errors' dictionary is indexed by possible wrong* * *label assignments.*