Skip to content

pydvl.valuation.methods.loo

This module implements Leave-One-Out (LOO) valuation.

This is the simplest example of marginal-contribution-based valuation method. It is defined as:

\[ v_ ext{LOO}(i) = U(I) - U(I \setminus \{i\}), \]

where \(U\) is the utility function, \(I\) is the set of all data points, and \(i\) is the data point of interest.

Strictly speaking, LOO can be seen as a [semivalue][pydvl.valuation.semivalue] where the coefficients are zero except for \(k=|D|-1\),

LOOValuation 

LOOValuation(utility: UtilityBase, progress: bool = False)

Bases: SemivalueValuation

Computes LOO values for a dataset.

Source code in src/pydvl/valuation/methods/loo.py 
35
36
37
38
39
40
41
42
43
def __init__(self, utility: UtilityBase, progress: bool = False):
    self.result: ValuationResult | None = None
    super().__init__(
        utility,
        LOOSampler(),
        # LOO is done when every index has been updated once
        MinUpdates(n_updates=1),
        progress=progress,
    )

values 

values(sort: bool = False) -> ValuationResult

Returns a copy of the valuation result.

The valuation must have been run with fit() before calling this method.

PARAMETER DESCRIPTION
sort

Whether to sort the valuation result before returning it.

TYPE: bool DEFAULT: False

Returns: The result of the valuation.

Source code in src/pydvl/valuation/base.py 
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
def values(self, sort: bool = False) -> ValuationResult:
    """Returns a copy of the valuation result.

    The valuation must have been run with `fit()` before calling this method.

    Args:
        sort: Whether to sort the valuation result before returning it.
    Returns:
        The result of the valuation.
    """
    if not self.is_fitted:
        raise NotFittedException(type(self))
    assert self.result is not None

    from copy import copy

    r = copy(self.result)
    if sort:
        r.sort()
    return r

coefficient 

coefficient(n: int, k: int) -> float

This is never actually used to filter out sets, because the LOOSampler returns only complements of {idx}, but it is required by the abstract class.

Source code in src/pydvl/valuation/methods/loo.py 
45
46
47
48
49
50
def coefficient(self, n: int, k: int) -> float:
    """
    This is never actually used to filter out sets, because the LOOSampler returns
    only complements of {idx}, but it is required by the abstract class.
    """
    return 1 if k == n - 1 else 0