Merge branch 'main' into dependabot/github_actions/actions/upload-artifact-5

Author: mmschlk

CHANGELOG.md

@@ -2,13 +2,21 @@
 
 ## Development
 
+### Introducing ProxySPEX
+Adds the ProxySPEX approximator for efficient computation of sparse interaction values using the new ProxySPEX algorithm.
+For further details refer to: Butler, L., Kang, J.S., Agarwal, A., Erginbas, Y.E., Yu, Bin, Ramchandran, K. (2025). ProxySPEX: Inference-Efficient Interpretability via Sparse Feature Interactions in LLMs https://arxiv.org/pdf/2505.17495
+
+
+### Introducing ProductKernelExplainer
+The ProductKernelExplainer is a new model-specific explanation method for Product Kernel based machine learning model, such as  Gaussian Processes or Support Vector Machines.
+For further details refer to:  https://arxiv.org/abs/2505.16516
+
 ### Shapiq Statically Typechecked [#430](https://github.com/mmschlk/shapiq/pull/430)
 We have introduced static type checking to `shapiq` using [Pyright](https://github.com/microsoft/pyright), and integrated it into our `pre-commit` hooks.
 This ensures that type inconsistencies are caught early during development, improving code quality and maintainability.
 Developers will now benefit from immediate feedback on type errors, making the codebase more robust and reliable as it evolves.
 
 ### Separation of `shapiq` into `shapiq`, `shapiq_games`, and `shapiq-benchmark`
-
 We have begun the process of modularizing the `shapiq` package by splitting it into three distinct packages: `shapiq`, `shapiq_games`, and `shapiq-benchmark`.
 
 - The `shapiq` package now serves as the core library. It contains the main functionality, including approximators, explainers, computation routines, interaction value logic, and plotting utilities.
@@ -28,8 +36,10 @@ This restructuring aims to improve maintainability and development scalability.
 ### Bugfixes
 - fixes a bug where RegressionFBII approximator was throwing an error when the index was `'BV'` or `'FBII'`.[#420](https://github.com/mmschlk/shapiq/pull/420)
 
-### New Features
+### All New Features
 - adds the ProxySPEX (Proxy Sparse Explanation) module in `approximator.sparse` for even more efficient computation of sparse interaction values [#442](https://github.com/mmschlk/shapiq/pull/442)
+- uses `predict_logits` method of sklearn-like classifiers if available in favor of `predict_proba` to support models that also offer logit outputs like TabPFNClassifier for better interpretability of the explanations [#426](https://github.com/mmschlk/shapiq/issues/426)
+- adds the `shapiq.explainer.ProductKernelExplainer` for model-specific explanation of Product Kernel based models like Gaussian Processes and Support Vector Machines. [#431](https://github.com/mmschlk/shapiq/pull/431)
 
 ### Removed Features
 - removes the ability to load `InteractionValues` from pickle files. This is now deprecated and will be removed in the next release. Use `InteractionValues.save(..., as_json=True)` to save interaction values as JSON files instead. [#413](https://github.com/mmschlk/shapiq/issues/413)

pyproject.toml

@@ -221,11 +221,11 @@ pythonPlatform = "Linux"
 
 [dependency-groups]
 all_ml = [
-    "tabpfn>=2.0.7",
+    "tabpfn>=2.1.3",
     "torchvision",
     "torch",
     "xgboost",
-    "lightgbm; platform_system != 'Darwin'",    # lightgbm has install problems on macOS
+    "lightgbm; platform_system != 'Darwin'",    # lightgbm has install problems on macOS in github actions
     "transformers",
     "scikit-image",
     "tensorflow; python_version < '3.13' and platform_system != 'Windows'", # only up to py 3.12

src/shapiq/approximator/regression/base.py

@@ -111,7 +111,10 @@ def _init_kernel_weights(self, interaction_size: int) -> FloatVector:
                 else:
                     weight_vector[coalition_size] = 1 / (
                         (self.n - 2 * interaction_size + 1)
-                        * binom(self.n - 2 * interaction_size, coalition_size - interaction_size)
+                        * binom(
+                            self.n - 2 * interaction_size,
+                            coalition_size - interaction_size,
+                        )
                     )
             return weight_vector
         msg = f"Index {self.index} not available for Regression Approximator."

src/shapiq/explainer/custom_types.py

@@ -5,3 +5,4 @@
 from typing import Literal
 
 ExplainerIndices = Literal["SV", "SII", "k-SII", "STII", "FSII", "BV", "BII", "FBII"]
+ValidProductKernelExplainerIndices = Literal["SV"]

src/shapiq/explainer/product_kernel/__init__.py

@@ -0,0 +1,7 @@
+"""Implementation of the ProductKernelComputer and the ProductKernelExplainer."""
+
+from .base import ProductKernelModel
+from .explainer import ProductKernelExplainer
+from .product_kernel import ProductKernelComputer
+
+__all__ = ["ProductKernelModel", "ProductKernelExplainer", "ProductKernelComputer"]

src/shapiq/explainer/product_kernel/base.py

@@ -0,0 +1,31 @@
+"""The base class for product kernel model conversion."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    import numpy as np
+
+
+@dataclass
+class ProductKernelModel:
+    """A dataclass for storing the information of a product kernel model.
+
+    Attributes:
+         alpha: The alpha parameter of the product kernel model.
+         X_train: The training data used to fit the product kernel model.
+         n: The number of samples in the training data.
+         d: The number of features in the training data.
+         gamma: The gamma parameter of the product kernel model.
+        intercept: The intercept term of the product kernel model. For Gaussian Processes this should be zero, but support vectors have often non-zero intercepts.
+    """
+
+    X_train: np.ndarray
+    alpha: np.ndarray
+    n: int
+    d: int
+    gamma: float | None = None
+    kernel_type: str = "rbf"
+    intercept: float = 0.0

src/shapiq/explainer/product_kernel/conversion.py

@@ -0,0 +1,88 @@
+"""Functions for converting scikit-learn models to a format used by shapiq."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+import numpy as np
+
+from shapiq.explainer.product_kernel.base import ProductKernelModel
+
+if TYPE_CHECKING:
+    from sklearn.gaussian_process import GaussianProcessRegressor
+    from sklearn.svm import SVC, SVR
+
+
+def convert_svm(model: SVC | SVR) -> ProductKernelModel:
+    """Converts a scikit-learn SVM model to the product kernel format used by shapiq.
+
+    Args:
+        model: The scikit-learn SVM model to convert. Can be either a binary support vector classifier (SVC) or a support vector regressor (SVR).
+
+    Returns:
+        ProductKernelModel: The converted model in the product kernel format.
+
+    """
+    X_train = model.support_vectors_
+    n, d = X_train.shape
+
+    if hasattr(model, "kernel"):
+        kernel_type = model.kernel  # pyright: ignore[reportAttributeAccessIssue]
+        if kernel_type != "rbf":
+            msg = "Currently only RBF kernel is supported for SVM models."
+            raise ValueError(msg)
+    else:
+        msg = "Kernel type not found in the model. Ensure the model is a valid SVC or SVR."
+        raise ValueError(msg)
+
+    return ProductKernelModel(
+        alpha=model.dual_coef_.flatten(),  # pyright: ignore[reportAttributeAccessIssue]
+        X_train=X_train,
+        n=n,
+        d=d,
+        gamma=model._gamma,  # pyright: ignore[reportArgumentType, reportAttributeAccessIssue] # noqa: SLF001
+        kernel_type=kernel_type,
+        intercept=model.intercept_[0],
+    )
+
+
+def convert_gp_reg(model: GaussianProcessRegressor) -> ProductKernelModel:
+    """Converts a scikit-learn Gaussian Process Regression model to the product kernel format used by shapiq.
+
+    Args:
+        model: The scikit-learn Gaussian Process Regression model to convert.
+
+    Returns:
+        ProductKernelModel: The converted model in the product kernel format.
+
+    """
+    X_train = np.array(model.X_train_)
+    n, d = X_train.shape
+
+    if hasattr(model, "kernel"):
+        kernel_type = model.kernel_.__class__.__name__.lower()  # Get the kernel type as a string
+        if kernel_type != "rbf":
+            msg = "Currently only RBF kernel is supported for Gaussian Process Regression models."
+            raise ValueError(msg)
+    else:
+        msg = "Kernel type not found in the model. Ensure the model is a valid Gaussian Process Regressor."
+        raise ValueError(msg)
+
+    alphas = np.array(model.alpha_).flatten()
+    parameters = (
+        model.kernel_.get_params()  # pyright: ignore[reportAttributeAccessIssue]
+    )
+    if "length_scale" in parameters:
+        length_scale = parameters["length_scale"]
+    else:
+        msg = "Length scale parameter not found in the kernel."
+        raise ValueError(msg)
+
+    return ProductKernelModel(
+        alpha=alphas,
+        X_train=X_train,
+        n=n,
+        d=d,
+        gamma=(2 * (length_scale**2)) ** -1,
+        kernel_type=kernel_type,
+    )

src/shapiq/explainer/product_kernel/explainer.py

@@ -0,0 +1,139 @@
+"""Implementation of the ProductKernelExplainer class."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Any
+
+from shapiq import InteractionValues
+from shapiq.explainer.base import Explainer
+from shapiq.game_theory import get_computation_index
+
+from .product_kernel import ProductKernelComputer, ProductKernelSHAPIQIndices
+from .validation import validate_pk_model
+
+if TYPE_CHECKING:
+    import numpy as np
+    from sklearn.gaussian_process import GaussianProcessRegressor
+    from sklearn.svm import SVC, SVR
+
+    from shapiq.typing import Model
+
+    from .base import ProductKernelModel
+
+
+class ProductKernelExplainer(Explainer):
+    """The ProductKernelExplainer class for product kernel-based models.
+
+    The ProductKernelExplainer can be used with a variety of product kernel-based models. The explainer can handle both regression and
+    classification models. See [pkex-shapley]_ for details.
+
+
+    References:
+        .. [pkex-shapley] Majid Mohammadi and Siu Lun Chau, Krikamol Muandet. (2025). Computing Exact Shapley Values in Polynomial Time for Product-Kernel Methods. https://arxiv.org/abs/2505.16516
+
+    Attributes:
+        model: The product kernel model to explain. Can be a dictionary, a ProductKernelModel, or a list of ProductKernelModels.
+             Note that the model will be converted to a ProductKernelModel if it is not already in that format.
+             Supported models include scikit-learn's SVR, SVC (binary classification only), and GaussianProcessRegressor.
+             Beware that for classification models, the class to explain is set to the predicted class of the model.
+             For further details, see the `validate_pk_model` function in `shapiq.explainer.product_kernel.validation`.
+        max_order: The maximum interaction order to be computed. Defaults to ``1``.
+        min_order: The minimum interaction order to be computed. Defaults to ``0``.
+        index: The type of interaction to be computed. Currently, only ``"SV"`` is supported.
+    """
+
+    def __init__(
+        self,
+        model: (
+            ProductKernelModel | Model | SVR | SVC | GaussianProcessRegressor  # pyright: ignore[reportInvalidTypeVarUse]
+        ),
+        *,
+        min_order: int = 0,
+        max_order: int = 1,
+        index: ProductKernelSHAPIQIndices = "SV",
+        **kwargs: Any,  # noqa: ARG002
+    ) -> None:
+        """Initializes the ProductKernelExplainer.
+
+        Args:
+            model: A product kernel-based model to explain.
+
+            min_order: The minimum interaction order to be computed. Defaults to ``0``.
+
+            max_order: The maximum interaction order to be computed. An interaction order of ``1``
+                corresponds to the Shapley value. Defaults to ``1``.
+
+            index: The type of interaction to be computed. Currently, only ``"SV"`` is supported.
+
+            class_index: The class index of the model to explain. Defaults to ``None``, which will
+                set the class index to ``1`` per default for classification models and is ignored
+                for regression models.
+
+            **kwargs: Additional keyword arguments are ignored.
+
+        """
+        if max_order > 1:
+            msg = "ProductKernelExplainer currently only supports max_order=1."
+            raise ValueError(msg)
+
+        super().__init__(model, index=index, max_order=max_order)
+
+        self._min_order = min_order
+        self._max_order = max_order
+
+        self._index = index
+        self._base_index: str = get_computation_index(self._index)
+
+        # validate model
+        self.converted_model = validate_pk_model(model)
+
+        self.explainer = ProductKernelComputer(
+            model=self.converted_model,
+            max_order=max_order,
+            index=index,
+        )
+
+        self.empty_prediction = self._compute_baseline_value()
+
+    def explain_function(
+        self,
+        x: np.ndarray,
+        **kwargs: Any,  # noqa: ARG002
+    ) -> InteractionValues:
+        """Compute Shapley values for all features of an instance.
+
+        Args:
+           x: The instance (1D array) for which to compute Shapley values.
+           **kwargs: Additional keyword arguments are ignored.
+
+        Returns:
+           The interaction values for the instance.
+        """
+        n_players = self.converted_model.d
+
+        # compute the kernel vectors for the instance x
+        kernel_vectors = self.explainer.compute_kernel_vectors(self.converted_model.X_train, x)
+
+        shapley_values = {}
+        for j in range(self.converted_model.d):
+            shapley_values.update({(j,): self.explainer.compute_shapley_value(kernel_vectors, j)})
+
+        return InteractionValues(
+            values=shapley_values,
+            index=self._base_index,
+            min_order=self._min_order,
+            max_order=self.max_order,
+            n_players=n_players,
+            estimated=False,
+            baseline_value=self.empty_prediction,
+            target_index=self._index,
+        )
+
+    def _compute_baseline_value(self) -> float:
+        """Computes the baseline value for the explainer.
+
+        Returns:
+            The baseline value for the explainer.
+
+        """
+        return self.converted_model.alpha.sum() + self.converted_model.intercept