Researchers have proposed metrics for verifying mechanistic interpretability

Researchers presented a new approach to mechanistic interpretability, defining it as a task that can be strictly verified. The main idea is to replace individual components of a neural network with simplified analogs and then measure the loss during the reconstruction of output data.

To evaluate the quality of interpretation, the authors suggest using quantitative metrics based on the number of nodes and edges in the model graph, as well as feature reuse indicators. During the study, it was found that classical methods, such as tensor networks and low-rank approximations, do not provide sufficient accuracy for full verification of the model's internal processes.

The method requires that any interpretation of the neural network structure allow restoring the original behavior of the model with minimal deviations. If replacing part of the weights with a simplified scheme leads to a sharp increase in reconstruction loss, the interpretation is considered incomplete or incorrect. This allows formalizing the process of analyzing the "black box" through reproducible experiments.

This approach is aimed at solving the problem of subjectivity in AI interpretation. Instead of a qualitative description of the work of neurons, it is proposed to use mathematically verifiable criteria of simplicity and functional equivalence. This allows researchers to objectively compare different hypotheses about how exactly the model processes information at the level of individual weights and activations.