Skip to content

[Repo Assist] fix: compute proper chi-squared p-value for CMI independence test in GraphRefuter #1431

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
github-actions wants to merge 2 commits into
base: main
Choose a base branch
from
Open

[Repo Assist] fix: compute proper chi-squared p-value for CMI independence test in GraphRefuter #1431

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
055c7a6
fix: compute proper chi-squared p-value for CMI independence test
github-actions[bot] Mar 31, 2026
2e79e9f
fix: address review comments - fix conditional_MI iteration, cardinal…
Copilot Apr 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 19 additions & 4 deletions dowhy/causal_refuters/graph_refuter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
import logging
from math import log

import numpy as np
from scipy.stats import chi2

from dowhy.causal_refuter import CausalRefutation, CausalRefuter
from dowhy.utils.cit import conditional_MI, partial_corr
Expand Down Expand Up @@ -56,14 +58,27 @@ def partial_correlation(self, x=None, y=None, z=None):
self._results[key] = [p_value, True]

def conditional_mutual_information(self, x=None, y=None, z=None):
cmi_val = conditional_MI(data=self._data, x=x, y=y, z=list(z))
cmi_bits = conditional_MI(data=self._data, x=x, y=y, z=list(z))
key = (x, y) + (z,)
Comment thread
emrekiciman marked this conversation as resolved.
if cmi_val <= 0.05:

n = len(self._data)
# Convert CMI (bits) to G-test statistic (asymptotically chi-squared under H0)
g_stat = 2 * n * cmi_bits * log(2)

# Degrees of freedom: (|X| - 1)(|Y| - 1) * number of distinct Z combinations
x_card = self._data[x].nunique()
y_card = self._data[y].nunique()
z_card = self._data[list(z)].drop_duplicates().shape[0] if z else 1
df = max(1, (x_card - 1) * (y_card - 1) * z_card)
Copy link

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The degrees-of-freedom calculation uses cardinalities from the raw dataframe (nunique() / drop_duplicates()), but conditional_MI internally casts variables to int before computing CMI. For non-int columns (e.g., continuous or discretized-on-the-fly cases), this can make g_stat reflect the int-cast data while df reflects the original data, producing invalid p-values. Compute x_card, y_card, and z_card from the same transformed/discretized data that is used to compute CMI (or refactor to compute both CMI and df from a shared contingency table).

Copilot uses AI. Check for mistakes.

p_value = float(chi2.sf(g_stat, df=df))
Copy link

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using df = max(1, ...) masks degenerate cases where x_card <= 1 or y_card <= 1 (or z_card == 0), for which the chi-squared approximation isn’t meaningful. Consider explicitly handling constant variables/empty strata (e.g., treat as non-rejection with p_value=1.0, or return NotImplemented) rather than forcing df=1.

Suggested change
df = max(1, (x_card - 1) * (y_card - 1) * z_card)
p_value = float(chi2.sf(g_stat, df=df))
df = (x_card - 1) * (y_card - 1) * z_card
if x_card <= 1 or y_card <= 1 or df <= 0:
# Degenerate contingency structure: the chi-squared approximation is not meaningful.
# Treat this as a non-rejection instead of forcing df=1.
p_value = 1.0
else:
p_value = float(chi2.sf(g_stat, df=df))

Copilot uses AI. Check for mistakes.

if p_value >= 0.05:
self._true_implications.append([x, y, z])
self._results[key] = [cmi_val, True]
self._results[key] = [p_value, True]
else:
self._false_implications.append([x, y, z])
self._results[key] = [cmi_val, False]
self._results[key] = [p_value, False]
Comment on lines +64 to +87
Copy link

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change alters the statistical decision rule by introducing a chi-squared p-value computation, but existing tests only assert overall pass/fail outcomes for random graphs. Adding a focused unit test that checks (1) an (approximately) independent discrete pair yields p_value >= 0.05 and (2) a dependent pair yields p_value < 0.05 would prevent regressions in the p-value computation (including df and unit conversions).

Copilot uses AI. Check for mistakes.

def refute_model(self, independence_constraints):
"""
Expand Down
Loading