Know what your
model leaks.
One function call audits any PyTorch model for membership inference, model inversion, and attribute inference attacks — with an interactive HTML report.
# Split training set → members / non-members
m, nm = auditml.split_loaders(train_dataset)
# Audit any nn.Module
results = auditml.audit(model, m, nm)
print(results.summary())
# mia_threshold AUC 0.641 ⚠ leakage detected
# model_inversion AUC 0.571
# Open interactive HTML report in browser
results.report("./report", open_browser=True)
Four attacks. One unified API.
Each attack exposes a different kind of leakage. Run them all or pick only what you need.
Threshold MIA
Classifies members vs non-members by thresholding the model's loss or confidence. Fast and surprisingly effective.
Shadow Model MIA
Trains shadow models with known membership labels, then builds an MLP attack classifier on their outputs.
Model Inversion
Reconstructs representative training images via gradient ascent — exposing what the model has memorised.
Attribute Inference
Predicts sensitive group membership from softmax outputs — revealing whether the model leaks protected attributes.
Up and running
in 60 seconds.
No account. No API key. Install and call
auditml.audit()
on your model.
pip install auditml
Python 3.10+ · PyTorch 2.x
Full installation guide →
m, nm = auditml.split_loaders(train_dataset)
results = auditml.audit(model, m, nm)
# Per-attack metrics
results["mia_threshold"].auc_roc # → 0.641
results.is_vulnerable() # → True
# HTML report + save / reload
results.report("./report", open_browser=True)
results.save("results.json")