Compare model checkpoints to see exactly what changed during training. Debug issues, analyze fine-tuning, and validate training runs.
Everything you need to understand model changes
Load PyTorch, SafeTensors, NumPy, and HuggingFace checkpoints seamlessly.
L2 diff, cosine similarity, relative change, and element-wise statistics.
Automatically identify training patterns like gradient flow and layer freezing.
Categorize and compare attention, MLP, embedding, and other layer types.
Get automated insights about frozen layers, stability, and architecture changes.
Generate HTML, JSON, or text reports with visualizations and summaries.
For each parameter, we compute comprehensive statistics
Automatically detect how your model changed during training
Earlier layers changed more than later layers - common in transfer learning
Later layers changed more - typical for fine-tuning on new tasks
Attention mechanisms changed most - learning new relationships
Feed-forward layers changed most - learning new features
Very few layers changed - possible frozen layers or early stopping
All layers changed equally - full model training
Compare checkpoints in just a few lines of code
from model_diff import compare_checkpoints, DiffAnalyzer, create_diff_report # Compare two checkpoints diffs, summary = compare_checkpoints( "model_v1.pt", "model_v2.pt" ) # Print summary print(f"Modified: {summary.modified_parameters}") print(f"Cosine Similarity: {summary.avg_cosine_similarity:.4f}") # Analyze the differences analyzer = DiffAnalyzer() analysis = analyzer.analyze(diffs, summary) print(f"Pattern: {analysis['change_pattern']}") # Generate HTML report report = create_diff_report( diffs, summary, analysis, format="html", old_name="Base", new_name="Fine-tuned" ) with open("report.html", "w") as f: f.write(report)
From checkpoints to insights in four simple steps
Load checkpoints from any format
Compute differences between parameters
Detect patterns and generate insights
Create beautiful visualizations