Skip to content
Draft
Show file tree
Hide file tree
Changes from 143 commits
Commits
Show all changes
196 commits
Select commit Hold shift + click to select a range
8ac58f5
Temp
FBumann Dec 13, 2025
e579a11
Add n_segments
FBumann Dec 13, 2025
16cffe1
Update CHANGELOG.md
FBumann Dec 13, 2025
139dc89
Use deep copy
FBumann Dec 13, 2025
60dd670
Add notebook for clustering
FBumann Dec 13, 2025
8c03f64
Update notebook
FBumann Dec 13, 2025
9fdc53d
Fix multi period and multi scenario clsutering
FBumann Dec 14, 2025
a9a442d
Improve
FBumann Dec 14, 2025
b29460b
Exclude solution when clustering
FBumann Dec 14, 2025
44aa5db
Use pre-buildt flow_system
FBumann Dec 14, 2025
fe87396
Improve notebook
FBumann Dec 14, 2025
584e907
Added new system to notebook defaults
FBumann Dec 14, 2025
f47ef38
Use realistic flow system in notebooks
FBumann Dec 14, 2025
b0c9166
Merge branch 'feature/flow-system-first' into feature/better-tsam-int…
FBumann Dec 14, 2025
1bfdc56
add segmentation to notebooks
FBumann Dec 14, 2025
6aebd18
fix cluster_multi_dimensional_data
FBumann Dec 14, 2025
c0c7c45
fix notebooks to only create flow_system if needed
FBumann Dec 14, 2025
db3e37e
Fix inter-cluster segmentation
FBumann Dec 14, 2025
bcf7136
Improve notebook to use more segments
FBumann Dec 14, 2025
6b5a638
Improve notebook to use more segments
FBumann Dec 14, 2025
e20775f
Fix Data_DIR in notebooks
FBumann Dec 15, 2025
6fdd684
Group constraints and varaibles form clustering together
FBumann Dec 15, 2025
76af019
Only equalize SOME variables
FBumann Dec 15, 2025
b7c5d60
Always fix binaries for better pre-solve
FBumann Dec 15, 2025
737f354
Group constraints and varaibles form clustering together
FBumann Dec 15, 2025
79db532
Improve readybility of clustering equations
FBumann Dec 15, 2025
2f38e7c
Add IO for clustering
FBumann Dec 15, 2025
96e0826
Add IO for clustering
FBumann Dec 15, 2025
e8ab4b8
Add IO for clustering
FBumann Dec 15, 2025
c74c5e7
Improve clustering
FBumann Dec 16, 2025
933c878
Improve clustering organization
FBumann Dec 16, 2025
f96d9f2
Improve clustering organization
FBumann Dec 16, 2025
2eb0ee4
Improve clustering organization
FBumann Dec 16, 2025
5f114f5
Improve clustering organization
FBumann Dec 16, 2025
529ef50
Improve clustering organization
FBumann Dec 16, 2025
f253832
Add external tsam support with data aggregation
FBumann Dec 18, 2025
ae37039
Add external tsam support with data aggregation
FBumann Dec 18, 2025
088bf4d
fix: Multi period clustering: Each period now gets its own clustering…
FBumann Dec 18, 2025
189b7d0
Temp
FBumann Dec 18, 2025
f92311a
Add expand_solution()
FBumann Dec 18, 2025
68be8f5
fix
FBumann Dec 18, 2025
4102148
Add notebook
FBumann Dec 18, 2025
0eac20a
fix
FBumann Dec 18, 2025
85a73cb
Improve wording
FBumann Dec 18, 2025
e727652
Temp
FBumann Dec 18, 2025
1827578
Temp
FBumann Dec 18, 2025
f33d31b
Weight Concepts (Final Naming)
FBumann Dec 18, 2025
7050db3
Temp
FBumann Dec 18, 2025
4673bc5
Temp
FBumann Dec 18, 2025
b2e71d7
Fix script for docs
FBumann Dec 18, 2025
05dd8da
Update notebook
FBumann Dec 18, 2025
4341cec
Fix clsuter weight applying
FBumann Dec 18, 2025
87fd661
Fix storage initial=final issue in clustering
FBumann Dec 18, 2025
2dc39e4
Improve notebooks
FBumann Dec 18, 2025
3c0e986
clsuter reduce does remoce regular cyclic storage behaviour
FBumann Dec 18, 2025
f2272d0
Fix cluster weightings when expanding and in statistics_accessor.py
FBumann Dec 18, 2025
c7155e8
Improve cluster weighting
FBumann Dec 18, 2025
7126207
1. Each period gets its own cluster weights based on its clustering
FBumann Dec 18, 2025
acff2b1
Improve code
FBumann Dec 18, 2025
85c2d7e
Add new aggregation module
FBumann Dec 18, 2025
78639df
Improve terminology
FBumann Dec 18, 2025
97befe1
Remove legacy clustering
FBumann Dec 18, 2025
b987bfd
Update CHANGELOG.md
FBumann Dec 18, 2025
bbebb96
Update notebooks
FBumann Dec 18, 2025
e8afe18
Update notebooks
FBumann Dec 18, 2025
00e7508
Update notebooks and fix storage inital=equal
FBumann Dec 18, 2025
0badd14
Update notebooks and fix storage inital=equal
FBumann Dec 18, 2025
aa20391
Remove old clustering code
FBumann Dec 18, 2025
cf54be5
Remove old clustering code
FBumann Dec 18, 2025
71d010a
Remove old clustering code
FBumann Dec 18, 2025
bfc81d2
Update notebooks
FBumann Dec 18, 2025
a56d1f8
Update notebooks
FBumann Dec 18, 2025
1faaaf0
⎿  ☒ Rename cluster_reduce() to cluster() in transform_accessor.py
FBumann Dec 18, 2025
3b88722
⏺ Summary
FBumann Dec 18, 2025
f653269
terminology unification is complete. All references have been renamed:
FBumann Dec 18, 2025
6cb06b1
Made cluster_order and timestep_mapping multi-dimensional to store p…
FBumann Dec 18, 2025
3e55faa
fix
FBumann Dec 18, 2025
d0f37cf
Update notebooks
FBumann Dec 18, 2025
d4bbcfe
simplified:
FBumann Dec 18, 2025
83a4603
Improve notebook
FBumann Dec 18, 2025
04a8030
Improve plotting
FBumann Dec 19, 2025
486929a
Improve plotting
FBumann Dec 19, 2025
4d6dcc0
Improve repr
FBumann Dec 19, 2025
33dd89b
Simplify notebook
FBumann Dec 19, 2025
8177c2e
Update API
FBumann Dec 19, 2025
126cbf4
Update API
FBumann Dec 19, 2025
88248db
Update API
FBumann Dec 19, 2025
e0f28a8
Update API
FBumann Dec 19, 2025
56b1ae2
Update API
FBumann Dec 19, 2025
5319077
Update notebooks
FBumann Dec 19, 2025
259e02b
Update plotting
FBumann Dec 19, 2025
cf512bc
Parameter Naming (Aligned with StatisticsPlotAccessor)
FBumann Dec 19, 2025
94fba48
Update low api method
FBumann Dec 19, 2025
d7e7faf
Update heatmap t accept animation frame
FBumann Dec 19, 2025
11a6304
Simplify compare method in clustering
FBumann Dec 19, 2025
a810878
Simplify compare method in clustering
FBumann Dec 19, 2025
6dc1979
Simplify compare method in clustering
FBumann Dec 19, 2025
f1ceac5
Simplify compare method in clustering
FBumann Dec 19, 2025
be103b8
improve API
FBumann Dec 19, 2025
ccec025
Add intra_cluster_mask to clustering for storage linking
FBumann Dec 19, 2025
f56f23e
Add intra_cluster_mask to clustering for storage linking
FBumann Dec 19, 2025
e49ed1b
Temp
FBumann Dec 19, 2025
2b084b3
Revert "Temp"
FBumann Dec 19, 2025
8fae0f0
fix intra_cluster_mask
FBumann Dec 19, 2025
d202b18
Change to cluster_start_mask
FBumann Dec 19, 2025
9d95ae0
Fix storage cluster decoupling
FBumann Dec 19, 2025
6b030c7
Switch cluster start indexing
FBumann Dec 19, 2025
b4dd428
Improve cluster indexing in Storage
FBumann Dec 19, 2025
ecb2c9d
Improve cluster indexing in Storage
FBumann Dec 19, 2025
355a205
Add more storag options when clustering
FBumann Dec 19, 2025
3c0203a
Add more storag options when clustering
FBumann Dec 19, 2025
1612599
Add more storag options when clustering
FBumann Dec 19, 2025
d6b82b5
Add more storag options when clustering
FBumann Dec 19, 2025
97ba243
Add more storag options when clustering
FBumann Dec 19, 2025
1b62c2c
Fix storage masking
FBumann Dec 19, 2025
8d026b5
Fix semantics: aggregation-> clsutering
FBumann Dec 20, 2025
ac6e58a
Add seasonal storage example
FBumann Dec 20, 2025
44e1832
Update notebook to show off storage modes
FBumann Dec 20, 2025
a59ef62
⏺ All changes complete. Here's a summary:
FBumann Dec 20, 2025
1c0e678
Update the notebooks
FBumann Dec 20, 2025
a6e5bd4
Update parameter name
FBumann Dec 20, 2025
a84543b
FIx clustered Storage
FBumann Dec 20, 2025
a8db4d7
FIx clustered Storage
FBumann Dec 20, 2025
c0ddab3
FIx clustered Storage
FBumann Dec 20, 2025
325534e
The refactoring is complete. Here's a summary of what was done:
FBumann Dec 20, 2025
2a36a64
The inter-cluster linking in StorageModel has been refactored to be …
FBumann Dec 20, 2025
0142272
Temp
FBumann Dec 20, 2025
b334699
Temp
FBumann Dec 20, 2025
db6e89a
Set start SOC in intercluster model when non cyclic
FBumann Dec 20, 2025
55fb607
Simplify cyclic constraint
FBumann Dec 20, 2025
d610da7
Improve Storage Model
FBumann Dec 22, 2025
5a2f08e
Add plan
FBumann Dec 25, 2025
c099b0b
Update plan
FBumann Dec 25, 2025
c048496
Update plan
FBumann Dec 25, 2025
d126511
Update plan
FBumann Dec 25, 2025
8f9bae8
Update plan
FBumann Dec 25, 2025
aa9653b
Update plan
FBumann Dec 25, 2025
06ad42b
Update plan I've completed the infrastructure for true (cluster, tim…
FBumann Dec 25, 2025
934c97d
Temp
FBumann Dec 25, 2025
949c903
Temp
FBumann Dec 25, 2025
923cd47
Temp
FBumann Dec 25, 2025
c18a126
Temp
FBumann Dec 25, 2025
11d90e9
Temp
FBumann Dec 25, 2025
4c7aeca
Temp
FBumann Dec 25, 2025
322d3c1
Temp
FBumann Dec 25, 2025
611c752
Temp
FBumann Dec 25, 2025
f232e0b
nitpicks and fixes
FBumann Dec 25, 2025
e200441
Temp
FBumann Dec 25, 2025
19f149f
Temp
FBumann Dec 25, 2025
82ac4c2
Temp
FBumann Dec 25, 2025
d1baccb
Temp
FBumann Dec 25, 2025
78ab16c
Temp
FBumann Dec 25, 2025
97a3440
Temp
FBumann Dec 26, 2025
5e4a856
Temp
FBumann Dec 26, 2025
d91c7e4
Better plotting defaults
FBumann Dec 26, 2025
3e07bca
timesteps (96) and timesteps_extra (97) correctly match the 2D (clust…
FBumann Dec 26, 2025
e5c3cf3
Revert storage energy balance reorder
FBumann Dec 26, 2025
d78179b
⏺ Both clustered and non-clustered FlowSystems work correctly.
FBumann Dec 26, 2025
8815eaa
minor simplifications
FBumann Dec 26, 2025
5df515f
Add options for states in cluster mode
FBumann Dec 26, 2025
fcf2269
Revert "Add options for states in cluster mode"
FBumann Dec 27, 2025
820fe6a
Update tests
FBumann Dec 27, 2025
12950f6
Fix expand_solution()
FBumann Dec 27, 2025
f4ef741
Fix Storage
FBumann Dec 27, 2025
937f4e2
Recombine charge state in intercluster mode
FBumann Dec 27, 2025
116415a
Fix storage model
FBumann Dec 27, 2025
358af3c
Summary of Changes
FBumann Dec 27, 2025
b8540ed
Feature/better data for notebooks (#542)
FBumann Dec 28, 2025
d488e85
Fix notebook
FBumann Dec 28, 2025
14abf85
add catchwarning and fix example system
FBumann Dec 28, 2025
dcea667
improve notebook
FBumann Dec 28, 2025
7014689
Improve docs, changelog and add tests
FBumann Dec 28, 2025
974df60
Merge remote-tracking branch 'origin/feature/aggregate-rework-v2' int…
FBumann Dec 28, 2025
702803a
Fix plotting animation handling
FBumann Dec 28, 2025
ab9a519
fix: cluster handling in temporal shares
FBumann Dec 28, 2025
300ea33
Change logging in clustering
FBumann Dec 28, 2025
9a8cd27
Fix imports and warnings
FBumann Dec 28, 2025
8e80478
Merge remote-tracking branch 'origin/feature/aggregate-rework-v2' int…
FBumann Dec 28, 2025
3ba9730
Fix netcdf warnings
FBumann Dec 28, 2025
325f4eb
Fix notebooks to not save to netcdf
FBumann Jan 3, 2026
282594c
Warning Handling Refactored
FBumann Jan 3, 2026
b178598
Feature: Clustering IO, single selection, and weights API improvement…
FBumann Jan 3, 2026
93e7152
Feature: fxplot Plotting Accessor (#548)
FBumann Jan 3, 2026
bf6962c
ci: Speedup Notebook execution (#551)
FBumann Jan 4, 2026
d4dd58b
Simplify notebooks
FBumann Jan 4, 2026
7b74846
Created helper module: docs/notebooks/data/tutorial_data.py
FBumann Jan 4, 2026
b1fa086
Removed animation_frame from the method signature, docstring, and up…
FBumann Jan 4, 2026
5f37003
Add missing notebooks to docs
FBumann Jan 4, 2026
184513e
Fix ci
FBumann Jan 4, 2026
0439d10
Fix ci
FBumann Jan 4, 2026
cfe8b9c
Add tutorial_data.py
FBumann Jan 5, 2026
c9bd406
Retrigger ci
FBumann Jan 5, 2026
7dcacfd
Merge branch 'main' into feature/aggregate-rework-v2
FBumann Jan 5, 2026
c94f6da
Clustering Parameter Refinements & Unified Slot Assignment (#552)
FBumann Jan 5, 2026
4f1d827
Add tutorial data back to ntoebooks
FBumann Jan 5, 2026
5ea8651
Feature/comparison (#550)
FBumann Jan 5, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 128 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,134 @@ If upgrading from v2.x, see the [v3.0.0 release notes](https://github.com/flixOp

Until here -->

## [5.1.0] - Upcoming

**Summary**: This release introduces a new **aggregation abstraction layer** for time series clustering, making flixopt future-proof for alternative clustering methods beyond TSAM. The API is simplified to focus on timestep reduction (`cluster_reduce`), removing the constraint-based clustering approach.

### ✨ Added

**New Aggregation Module** (`flixopt.aggregation`): A backend-agnostic abstraction for time series aggregation:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check what's actually exported from flixopt package
rg -n "from.*clustering" flixopt/__init__.py
rg -n "aggregation" flixopt/__init__.py
# Check if there's an aggregation module
fd -t f "aggregation" flixopt/

Repository: flixOpt/flixopt

Length of output: 249


Fix module name in CHANGELOG.md to match actual exports.

The changelog documents flixopt.aggregation as the new module, but the actual code exports flixopt.clustering (see flixopt/init.py:16). Update the changelog to reference the correct module name.

🤖 Prompt for AI Agents
In CHANGELOG.md around line 60, the changelog references `flixopt.aggregation`
but the package actually exports `flixopt.clustering` (see
flixopt/__init__.py:16); update the line to replace `flixopt.aggregation` with
`flixopt.clustering` so the changelog matches the actual exported module name.


```python
from flixopt import aggregation

# Available backends
aggregation.list_backends() # ['tsam', 'manual']

# Core data structures for any aggregation method
aggregation.AggregationResult # Universal result format
aggregation.ClusterStructure # For storage inter-cluster linking
aggregation.Aggregator # Protocol for custom backends
```

**Unified Aggregation API**: New `transform.aggregate()` method supporting multiple backends:

```python
# TSAM clustering (default) - clusters 365 days into 8 typical days
fs_reduced = flow_system.transform.aggregate(
method='tsam',
n_representatives=8,
cluster_duration='1D',
)
fs_reduced.optimize(solver)

# Expand back to full resolution
fs_expanded = fs_reduced.transform.expand_solution()
```

**TimeSeriesWeights Class**: PyPSA-inspired unified weighting system:

```python
# Access weights on any FlowSystem
weights = flow_system.weights

# temporal = timestep_duration × cluster_weight
weights.temporal # Applied to objective and constraints
weights.effective_objective # For objective function (with optional override)

# Convenience method for weighted summation
total_energy = weights.sum_over_time(flow_rates)
```

**Manual Aggregation Backend**: Enables PyPSA-style workflow with external clustering tools:

```python
from flixopt.aggregation import ManualBackend, create_manual_backend_from_labels

# Use sklearn or any clustering algorithm
from sklearn.cluster import KMeans
# ... perform clustering, get labels ...

# Create backend from cluster labels
backend = create_manual_backend_from_labels(labels, timesteps_per_cluster=24)

# Or directly with mapping and weights
backend = ManualBackend(
timestep_mapping=my_mapping, # xr.DataArray: original → representative
representative_weights=my_weights, # xr.DataArray: weight per representative
)
```

**set_aggregation() Method** (placeholder): Future PyPSA-style manual aggregation:

```python
# Coming soon - apply external clustering results directly
fs_agg = flow_system.transform.set_aggregation(
timestep_mapping=mapping,
weights=weights,
)
```

### 💥 Breaking Changes

**Removed `transform.cluster()` method**: The constraint-based clustering approach has been removed. Use `cluster_reduce()` instead:

```python
# Old (removed):
clustered_fs = flow_system.transform.cluster(
n_clusters=8,
cluster_duration='1D',
)

# New (use cluster_reduce instead):
reduced_fs = flow_system.transform.cluster_reduce(
n_clusters=8,
cluster_duration='1D',
)
```

**Removed constraint-based clustering infrastructure**:
- `transform.cluster()` - removed (use `cluster_reduce()`)
- `transform.add_clustering()` - removed
- `FlowSystem._clustering_info` - removed (only `_cluster_info` for `cluster_reduce` remains)
- `FlowSystem._add_clustering_constraints()` - removed

### ♻️ Changed

**Terminology clarification** in aggregation module:
- "cluster" = a group of similar time chunks (e.g., similar days grouped together)
- "typical period" = a representative time chunk for a cluster (TSAM terminology)
- "cluster duration" = the length of each time chunk (e.g., 24h for daily clustering)

Note: This is separate from the model's "period" dimension (years/months) and "scenario" dimension.

**xarray-native data structures**: All aggregation interfaces use `xr.DataArray` and `xr.Dataset` for proper coordinate handling.

### 🔥 Removed

- `transform.cluster()` method (constraint-based clustering)
- `transform.add_clustering()` method
- `ClusteringModel` constraint generation (internal)
- `_clustering_info` storage on FlowSystem

### 📝 Docs

- Improved terminology: clarified distinction between clustering "typical periods" and model "period" dimension
- Added aggregation module documentation with backend examples

---


## [5.0.3] - 2025-12-18

**Summary**: Cleaner notebook outputs and improved `CONFIG.notebook()` preset.
Expand Down
Loading
Loading