Skip to content

Remove eoscale vegmask#69

Merged
ytanguy merged 8 commits intoremove_eoscalefrom
remove_eoscale_vegmask
Apr 2, 2026
Merged

Remove eoscale vegmask#69
ytanguy merged 8 commits intoremove_eoscalefrom
remove_eoscale_vegmask

Conversation

@namsellem
Copy link
Copy Markdown
Collaborator

@namsellem namsellem commented Apr 1, 2026

Remove eoscale in vegetation mask

eoscale removed in functions called :

  • segmentation task
  • clean_task
  • compute_stats_image
  • finalize_task

Map reduce logic added to slurp executor and implemented for segmentation task needs.

Reported #44 in vegetation algorithm compliant with new signature in clean_task function

Results

For vegetation mask results refer to issue : #52

Tests

Name                                        Stmts   Miss  Cover
---------------------------------------------------------------
slurp/__init__.py                               7      0   100%
slurp/eomultiprocessing/__init__.py             0      0   100%
slurp/eomultiprocessing/slurp_executor.py     268    200    25%
slurp/eomultiprocessing/slurp_manager.py       47     16    66%
slurp/eomultiprocessing/utils.py               52     17    67%
slurp/masks/__init__.py                         7      0   100%
slurp/masks/shadowmask.py                     135      7    95%
slurp/masks/stack_masks.py                    220     13    94%
slurp/masks/urbanmask.py                      265     31    88%
slurp/masks/vegetationmask.py                 407     37    91%
slurp/masks/watermask.py                      360     31    91%
slurp/post_process/__init__.py                  0      0   100%
slurp/post_process/morphology.py               32      4    88%
slurp/prepare/__init__.py                       7      0   100%
slurp/prepare/analyse_glcm.py                  73      9    88%
slurp/prepare/aux_files.py                      7      0   100%
slurp/prepare/geometry.py                     110      2    98%
slurp/prepare/prepare.py                      227     29    87%
slurp/prepare/primitives.py                    29      0   100%
slurp/prepare/validity.py                       7      1    86%
slurp/tools/__init__.py                         7      0   100%
slurp/tools/constant.py                         7      0   100%
slurp/tools/eoscale_utils.py                   80     80     0%
slurp/tools/io_utils.py                        30      7    77%
slurp/tools/misc.py                            26     26     0%
slurp/tools/mylogger.py                        27     14    48%
slurp/tools/profile_utils.py                   78     14    82%
slurp/tools/pydantic_class.py                 144      0   100%
slurp/tools/random_forest_utils.py             23      0   100%
slurp/tools/utils.py                           56      5    91%
---------------------------------------------------------------
TOTAL                                        2738    543    80%
Coverage HTML written to dir htmlcov
====== 42 passed, 23 deselected, 106 warnings in 630.39s (0:10:30) =======

@namsellem namsellem requested a review from ytanguy April 1, 2026 12:54
@namsellem namsellem mentioned this pull request Apr 2, 2026
6 tasks
@ytanguy ytanguy merged commit 7816dfe into remove_eoscale Apr 2, 2026
4 checks passed
ytanguy added a commit that referenced this pull request Apr 21, 2026
* Fix bug - new mask_valid_indices array to enumerate valid segments (WIP : still some debug messages to remove)

* Fix bug - new mask_valid_indices array to enumerate valid segments (WIP : still some debug messages to remove)

* New Cython function to produce valid indices mask, apply in the clustering steps

* NDVI/NDWI are limited to -1/1 interval : correct rare issues when some pixels have a negative value

* Fix : NDVI/NDWI values are -1000/1000 and not -1 / 1

* Fix NDVI/NDWI, and take into account NODATA to compute number of segment to compute

* remove eoscale on first stage valid stack

* add eomultiprocessing component

* rm eoscale from ndvi & ndwi functions

* rm eoscale from texture functions & adapt docstrings

* rm out directories from slurp manager & spread profiles along workflow

* add logger info step

* split eoscale utils & profile utils

* fix texture profile forom int16 to uint16

* fix key cloud mask use case

* add concatenate samples in utils & rm from eoscales/profiles utils

* add mp_n_to_m_scalars function

* watermask to slurp exec & manager mp framework

* update mask variable names

* fix concatenate samples function & concat samples

* add rgba bands to rf pred

* fix pekel filter in post_process

* Add NDWI threshold use cases

* Add handling of Aux files

* Add some docstrings

* Add some docstrings

* Fix type mismatch : remove [ ], otherwise, a call to prepare with existing valid stack but new NDxI files fail... (#55)

* First Urbanmask version & Slurp executor aux files handling

* Fix info message + report Pekel threshold modification used in post-processing (#59)

* Fix info message + report Pekel threshold modification used in post-processing

* Fix documentation #57

* update logs urbanmask

* integ modifs in watermask, logs & aux inputs handling

* update logs in urbanmask

* write tif in all urban use cases & diff valid stack & OG if no water/veg Masks

* reduce McCabe Complexity through orchestration pattern

* phr to vhr

* swap vhr phr watermask

* first version shadowmask & stack masks with eomultiprocessing component

* remove eoscale imports

* add docstrings

* phr to vhr

* adding support to n bands images in slurp mp executor

* handle random input order in n bands images (h, w, c), (c, w, h),... (#67)

* handle tile_max_size args

* Revert accidental change in veg mask

* handle tile_max_size in remaining masks

* Remove eoscale vegmask (#69)

* first vegetation mask version

* veg mask handle tile_max size

* debug segmentation processing in vegetationmask with dev of func  mp_n_to_m_images_with_mapping

* fix stats in vegetationmask, first correct version

* adapt some docstrings in veg mask

* refactor mp_n_to_m_scalars

* before_clean.tif optionnal if debug

* report #44 in veg mask

* removed all eoscale references

* --version option added

* fix --version argparse

* first test version for mnh with urbanmask

* add test with mnh

* logging in veg mask & improve code style w/ pylint

* Update file paths in config_test_mnh.json

* Update file paths in main_MNH_config.json

* Add tqdm to package dependencies

* replace display infos function

* format w/ Black compliance

---------

Co-authored-by: Yannick TANGUY <yannick.tanguy@cnes.fr>
Co-authored-by: Yannick TANGUY <45934446+ytanguy@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants