Skip to content

feat(alerts): display detection crop overlay on main image#209

Open
MateoLostanlen wants to merge 9 commits into
mainfrom
feat/detection-crop-overlay
Open

feat(alerts): display detection crop overlay on main image#209
MateoLostanlen wants to merge 9 commits into
mainfrom
feat/detection-crop-overlay

Conversation

@MateoLostanlen
Copy link
Copy Markdown
Member

@MateoLostanlen MateoLostanlen commented Apr 27, 2026

Why

Make faster decisions on small fires from afar. The crop gives reviewers a tight, zoomed-in view of the smoke so they can confirm or dismiss without waiting on the full frame.

Companion PRs:

Demo: https://www.loom.com/share/0f5a3c5310044779abf3bb9dcdbc02ec

Note on the demo: the source images are already compressed HD frames, so the actual resolution benefit of the crop isn't really visible in the video. On real raw frames the close-up will be noticeably sharper.

What changes

  • Surface the new optional crop_url returned by GET /sequences/{id}/detections (with with_crop=true) in the detection schema.
  • Render the crop as a small (~150px) thumbnail pinned to a top corner of the detection viewer, flipping between top-left and top-right based on the bbox center so it never sits on the smoke.
  • Add a toggle button next to the existing bbox toggle so reviewers can hide the close-up. The overlay is automatically skipped for older detections without a crop.
  • Add buttonDisplayCrop / buttonHideCrop labels in en/fr/es/de (close-up / gros plan / primer plano / Nahaufnahme).

How to test

Run against the dev env with the engine and api PR branches. Heads-up: the current engine PR loops the same image, which makes the demo less convincing. For testing prefer the send_detection_crop_changing_images branch on pyro-engine, which is the same PR plus image rotation. A separate PR will fold that change into engine soon.

Note

This is a first proposal, fully open to discussion on both design and implementation. Feel free to commit directly on this branch if needed.

Surface optional crop_url returned by /sequences/{id}/detections
when with_crop=true. Field is nullable for older detections without
a crop.
Display the optional crop image returned by the API as a small
thumbnail pinned to the top-left of the detection viewer, with a
toggle button alongside the existing bbox toggle. The overlay is
only rendered when crop_url is present.
Add buttonDisplayCrop / buttonHideCrop in en/fr/es/de under the
alerts namespace.
Replace 'crop' with 'recadrage' which is the correct French term.
Place the crop preview opposite the detection: top-left when the
bbox center is on the right half of the image, top-right otherwise.
Avoids the overlay sitting on top of the smoke.
Use close-up / gros plan / primer plano / Nahaufnahme to better
convey that the inset shows a zoomed-in view of the smoke.
Render the crop at its native 224px width to match the model input
size, so reviewers see the full-resolution close-up.
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 27, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
new-pyro-platform Ready Ready Preview, Comment May 1, 2026 5:20am

@MateoLostanlen
Copy link
Copy Markdown
Member Author

I have a few questions to discuss together.
The crop moves from right to left depending on the detection position, which is good, but it hides the little window that shows us where we're zooming. How do we handle this, do we shift the crop, or maybe the crop disappears as soon as we zoom in?
What window size should we use? 150 here, the base image is 224x224.
The name, we say "crop" in English normally, I'm not really sure how to translate it

@MateoLostanlen MateoLostanlen requested review from fe51 and m-julio April 27, 2026 06:34
@fe51
Copy link
Copy Markdown
Member

fe51 commented Apr 28, 2026

Hi Mateo, thanks for making a first proposition to #203 !

At this stage of the discussion, provided the crop dimensions remain 224×224, I will try to include the full crop !

Now, as discussed and pointed out in your comment, the crop image must not alter the "map zoom" (At least we need to avoid it)

What about a display that’s always at the bottom right if the light is mainly in the left-hand area (the centre of the bounding boxes of the sequences? Or just the first one?), and vice versa?

So, when might this approach cause problems?
If a bounding box is so wide that, even though it is mainly on one side, it overlaps onto the other side, and the bounding box and the crop area overlap?
On the one hand, this is a situation where we would need to count the number of occurrences; on the other hand, let us remember that the crop area is not displayed by default at this stage (and if it were, one would simply need to click to hide it).

…verlay

# Conflicts:
#	src/components/Alerts/AlertDetails/AlertImages/AlertImages.tsx
#	src/services/alerts.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants