Skip to content

feat: prevent future events#133

Open
sl-aurelien wants to merge 3 commits intoMeltanoLabs:mainfrom
sl-aurelien:feat-prevent-future-events
Open

feat: prevent future events#133
sl-aurelien wants to merge 3 commits intoMeltanoLabs:mainfrom
sl-aurelien:feat-prevent-future-events

Conversation

@sl-aurelien
Copy link
Copy Markdown

We observed rare case instances of Klaviyo API returning events way in the future (events API).
Those events will update the state to the future as well, meaning that every future call will fail because a greater-than filter in the API call isn't allowed.

This PR is a suggestion that would eliminate the issue by filtering out future events.
Feel free to suggest alternative approaches.

Example of a payload queried on 2024-04-12. Note that datetime (which is the Event's stream replication key) is 2025-04-21:

 {
            "type": "event",
            "id": "xxx",
            "attributes": {
                "timestamp": 1745246922,
                "event_properties": {
                    "utm_medium": "xxx",
                    "os": "Windows",
                    "initial_page_path": "/",
                    "browser": "Chrome",
                    "page": "xxx",
                    "utm_source": "xxx",
                    "rfsn": "xxx",
                    "$event_id": "1745246922"
                },
                "datetime": "2025-04-21T14:48:42+00:00",
                "uuid": "xxx"
            },
            "relationships": {
                "profile": {
                    "data": {
                        "type": "profile",
                        "id": "xxx"
                    },
                    "links": {
                        "self": "https://a.klaviyo.com/api/events/xxx/relationships/profile/",
                        "related": "https://a.klaviyo.com/api/events/xxx/profile/"
                    }
                },
                "metric": {
                    "data": {
                        "type": "metric",
                        "id": "xxx"
                    },
                    "links": {
                        "self": "https://a.klaviyo.com/api/events/xxx/relationships/metric/",
                        "related": "https://a.klaviyo.com/api/events/xxx/metric/"
                    }
                }
            },
            "links": {
                "self": "https://a.klaviyo.com/api/events/xxx/"
            }
        }
    ],
    "links": {
        "self": "https://a.klaviyo.com/api/events?sort=datetime&filter=greater-than%28datetime%2C2025-04-14+02%3A39%3A06%2B00%3A00%29",
        "next": null,
        "prev": null
    }

sl-aurelien and others added 3 commits April 14, 2025 10:22
* feat: add requester

* feat: use builtin backoffs methods

* force api to fail for testing purposes

* add explicit list of exceptions

* use validate_response approach

* move super validate_response to after our custom logic

* use retry_statuses config instead of hardcoded values

* Revert "force api to fail for testing purposes"

This reverts commit 42f0c99.

* chore: minor cleanup

* [pre-commit.ci] auto fixes

* fix: pre-commit

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant