All changes that impact users of this module are documented in this file, in the Common Changelog format with some additional specifications defined in the CONTRIBUTING file. This codebase adheres to Semantic Versioning.
Development of this release was supported by Reset Tech.
- Fix snapshot storage failure when source document URLs contain characters forbidden in filenames
- Fix duplicate file extension in snapshot filenames when source document URLs end with an extension
Development of this release was supported by Reset Tech.
- Update JSDOM to fix extraction of terms with large inline stylesheets
Development of this release was supported by the NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.
- Remove Notifier module
- Breaking: Remove all default SMTP values (
host,port,username) from logger configuration; operators enablingsendMailOnErrormust now explicitly set all three in@opentermsarchive/engine.logger.smtp
Development of this release was supported by Reset Tech.
- Fix declaration validation when a new service is added along with a filters file
Development of this release was supported by Reset Tech.
- Treat navigation timeout errors as transient, enabling automatic retry when pages take too long to load
Development of this release was supported by Reset Tech.
- Add PDF file support in full DOM fetcher
Development of this release was supported by Reset Tech.
- Make SMTP port configurable and switch from implicit SSL to STARTTLS, enabling use of alternative ports (like 2525) on servers where standard SMTP ports are blocked
Development of this release was supported by Reset Tech.
- Fix dataset publishing failure for large files by executing uploads sequentially instead of in parallel
Development of this release was supported by Reset Tech.
- Fix error messages when the start and end selectors of range selectors match elements on the page but the selected range or extracted content is empty
Development of this release was supported by Reset Tech.
- Fix browser and request language with full DOM fetcher
Development of this release was made on a volunteer basis by @Ndpnt.
- Set realistic viewport size for full DOM fetcher to avoid detection based on default Puppeteer dimensions
- Improve isolation between fetches by using isolated browser contexts
Full changeset and discussions: #1219.
Development of this release was supported by Reset Tech.
- Add
Advertising Content Policyterms type, introduced interms-types@2.2.0
Full changeset and discussions: #1217.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Validate presence of required configuration for dataset publication to data.gouv.fr
Full changeset and discussions: #1212.
Development of this release was supported by Reset Tech.
- Exclude development and local config files from the published package to avoid unintentionally overriding default configuration
Full changeset and discussions: #1209.
Development of this release was supported by the European Commission for its VLOPs/VLOSEs instance.
- Support proxies in GitLab issues reporting
Full changeset and discussions: #1202.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Add support for publishing datasets to data.gouv.fr; configure
dataset.datagouv.datasetIdordataset.datagouv.organizationIdOrSlugin configuration file and setOTA_ENGINE_DATAGOUV_API_KEYenvironment variable - Add ability to publish datasets to multiple platforms simultaneously; datasets can now be published to GitHub (or GitLab) and data.gouv.fr in parallel
Full changeset and discussions: #1208.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Prevent external hosts from accessing the collection API directly
Full changeset and discussions: #1207.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Add
ota apply-technical-upgradesCLI command to apply technical upgrades independently
- Breaking: Remove
--extract-onlyoption fromota trackcommand; use the newota apply-technical-upgradescommand instead
- Fix incorrect versioning that occurred when adding new source documents to combined terms declarations
Full changeset and discussions: #1204.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix email reporting on collection API errors
Full changeset and discussions: #1203.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix saving records whose content is not yet loaded when using MongoDB storage.
Full changeset and discussions: #1206.
Development of this release was supported by the European Commission for its VLOPs/VLOSEs instance.
- Fix tracking when proxy configuration is provided
Full changeset and discussions: #1173.
Development of this release was supported by the European Commission for its VLOPs/VLOSEs instance.
- Add proxy support for fetching documents behind firewalls or restricted networks; configure using
HTTP_PROXYandHTTPS_PROXY(orhttp_proxyandhttps_proxy) environment variables - Add debugging options to disable headless mode for visual troubleshooting during development; set
OTA_ENGINE_FETCHER_NO_HEADLESS=1to show browser window - Add sandbox control for improved compatibility with Docker and containerized environments; set
OTA_ENGINE_FETCHER_NO_SANDBOX=1when running in containers
Full changeset and discussions: #1199.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Update Node compatibility metadata; actual compatibility was added in v7.2.1
Full changeset and discussions: #1198.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Increase robustness of PDF content type detection
Full changeset and discussions: #1197.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Improve memory management when tracking terms with multiple source documents
- All available snapshots are now stored in multi-documents terms, even if some are unavailable.
Full changeset and discussions: #1196.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Breaking: Update collection governance roles by adding
contributor,analystanddiffuserand replacingadministratorwithsysadmin; update roles in your metadata file at the root of your collection folder
Full changeset and discussions: #1193.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Breaking: Update minimum supported Node.js version to v20
- Add
Content Monetisation Policyterms type, introduced interms-types@2.1.0 - Update dependencies
Full changeset and discussions: #1195.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix built-in
removeQueryParamsfilter to not modify URLs without target parameters
Full changeset and discussions: #1192.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix declarations validation when filters history is modified
Full changeset and discussions: #1191.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix validation of declaration files with filters
Full changeset and discussions: #1189.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Add support for Node.js v24
Full changeset and discussions: #1178.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Add support for parameters in filters; see more in the filters documentation
- Add
removeQueryParamsbuilt-in filter to remove query parameters from links and images; see more in the built-in filters documentation
Full changeset and discussions: #1188.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Improve readability of multi-document terms
Full changeset and discussions: #1175.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Breaking: Update issue labels to improve clarity; existing labels will be automatically updated on the first run
Full changeset and discussions: #1176.
Development of this release was supported by the Lab Platform Governance, Media and Technology (PGMT), Centre for Media, Communication and Information Research (ZeMKI), University of Bremen as part of the project Governance: Private ordering of ComAI through corporate communication and policies in the research unit Communicative AI, funded by the German Research Foundation (DFG) (Grant No. 516511468).
- Fetch multiple source documents sequentially to prevent bot detection and improve tracking success rate
Full changeset and discussions: #1171.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Ensure labels are properly recreated after removing deprecated labels in GitHub and GitLab reporters
Full changeset and discussions: #1170.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Breaking: Replace generic issue labels with more descriptive names to better understand why terms tracking was interrupted; existing labels will be automatically updated on the first run
- Add
⚠ needs interventionlabel to highlight critical issues requiring manual action to restore terms tracking
Full changeset and discussions: #1169.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Improve headless browser context isolation when fetching pages by disabling cache and clearing cookies between requests to prevent session persistence across different URLs and to improve tracking success rate
Full changeset and discussions: #1168.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix automatic labeling of GitHub issues when tracking fails due to DNS resolution errors
Full changeset and discussions: #1166.
Development of this release was supported by the Lab Platform Governance, Media and Technology (PGMT), Centre for Media, Communication and Information Research (ZeMKI), University of Bremen as part of the project Governance: Private ordering of ComAI through corporate communication and policies in the research unit Communicative AI, funded by the German Research Foundation (DFG) (Grant No. 516511468).
- Extend automatic retry mechanism for failed tracking attempts due to likely bot blocking errors to improve tracking success rate
Full changeset and discussions: #1159.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Add content validation to ensure selectors contain actual text before considering them loaded to improve tracking success rate
Full changeset and discussions: #1162.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix engine version in logs, records metadata and collection API
Full changeset and discussions: #1156.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix network system error reporting
Full changeset and discussions: #1155.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Improve email error and warning reporting to provide more detailed information and better readability
Full changeset and discussions: #1154.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Handle connection resets as bot protection to improve tracking success rate
Full changeset and discussions: #1152.
Development of this release was supported by the Lab Platform Governance, Media and Technology (PGMT), Centre for Media, Communication and Information Research (ZeMKI), University of Bremen as part of the project Governance: Private ordering of ComAI through corporate communication and policies in the research unit Communicative AI, funded by the German Research Foundation (DFG) (Grant No. 516511468).
- Add an automatic fetching fallback to headless browser when bot detection is encountered to improve tracking success rate
Full changeset and discussions: #1151.
Development of this release was supported by the Lab Platform Governance, Media and Technology (PGMT), Centre for Media, Communication and Information Research (ZeMKI), University of Bremen as part of the project Governance: Private ordering of ComAI through corporate communication and policies in the research unit Communicative AI, funded by the German Research Foundation (DFG) (Grant No. 516511468).
-
Add an automatic retry mechanism for failed tracking attempts due to likely transient errors to improve tracking success rate
⚠️ Version correction notice: Version 5.1.1 was incorrectly published as a patch release, violating semantic versioning principles. This version (5.2.0) contains the same changes but is published as a minor version to properly reflect the addition of the new automatic retry mechanism feature.
Full changeset and discussions: #1150.
Development of this release was supported by the Lab Platform Governance, Media and Technology (PGMT), Centre for Media, Communication and Information Research (ZeMKI), University of Bremen as part of the project Governance: Private ordering of ComAI through corporate communication and policies in the research unit Communicative AI, funded by the German Research Foundation (DFG) (Grant No. 516511468).
- Add an automatic retry mechanism for failed tracking attempts due to likely transient errors to improve tracking success rate
Full changeset and discussions: #1149.
Development of this release was supported by the Lab Platform Governance, Media and Technology (PGMT), Centre for Media, Communication and Information Research (ZeMKI), University of Bremen as part of the project Governance: Private ordering of ComAI through corporate communication and policies in the research unit Communicative AI, funded by the German Research Foundation (DFG) (Grant No. 516511468).
- Reduce navigation timeout errors and enhance performance of headless browser fetching
Full changeset and discussions: #1148.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Update links to documentation in tracking issues reports
Full changeset and discussions: #1147.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Enable declarations linting in production environments
Full changeset and discussions: #1146.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix creation of issues labels for GitHub repositories with over 30 labels
Full changeset and discussions: #1141.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix MongoDB repository support of terms with multiple source documents
Full changeset and discussions: #1140.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix validation of the collection metadata file
Full changeset and discussions: #1139.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Validate required fields in collection metadata file
Full changeset and discussions: #1137.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
-
Breaking: Rename
documentskey totermsin service declarations for vocabulary consistency; to update your declarations:For Unix-like systems:
find . -name "*.json" -type f -exec sed -i 's/"documents":/"terms":/g' {} +
For macOS:
find . -name "*.json" -type f -exec sed -i '' 's/"documents":/"terms":/g' {} +
⚠️ Note: when updating the engine, CI checks of modified declarations will fail once. This is expected and can be safely ignored
Full changeset and discussions: #1135.
Development of this release was supported by the European Union.
- Add support for nested subgroups in Gitlab reporter repository paths
- Fix error logging in Gitlab reporter
Full changeset and discussions: #1134.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Add
ota validate metadatacommand to the CLI to validate the collection metadata file
Full changeset and discussions: #1133.
Development of this release was supported by the European Union.
- Fix functionality of filtering by status for the issues list on GitLab
Full changeset and discussions: #1124.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix documentation of collection metadata in the collection API
Full changeset and discussions: #1123.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Breaking: Expose collection metadata through the collection API; ensure to add the required metadata file at the root of your collection folder
- Breaking: Replace
@opentermsarchive/engine.services.declarationsPathwith@opentermsarchive/engine.collectionPath; ensure your declarations are located in./declarationsin your collection folder
Full changeset and discussions: #1122.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Breaking: Updated peer dependency
@opentermsarchive/terms-typestov2.0.0; in your declarations, if you define a specific version of this library, ensure to also upgrade it tov2.0.0to avoid compatibility issues; see terms-types@2.0.0 for details
Full changeset and discussions: #1121.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Optimize performance for Git storage
Full changeset and discussions: #1120.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix timestamp prefix format in logs
Full changeset and discussions: #1118.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Add configuration option to toggle timestamp prefix in logs; set
@opentermsarchive/engine.logger.timestampPrefixtotrueorfalsein your configuration file to control this feature.
Full changeset and discussions: #1116.
Development of this release was supported by the European Union and the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Add support for GitLab for issue reporting
- Add support for GitLab Releases for publishing datasets
Full changeset and discussions: #1115.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Add script to remove duplicate issues in GitHub reports
Full changeset and discussions: #1114.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Add
Service Level Agreementterms type, introduced interms-types@1.4.0
Full changeset and discussions: #1113.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Reduce GitHub API calls in issues reporter
Full changeset and discussions: #1112.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Reduce API calls and avoid inconsistent states on GitHub issues reports
Full changeset and discussions: #1111.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Prevent duplicate issues on GitHub reports
Full changeset and discussions: #1110.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Bump
@opentermsarchive/terms-typespeer dependency to support new types introduced in v1.3.0
Full changeset and discussions: #1109.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Prevent the engine from exiting on SMTP errors from the logger
Full changeset and discussions: #1088.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Display engine version in logs
- Fix the number of terms displayed in logs
- Fix Reporter module instantiation
Full changeset and discussions: #1087.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Set default tracking frequency to two per day
Full changeset and discussions: #1086.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Make tracking schedule configurable
- Make publishing schedule configurable
- Add tracking overrun protection
Full changeset and discussions: #1085.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Corrected a missing negation in a Reporter error log
Full changeset and discussions: #1071.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Breaking: Nest all configurations under the
@opentermsarchive/enginekey; wrap the content ofconfig/production.jsonlike this:{ @opentermsarchive/engine: { < your previous configuration >}} - Breaking: Rename the
apiconfiguration key tocollection-api; update this in yourconfig/production.json - Breaking: Remove default values for
portandbasePathincollection-api; specify these configurations explicitly in yourconfig/production.json - Breaking: Prefix all environment variables with
OTA_ENGINE_; rename variables such asSENDINBLUE_API_KEYtoOTA_ENGINE_SENDINBLUE_API_KEY,SMTP_PASSWORDtoOTA_ENGINE_SMTP_PASSWORD, andGITHUB_TOKENtoOTA_ENGINE_GITHUB_TOKEN
Full changeset and discussions: #1076.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Enable a service to be renamed into several services when exporting a dataset
Full changeset and discussions: #1075.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Exclude non-record files (e.g., README, LICENSE) from dataset
Full changeset and discussions: #1070.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix
ota lintcommand
Full changeset and discussions: #1069.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Add label
empty contentto reported issues on GitHub when server returns empty content or when PDF content is unextractable - Add label
invalid selectorto reported issues on GitHub when CSS selector is invalid
- Consider extraction errors as expected operational errors instead of crashing the engine
- Do not crash when source documents links point to invalid URLs
- Take into account the
--typesCLI option - Fix counts of terms when they are specified via the CLI option
--types - Fix display issues for service ID and terms type associated with errors
Full changeset and discussions: #1068.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix validation of modified declarations when declaration history is modified; bug introduced in v1.1.2
Full changeset and discussions: #1066.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix validation of modified declarations when new services were added
- Fix validation of modified declarations when filters were modified
Full changeset and discussions: #1065.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Ensure to reopen associated issue if terms become inaccessible again
Full changeset and discussions: #1063.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Introduce
pluginErrorerror type to isolate plugin errors from the engine and increase robustness
Full changeset and discussions: #1061.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
No code changes were made in this release, it simply signifies that the public API can now be considered stable.
Full changeset and discussions: #1056.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix support of service names with special character in the validation CLI
Full changeset and discussions: #1054.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Minimize required privileges associated to the GitHub token to create issues in the declarations repository
Full changeset and discussions: #1053.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Log a warning in case log emails cannot be sent because of a missing config
Full changeset and discussions: #1052.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Breaking: Base the loading of the Notifier module on the presence of required configuration, not on the environment; only define the
SENDINBLUE_API_KEYenvironment variable if you want the Notifier to be loaded, avoid relying onNODE_ENV=production
Full changeset and discussions: #1051.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Consider EAI_AGAIN errors as a legitimate part of the tracking lifecycle rather than causing the engine to crash
Full changeset and discussions: #1050.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix engine hang with non-existent service ID
Full changeset and discussions: #1045.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix support of special character in service names
Full changeset and discussions: #1037.
Development of this release was made on a volunteer basis by @Ndpnt.
- Improve performance for large-scale MongoDB databases
Full changeset and discussions: #1033.
Development of this release was supported by the NGI0 Entrust Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.
- Expose versions data through the collection API (#1003). When using Git as storage for versions, this API relies on the assumption that the commit date matches the author date, introduced in the engine in June 2022 (#875). If your collection was created before this date, inconsistencies in the API results may arise. You can verify if your version history includes commits with commit dates differing from author dates by executing the following command at the root of your versions repository:
git log --format="%H %ad %cI" --date=iso-strict | awk '{if ($2 != $3) print "Author date", $2, "and commit date", $3, "mismatch for commit", $1 }'. You can correct the history with the command:git rebase --committer-date-is-author-date $(git rev-list --max-parents=0 HEAD). Since the entire history will be rewritten, a force push may be required for distributed repositories
- Provide a succinct JSON-formatted error message as response in API errors
Full changeset and discussions: #1032.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Update ESLint to use the stylistic plugin following deprecation of formatting rules, correcting a randomly-appearing
Configuration for rule "lines-between-class-members" is invaliderror
Full changeset and discussions: #1031.
Development of this release was supported by the NGI0 Entrust Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.
- Add
termsattribute to/servicesAPI response, containing declared term types for each service
Full changeset and discussions: #1026.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Ensure
Reporterconfiguration is defined before instantiating it - Ensure to add links to
versionsandsnapshotsin issues only if repositories are defined in configuration
- No longer define default configuration for
Reportermodule
Full changeset and discussions: #1025.
Development of this release was supported by GitHub Social Impact through its DPG Open Source Community Manager Program.
- Auto create labels for declarations repository on GitHub
- Assign a label based on the error type to issues managed by the engine
- Breaking: Revise the formatting of reported issues, with notable adjustments to the title, transitioning from
Fix <service_id> - <terms_type>to<service_id> ‧ <terms_type> ‧ not tracked anymore - Breaking: Rename
trackermodule intoreporter; update your configuration file by renamingtrackerintoreporter - Breaking: Change configuration for
reportermodule; update yourreporterconfiguration accordingly
- Breaking: No longer assign the label
bot-reportto issues managed by the engine
Full changeset and discussions: #1024.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Upgrade dependencies patches and ensure compatibility with Node v20
Full changeset and discussions: #1021.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Breaking: Update
opentermsarchive/terms-typesdependency; update your declarations following the instructions in the changelog for validating declarations
Full changeset and discussions: #1020.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix validation for removed terms types
Full changeset and discussions: #1015.
Development of this release was supported by the NGI0 Entrust Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.
- Embed Swagger UI for graphical user interface documentation of the API; access it on
/docs
- Breaking: Change path from
/services/:serviceIdto/service/:serviceIdfor direct access to aserviceresource; update paths accordingly in your own codebase - Breaking: Change path from
/specsto/docsfor OpenAPI specification; update paths accordingly in your own codebase and set the HTTP headerAccept: application/json
Full changeset and discussions: #1013.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Fix issue creation on GitHub tracker (#1012)
Full changeset and discussions: #1011.
Development of this release was supported by the NGI0 Entrust Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG CNECT under grant agreement N°101069594.
- Add collection metadata API (#1003)
- Add
ota servecommand to CLI to start the server and expose the API
- Include collection name in dataset name
- Clean up README
Full changeset and discussions: #996, #999, #1000.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Breaking: Rename CLI option
--terms-typesto--typesin API; simply rename accordingly in your own codebase - Breaking: Rename CLI option
--refilter-only,-rto--extract-only,-ein API; simply rename accordingly in your own codebase - Breaking: Rename class
PageDeclarationtoSourceDocumentand its atributenoiseSelectorstoinsignificantContentSelectorsin API; simply rename accordingly in your own codebase - Breaking: Rename function and its parameters
filter({ content, mimeType, pageDeclaration })toextract(sourceDocument)in API;contentandmimeTypeare embeddedsourceDocumentattributes; rename accordingly in your own codebase and setcontentandmimeTypein thesourceDocumentpassed as a parameter to the function - Breaking: Rephrase commit messages in Git storage:
Start trackingis changed toFirst record of,RefiltertoApply technical or declaration upgrade onandUpdatetoRecord new changes of; existing data will still be loaded, but new commits will use these new messages, if you have scripts that parse commit messages directly, update them accordingly - Breaking: Rename document attribute
isRefiltertoisExtractOnlyin MongoDB storage; existing data will still be loaded, but new entries will use this new attribute, if you have scripts that query the Mongo database directly, update them accordingly - Make vocabulary consistent throughout the codebase (#971)
- Breaking: Remove
npm run extractcommand; usenpm run start -- --extract-onlyinstead
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Change URL to engine repo from
ambanum/OpenTermsArchivetoOpenTermsArchive/engine - Replace documentation with links to the documentation website to avoid redundancy
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Breaking: Split validation and linting of declaration files in 2 commands.
ota validatedoes not test the linting of files anymore. - Breaking:
ota lintdoes not fix files by default.ota lint --fixmust be used for that.
Full changeset and discussions: #981.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Specification of changelog format
- Specification of changelog format is now in CONTRIBUTING
- Improve documentation for CLI commands
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Breaking: Replace behaviour of
ota track --schedule. It now tracks the changes only on scheduled hours. It was before running also on launch.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Trigger a release event on CI to deploy documentation website
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Breaking: Remove obsolete ansible deployment recipes as it was extracted in a dedicated repository. Look at the README to know how to deploy the engine.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Replace embedded terms types list with the one defined in the dedicated repository
@opentermsarchive/terms-types.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Add dataset command to CLI; this command can be discovered in the documentation and by running
ota dataset help
Full changeset and discussions: #959.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Improved reliability and expanded coverage of email protection global filter
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Add missing configuration for production
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Breaking: Unify all CLI commands as subcommands of one single
otacommand and rename some options; the new CLI can be discovered in the documentation and by runningota help(#978)
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Ensure paths for given Git storage configuration are relative to current working directory instead of engine module folder
Full changeset and discussions: #979.
- Add missing configuration for production
Full changeset and discussions: #976.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Load both the configurations defined in this module and the configurations defined in the module which use it as dependency
Full changeset and discussions: #977.
- Add missing file required for CLI in the packaging process
Full changeset and discussions: #971.
- Rewrite documentation to bring it up to date with current behaviour
Full changeset and discussions: #957.
Development of this release was supported by the French Ministry for Foreign Affairs through its ministerial State Startups incubator under the aegis of the Ambassador for Digital Affairs.
- Publish package on NPM under name
@opentermsarchive/engine - Export
filter,pageDeclarationandfetchin NPM module - Add changelog