From 9f9474873b810ef01f8d03caaea56d8b039f782e Mon Sep 17 00:00:00 2001 From: Dave Marion Date: Fri, 18 Apr 2025 14:12:36 +0000 Subject: [PATCH 1/3] Updated upgrade documentation for new upgrade process. Closes #455 --- _docs-2/administration/upgrading.md | 42 +++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/_docs-2/administration/upgrading.md b/_docs-2/administration/upgrading.md index 3475ac378..4d1ec469d 100644 --- a/_docs-2/administration/upgrading.md +++ b/_docs-2/administration/upgrading.md @@ -4,6 +4,48 @@ category: administration order: 7 --- +## Changes to the upgrade process + +In upgrade notes for prior releases we have advised that users should ensure that no FATE +transactions exist (see `Upgrading from 1.10 or 2.0 to 2.1` below). This is due to the +fact that the internal serialization of FATE transactions is not guaranteed to be +compatible between versions. Accumulo never provided any tooling around the upgrade process +and left it up the user, which can cause problems if older versions of FATE transactions +exist and the user already deployed the new version of software. The user would have to +re-install the old version of software to remove the FATE transactions. + +Starting with Accumulo 4.0 we are modifying the upgrade process steps in an attempt to +make it easier for the user to validate their instance is ready for upgrade. In earlier +versions the upgrade process would start when the user started the instance with the +new version of software. The process ran entirely in the Manager and it was the users +responsibility to read the upgrade notes to perform any necessary pre-upgrade steps. + +The new upgrade process introduces a new `accumulo upgrade` command which will be used +after shutting down the instance with the old version of software and before starting +the instance with the new version of software. The `upgrade` command has two options, +`--prepare` and `--start`. `--prepare` is designed to be executed by the user after shutting +down the instance. This option will check that the Manager is down, validate that there +are no existing FATE transactions, remove all of the server locks in ZooKeeper, and place +a node in ZooKeeper that will prevent server processes from being started again. If there +are FATE transactions, the command will fail giving the user the opportunity to clean them +up. The `--prepare` option can then be run again. + +The `--start` option is designed to be executed by the user before starting the instance +with the newer version of software. The `--start` option will perform any necessary pre-upgrade +validation, any changes that are necessary for the new version of software to start, seed +an upgrade progress tracker node in ZooKeeper, and then finally remove the node in ZooKeeper +created by the `--prepare` step so that the server processes can be started. If the user did +not run the `--prepare` step with the older version of software, then the `--start` option +will fail unless the `--force` option is used. Running `--start` with the `--force` option +will perform the same checks that `--prepare` executes, but if FATE transactions are found +then the user will need to remove them using the older version of software. Once the `--start` +option completes, then the user can start the server processes to complete the upgrade +process. The user will want to check the contents of the Manager log to observe the progress +of the upgrade. + +The `accumulo upgrade --prepare` command will be included with the 2.1.4 and 3.1.0 releases +to assist users in upgrading to the 4.0 release. + ## Upgrading from 1.10 or 2.0 to 2.1 Please read these directions in their entirety before beginning. Please [contact] us with any From 85d23a967c62cb6e4e5433e82704f861c7f2d3c2 Mon Sep 17 00:00:00 2001 From: Dave Marion Date: Mon, 21 Apr 2025 14:24:44 -0400 Subject: [PATCH 2/3] Update _docs-2/administration/upgrading.md Co-authored-by: Dom G. --- _docs-2/administration/upgrading.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_docs-2/administration/upgrading.md b/_docs-2/administration/upgrading.md index 4d1ec469d..fcf971cc1 100644 --- a/_docs-2/administration/upgrading.md +++ b/_docs-2/administration/upgrading.md @@ -32,7 +32,7 @@ up. The `--prepare` option can then be run again. The `--start` option is designed to be executed by the user before starting the instance with the newer version of software. The `--start` option will perform any necessary pre-upgrade -validation, any changes that are necessary for the new version of software to start, seed +validation, make any changes that are necessary for the new version of software to start, seed an upgrade progress tracker node in ZooKeeper, and then finally remove the node in ZooKeeper created by the `--prepare` step so that the server processes can be started. If the user did not run the `--prepare` step with the older version of software, then the `--start` option From f33521937fe5103177ff7aebb59d7e82d153d936 Mon Sep 17 00:00:00 2001 From: Dave Marion Date: Mon, 21 Apr 2025 18:42:51 +0000 Subject: [PATCH 3/3] Copy changes to _docs-4 directory to keep in sync --- _docs-4/administration/upgrading.md | 42 +++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/_docs-4/administration/upgrading.md b/_docs-4/administration/upgrading.md index 3475ac378..fcf971cc1 100644 --- a/_docs-4/administration/upgrading.md +++ b/_docs-4/administration/upgrading.md @@ -4,6 +4,48 @@ category: administration order: 7 --- +## Changes to the upgrade process + +In upgrade notes for prior releases we have advised that users should ensure that no FATE +transactions exist (see `Upgrading from 1.10 or 2.0 to 2.1` below). This is due to the +fact that the internal serialization of FATE transactions is not guaranteed to be +compatible between versions. Accumulo never provided any tooling around the upgrade process +and left it up the user, which can cause problems if older versions of FATE transactions +exist and the user already deployed the new version of software. The user would have to +re-install the old version of software to remove the FATE transactions. + +Starting with Accumulo 4.0 we are modifying the upgrade process steps in an attempt to +make it easier for the user to validate their instance is ready for upgrade. In earlier +versions the upgrade process would start when the user started the instance with the +new version of software. The process ran entirely in the Manager and it was the users +responsibility to read the upgrade notes to perform any necessary pre-upgrade steps. + +The new upgrade process introduces a new `accumulo upgrade` command which will be used +after shutting down the instance with the old version of software and before starting +the instance with the new version of software. The `upgrade` command has two options, +`--prepare` and `--start`. `--prepare` is designed to be executed by the user after shutting +down the instance. This option will check that the Manager is down, validate that there +are no existing FATE transactions, remove all of the server locks in ZooKeeper, and place +a node in ZooKeeper that will prevent server processes from being started again. If there +are FATE transactions, the command will fail giving the user the opportunity to clean them +up. The `--prepare` option can then be run again. + +The `--start` option is designed to be executed by the user before starting the instance +with the newer version of software. The `--start` option will perform any necessary pre-upgrade +validation, make any changes that are necessary for the new version of software to start, seed +an upgrade progress tracker node in ZooKeeper, and then finally remove the node in ZooKeeper +created by the `--prepare` step so that the server processes can be started. If the user did +not run the `--prepare` step with the older version of software, then the `--start` option +will fail unless the `--force` option is used. Running `--start` with the `--force` option +will perform the same checks that `--prepare` executes, but if FATE transactions are found +then the user will need to remove them using the older version of software. Once the `--start` +option completes, then the user can start the server processes to complete the upgrade +process. The user will want to check the contents of the Manager log to observe the progress +of the upgrade. + +The `accumulo upgrade --prepare` command will be included with the 2.1.4 and 3.1.0 releases +to assist users in upgrading to the 4.0 release. + ## Upgrading from 1.10 or 2.0 to 2.1 Please read these directions in their entirety before beginning. Please [contact] us with any