Skip to content

Updated upgrade documentation for new upgrade process. #456

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dlmarion merged 4 commits into from
Apr 22, 2025
Merged

Updated upgrade documentation for new upgrade process. #456

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9f94748
Updated upgrade documentation for new upgrade process.
dlmarion Apr 18, 2025
85d23a9
Update _docs-2/administration/upgrading.md
dlmarion Apr 21, 2025
8ec9701
Merge branch 'main' into 455-2-1-upgrade-prepare
dlmarion Apr 21, 2025
f335219
Copy changes to _docs-4 directory to keep in sync
dlmarion Apr 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions _docs-2/administration/upgrading.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,48 @@ category: administration
order: 7
---

## Changes to the upgrade process

In upgrade notes for prior releases we have advised that users should ensure that no FATE
transactions exist (see `Upgrading from 1.10 or 2.0 to 2.1` below). This is due to the
fact that the internal serialization of FATE transactions is not guaranteed to be
compatible between versions. Accumulo never provided any tooling around the upgrade process
and left it up the user, which can cause problems if older versions of FATE transactions
exist and the user already deployed the new version of software. The user would have to
re-install the old version of software to remove the FATE transactions.

Starting with Accumulo 4.0 we are modifying the upgrade process steps in an attempt to
make it easier for the user to validate their instance is ready for upgrade. In earlier
versions the upgrade process would start when the user started the instance with the
new version of software. The process ran entirely in the Manager and it was the users
responsibility to read the upgrade notes to perform any necessary pre-upgrade steps.

The new upgrade process introduces a new `accumulo upgrade` command which will be used
Copy link
Copy Markdown
Member

@DomGarguilo DomGarguilo Apr 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"will be used"
Would "should be" or "needs to be" fit better here?

Copy link
Copy Markdown
Contributor Author

@dlmarion dlmarion Apr 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Starting with 4.0 it will be, the user will have to run accumulo upgrade --start before they start the Manager.

after shutting down the instance with the old version of software and before starting
the instance with the new version of software. The `upgrade` command has two options,
`--prepare` and `--start`. `--prepare` is designed to be executed by the user after shutting
down the instance. This option will check that the Manager is down, validate that there
are no existing FATE transactions, remove all of the server locks in ZooKeeper, and place
a node in ZooKeeper that will prevent server processes from being started again. If there
Copy link
Copy Markdown
Member

@DomGarguilo DomGarguilo Apr 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
a node in ZooKeeper that will prevent server processes from being started again. If there
a node in ZooKeeper that will prevent server processes from being started again before the upgrade is complete. If there

Copy link
Copy Markdown
Contributor Author

@dlmarion dlmarion Apr 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The node created by --prepare just prevents the server processes from starting. If the user decides to abort the upgrade, then they can manually delete this node. These instructions are provided if there are fate transactions. I point this out to say that completing the upgrade is not necessarily the only way to start the server processes again.

are FATE transactions, the command will fail giving the user the opportunity to clean them
up. The `--prepare` option can then be run again.

The `--start` option is designed to be executed by the user before starting the instance
Copy link
Copy Markdown
Member

@DomGarguilo DomGarguilo Apr 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The `--start` option is designed to be executed by the user before starting the instance
The `--start` option is designed to be executed by the user before starting the instance (and after running the `--prepare` step)

I'm not too sure if this should be included since we say below that its not absolutely necessary for --prepare to be run

Copy link
Copy Markdown
Contributor Author

@dlmarion dlmarion Apr 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running --prepare with the old version of code is certainly preferred.

with the newer version of software. The `--start` option will perform any necessary pre-upgrade
validation, any changes that are necessary for the new version of software to start, seed
an upgrade progress tracker node in ZooKeeper, and then finally remove the node in ZooKeeper
created by the `--prepare` step so that the server processes can be started. If the user did
not run the `--prepare` step with the older version of software, then the `--start` option
will fail unless the `--force` option is used. Running `--start` with the `--force` option
will perform the same checks that `--prepare` executes, but if FATE transactions are found
then the user will need to remove them using the older version of software. Once the `--start`
option completes, then the user can start the server processes to complete the upgrade
process. The user will want to check the contents of the Manager log to observe the progress
of the upgrade.

The `accumulo upgrade --prepare` command will be included with the 2.1.4 and 3.1.0 releases
to assist users in upgrading to the 4.0 release.

## Upgrading from 1.10 or 2.0 to 2.1

Please read these directions in their entirety before beginning. Please [contact] us with any
Expand Down
Loading