Skip to content

Normalize the number of workers when performing parallel operations #9400

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Edouard-chin merged 2 commits into from
Mar 18, 2026
+13 −15
Merged

Normalize the number of workers when performing parallel operations #9400

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
170c9d7
Normalize the number of workers:
Edouard-chin Mar 16, 2026
36f572b
Update the `bundle config --help` man page:
Edouard-chin Mar 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion bundler/lib/bundler/cli/pristine.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ def run
true
end.map(&:name)

jobs = installer.send(:installation_parallelization)
jobs = Bundler.settings.installation_parallelization
pristine_count = definition.specs.count - installed_specs.count
# allow a pristining a single gem to skip the parallel worker
jobs = [jobs, pristine_count].min
Expand Down
4 changes: 3 additions & 1 deletion bundler/lib/bundler/definition.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1122,7 +1122,9 @@ def source_requirements
end

def preload_git_source_worker
@preload_git_source_worker ||= Bundler::Worker.new(5, "Git source preloading", ->(source, _) { source.specs })
workers = Bundler.settings.installation_parallelization

@preload_git_source_worker ||= Bundler::Worker.new(workers, "Git source preloading", ->(source, _) { source.specs })
end

def preload_git_sources
Expand Down
2 changes: 1 addition & 1 deletion bundler/lib/bundler/fetcher/gem_remote_fetcher.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ class GemRemoteFetcher < Gem::RemoteFetcher
def initialize(*)
super

@pool_size = 5
@pool_size = Bundler.settings.installation_parallelization
end

def request(*args)
Expand Down
10 changes: 1 addition & 9 deletions bundler/lib/bundler/installer.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -189,21 +189,13 @@ def install(options)
standalone = options[:standalone]
force = options[:force]
local = options[:local] || options[:"prefer-local"]
jobs = installation_parallelization
jobs = Bundler.settings.installation_parallelization
spec_installations = ParallelInstaller.call(self, @definition.specs, jobs, standalone, force, local: local)
spec_installations.each do |installation|
post_install_messages[installation.name] = installation.post_install_message if installation.has_post_install_message?
end
end

def installation_parallelization
if jobs = Bundler.settings[:jobs]
return jobs
end

Bundler.settings.processor_count
end

def load_plugins
Gem.load_plugins

Expand Down
2 changes: 1 addition & 1 deletion bundler/lib/bundler/man/bundle-config.1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ When set, no post install messages will be printed\. To silence a single gem, us
Generate a \fBgems\.rb\fR instead of a \fBGemfile\fR when running \fBbundle init\fR\.
.TP
\fBjobs\fR (\fBBUNDLE_JOBS\fR)
The number of gems Bundler can install in parallel\. Defaults to the number of available processors\.
The number of gems Bundler can download and install in parallel\. Defaults to the number of available processors\.
.TP
\fBlockfile\fR (\fBBUNDLE_LOCKFILE\fR)
The path to the lockfile that bundler should use\. By default, Bundler adds \fB\.lock\fR to the end of the \fBgemfile\fR entry\. Can be set to \fBfalse\fR in the Gemfile to disable lockfile creation entirely (see gemfile(5))\.
Expand Down
4 changes: 2 additions & 2 deletions bundler/lib/bundler/man/bundle-config.1.ronn
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,8 +192,8 @@ learn more about their operation in [bundle install(1)](bundle-install.1.html).
* `init_gems_rb` (`BUNDLE_INIT_GEMS_RB`):
Generate a `gems.rb` instead of a `Gemfile` when running `bundle init`.
* `jobs` (`BUNDLE_JOBS`):
The number of gems Bundler can install in parallel. Defaults to the number of
available processors.
The number of gems Bundler can download and install in parallel.
Defaults to the number of available processors.
* `lockfile` (`BUNDLE_LOCKFILE`):
The path to the lockfile that bundler should use. By default, Bundler adds
`.lock` to the end of the `gemfile` entry. Can be set to `false` in the
Expand Down
4 changes: 4 additions & 0 deletions bundler/lib/bundler/settings.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -303,6 +303,10 @@ def app_cache_path
@app_cache_path ||= self[:cache_path] || "vendor/cache"
end

def installation_parallelization
Copy link
Member

@colby-swandale colby-swandale Mar 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any documentation that we need to update to include this information?

Copy link
Collaborator Author

@Edouard-chin Edouard-chin Mar 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks good call. I updated the man page to reflect that BUNDLE_JOBS affects the number of parallel downloads, lmk what you think.

self[:jobs] || processor_count
end

def validate!
all.each do |raw_key|
[@local_config, @env_config, @global_config].each do |settings|
Expand Down
Loading