From a4d9c62f9d0465c51dd1069204d2c9c7c568ed61 Mon Sep 17 00:00:00 2001 From: Colin Casey Date: Thu, 29 Jan 2026 13:56:00 -0400 Subject: [PATCH 1/8] Layer cleanup to prevent runtime layer invalidation During automation runs for new Node.js runtimes, occasionally we'd see a PR opened where no new runtimes were added but one of the test snapshots had changed. The specific snapshot was always [the pnpm test for rebuilding a native module](https://github.com/heroku/buildpacks-nodejs/blob/cf9cb431fea999e30ac7528a93d3fe841b853f9c/tests/snapshots/test_pnpm_native_modules_are_recompiled_even_on_cache_restore.snap) and it would toggle back and forth between the following lines during a rebuild: - `Adding layer 'heroku/nodejs:virtual'` - `Reusing layer 'heroku/nodejs:virtual'` This indicated that files were present in this layer with content that was changing between runs in a way that could trigger the runtime layer to be invalidated and it was trigger by the `node-gyp` compilation. Both npm and Yarn have equivalent tests for native module recompilation, so I was curious if they were also affected. This led me to notice that the npm test snapshot reported `Adding layer 'heroku/nodejs:dist'` during a rebuild instead of `Reusing layer 'heroku/nodejs:dist'`. Yarn did not seem to be affected. So I spent some time analyzing the contents of these layers to see if I could track down what exactly was changing between builds. This involved: - adding debug output to the local code to print out the `sha256sum` of every file in the `/layers` directory - creating a test harness script to execute the native module rebuild tests for each package manager and collect the logs - creating an analysis script to extract the `(hash filename)` pairs from the raw logs and put them into a map of `(filename, [hash, hash, ...])` values where any entry that didn't contain hashes that were all the same would be reported The results of this analysis were, when a native module would be compiled, the layer containing the `node-gyp` binary (`dist` for bundled npm, `npm_engine` for installed version, `pnpm` for installed version) would contain `__pycache__` artifacts. E.g.; - `/layers/heroku_nodejs/dist/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/__pycache__/__init__.cpython-312.pyc` - ``/layers/heroku_nodejs/dist/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/__pycache__/common.cpython-312.pyc` - (and several others consistent across all these layers) And, for pnpm's `virtual` layer containing the modules installed to the virtual store, `node-gyp` will execute in the module's directory and is hardcoded to create a `build` folder. The file consistently triggering invalidation would be a `Makefile` like the following: - `/layers/heroku_nodejs/virtual/store/dtrace-provider@0.8.8/node_modules/dtrace-provider/build/Makefile` > [!NOTE] > Neither npm nor Yarn are impacted because those tools install modules directly into the workspace directory, not into a layer. This PR attempts to remove these specific build artifacts by collecting layers known to contain these files during the build process and then traversing those layer folders at the end of the build to remove the target files. This will be reported in buildpack output as: ``` - Cleaning up Node.js installation layer - Cleaning up npm installation layer - Removed 3 __pycache__ directories ``` Where the layer to be cleaned will always be reported. If any of the target files are detected, there will be extra output indicating how many of these files were removed. --- CHANGELOG.md | 4 + Cargo.lock | 1 + Cargo.toml | 1 + crates/test_support/src/snapshot_filters.rs | 9 +- src/context.rs | 36 ++++ src/layer_cleanup.rs | 179 ++++++++++++++++++ src/main.rs | 21 +- src/package_managers/npm.rs | 14 +- src/package_managers/pnpm.rs | 20 +- src/package_managers/yarn.rs | 5 +- src/runtimes/nodejs.rs | 7 + src/utils/npm_registry.rs | 6 +- tests/npm_integration_test.rs | 21 ++ tests/snapshots/node_24.snap | 1 + tests/snapshots/node_25.snap | 1 + tests/snapshots/npm_engine_install.snap | 2 + .../npm_package_manager_install.snap | 2 + tests/snapshots/pnpm_7_pnp.snap | 5 +- tests/snapshots/pnpm_8_hoist.snap | 5 +- tests/snapshots/pnpm_8_nuxt.snap | 5 +- tests/snapshots/pnpm_install_engine.snap | 5 +- .../pnpm_install_package_manager.snap | 5 +- .../reinstalls_node_if_version_changes.snap | 2 + tests/snapshots/simple_indexjs.snap | 1 + tests/snapshots/simple_serverjs.snap | 1 + ...n_change_invalidates_npm_engine_cache.snap | 4 + tests/snapshots/test_npm_build_scripts.snap | 1 + ...ripts_prefers_heroku_build_over_build.snap | 1 + ...tration_is_skipped_if_procfile_exists.snap | 1 + tests/snapshots/test_npm_engine_caching.snap | 4 + ..._are_recompiled_even_on_cache_restore.snap | 171 +++++++++++++++++ tests/snapshots/test_npm_install_caching.snap | 2 + .../test_npm_install_new_package.snap | 2 + .../test_npm_install_with_lockfile.snap | 1 + ..._are_recompiled_even_on_cache_restore.snap | 7 +- ...est_npm_prune_dev_dependencies_config.snap | 1 + ...npm_skip_build_scripts_from_buildplan.snap | 1 + ...script_creates_a_web_process_launcher.snap | 1 + ...n_change_invalidates_npm_engine_cache.snap | 4 + ...tration_is_skipped_if_procfile_exists.snap | 5 +- ..._are_recompiled_even_on_cache_restore.snap | 17 +- ...st_pnpm_prune_dev_dependencies_config.snap | 5 +- ...npm_skip_build_scripts_from_buildplan.snap | 5 +- ...tration_is_skipped_if_procfile_exists.snap | 1 + ..._are_recompiled_even_on_cache_restore.snap | 2 + ...st_yarn_prune_dev_dependencies_config.snap | 1 + ...arn_skip_build_scripts_from_buildplan.snap | 1 + tests/snapshots/yarn_1_typescript.snap | 1 + tests/snapshots/yarn_2_modules_nonzero.snap | 1 + tests/snapshots/yarn_2_pnp_zero.snap | 1 + tests/snapshots/yarn_3_modules_zero.snap | 1 + tests/snapshots/yarn_3_pnp_nonzero.snap | 1 + tests/snapshots/yarn_4_modules_zero.snap | 1 + tests/snapshots/yarn_4_pnp_nonzero.snap | 1 + 54 files changed, 575 insertions(+), 29 deletions(-) create mode 100644 src/context.rs create mode 100644 src/layer_cleanup.rs create mode 100644 tests/snapshots/test_npm_engine_native_modules_are_recompiled_even_on_cache_restore.snap diff --git a/CHANGELOG.md b/CHANGELOG.md index 2670ba466..81d671345 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- Cleanup of non-deterministic files in Node.js, npm, and pnpm layers to prevent unnecessary invalidation of runtime layers for export. ([#1274](https://github.com/heroku/buildpacks-nodejs/pull/1274)) + ## [5.3.5] - 2026-01-27 ### Added diff --git a/Cargo.lock b/Cargo.lock index f95a16aee..da59f4fb7 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2270,6 +2270,7 @@ dependencies = [ "toml", "toml_edit", "tracing", + "walkdir", "wiremock", "yaml-rust2", ] diff --git a/Cargo.toml b/Cargo.toml index 490eebf3d..f9e7b845d 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -35,6 +35,7 @@ toml = "0.9" toml_edit = "0.23" tracing = "0.1" yaml-rust2 = "0.10" +walkdir = "2.5.0" [dev-dependencies] insta = "1" diff --git a/crates/test_support/src/snapshot_filters.rs b/crates/test_support/src/snapshot_filters.rs index 8503f8eb1..aba6088e1 100644 --- a/crates/test_support/src/snapshot_filters.rs +++ b/crates/test_support/src/snapshot_filters.rs @@ -328,10 +328,11 @@ pub(super) fn create_snapshot_filters() -> Vec<(String, String)> { // TODO: investigate why the `...native_modules_are_recompiled...`-based tests (mostly the pnpm version) // occassionally switch from Adding layer 'heroku/nodejs:xyz' to Reusing layer 'heroku/nodejs:xyz' - filters.push(( - r"(?:Adding|Reusing) layer 'heroku/nodejs:virtual", - " layer 'heroku/nodejs:virtual", - )); + // TEMPORARILY DISABLED FOR INVESTIGATION + // filters.push(( + // r"(?:Adding|Reusing) layer 'heroku/nodejs:virtual", + // " layer 'heroku/nodejs:virtual", + // )); filters .into_iter() diff --git a/src/context.rs b/src/context.rs new file mode 100644 index 000000000..1fb2fe841 --- /dev/null +++ b/src/context.rs @@ -0,0 +1,36 @@ +use crate::NodeJsBuildpack; +use crate::layer_cleanup::LayerCleanupTarget; +use std::cell::RefCell; +use std::ops::Deref; + +/// Wrapper around libcnb `BuildContext` that tracks layers needing cleanup +/// of non-deterministic build artifacts (Python bytecode, Makefiles) +pub(crate) struct NodeJsBuildContext { + inner: libcnb::build::BuildContext, + cleanup_registry: RefCell>, +} + +impl NodeJsBuildContext { + pub(crate) fn new(inner: libcnb::build::BuildContext) -> Self { + Self { + inner, + cleanup_registry: RefCell::new(Vec::new()), + } + } + + pub(crate) fn register_layer_for_cleanup(&self, target: LayerCleanupTarget) { + self.cleanup_registry.borrow_mut().push(target); + } + + pub(crate) fn layers_to_cleanup(&self) -> Vec { + self.cleanup_registry.borrow().iter().cloned().collect() + } +} + +impl Deref for NodeJsBuildContext { + type Target = libcnb::build::BuildContext; + + fn deref(&self) -> &Self::Target { + &self.inner + } +} diff --git a/src/layer_cleanup.rs b/src/layer_cleanup.rs new file mode 100644 index 000000000..ff85ffac1 --- /dev/null +++ b/src/layer_cleanup.rs @@ -0,0 +1,179 @@ +use bullet_stream::global::print; +use std::fs; +use std::path::{Path, PathBuf}; +use walkdir::WalkDir; + +#[derive(Debug, Clone)] +pub(crate) enum LayerKind { + /// Node.js distribution layer (contains npm with node-gyp) + Dist, + /// Custom npm version layer (contains npm with node-gyp) + NpmEngine, + /// pnpm distribution layer (contains pnpm with node-gyp) + Pnpm, + /// pnpm virtual store layer (contains native module builds with Makefiles) + Virtual, +} + +#[derive(Debug, Clone)] +pub(crate) struct LayerCleanupTarget { + pub(crate) path: PathBuf, + pub(crate) kind: LayerKind, +} + +/// Remove Python bytecode cache files (__pycache__/*.pyc) from node-gyp directories +/// These files are generated during native module compilation and are non-deterministic +fn remove_python_bytecode_cache(base_path: &Path) -> Result { + let mut removed_count = 0; + + // Look for node-gyp/gyp/pylib directory containing Python source + let pylib_path = base_path.join("node_modules/node-gyp/gyp/pylib"); + + if !pylib_path.exists() { + return Ok(0); + } + + // Walk the pylib tree looking for __pycache__ directories + for entry in WalkDir::new(&pylib_path) + .into_iter() + .filter_map(Result::ok) + .filter(|e| e.file_type().is_dir() && e.file_name() == "__pycache__") + { + // Remove the entire __pycache__ directory + fs::remove_dir_all(entry.path())?; + removed_count += 1; + } + + Ok(removed_count) +} + +/// Remove Makefile and *.mk files from native module build directories +/// These files have non-deterministic dependency ordering causing layer invalidation +fn remove_build_makefiles(base_path: &Path) -> Result { + let mut removed_count = 0; + + // Walk directory tree looking for build/Makefile patterns + for entry in WalkDir::new(base_path) + .into_iter() + .filter_map(Result::ok) + .filter(|e| { + if !e.file_type().is_file() { + return false; + } + + // Check if this is a Makefile or .mk file in a build/ directory + let path = e.path(); + if let Some(parent) = path.parent() + && parent.file_name() == Some(std::ffi::OsStr::new("build")) + && let Some(filename) = path.file_name() + { + return filename.to_string_lossy() == "Makefile"; + } + + false + }) + { + fs::remove_file(entry.path())?; + removed_count += 1; + } + + Ok(removed_count) +} + +/// Clean up non-deterministic build artifacts from a layer +pub(crate) fn cleanup_layer(target: &LayerCleanupTarget) -> Result<(), std::io::Error> { + let path = &target.path; + + if !path.exists() { + // Layer doesn't exist, nothing to clean + return Ok(()); + } + + match target.kind { + LayerKind::Dist => { + // npm dist layer: contains bundled npm with node-gyp + // Clean Python bytecode from: dist/lib/node_modules/npm/node_modules/node-gyp/ + print::bullet("Cleaning up Node.js installation layer"); + let npm_modules = path.join("lib/node_modules/npm"); + if npm_modules.exists() { + let removed = remove_python_bytecode_cache(&npm_modules)?; + if removed > 0 { + print::sub_bullet(format!("Removed {removed} __pycache__ directories")); + } + } + } + LayerKind::NpmEngine => { + // npm_engine layer: custom npm version installed via npm registry + // Clean Python bytecode from: npm_engine/node_modules/node-gyp/ + print::bullet("Cleaning up npm installation layer"); + let removed = remove_python_bytecode_cache(path)?; + if removed > 0 { + print::sub_bullet(format!("Removed {removed} __pycache__ directories")); + } + } + LayerKind::Pnpm => { + // pnpm layer: contains pnpm distribution with node-gyp + // Clean Python bytecode from: pnpm/dist/node_modules/node-gyp/ + print::bullet("Cleaning up pnpm installation layer"); + let pnpm_dist = path.join("dist"); + if pnpm_dist.exists() { + let removed = remove_python_bytecode_cache(&pnpm_dist)?; + if removed > 0 { + print::sub_bullet(format!("Removed {removed} __pycache__ directories")); + } + } + } + LayerKind::Virtual => { + // pnpm virtual store: contains symlinked packages with native module builds + // Clean Makefiles from: virtual/store/*/node_modules/*/build/ + print::bullet("Cleaning up pnpm virtual store layer"); + let removed = remove_build_makefiles(path)?; + if removed > 0 { + print::sub_bullet(format!("Removed {removed} Makefile artifacts")); + } + } + } + + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::fs; + use tempfile::TempDir; + + #[test] + fn test_remove_python_bytecode_cache() { + let temp = TempDir::new().unwrap(); + let base = temp.path(); + + // Create directory structure with __pycache__ + let pycache_path = base.join("node_modules/node-gyp/gyp/pylib/gyp/__pycache__"); + fs::create_dir_all(&pycache_path).unwrap(); + fs::write(pycache_path.join("test.cpython-312.pyc"), b"bytecode").unwrap(); + + let removed = remove_python_bytecode_cache(base).unwrap(); + + assert_eq!(removed, 1); + assert!(!pycache_path.exists()); + } + + #[test] + fn test_remove_build_makefiles() { + let temp = TempDir::new().unwrap(); + let base = temp.path(); + + // Create build directory with Makefile + let build_dir = base.join("node_modules/some-package/build"); + fs::create_dir_all(&build_dir).unwrap(); + fs::write(build_dir.join("Makefile"), b"makefile content").unwrap(); + fs::write(build_dir.join("output.o"), b"binary").unwrap(); // Should not be removed + + let removed = remove_build_makefiles(base).unwrap(); + + assert_eq!(removed, 1); // Makefile + assert!(!build_dir.join("Makefile").exists()); + assert!(build_dir.join("output.o").exists()); // Not a makefile + } +} diff --git a/src/main.rs b/src/main.rs index 5e5f91fb6..4667b4cda 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1,4 +1,6 @@ use crate::buildpack_config::{ConfigValue, ConfigValueSource}; +use crate::context::NodeJsBuildContext; +use crate::layer_cleanup::cleanup_layer; use crate::o11y::*; use crate::package_manager::InstalledPackageManager; use crate::utils::error_handling::{ErrorMessage, on_framework_error}; @@ -16,6 +18,8 @@ use libcnb_test as _; use toml::Table; mod buildpack_config; +mod context; +mod layer_cleanup; mod o11y; mod package_json; mod package_manager; @@ -25,7 +29,7 @@ mod runtimes; mod utils; type BuildpackDetectContext = libcnb::detect::DetectContext; -type BuildpackBuildContext = libcnb::build::BuildContext; +type BuildpackBuildContext = NodeJsBuildContext; type BuildpackError = libcnb::Error; type BuildpackResult = Result; @@ -68,8 +72,11 @@ impl libcnb::Buildpack for NodeJsBuildpack { #[allow(clippy::too_many_lines)] fn build( &self, - context: BuildpackBuildContext, + context: libcnb::build::BuildContext, ) -> libcnb::Result { + // Wrap the context to track layers needing cleanup + let context = NodeJsBuildContext::new(context); + let buildpack_start = print::buildpack( context .buildpack_descriptor @@ -215,6 +222,16 @@ impl libcnb::Buildpack for NodeJsBuildpack { } } + // Clean up non-deterministic build artifacts from registered layers + let layers_to_cleanup = context.layers_to_cleanup(); + if !layers_to_cleanup.is_empty() { + for layer_to_cleanup in layers_to_cleanup { + if let Err(e) = cleanup_layer(&layer_to_cleanup) { + print::sub_bullet(format!("- Error during cleanup: {e}")); + } + } + } + print::all_done(&Some(buildpack_start)); build_result_builder.store(store).build() diff --git a/src/package_managers/npm.rs b/src/package_managers/npm.rs index f27efd130..f1debc7ce 100644 --- a/src/package_managers/npm.rs +++ b/src/package_managers/npm.rs @@ -1,3 +1,4 @@ +use crate::layer_cleanup::{LayerCleanupTarget, LayerKind}; use crate::utils::error_handling::ErrorType::Internal; use crate::utils::error_handling::{ ErrorMessage, ErrorType, SuggestRetryBuild, SuggestSubmitIssue, error_message, @@ -69,14 +70,21 @@ pub(crate) fn install_npm( npm_packument: &npm_registry::PackagePackument, node_version: &Version, ) -> BuildpackResult<()> { - npm_registry::install_package_layer( + let npm_engine_layer_path = npm_registry::install_package_layer( layer_name!("npm_engine"), context, env, npm_packument, node_version, - ) - .map_err(Into::into) + )?; + + // Register npm_engine layer for cleanup of non-deterministic Python bytecode + context.register_layer_for_cleanup(LayerCleanupTarget { + path: npm_engine_layer_path, + kind: LayerKind::NpmEngine, + }); + + Ok(()) } pub(crate) fn install_npm_dependencies( diff --git a/src/package_managers/pnpm.rs b/src/package_managers/pnpm.rs index b56e4e399..7b6862ca3 100644 --- a/src/package_managers/pnpm.rs +++ b/src/package_managers/pnpm.rs @@ -1,3 +1,4 @@ +use crate::layer_cleanup::{LayerCleanupTarget, LayerKind}; use crate::utils::error_handling::{ ErrorMessage, ErrorType, SuggestRetryBuild, SuggestSubmitIssue, error_message, file_value, }; @@ -38,14 +39,21 @@ pub(crate) fn install_pnpm( pnpm_packument: &PackagePackument, node_version: &Version, ) -> BuildpackResult<()> { - utils::npm_registry::install_package_layer( + let pnpm_layer_path = utils::npm_registry::install_package_layer( layer_name!("pnpm"), context, env, pnpm_packument, node_version, - ) - .map_err(Into::into) + )?; + + // Register pnpm layer for cleanup of non-deterministic Python bytecode + context.register_layer_for_cleanup(LayerCleanupTarget { + path: pnpm_layer_path, + kind: LayerKind::Pnpm, + }); + + Ok(()) } pub(crate) fn install_dependencies( @@ -182,6 +190,12 @@ fn create_virtual_store_directory(context: &BuildpackBuildContext) -> BuildpackR Err(create_node_modules_symlink_error(&error))?; } + // Register virtual layer for cleanup of non-deterministic Makefiles + context.register_layer_for_cleanup(LayerCleanupTarget { + path: pnpm_virtual_store_layer.path().clone(), + kind: LayerKind::Virtual, + }); + Ok(virtual_store_dir) } diff --git a/src/package_managers/yarn.rs b/src/package_managers/yarn.rs index 2af27aa80..5b632b3d6 100644 --- a/src/package_managers/yarn.rs +++ b/src/package_managers/yarn.rs @@ -57,14 +57,15 @@ pub(crate) fn install_yarn( yarn_packument: &PackagePackument, node_version: &Version, ) -> BuildpackResult<()> { + // Note: yarn layer path is returned but not used for cleanup registration utils::npm_registry::install_package_layer( layer_name!("yarn"), context, env, yarn_packument, node_version, - ) - .map_err(Into::into) + )?; + Ok(()) } #[instrument(skip_all)] diff --git a/src/runtimes/nodejs.rs b/src/runtimes/nodejs.rs index 2bcea9444..75e8b2413 100644 --- a/src/runtimes/nodejs.rs +++ b/src/runtimes/nodejs.rs @@ -1,3 +1,4 @@ +use crate::layer_cleanup::{LayerCleanupTarget, LayerKind}; use crate::utils::error_handling::ErrorType::{Internal, UserFacing}; use crate::utils::error_handling::{ ErrorMessage, SuggestRetryBuild, SuggestSubmitIssue, error_message, file_value, @@ -110,6 +111,12 @@ pub(crate) fn install( env.clone_from(&distribution_layer.read_env()?.apply(Scope::Build, env)); + // Register dist layer for cleanup of non-deterministic Python bytecode + context.register_layer_for_cleanup(LayerCleanupTarget { + path: distribution_layer.path().clone(), + kind: LayerKind::Dist, + }); + Ok(()) } diff --git a/src/utils/npm_registry.rs b/src/utils/npm_registry.rs index 8cf4ba857..3c80624f7 100644 --- a/src/utils/npm_registry.rs +++ b/src/utils/npm_registry.rs @@ -20,7 +20,7 @@ use libcnb::layer_env::Scope; use serde::{Deserialize, Serialize}; use std::collections::HashMap; use std::fs::File; -use std::path::Path; +use std::path::{Path, PathBuf}; use std::{fs, io}; const NPMJS_ORG_HOST: &str = "https://registry.npmjs.org"; @@ -335,7 +335,7 @@ pub(crate) fn install_package_layer( env: &mut Env, package_packument: &PackagePackument, node_version: &Version, -) -> Result<(), InstallPackageLayerError> { +) -> Result { let package_name = &package_packument.name; let package_version = &package_packument.version; @@ -433,7 +433,7 @@ pub(crate) fn install_package_layer( env.clone_from(&layer_env.apply(Scope::Build, env)); - Ok(()) + Ok(install_package_layer.path().clone()) } pub(crate) enum InstallPackageLayerError { diff --git a/tests/npm_integration_test.rs b/tests/npm_integration_test.rs index 1300de087..dd885b22b 100644 --- a/tests/npm_integration_test.rs +++ b/tests/npm_integration_test.rs @@ -231,6 +231,27 @@ fn test_npm_native_modules_are_recompiled_even_on_cache_restore() { ); } +#[test] +#[ignore = "integration test"] +fn test_npm_engine_native_modules_are_recompiled_even_on_cache_restore() { + nodejs_integration_test_with_config( + "./fixtures/npm-project-with-native-module", + |config| { + config.env("npm_config_foreground-scripts", "true"); + config.app_dir_preprocessor(|app_dir| { + set_npm_engine(&app_dir, "11.x"); + }); + }, + |ctx| { + let build_snapshot = create_build_snapshot(&ctx.pack_stdout); + let config = ctx.config.clone(); + ctx.rebuild(config, |ctx| { + build_snapshot.rebuild_output(&ctx.pack_stdout).assert(); + }); + }, + ); +} + #[test] #[ignore = "integration test"] fn test_npm_skip_build_scripts_from_buildplan() { diff --git a/tests/snapshots/node_24.snap b/tests/snapshots/node_24.snap index fce2142e2..d3bf5a40d 100644 --- a/tests/snapshots/node_24.snap +++ b/tests/snapshots/node_24.snap @@ -21,6 +21,7 @@ heroku/nodejs - Verifying checksum - Extracting Node.js `24.13.0 ()` - Installing Node.js `24.13.0 ()` ... () +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/node_25.snap b/tests/snapshots/node_25.snap index 9defe317b..0f06c22de 100644 --- a/tests/snapshots/node_25.snap +++ b/tests/snapshots/node_25.snap @@ -21,6 +21,7 @@ heroku/nodejs - Verifying checksum - Extracting Node.js `25.5.0 ()` - Installing Node.js `25.5.0 ()` ... () +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/npm_engine_install.snap b/tests/snapshots/npm_engine_install.snap index cd3a1dd95..8b00d5c30 100644 --- a/tests/snapshots/npm_engine_install.snap +++ b/tests/snapshots/npm_engine_install.snap @@ -66,6 +66,8 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer +- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/npm_package_manager_install.snap b/tests/snapshots/npm_package_manager_install.snap index 9e02fa463..85207ea31 100644 --- a/tests/snapshots/npm_package_manager_install.snap +++ b/tests/snapshots/npm_package_manager_install.snap @@ -54,6 +54,8 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer +- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/pnpm_7_pnp.snap b/tests/snapshots/pnpm_7_pnp.snap index 56a5d3bd2..f6231c95a 100644 --- a/tests/snapshots/pnpm_7_pnp.snap +++ b/tests/snapshots/pnpm_7_pnp.snap @@ -64,12 +64,15 @@ heroku/nodejs devDependencies: skipped - Done () +- Cleaning up Node.js installation layer +- Cleaning up pnpm installation layer +- Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' Adding layer 'heroku/nodejs:dist' Adding layer 'heroku/nodejs:pnpm' - layer 'heroku/nodejs:virtual' +Adding layer 'heroku/nodejs:virtual' Adding layer 'heroku/nodejs:web_env' Adding layer 'buildpacksio/lifecycle:launch.sbom' Added 1/1 app layer(s) diff --git a/tests/snapshots/pnpm_8_hoist.snap b/tests/snapshots/pnpm_8_hoist.snap index 7f92cc798..3514d29af 100644 --- a/tests/snapshots/pnpm_8_hoist.snap +++ b/tests/snapshots/pnpm_8_hoist.snap @@ -67,12 +67,15 @@ heroku/nodejs devDependencies: skipped - Done () +- Cleaning up Node.js installation layer +- Cleaning up pnpm installation layer +- Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' Adding layer 'heroku/nodejs:dist' Adding layer 'heroku/nodejs:pnpm' - layer 'heroku/nodejs:virtual' +Adding layer 'heroku/nodejs:virtual' Adding layer 'heroku/nodejs:web_env' Adding layer 'buildpacksio/lifecycle:launch.sbom' Added 1/1 app layer(s) diff --git a/tests/snapshots/pnpm_8_nuxt.snap b/tests/snapshots/pnpm_8_nuxt.snap index 94d27eff5..b3d0f9555 100644 --- a/tests/snapshots/pnpm_8_nuxt.snap +++ b/tests/snapshots/pnpm_8_nuxt.snap @@ -94,12 +94,15 @@ heroku/nodejs ! ! Since pruning can't be done safely for your build, it will be skipped. To fix this you must upgrade your version of pnpm to 8.15.6 or higher. +- Cleaning up Node.js installation layer +- Cleaning up pnpm installation layer +- Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' Adding layer 'heroku/nodejs:dist' Adding layer 'heroku/nodejs:pnpm' - layer 'heroku/nodejs:virtual' +Adding layer 'heroku/nodejs:virtual' Adding layer 'heroku/nodejs:web_env' Adding layer 'buildpacksio/lifecycle:launch.sbom' Added 1/1 app layer(s) diff --git a/tests/snapshots/pnpm_install_engine.snap b/tests/snapshots/pnpm_install_engine.snap index c3c4ba3a1..b6c9f805b 100644 --- a/tests/snapshots/pnpm_install_engine.snap +++ b/tests/snapshots/pnpm_install_engine.snap @@ -67,12 +67,15 @@ heroku/nodejs devDependencies: skipped - Done () +- Cleaning up Node.js installation layer +- Cleaning up pnpm installation layer +- Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' Adding layer 'heroku/nodejs:dist' Adding layer 'heroku/nodejs:pnpm' - layer 'heroku/nodejs:virtual' +Adding layer 'heroku/nodejs:virtual' Adding layer 'heroku/nodejs:web_env' Adding layer 'buildpacksio/lifecycle:launch.sbom' Added 1/1 app layer(s) diff --git a/tests/snapshots/pnpm_install_package_manager.snap b/tests/snapshots/pnpm_install_package_manager.snap index 7cdfdd2a9..cfa9d75af 100644 --- a/tests/snapshots/pnpm_install_package_manager.snap +++ b/tests/snapshots/pnpm_install_package_manager.snap @@ -67,12 +67,15 @@ heroku/nodejs devDependencies: skipped - Done () +- Cleaning up Node.js installation layer +- Cleaning up pnpm installation layer +- Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' Adding layer 'heroku/nodejs:dist' Adding layer 'heroku/nodejs:pnpm' - layer 'heroku/nodejs:virtual' +Adding layer 'heroku/nodejs:virtual' Adding layer 'heroku/nodejs:web_env' Adding layer 'buildpacksio/lifecycle:launch.sbom' Added 1/1 app layer(s) diff --git a/tests/snapshots/reinstalls_node_if_version_changes.snap b/tests/snapshots/reinstalls_node_if_version_changes.snap index 662f65960..7b00355e3 100644 --- a/tests/snapshots/reinstalls_node_if_version_changes.snap +++ b/tests/snapshots/reinstalls_node_if_version_changes.snap @@ -21,6 +21,7 @@ heroku/nodejs - Verifying checksum - Extracting Node.js `14.21.3 ()` - Installing Node.js `14.21.3 ()` ... () +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -62,6 +63,7 @@ heroku/nodejs - Verifying checksum - Extracting Node.js `16.20.2 ()` - Installing Node.js `16.20.2 ()` ... () +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/simple_indexjs.snap b/tests/snapshots/simple_indexjs.snap index 89bd120ae..8ee2bb1cd 100644 --- a/tests/snapshots/simple_indexjs.snap +++ b/tests/snapshots/simple_indexjs.snap @@ -21,6 +21,7 @@ heroku/nodejs - Verifying checksum - Extracting Node.js `24.13.0 ()` - Installing Node.js `24.13.0 ()` ... () +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/simple_serverjs.snap b/tests/snapshots/simple_serverjs.snap index 2b1e16dec..ef9d406ae 100644 --- a/tests/snapshots/simple_serverjs.snap +++ b/tests/snapshots/simple_serverjs.snap @@ -21,6 +21,7 @@ heroku/nodejs - Verifying checksum - Extracting Node.js `16.0.0 ()` - Installing Node.js `16.0.0 ()` ... () +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_node_version_change_invalidates_npm_engine_cache.snap b/tests/snapshots/test_node_version_change_invalidates_npm_engine_cache.snap index 94b5a8f41..906daa8e7 100644 --- a/tests/snapshots/test_node_version_change_invalidates_npm_engine_cache.snap +++ b/tests/snapshots/test_node_version_change_invalidates_npm_engine_cache.snap @@ -66,6 +66,8 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer +- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -158,6 +160,8 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer +- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_build_scripts.snap b/tests/snapshots/test_npm_build_scripts.snap index 812e432db..8e9b9a94c 100644 --- a/tests/snapshots/test_npm_build_scripts.snap +++ b/tests/snapshots/test_npm_build_scripts.snap @@ -72,6 +72,7 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_build_scripts_prefers_heroku_build_over_build.snap b/tests/snapshots/test_npm_build_scripts_prefers_heroku_build_over_build.snap index ce915bb6c..60fed1915 100644 --- a/tests/snapshots/test_npm_build_scripts_prefers_heroku_build_over_build.snap +++ b/tests/snapshots/test_npm_build_scripts_prefers_heroku_build_over_build.snap @@ -54,6 +54,7 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_default_web_process_registration_is_skipped_if_procfile_exists.snap b/tests/snapshots/test_npm_default_web_process_registration_is_skipped_if_procfile_exists.snap index 111acde10..07fbc101e 100644 --- a/tests/snapshots/test_npm_default_web_process_registration_is_skipped_if_procfile_exists.snap +++ b/tests/snapshots/test_npm_default_web_process_registration_is_skipped_if_procfile_exists.snap @@ -46,6 +46,7 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (Procfile detected) +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_engine_caching.snap b/tests/snapshots/test_npm_engine_caching.snap index 46ca0cac9..dc460551c 100644 --- a/tests/snapshots/test_npm_engine_caching.snap +++ b/tests/snapshots/test_npm_engine_caching.snap @@ -66,6 +66,8 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer +- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -151,6 +153,8 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer +- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_engine_native_modules_are_recompiled_even_on_cache_restore.snap b/tests/snapshots/test_npm_engine_native_modules_are_recompiled_even_on_cache_restore.snap new file mode 100644 index 000000000..149c66181 --- /dev/null +++ b/tests/snapshots/test_npm_engine_native_modules_are_recompiled_even_on_cache_restore.snap @@ -0,0 +1,171 @@ +--- +source: crates/test_support/src/lib.rs +--- +===> ANALYZING +Image with name "" not found +===> DETECTING +heroku/nodejs +===> RESTORING + +===> BUILDING + +## Heroku Node.js + +- Checking Node.js version + - Detected Node.js version range: `^20.0` + - Resolved Node.js version: `20.20.0` +- Installing Node.js distribution + - GET https://nodejs.org/download/release/v20.20.0/node-v20.20.0-.tar.gz ... () + - Validating ... () + - Extracting ... () + - Verifying checksum + - Extracting Node.js `20.20.0 ()` + - Installing Node.js `20.20.0 ()` ... () +- Determining npm package information + - Found `engines.npm` version `11.x` declared in `package.json` + - GET https://registry.npmjs.org/npm ... () + - Resolved npm version `11.x` to `11.8.0` +- Installing npm + - GET https://registry.npmjs.org/npm/-/npm-11.8.0.tgz ... () + - Extracting ... () + - Successfully installed `npm@11.8.0` +- Installing node modules + - Using npm version `11.8.0` + - Creating npm cache + - Configuring npm cache directory + - Running `npm ci` + + + > dtrace-provider@0.8.8 install + > node-gyp rebuild || node suppress-error.js + + + + added packages, and audited packages in + + found 0 vulnerabilities + + - Done () +- Running scripts + - No build scripts found +- Pruning dev dependencies + - Running `npm prune` + + + up to date, audited packages in + + found 0 vulnerabilities + + - Done () +- Configuring default processes + - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer +- Cleaning up npm installation layer + - Removed 3 __pycache__ directories +- Done (finished in ) +===> EXPORTING +Adding layer 'heroku/nodejs:available_parallelism' +Adding layer 'heroku/nodejs:dist' +Adding layer 'heroku/nodejs:npm_engine' +Adding layer 'heroku/nodejs:npm_runtime_config' +Adding layer 'heroku/nodejs:web_env' +Adding layer 'buildpacksio/lifecycle:launch.sbom' +Added 1/1 app layer(s) +Adding layer 'buildpacksio/lifecycle:launcher' +Adding layer 'buildpacksio/lifecycle:config' +Adding label 'io.buildpacks.lifecycle.metadata' +Adding label 'io.buildpacks.build.metadata' +Adding label 'io.buildpacks.project.metadata' +no default process type +Saving ... +*** Images (): + +Adding cache layer 'heroku/nodejs:dist' +Adding cache layer 'heroku/nodejs:npm_cache' +Adding cache layer 'heroku/nodejs:npm_engine' +Adding cache layer 'heroku/nodejs:npm_packument' +Successfully built image '' + +--------------------------------------------- REBUILD --------------------------------------------- +===> ANALYZING +Restoring data for SBOM from previous image +===> DETECTING +heroku/nodejs +===> RESTORING + +===> BUILDING + +## Heroku Node.js + +- Checking Node.js version + - Detected Node.js version range: `^20.0` + - Resolved Node.js version: `20.20.0` +- Installing Node.js distribution + - Reusing Node.js 20.20.0 () +- Determining npm package information + - Found `engines.npm` version `11.x` declared in `package.json` + - GET https://registry.npmjs.org/npm ... (Not Modified) + - Using cached packument for npm + - Resolved npm version `11.x` to `11.8.0` +- Installing npm + - Using cached version of npm + - Successfully installed `npm@11.8.0` +- Installing node modules + - Using npm version `11.8.0` + - Restoring npm cache + - Configuring npm cache directory + - Running `npm ci` + + + > dtrace-provider@0.8.8 install + > node-gyp rebuild || node suppress-error.js + + + + added packages, and audited packages in + + found 0 vulnerabilities + + - Done () +- Running scripts + - No build scripts found +- Pruning dev dependencies + - Running `npm prune` + + + up to date, audited packages in + + found 0 vulnerabilities + + - Done () +- Configuring default processes + - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer +- Cleaning up npm installation layer + - Removed 3 __pycache__ directories +- Done (finished in ) +===> EXPORTING +Reusing layer 'heroku/nodejs:available_parallelism' +Reusing layer 'heroku/nodejs:dist' +Reusing layer 'heroku/nodejs:npm_engine' +Reusing layer 'heroku/nodejs:npm_runtime_config' +Reusing layer 'heroku/nodejs:web_env' +Reusing layer 'buildpacksio/lifecycle:launch.sbom' +Reused 1/1 app layer(s) +Reusing layer 'buildpacksio/lifecycle:launcher' +Reusing layer 'buildpacksio/lifecycle:config' +Adding label 'io.buildpacks.lifecycle.metadata' +Adding label 'io.buildpacks.build.metadata' +Adding label 'io.buildpacks.project.metadata' +no default process type +Saving ... +*** Images (): + +Reusing cache layer 'heroku/nodejs:dist' +Adding cache layer 'heroku/nodejs:dist' +Adding cache layer 'heroku/nodejs:npm_cache' +Reusing cache layer 'heroku/nodejs:npm_engine' +Adding cache layer 'heroku/nodejs:npm_engine' +Reusing cache layer 'heroku/nodejs:npm_packument' +Adding cache layer 'heroku/nodejs:npm_packument' +Successfully built image '' diff --git a/tests/snapshots/test_npm_install_caching.snap b/tests/snapshots/test_npm_install_caching.snap index 3ec7e45de..67779370b 100644 --- a/tests/snapshots/test_npm_install_caching.snap +++ b/tests/snapshots/test_npm_install_caching.snap @@ -46,6 +46,7 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -108,6 +109,7 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_install_new_package.snap b/tests/snapshots/test_npm_install_new_package.snap index ede98be9e..3d6043233 100644 --- a/tests/snapshots/test_npm_install_new_package.snap +++ b/tests/snapshots/test_npm_install_new_package.snap @@ -46,6 +46,7 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -108,6 +109,7 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_install_with_lockfile.snap b/tests/snapshots/test_npm_install_with_lockfile.snap index 8003eb4a5..bcdecc770 100644 --- a/tests/snapshots/test_npm_install_with_lockfile.snap +++ b/tests/snapshots/test_npm_install_with_lockfile.snap @@ -46,6 +46,7 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_native_modules_are_recompiled_even_on_cache_restore.snap b/tests/snapshots/test_npm_native_modules_are_recompiled_even_on_cache_restore.snap index 683e94951..dc28edf07 100644 --- a/tests/snapshots/test_npm_native_modules_are_recompiled_even_on_cache_restore.snap +++ b/tests/snapshots/test_npm_native_modules_are_recompiled_even_on_cache_restore.snap @@ -51,6 +51,8 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer + - Removed 3 __pycache__ directories - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -118,10 +120,12 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer + - Removed 3 __pycache__ directories - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' -Adding layer 'heroku/nodejs:dist' +Reusing layer 'heroku/nodejs:dist' Reusing layer 'heroku/nodejs:npm_runtime_config' Reusing layer 'heroku/nodejs:web_env' Reusing layer 'buildpacksio/lifecycle:launch.sbom' @@ -135,6 +139,7 @@ no default process type Saving ... *** Images (): +Reusing cache layer 'heroku/nodejs:dist' Adding cache layer 'heroku/nodejs:dist' Adding cache layer 'heroku/nodejs:npm_cache' Successfully built image '' diff --git a/tests/snapshots/test_npm_prune_dev_dependencies_config.snap b/tests/snapshots/test_npm_prune_dev_dependencies_config.snap index 14dd9f229..0246b90f1 100644 --- a/tests/snapshots/test_npm_prune_dev_dependencies_config.snap +++ b/tests/snapshots/test_npm_prune_dev_dependencies_config.snap @@ -43,6 +43,7 @@ heroku/nodejs ! Warning: Experimental configuration `com.heroku.buildpacks.nodejs.actions.prune_dev_dependencies` found in `project.toml`. This feature may change unexpectedly in the future. +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_skip_build_scripts_from_buildplan.snap b/tests/snapshots/test_npm_skip_build_scripts_from_buildplan.snap index bf3dc35ef..5afbab819 100644 --- a/tests/snapshots/test_npm_skip_build_scripts_from_buildplan.snap +++ b/tests/snapshots/test_npm_skip_build_scripts_from_buildplan.snap @@ -45,6 +45,7 @@ test/skip-build-scripts 0.0.0 ! Warning: Experimental configuration `node_build_scripts.metadata.skip_pruning` was added to the buildplan by a later buildpack. This feature may change unexpectedly in the future. +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_start_script_creates_a_web_process_launcher.snap b/tests/snapshots/test_npm_start_script_creates_a_web_process_launcher.snap index d3dd8f15a..2b691154b 100644 --- a/tests/snapshots/test_npm_start_script_creates_a_web_process_launcher.snap +++ b/tests/snapshots/test_npm_start_script_creates_a_web_process_launcher.snap @@ -46,6 +46,7 @@ heroku/nodejs - Done () - Configuring default processes - Adding default web process for `npm start` +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_version_change_invalidates_npm_engine_cache.snap b/tests/snapshots/test_npm_version_change_invalidates_npm_engine_cache.snap index ab0fb066a..18815a693 100644 --- a/tests/snapshots/test_npm_version_change_invalidates_npm_engine_cache.snap +++ b/tests/snapshots/test_npm_version_change_invalidates_npm_engine_cache.snap @@ -66,6 +66,8 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer +- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -153,6 +155,8 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) +- Cleaning up Node.js installation layer +- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_pnpm_default_web_process_registration_is_skipped_if_procfile_exists.snap b/tests/snapshots/test_pnpm_default_web_process_registration_is_skipped_if_procfile_exists.snap index bd8fc3d0f..e421decb8 100644 --- a/tests/snapshots/test_pnpm_default_web_process_registration_is_skipped_if_procfile_exists.snap +++ b/tests/snapshots/test_pnpm_default_web_process_registration_is_skipped_if_procfile_exists.snap @@ -70,12 +70,15 @@ heroku/nodejs - Done () - Skipping default web process (Procfile detected) +- Cleaning up Node.js installation layer +- Cleaning up pnpm installation layer +- Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' Adding layer 'heroku/nodejs:dist' Adding layer 'heroku/nodejs:pnpm' - layer 'heroku/nodejs:virtual' +Adding layer 'heroku/nodejs:virtual' Adding layer 'heroku/nodejs:web_env' Adding layer 'buildpacksio/lifecycle:launch.sbom' Added 1/1 app layer(s) diff --git a/tests/snapshots/test_pnpm_native_modules_are_recompiled_even_on_cache_restore.snap b/tests/snapshots/test_pnpm_native_modules_are_recompiled_even_on_cache_restore.snap index 8f7df965e..075030016 100644 --- a/tests/snapshots/test_pnpm_native_modules_are_recompiled_even_on_cache_restore.snap +++ b/tests/snapshots/test_pnpm_native_modules_are_recompiled_even_on_cache_restore.snap @@ -67,12 +67,17 @@ heroku/nodejs devDependencies: skipped - Done () +- Cleaning up Node.js installation layer +- Cleaning up pnpm installation layer + - Removed 3 __pycache__ directories +- Cleaning up pnpm virtual store layer + - Removed 2 Makefile artifacts - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' Adding layer 'heroku/nodejs:dist' Adding layer 'heroku/nodejs:pnpm' - layer 'heroku/nodejs:virtual' +Adding layer 'heroku/nodejs:virtual' Adding layer 'heroku/nodejs:web_env' Adding layer 'buildpacksio/lifecycle:launch.sbom' Added 1/1 app layer(s) @@ -145,12 +150,17 @@ heroku/nodejs devDependencies: skipped - Done () +- Cleaning up Node.js installation layer +- Cleaning up pnpm installation layer + - Removed 3 __pycache__ directories +- Cleaning up pnpm virtual store layer + - Removed 2 Makefile artifacts - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' Reusing layer 'heroku/nodejs:dist' -Adding layer 'heroku/nodejs:pnpm' - layer 'heroku/nodejs:virtual' +Reusing layer 'heroku/nodejs:pnpm' +Reusing layer 'heroku/nodejs:virtual' Reusing layer 'heroku/nodejs:web_env' Reusing layer 'buildpacksio/lifecycle:launch.sbom' Added 1/1 app layer(s) @@ -166,6 +176,7 @@ Saving ... Adding cache layer 'heroku/nodejs:addressable' Reusing cache layer 'heroku/nodejs:dist' Adding cache layer 'heroku/nodejs:dist' +Reusing cache layer 'heroku/nodejs:pnpm' Adding cache layer 'heroku/nodejs:pnpm' Reusing cache layer 'heroku/nodejs:pnpm_packument' Adding cache layer 'heroku/nodejs:pnpm_packument' diff --git a/tests/snapshots/test_pnpm_prune_dev_dependencies_config.snap b/tests/snapshots/test_pnpm_prune_dev_dependencies_config.snap index f88f59c1b..6b63322a5 100644 --- a/tests/snapshots/test_pnpm_prune_dev_dependencies_config.snap +++ b/tests/snapshots/test_pnpm_prune_dev_dependencies_config.snap @@ -64,12 +64,15 @@ heroku/nodejs ! Warning: Experimental configuration `com.heroku.buildpacks.nodejs.actions.prune_dev_dependencies` found in `project.toml`. This feature may change unexpectedly in the future. +- Cleaning up Node.js installation layer +- Cleaning up pnpm installation layer +- Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' Adding layer 'heroku/nodejs:dist' Adding layer 'heroku/nodejs:pnpm' - layer 'heroku/nodejs:virtual' +Adding layer 'heroku/nodejs:virtual' Adding layer 'heroku/nodejs:web_env' Adding layer 'buildpacksio/lifecycle:launch.sbom' Added 1/1 app layer(s) diff --git a/tests/snapshots/test_pnpm_skip_build_scripts_from_buildplan.snap b/tests/snapshots/test_pnpm_skip_build_scripts_from_buildplan.snap index 937c0a9dc..f6f6524f6 100644 --- a/tests/snapshots/test_pnpm_skip_build_scripts_from_buildplan.snap +++ b/tests/snapshots/test_pnpm_skip_build_scripts_from_buildplan.snap @@ -66,12 +66,15 @@ test/skip-build-scripts 0.0.0 ! Warning: Experimental configuration `node_build_scripts.metadata.skip_pruning` was added to the buildplan by a later buildpack. This feature may change unexpectedly in the future. +- Cleaning up Node.js installation layer +- Cleaning up pnpm installation layer +- Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' Adding layer 'heroku/nodejs:dist' Adding layer 'heroku/nodejs:pnpm' - layer 'heroku/nodejs:virtual' +Adding layer 'heroku/nodejs:virtual' Adding layer 'heroku/nodejs:web_env' Adding layer 'buildpacksio/lifecycle:launch.sbom' Added 1/1 app layer(s) diff --git a/tests/snapshots/test_yarn_default_web_process_registration_is_skipped_if_procfile_exists.snap b/tests/snapshots/test_yarn_default_web_process_registration_is_skipped_if_procfile_exists.snap index fd00b8de1..d94c3e4dd 100644 --- a/tests/snapshots/test_yarn_default_web_process_registration_is_skipped_if_procfile_exists.snap +++ b/tests/snapshots/test_yarn_default_web_process_registration_is_skipped_if_procfile_exists.snap @@ -68,6 +68,7 @@ heroku/nodejs - Done () - Skipping default web process (Procfile detected) +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_yarn_native_modules_are_recompiled_even_on_cache_restore.snap b/tests/snapshots/test_yarn_native_modules_are_recompiled_even_on_cache_restore.snap index d8a42fa43..569a47bf6 100644 --- a/tests/snapshots/test_yarn_native_modules_are_recompiled_even_on_cache_restore.snap +++ b/tests/snapshots/test_yarn_native_modules_are_recompiled_even_on_cache_restore.snap @@ -72,6 +72,7 @@ heroku/nodejs ➤ YN0000: Done with warnings in - Done () +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -163,6 +164,7 @@ heroku/nodejs ➤ YN0000: Done with warnings in - Done () +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_yarn_prune_dev_dependencies_config.snap b/tests/snapshots/test_yarn_prune_dev_dependencies_config.snap index fa65d2dff..dfa726dc3 100644 --- a/tests/snapshots/test_yarn_prune_dev_dependencies_config.snap +++ b/tests/snapshots/test_yarn_prune_dev_dependencies_config.snap @@ -61,6 +61,7 @@ heroku/nodejs ! Warning: Experimental configuration `com.heroku.buildpacks.nodejs.actions.prune_dev_dependencies` found in `project.toml`. This feature may change unexpectedly in the future. +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_yarn_skip_build_scripts_from_buildplan.snap b/tests/snapshots/test_yarn_skip_build_scripts_from_buildplan.snap index 0f2e22750..e52546345 100644 --- a/tests/snapshots/test_yarn_skip_build_scripts_from_buildplan.snap +++ b/tests/snapshots/test_yarn_skip_build_scripts_from_buildplan.snap @@ -63,6 +63,7 @@ test/skip-build-scripts 0.0.0 ! Warning: Experimental configuration `node_build_scripts.metadata.skip_pruning` was added to the buildplan by a later buildpack. This feature may change unexpectedly in the future. +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/yarn_1_typescript.snap b/tests/snapshots/yarn_1_typescript.snap index c478584d7..ec8284da9 100644 --- a/tests/snapshots/yarn_1_typescript.snap +++ b/tests/snapshots/yarn_1_typescript.snap @@ -68,6 +68,7 @@ heroku/nodejs Done in . - Done () +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/yarn_2_modules_nonzero.snap b/tests/snapshots/yarn_2_modules_nonzero.snap index 440bc9ecc..3bea1bd8e 100644 --- a/tests/snapshots/yarn_2_modules_nonzero.snap +++ b/tests/snapshots/yarn_2_modules_nonzero.snap @@ -67,6 +67,7 @@ heroku/nodejs ➤ YN0000: Done in - Done () +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/yarn_2_pnp_zero.snap b/tests/snapshots/yarn_2_pnp_zero.snap index 93aee517e..d0eec99af 100644 --- a/tests/snapshots/yarn_2_pnp_zero.snap +++ b/tests/snapshots/yarn_2_pnp_zero.snap @@ -61,6 +61,7 @@ heroku/nodejs ➤ YN0000: Done in - Done () +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/yarn_3_modules_zero.snap b/tests/snapshots/yarn_3_modules_zero.snap index 71c06113e..61281c8f1 100644 --- a/tests/snapshots/yarn_3_modules_zero.snap +++ b/tests/snapshots/yarn_3_modules_zero.snap @@ -61,6 +61,7 @@ heroku/nodejs ➤ YN0000: Done in - Done () +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/yarn_3_pnp_nonzero.snap b/tests/snapshots/yarn_3_pnp_nonzero.snap index 4bd22f3a9..23f99c945 100644 --- a/tests/snapshots/yarn_3_pnp_nonzero.snap +++ b/tests/snapshots/yarn_3_pnp_nonzero.snap @@ -65,6 +65,7 @@ heroku/nodejs ➤ YN0000: Done in - Done () +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/yarn_4_modules_zero.snap b/tests/snapshots/yarn_4_modules_zero.snap index d7e65e39d..c2427b469 100644 --- a/tests/snapshots/yarn_4_modules_zero.snap +++ b/tests/snapshots/yarn_4_modules_zero.snap @@ -62,6 +62,7 @@ heroku/nodejs ➤ YN0000: Done in - Done () +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/yarn_4_pnp_nonzero.snap b/tests/snapshots/yarn_4_pnp_nonzero.snap index e865dd3e0..3cedbd547 100644 --- a/tests/snapshots/yarn_4_pnp_nonzero.snap +++ b/tests/snapshots/yarn_4_pnp_nonzero.snap @@ -67,6 +67,7 @@ heroku/nodejs ➤ YN0000: Done in - Done () +- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' From 2ec02f33345e7820ad32cd6e262a355f23f17b7b Mon Sep 17 00:00:00 2001 From: Colin Casey Date: Thu, 29 Jan 2026 14:01:25 -0400 Subject: [PATCH 2/8] Remove commented out filter --- crates/test_support/src/snapshot_filters.rs | 8 -------- 1 file changed, 8 deletions(-) diff --git a/crates/test_support/src/snapshot_filters.rs b/crates/test_support/src/snapshot_filters.rs index aba6088e1..8b53704eb 100644 --- a/crates/test_support/src/snapshot_filters.rs +++ b/crates/test_support/src/snapshot_filters.rs @@ -326,14 +326,6 @@ pub(super) fn create_snapshot_filters() -> Vec<(String, String)> { "${1}", )); - // TODO: investigate why the `...native_modules_are_recompiled...`-based tests (mostly the pnpm version) - // occassionally switch from Adding layer 'heroku/nodejs:xyz' to Reusing layer 'heroku/nodejs:xyz' - // TEMPORARILY DISABLED FOR INVESTIGATION - // filters.push(( - // r"(?:Adding|Reusing) layer 'heroku/nodejs:virtual", - // " layer 'heroku/nodejs:virtual", - // )); - filters .into_iter() .map(|(matcher, replacement)| (matcher.to_string(), replacement.to_string())) From d0b2a17d85d0b09fd77fdbf2b08e494a24ecefda Mon Sep 17 00:00:00 2001 From: Colin Casey Date: Thu, 29 Jan 2026 16:53:20 -0400 Subject: [PATCH 3/8] Fix snapshot --- ...m_native_modules_are_recompiled_even_on_cache_restore.snap | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/snapshots/test_pnpm_native_modules_are_recompiled_even_on_cache_restore.snap b/tests/snapshots/test_pnpm_native_modules_are_recompiled_even_on_cache_restore.snap index 075030016..d09463648 100644 --- a/tests/snapshots/test_pnpm_native_modules_are_recompiled_even_on_cache_restore.snap +++ b/tests/snapshots/test_pnpm_native_modules_are_recompiled_even_on_cache_restore.snap @@ -71,7 +71,7 @@ heroku/nodejs - Cleaning up pnpm installation layer - Removed 3 __pycache__ directories - Cleaning up pnpm virtual store layer - - Removed 2 Makefile artifacts + - Removed 1 Makefile artifacts - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -154,7 +154,7 @@ heroku/nodejs - Cleaning up pnpm installation layer - Removed 3 __pycache__ directories - Cleaning up pnpm virtual store layer - - Removed 2 Makefile artifacts + - Removed 1 Makefile artifacts - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' From 5739225d67a342ed5e2202d14f999719df8dc7c3 Mon Sep 17 00:00:00 2001 From: Colin Casey Date: Thu, 19 Feb 2026 13:17:55 -0400 Subject: [PATCH 4/8] Use `PYTHONDONTWRITEBYTECODE` to reduce cleanup surface (#1275) --- src/layer_cleanup.rs | 81 ------------------- src/main.rs | 9 +++ src/package_managers/npm.rs | 20 +++-- src/package_managers/pnpm.rs | 13 +-- src/package_managers/yarn.rs | 2 + src/runtimes/nodejs.rs | 7 -- src/utils/build_env.rs | 11 +++ tests/snapshots/node_24.snap | 1 - tests/snapshots/node_25.snap | 1 - tests/snapshots/npm_engine_install.snap | 2 - .../npm_package_manager_install.snap | 2 - tests/snapshots/pnpm_7_pnp.snap | 2 - tests/snapshots/pnpm_8_hoist.snap | 2 - tests/snapshots/pnpm_8_nuxt.snap | 2 - tests/snapshots/pnpm_install_engine.snap | 2 - .../pnpm_install_package_manager.snap | 2 - .../reinstalls_node_if_version_changes.snap | 2 - tests/snapshots/simple_indexjs.snap | 1 - tests/snapshots/simple_serverjs.snap | 1 - ...n_change_invalidates_npm_engine_cache.snap | 4 - tests/snapshots/test_npm_build_scripts.snap | 1 - ...ripts_prefers_heroku_build_over_build.snap | 1 - ...tration_is_skipped_if_procfile_exists.snap | 1 - tests/snapshots/test_npm_engine_caching.snap | 4 - ..._are_recompiled_even_on_cache_restore.snap | 6 -- tests/snapshots/test_npm_install_caching.snap | 2 - .../test_npm_install_new_package.snap | 2 - .../test_npm_install_with_lockfile.snap | 1 - ..._are_recompiled_even_on_cache_restore.snap | 4 - ...est_npm_prune_dev_dependencies_config.snap | 1 - ...npm_skip_build_scripts_from_buildplan.snap | 1 - ...script_creates_a_web_process_launcher.snap | 1 - ...n_change_invalidates_npm_engine_cache.snap | 4 - ...tration_is_skipped_if_procfile_exists.snap | 2 - ..._are_recompiled_even_on_cache_restore.snap | 6 -- ...st_pnpm_prune_dev_dependencies_config.snap | 2 - ...npm_skip_build_scripts_from_buildplan.snap | 2 - ...tration_is_skipped_if_procfile_exists.snap | 1 - ..._are_recompiled_even_on_cache_restore.snap | 2 - ...st_yarn_prune_dev_dependencies_config.snap | 1 - ...arn_skip_build_scripts_from_buildplan.snap | 1 - tests/snapshots/yarn_1_typescript.snap | 1 - tests/snapshots/yarn_2_modules_nonzero.snap | 1 - tests/snapshots/yarn_2_pnp_zero.snap | 1 - tests/snapshots/yarn_3_modules_zero.snap | 1 - tests/snapshots/yarn_3_pnp_nonzero.snap | 1 - tests/snapshots/yarn_4_modules_zero.snap | 1 - tests/snapshots/yarn_4_pnp_nonzero.snap | 1 - 48 files changed, 35 insertions(+), 185 deletions(-) diff --git a/src/layer_cleanup.rs b/src/layer_cleanup.rs index ff85ffac1..4aa031b13 100644 --- a/src/layer_cleanup.rs +++ b/src/layer_cleanup.rs @@ -5,12 +5,6 @@ use walkdir::WalkDir; #[derive(Debug, Clone)] pub(crate) enum LayerKind { - /// Node.js distribution layer (contains npm with node-gyp) - Dist, - /// Custom npm version layer (contains npm with node-gyp) - NpmEngine, - /// pnpm distribution layer (contains pnpm with node-gyp) - Pnpm, /// pnpm virtual store layer (contains native module builds with Makefiles) Virtual, } @@ -21,32 +15,6 @@ pub(crate) struct LayerCleanupTarget { pub(crate) kind: LayerKind, } -/// Remove Python bytecode cache files (__pycache__/*.pyc) from node-gyp directories -/// These files are generated during native module compilation and are non-deterministic -fn remove_python_bytecode_cache(base_path: &Path) -> Result { - let mut removed_count = 0; - - // Look for node-gyp/gyp/pylib directory containing Python source - let pylib_path = base_path.join("node_modules/node-gyp/gyp/pylib"); - - if !pylib_path.exists() { - return Ok(0); - } - - // Walk the pylib tree looking for __pycache__ directories - for entry in WalkDir::new(&pylib_path) - .into_iter() - .filter_map(Result::ok) - .filter(|e| e.file_type().is_dir() && e.file_name() == "__pycache__") - { - // Remove the entire __pycache__ directory - fs::remove_dir_all(entry.path())?; - removed_count += 1; - } - - Ok(removed_count) -} - /// Remove Makefile and *.mk files from native module build directories /// These files have non-deterministic dependency ordering causing layer invalidation fn remove_build_makefiles(base_path: &Path) -> Result { @@ -90,39 +58,6 @@ pub(crate) fn cleanup_layer(target: &LayerCleanupTarget) -> Result<(), std::io:: } match target.kind { - LayerKind::Dist => { - // npm dist layer: contains bundled npm with node-gyp - // Clean Python bytecode from: dist/lib/node_modules/npm/node_modules/node-gyp/ - print::bullet("Cleaning up Node.js installation layer"); - let npm_modules = path.join("lib/node_modules/npm"); - if npm_modules.exists() { - let removed = remove_python_bytecode_cache(&npm_modules)?; - if removed > 0 { - print::sub_bullet(format!("Removed {removed} __pycache__ directories")); - } - } - } - LayerKind::NpmEngine => { - // npm_engine layer: custom npm version installed via npm registry - // Clean Python bytecode from: npm_engine/node_modules/node-gyp/ - print::bullet("Cleaning up npm installation layer"); - let removed = remove_python_bytecode_cache(path)?; - if removed > 0 { - print::sub_bullet(format!("Removed {removed} __pycache__ directories")); - } - } - LayerKind::Pnpm => { - // pnpm layer: contains pnpm distribution with node-gyp - // Clean Python bytecode from: pnpm/dist/node_modules/node-gyp/ - print::bullet("Cleaning up pnpm installation layer"); - let pnpm_dist = path.join("dist"); - if pnpm_dist.exists() { - let removed = remove_python_bytecode_cache(&pnpm_dist)?; - if removed > 0 { - print::sub_bullet(format!("Removed {removed} __pycache__ directories")); - } - } - } LayerKind::Virtual => { // pnpm virtual store: contains symlinked packages with native module builds // Clean Makefiles from: virtual/store/*/node_modules/*/build/ @@ -143,22 +78,6 @@ mod tests { use std::fs; use tempfile::TempDir; - #[test] - fn test_remove_python_bytecode_cache() { - let temp = TempDir::new().unwrap(); - let base = temp.path(); - - // Create directory structure with __pycache__ - let pycache_path = base.join("node_modules/node-gyp/gyp/pylib/gyp/__pycache__"); - fs::create_dir_all(&pycache_path).unwrap(); - fs::write(pycache_path.join("test.cpython-312.pyc"), b"bytecode").unwrap(); - - let removed = remove_python_bytecode_cache(base).unwrap(); - - assert_eq!(removed, 1); - assert!(!pycache_path.exists()); - } - #[test] fn test_remove_build_makefiles() { let temp = TempDir::new().unwrap(); diff --git a/src/main.rs b/src/main.rs index 4667b4cda..0e7cd693b 100644 --- a/src/main.rs +++ b/src/main.rs @@ -129,6 +129,15 @@ impl libcnb::Buildpack for NodeJsBuildpack { available_parallelism::env_value(), )?; + // // To prevent .pyc files from being generated by node-gyp + // // ref: https://github.com/heroku/buildpacks-python/blob/a812275a1142a3ad104ac381f2aac61d7e005183/src/layers/python.rs#L213-L249 + // utils::build_env::set_default_env_var( + // &context, + // &mut env, + // "SOURCE_DATE_EPOCH", + // "315532801", + // )?; + // TODO: this code should be moved to the end of the build execution but can't until the package managers are cleaned up if let Some(path) = ["server.js", "index.js"] .map(|name| context.app_dir.join(name)) diff --git a/src/package_managers/npm.rs b/src/package_managers/npm.rs index f1debc7ce..f9ed6cebc 100644 --- a/src/package_managers/npm.rs +++ b/src/package_managers/npm.rs @@ -1,4 +1,4 @@ -use crate::layer_cleanup::{LayerCleanupTarget, LayerKind}; +use crate::utils::build_env::node_gyp_env; use crate::utils::error_handling::ErrorType::Internal; use crate::utils::error_handling::{ ErrorMessage, ErrorType, SuggestRetryBuild, SuggestSubmitIssue, error_message, @@ -70,20 +70,13 @@ pub(crate) fn install_npm( npm_packument: &npm_registry::PackagePackument, node_version: &Version, ) -> BuildpackResult<()> { - let npm_engine_layer_path = npm_registry::install_package_layer( + npm_registry::install_package_layer( layer_name!("npm_engine"), context, env, npm_packument, node_version, )?; - - // Register npm_engine layer for cleanup of non-deterministic Python bytecode - context.register_layer_for_cleanup(LayerCleanupTarget { - path: npm_engine_layer_path, - kind: LayerKind::NpmEngine, - }); - Ok(()) } @@ -113,8 +106,13 @@ pub(crate) fn install_npm_dependencies( .named_output() .map_err(|e| create_set_npm_cache_directory_command_error(&e))?; - print::sub_stream_cmd(Command::new("npm").args(["ci"]).envs(env)) - .map_err(|e| create_npm_install_error(&e))?; + print::sub_stream_cmd( + Command::new("npm") + .args(["ci"]) + .envs(env) + .envs(node_gyp_env()), + ) + .map_err(|e| create_npm_install_error(&e))?; Ok(()) } diff --git a/src/package_managers/pnpm.rs b/src/package_managers/pnpm.rs index 7b6862ca3..3599ef116 100644 --- a/src/package_managers/pnpm.rs +++ b/src/package_managers/pnpm.rs @@ -1,4 +1,5 @@ use crate::layer_cleanup::{LayerCleanupTarget, LayerKind}; +use crate::utils::build_env::node_gyp_env; use crate::utils::error_handling::{ ErrorMessage, ErrorType, SuggestRetryBuild, SuggestSubmitIssue, error_message, file_value, }; @@ -39,20 +40,13 @@ pub(crate) fn install_pnpm( pnpm_packument: &PackagePackument, node_version: &Version, ) -> BuildpackResult<()> { - let pnpm_layer_path = utils::npm_registry::install_package_layer( + utils::npm_registry::install_package_layer( layer_name!("pnpm"), context, env, pnpm_packument, node_version, )?; - - // Register pnpm layer for cleanup of non-deterministic Python bytecode - context.register_layer_for_cleanup(LayerCleanupTarget { - path: pnpm_layer_path, - kind: LayerKind::Pnpm, - }); - Ok(()) } @@ -74,7 +68,8 @@ pub(crate) fn install_dependencies( print::sub_stream_cmd( Command::new("pnpm") .args(["install", "--frozen-lockfile"]) - .envs(env), + .envs(env) + .envs(node_gyp_env()), ) .map_err(|e| create_pnpm_install_command_error(&e))?; diff --git a/src/package_managers/yarn.rs b/src/package_managers/yarn.rs index 5b632b3d6..e19b8f74d 100644 --- a/src/package_managers/yarn.rs +++ b/src/package_managers/yarn.rs @@ -1,4 +1,5 @@ use crate::o11y::*; +use crate::utils::build_env::node_gyp_env; use crate::utils::error_handling::{ ErrorMessage, ErrorType, SuggestRetryBuild, SuggestSubmitIssue, error_message, file_value, }; @@ -284,6 +285,7 @@ pub(crate) fn install_dependencies( print::bullet("Installing dependencies"); let mut yarn_install_command = Command::new("yarn"); yarn_install_command.envs(env); + yarn_install_command.envs(node_gyp_env()); yarn_install_command.arg("install"); if version.major() == 1 { yarn_install_command.args(["--production=false", "--frozen-lockfile"]); diff --git a/src/runtimes/nodejs.rs b/src/runtimes/nodejs.rs index 75e8b2413..2bcea9444 100644 --- a/src/runtimes/nodejs.rs +++ b/src/runtimes/nodejs.rs @@ -1,4 +1,3 @@ -use crate::layer_cleanup::{LayerCleanupTarget, LayerKind}; use crate::utils::error_handling::ErrorType::{Internal, UserFacing}; use crate::utils::error_handling::{ ErrorMessage, SuggestRetryBuild, SuggestSubmitIssue, error_message, file_value, @@ -111,12 +110,6 @@ pub(crate) fn install( env.clone_from(&distribution_layer.read_env()?.apply(Scope::Build, env)); - // Register dist layer for cleanup of non-deterministic Python bytecode - context.register_layer_for_cleanup(LayerCleanupTarget { - path: distribution_layer.path().clone(), - kind: LayerKind::Dist, - }); - Ok(()) } diff --git a/src/utils/build_env.rs b/src/utils/build_env.rs index d5ea75f11..5b6f96c01 100644 --- a/src/utils/build_env.rs +++ b/src/utils/build_env.rs @@ -34,3 +34,14 @@ pub(crate) fn set_default_env_var( Ok(()) } + +pub(crate) fn node_gyp_env() -> Vec<(String, String)> { + vec![ + // If this is set to a non-empty string, Python won’t try to write .pyc files on the import of source modules. + // see https://docs.python.org/3/using/cmdline.html#envvar-PYTHONDONTWRITEBYTECODE + // + // This is used to prevent node-gyp from generating files which invalidate runtime layers whenever + // native modules are compiled as `node-gyp` uses Python as part of the compilation process. + ("PYTHONDONTWRITEBYTECODE".to_string(), "1".to_string()), + ] +} diff --git a/tests/snapshots/node_24.snap b/tests/snapshots/node_24.snap index d3bf5a40d..fce2142e2 100644 --- a/tests/snapshots/node_24.snap +++ b/tests/snapshots/node_24.snap @@ -21,7 +21,6 @@ heroku/nodejs - Verifying checksum - Extracting Node.js `24.13.0 ()` - Installing Node.js `24.13.0 ()` ... () -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/node_25.snap b/tests/snapshots/node_25.snap index 0f06c22de..9defe317b 100644 --- a/tests/snapshots/node_25.snap +++ b/tests/snapshots/node_25.snap @@ -21,7 +21,6 @@ heroku/nodejs - Verifying checksum - Extracting Node.js `25.5.0 ()` - Installing Node.js `25.5.0 ()` ... () -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/npm_engine_install.snap b/tests/snapshots/npm_engine_install.snap index 8b00d5c30..cd3a1dd95 100644 --- a/tests/snapshots/npm_engine_install.snap +++ b/tests/snapshots/npm_engine_install.snap @@ -66,8 +66,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer -- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/npm_package_manager_install.snap b/tests/snapshots/npm_package_manager_install.snap index 85207ea31..9e02fa463 100644 --- a/tests/snapshots/npm_package_manager_install.snap +++ b/tests/snapshots/npm_package_manager_install.snap @@ -54,8 +54,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer -- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/pnpm_7_pnp.snap b/tests/snapshots/pnpm_7_pnp.snap index f6231c95a..905715111 100644 --- a/tests/snapshots/pnpm_7_pnp.snap +++ b/tests/snapshots/pnpm_7_pnp.snap @@ -64,8 +64,6 @@ heroku/nodejs devDependencies: skipped - Done () -- Cleaning up Node.js installation layer -- Cleaning up pnpm installation layer - Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING diff --git a/tests/snapshots/pnpm_8_hoist.snap b/tests/snapshots/pnpm_8_hoist.snap index 3514d29af..dd770fc32 100644 --- a/tests/snapshots/pnpm_8_hoist.snap +++ b/tests/snapshots/pnpm_8_hoist.snap @@ -67,8 +67,6 @@ heroku/nodejs devDependencies: skipped - Done () -- Cleaning up Node.js installation layer -- Cleaning up pnpm installation layer - Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING diff --git a/tests/snapshots/pnpm_8_nuxt.snap b/tests/snapshots/pnpm_8_nuxt.snap index b3d0f9555..08dd27982 100644 --- a/tests/snapshots/pnpm_8_nuxt.snap +++ b/tests/snapshots/pnpm_8_nuxt.snap @@ -94,8 +94,6 @@ heroku/nodejs ! ! Since pruning can't be done safely for your build, it will be skipped. To fix this you must upgrade your version of pnpm to 8.15.6 or higher. -- Cleaning up Node.js installation layer -- Cleaning up pnpm installation layer - Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING diff --git a/tests/snapshots/pnpm_install_engine.snap b/tests/snapshots/pnpm_install_engine.snap index b6c9f805b..9a9a4f395 100644 --- a/tests/snapshots/pnpm_install_engine.snap +++ b/tests/snapshots/pnpm_install_engine.snap @@ -67,8 +67,6 @@ heroku/nodejs devDependencies: skipped - Done () -- Cleaning up Node.js installation layer -- Cleaning up pnpm installation layer - Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING diff --git a/tests/snapshots/pnpm_install_package_manager.snap b/tests/snapshots/pnpm_install_package_manager.snap index cfa9d75af..951ddbf7c 100644 --- a/tests/snapshots/pnpm_install_package_manager.snap +++ b/tests/snapshots/pnpm_install_package_manager.snap @@ -67,8 +67,6 @@ heroku/nodejs devDependencies: skipped - Done () -- Cleaning up Node.js installation layer -- Cleaning up pnpm installation layer - Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING diff --git a/tests/snapshots/reinstalls_node_if_version_changes.snap b/tests/snapshots/reinstalls_node_if_version_changes.snap index 7b00355e3..662f65960 100644 --- a/tests/snapshots/reinstalls_node_if_version_changes.snap +++ b/tests/snapshots/reinstalls_node_if_version_changes.snap @@ -21,7 +21,6 @@ heroku/nodejs - Verifying checksum - Extracting Node.js `14.21.3 ()` - Installing Node.js `14.21.3 ()` ... () -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -63,7 +62,6 @@ heroku/nodejs - Verifying checksum - Extracting Node.js `16.20.2 ()` - Installing Node.js `16.20.2 ()` ... () -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/simple_indexjs.snap b/tests/snapshots/simple_indexjs.snap index 8ee2bb1cd..89bd120ae 100644 --- a/tests/snapshots/simple_indexjs.snap +++ b/tests/snapshots/simple_indexjs.snap @@ -21,7 +21,6 @@ heroku/nodejs - Verifying checksum - Extracting Node.js `24.13.0 ()` - Installing Node.js `24.13.0 ()` ... () -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/simple_serverjs.snap b/tests/snapshots/simple_serverjs.snap index ef9d406ae..2b1e16dec 100644 --- a/tests/snapshots/simple_serverjs.snap +++ b/tests/snapshots/simple_serverjs.snap @@ -21,7 +21,6 @@ heroku/nodejs - Verifying checksum - Extracting Node.js `16.0.0 ()` - Installing Node.js `16.0.0 ()` ... () -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_node_version_change_invalidates_npm_engine_cache.snap b/tests/snapshots/test_node_version_change_invalidates_npm_engine_cache.snap index 906daa8e7..94b5a8f41 100644 --- a/tests/snapshots/test_node_version_change_invalidates_npm_engine_cache.snap +++ b/tests/snapshots/test_node_version_change_invalidates_npm_engine_cache.snap @@ -66,8 +66,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer -- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -160,8 +158,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer -- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_build_scripts.snap b/tests/snapshots/test_npm_build_scripts.snap index 8e9b9a94c..812e432db 100644 --- a/tests/snapshots/test_npm_build_scripts.snap +++ b/tests/snapshots/test_npm_build_scripts.snap @@ -72,7 +72,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_build_scripts_prefers_heroku_build_over_build.snap b/tests/snapshots/test_npm_build_scripts_prefers_heroku_build_over_build.snap index 60fed1915..ce915bb6c 100644 --- a/tests/snapshots/test_npm_build_scripts_prefers_heroku_build_over_build.snap +++ b/tests/snapshots/test_npm_build_scripts_prefers_heroku_build_over_build.snap @@ -54,7 +54,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_default_web_process_registration_is_skipped_if_procfile_exists.snap b/tests/snapshots/test_npm_default_web_process_registration_is_skipped_if_procfile_exists.snap index 07fbc101e..111acde10 100644 --- a/tests/snapshots/test_npm_default_web_process_registration_is_skipped_if_procfile_exists.snap +++ b/tests/snapshots/test_npm_default_web_process_registration_is_skipped_if_procfile_exists.snap @@ -46,7 +46,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (Procfile detected) -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_engine_caching.snap b/tests/snapshots/test_npm_engine_caching.snap index dc460551c..46ca0cac9 100644 --- a/tests/snapshots/test_npm_engine_caching.snap +++ b/tests/snapshots/test_npm_engine_caching.snap @@ -66,8 +66,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer -- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -153,8 +151,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer -- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_engine_native_modules_are_recompiled_even_on_cache_restore.snap b/tests/snapshots/test_npm_engine_native_modules_are_recompiled_even_on_cache_restore.snap index 149c66181..e66867232 100644 --- a/tests/snapshots/test_npm_engine_native_modules_are_recompiled_even_on_cache_restore.snap +++ b/tests/snapshots/test_npm_engine_native_modules_are_recompiled_even_on_cache_restore.snap @@ -59,9 +59,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer -- Cleaning up npm installation layer - - Removed 3 __pycache__ directories - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -140,9 +137,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer -- Cleaning up npm installation layer - - Removed 3 __pycache__ directories - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_install_caching.snap b/tests/snapshots/test_npm_install_caching.snap index 67779370b..3ec7e45de 100644 --- a/tests/snapshots/test_npm_install_caching.snap +++ b/tests/snapshots/test_npm_install_caching.snap @@ -46,7 +46,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -109,7 +108,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_install_new_package.snap b/tests/snapshots/test_npm_install_new_package.snap index 3d6043233..ede98be9e 100644 --- a/tests/snapshots/test_npm_install_new_package.snap +++ b/tests/snapshots/test_npm_install_new_package.snap @@ -46,7 +46,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -109,7 +108,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_install_with_lockfile.snap b/tests/snapshots/test_npm_install_with_lockfile.snap index bcdecc770..8003eb4a5 100644 --- a/tests/snapshots/test_npm_install_with_lockfile.snap +++ b/tests/snapshots/test_npm_install_with_lockfile.snap @@ -46,7 +46,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_native_modules_are_recompiled_even_on_cache_restore.snap b/tests/snapshots/test_npm_native_modules_are_recompiled_even_on_cache_restore.snap index dc28edf07..5cc67b759 100644 --- a/tests/snapshots/test_npm_native_modules_are_recompiled_even_on_cache_restore.snap +++ b/tests/snapshots/test_npm_native_modules_are_recompiled_even_on_cache_restore.snap @@ -51,8 +51,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer - - Removed 3 __pycache__ directories - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -120,8 +118,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer - - Removed 3 __pycache__ directories - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_prune_dev_dependencies_config.snap b/tests/snapshots/test_npm_prune_dev_dependencies_config.snap index 0246b90f1..14dd9f229 100644 --- a/tests/snapshots/test_npm_prune_dev_dependencies_config.snap +++ b/tests/snapshots/test_npm_prune_dev_dependencies_config.snap @@ -43,7 +43,6 @@ heroku/nodejs ! Warning: Experimental configuration `com.heroku.buildpacks.nodejs.actions.prune_dev_dependencies` found in `project.toml`. This feature may change unexpectedly in the future. -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_skip_build_scripts_from_buildplan.snap b/tests/snapshots/test_npm_skip_build_scripts_from_buildplan.snap index 5afbab819..bf3dc35ef 100644 --- a/tests/snapshots/test_npm_skip_build_scripts_from_buildplan.snap +++ b/tests/snapshots/test_npm_skip_build_scripts_from_buildplan.snap @@ -45,7 +45,6 @@ test/skip-build-scripts 0.0.0 ! Warning: Experimental configuration `node_build_scripts.metadata.skip_pruning` was added to the buildplan by a later buildpack. This feature may change unexpectedly in the future. -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_start_script_creates_a_web_process_launcher.snap b/tests/snapshots/test_npm_start_script_creates_a_web_process_launcher.snap index 2b691154b..d3dd8f15a 100644 --- a/tests/snapshots/test_npm_start_script_creates_a_web_process_launcher.snap +++ b/tests/snapshots/test_npm_start_script_creates_a_web_process_launcher.snap @@ -46,7 +46,6 @@ heroku/nodejs - Done () - Configuring default processes - Adding default web process for `npm start` -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_npm_version_change_invalidates_npm_engine_cache.snap b/tests/snapshots/test_npm_version_change_invalidates_npm_engine_cache.snap index 18815a693..ab0fb066a 100644 --- a/tests/snapshots/test_npm_version_change_invalidates_npm_engine_cache.snap +++ b/tests/snapshots/test_npm_version_change_invalidates_npm_engine_cache.snap @@ -66,8 +66,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer -- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -155,8 +153,6 @@ heroku/nodejs - Done () - Configuring default processes - Skipping default web process (no start script defined) -- Cleaning up Node.js installation layer -- Cleaning up npm installation layer - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_pnpm_default_web_process_registration_is_skipped_if_procfile_exists.snap b/tests/snapshots/test_pnpm_default_web_process_registration_is_skipped_if_procfile_exists.snap index e421decb8..de327fc64 100644 --- a/tests/snapshots/test_pnpm_default_web_process_registration_is_skipped_if_procfile_exists.snap +++ b/tests/snapshots/test_pnpm_default_web_process_registration_is_skipped_if_procfile_exists.snap @@ -70,8 +70,6 @@ heroku/nodejs - Done () - Skipping default web process (Procfile detected) -- Cleaning up Node.js installation layer -- Cleaning up pnpm installation layer - Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING diff --git a/tests/snapshots/test_pnpm_native_modules_are_recompiled_even_on_cache_restore.snap b/tests/snapshots/test_pnpm_native_modules_are_recompiled_even_on_cache_restore.snap index d09463648..4084b00d0 100644 --- a/tests/snapshots/test_pnpm_native_modules_are_recompiled_even_on_cache_restore.snap +++ b/tests/snapshots/test_pnpm_native_modules_are_recompiled_even_on_cache_restore.snap @@ -67,9 +67,6 @@ heroku/nodejs devDependencies: skipped - Done () -- Cleaning up Node.js installation layer -- Cleaning up pnpm installation layer - - Removed 3 __pycache__ directories - Cleaning up pnpm virtual store layer - Removed 1 Makefile artifacts - Done (finished in ) @@ -150,9 +147,6 @@ heroku/nodejs devDependencies: skipped - Done () -- Cleaning up Node.js installation layer -- Cleaning up pnpm installation layer - - Removed 3 __pycache__ directories - Cleaning up pnpm virtual store layer - Removed 1 Makefile artifacts - Done (finished in ) diff --git a/tests/snapshots/test_pnpm_prune_dev_dependencies_config.snap b/tests/snapshots/test_pnpm_prune_dev_dependencies_config.snap index 6b63322a5..fc2b9cb37 100644 --- a/tests/snapshots/test_pnpm_prune_dev_dependencies_config.snap +++ b/tests/snapshots/test_pnpm_prune_dev_dependencies_config.snap @@ -64,8 +64,6 @@ heroku/nodejs ! Warning: Experimental configuration `com.heroku.buildpacks.nodejs.actions.prune_dev_dependencies` found in `project.toml`. This feature may change unexpectedly in the future. -- Cleaning up Node.js installation layer -- Cleaning up pnpm installation layer - Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING diff --git a/tests/snapshots/test_pnpm_skip_build_scripts_from_buildplan.snap b/tests/snapshots/test_pnpm_skip_build_scripts_from_buildplan.snap index f6f6524f6..9db556c35 100644 --- a/tests/snapshots/test_pnpm_skip_build_scripts_from_buildplan.snap +++ b/tests/snapshots/test_pnpm_skip_build_scripts_from_buildplan.snap @@ -66,8 +66,6 @@ test/skip-build-scripts 0.0.0 ! Warning: Experimental configuration `node_build_scripts.metadata.skip_pruning` was added to the buildplan by a later buildpack. This feature may change unexpectedly in the future. -- Cleaning up Node.js installation layer -- Cleaning up pnpm installation layer - Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING diff --git a/tests/snapshots/test_yarn_default_web_process_registration_is_skipped_if_procfile_exists.snap b/tests/snapshots/test_yarn_default_web_process_registration_is_skipped_if_procfile_exists.snap index d94c3e4dd..fd00b8de1 100644 --- a/tests/snapshots/test_yarn_default_web_process_registration_is_skipped_if_procfile_exists.snap +++ b/tests/snapshots/test_yarn_default_web_process_registration_is_skipped_if_procfile_exists.snap @@ -68,7 +68,6 @@ heroku/nodejs - Done () - Skipping default web process (Procfile detected) -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_yarn_native_modules_are_recompiled_even_on_cache_restore.snap b/tests/snapshots/test_yarn_native_modules_are_recompiled_even_on_cache_restore.snap index 569a47bf6..d8a42fa43 100644 --- a/tests/snapshots/test_yarn_native_modules_are_recompiled_even_on_cache_restore.snap +++ b/tests/snapshots/test_yarn_native_modules_are_recompiled_even_on_cache_restore.snap @@ -72,7 +72,6 @@ heroku/nodejs ➤ YN0000: Done with warnings in - Done () -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' @@ -164,7 +163,6 @@ heroku/nodejs ➤ YN0000: Done with warnings in - Done () -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Reusing layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_yarn_prune_dev_dependencies_config.snap b/tests/snapshots/test_yarn_prune_dev_dependencies_config.snap index dfa726dc3..fa65d2dff 100644 --- a/tests/snapshots/test_yarn_prune_dev_dependencies_config.snap +++ b/tests/snapshots/test_yarn_prune_dev_dependencies_config.snap @@ -61,7 +61,6 @@ heroku/nodejs ! Warning: Experimental configuration `com.heroku.buildpacks.nodejs.actions.prune_dev_dependencies` found in `project.toml`. This feature may change unexpectedly in the future. -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/test_yarn_skip_build_scripts_from_buildplan.snap b/tests/snapshots/test_yarn_skip_build_scripts_from_buildplan.snap index e52546345..0f2e22750 100644 --- a/tests/snapshots/test_yarn_skip_build_scripts_from_buildplan.snap +++ b/tests/snapshots/test_yarn_skip_build_scripts_from_buildplan.snap @@ -63,7 +63,6 @@ test/skip-build-scripts 0.0.0 ! Warning: Experimental configuration `node_build_scripts.metadata.skip_pruning` was added to the buildplan by a later buildpack. This feature may change unexpectedly in the future. -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/yarn_1_typescript.snap b/tests/snapshots/yarn_1_typescript.snap index ec8284da9..c478584d7 100644 --- a/tests/snapshots/yarn_1_typescript.snap +++ b/tests/snapshots/yarn_1_typescript.snap @@ -68,7 +68,6 @@ heroku/nodejs Done in . - Done () -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/yarn_2_modules_nonzero.snap b/tests/snapshots/yarn_2_modules_nonzero.snap index 3bea1bd8e..440bc9ecc 100644 --- a/tests/snapshots/yarn_2_modules_nonzero.snap +++ b/tests/snapshots/yarn_2_modules_nonzero.snap @@ -67,7 +67,6 @@ heroku/nodejs ➤ YN0000: Done in - Done () -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/yarn_2_pnp_zero.snap b/tests/snapshots/yarn_2_pnp_zero.snap index d0eec99af..93aee517e 100644 --- a/tests/snapshots/yarn_2_pnp_zero.snap +++ b/tests/snapshots/yarn_2_pnp_zero.snap @@ -61,7 +61,6 @@ heroku/nodejs ➤ YN0000: Done in - Done () -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/yarn_3_modules_zero.snap b/tests/snapshots/yarn_3_modules_zero.snap index 61281c8f1..71c06113e 100644 --- a/tests/snapshots/yarn_3_modules_zero.snap +++ b/tests/snapshots/yarn_3_modules_zero.snap @@ -61,7 +61,6 @@ heroku/nodejs ➤ YN0000: Done in - Done () -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/yarn_3_pnp_nonzero.snap b/tests/snapshots/yarn_3_pnp_nonzero.snap index 23f99c945..4bd22f3a9 100644 --- a/tests/snapshots/yarn_3_pnp_nonzero.snap +++ b/tests/snapshots/yarn_3_pnp_nonzero.snap @@ -65,7 +65,6 @@ heroku/nodejs ➤ YN0000: Done in - Done () -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/yarn_4_modules_zero.snap b/tests/snapshots/yarn_4_modules_zero.snap index c2427b469..d7e65e39d 100644 --- a/tests/snapshots/yarn_4_modules_zero.snap +++ b/tests/snapshots/yarn_4_modules_zero.snap @@ -62,7 +62,6 @@ heroku/nodejs ➤ YN0000: Done in - Done () -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' diff --git a/tests/snapshots/yarn_4_pnp_nonzero.snap b/tests/snapshots/yarn_4_pnp_nonzero.snap index 3cedbd547..e865dd3e0 100644 --- a/tests/snapshots/yarn_4_pnp_nonzero.snap +++ b/tests/snapshots/yarn_4_pnp_nonzero.snap @@ -67,7 +67,6 @@ heroku/nodejs ➤ YN0000: Done in - Done () -- Cleaning up Node.js installation layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' From c24ea15e1149e357ebb8510780f5b81046be5b64 Mon Sep 17 00:00:00 2001 From: Colin Casey Date: Thu, 19 Feb 2026 13:20:35 -0400 Subject: [PATCH 5/8] Update main.rs Removed commented out code Signed-off-by: Colin Casey --- src/main.rs | 9 --------- 1 file changed, 9 deletions(-) diff --git a/src/main.rs b/src/main.rs index 0e7cd693b..4667b4cda 100644 --- a/src/main.rs +++ b/src/main.rs @@ -129,15 +129,6 @@ impl libcnb::Buildpack for NodeJsBuildpack { available_parallelism::env_value(), )?; - // // To prevent .pyc files from being generated by node-gyp - // // ref: https://github.com/heroku/buildpacks-python/blob/a812275a1142a3ad104ac381f2aac61d7e005183/src/layers/python.rs#L213-L249 - // utils::build_env::set_default_env_var( - // &context, - // &mut env, - // "SOURCE_DATE_EPOCH", - // "315532801", - // )?; - // TODO: this code should be moved to the end of the build execution but can't until the package managers are cleaned up if let Some(path) = ["server.js", "index.js"] .map(|name| context.app_dir.join(name)) From e5b481b5551d7d709e026fa602b3bd81d5b021ae Mon Sep 17 00:00:00 2001 From: Colin Casey Date: Thu, 19 Feb 2026 13:32:49 -0400 Subject: [PATCH 6/8] Fix formatting errors --- src/main.rs | 4 ++-- src/package_managers/pnpm.rs | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/main.rs b/src/main.rs index 465673b02..7dc87d229 100644 --- a/src/main.rs +++ b/src/main.rs @@ -223,7 +223,7 @@ impl libcnb::Buildpack for NodeJsBuildpack { } } } - + let node_module_bins_layer = context.uncached_layer( layer_name!("z_node_module_bins"), UncachedLayerDefinition { @@ -242,7 +242,7 @@ impl libcnb::Buildpack for NodeJsBuildpack { context.app_dir.join("node_modules/.bin"), ), )?; - + // Clean up non-deterministic build artifacts from registered layers let layers_to_cleanup = context.layers_to_cleanup(); if !layers_to_cleanup.is_empty() { diff --git a/src/package_managers/pnpm.rs b/src/package_managers/pnpm.rs index 67c4bcd7a..7522855f6 100644 --- a/src/package_managers/pnpm.rs +++ b/src/package_managers/pnpm.rs @@ -1,6 +1,6 @@ use crate::layer_cleanup::{LayerCleanupTarget, LayerKind}; -use crate::utils::build_env::node_gyp_env; use crate::package_json::PackageJson; +use crate::utils::build_env::node_gyp_env; use crate::utils::error_handling::{ ErrorMessage, ErrorType, SuggestRetryBuild, SuggestSubmitIssue, error_message, file_value, }; From 0170f337b642b0b5e99230fabc7502491adf5d43 Mon Sep 17 00:00:00 2001 From: Colin Casey Date: Fri, 20 Feb 2026 17:01:15 -0400 Subject: [PATCH 7/8] Code review suggestions --- src/layer_cleanup.rs | 30 ++++++++---------------------- 1 file changed, 8 insertions(+), 22 deletions(-) diff --git a/src/layer_cleanup.rs b/src/layer_cleanup.rs index 4aa031b13..e4f03a56e 100644 --- a/src/layer_cleanup.rs +++ b/src/layer_cleanup.rs @@ -15,32 +15,18 @@ pub(crate) struct LayerCleanupTarget { pub(crate) kind: LayerKind, } -/// Remove Makefile and *.mk files from native module build directories +/// Remove Makefile files from native module build directories /// These files have non-deterministic dependency ordering causing layer invalidation -fn remove_build_makefiles(base_path: &Path) -> Result { - let mut removed_count = 0; - - // Walk directory tree looking for build/Makefile patterns - for entry in WalkDir::new(base_path) +fn remove_build_makefiles(base_path: &Path) -> std::io::Result { + let makefile_dir_entries = WalkDir::new(base_path) .into_iter() .filter_map(Result::ok) - .filter(|e| { - if !e.file_type().is_file() { - return false; - } + .filter(|dir_entry| { + dir_entry.file_type().is_file() && dir_entry.path().ends_with("build/Makefile") + }); - // Check if this is a Makefile or .mk file in a build/ directory - let path = e.path(); - if let Some(parent) = path.parent() - && parent.file_name() == Some(std::ffi::OsStr::new("build")) - && let Some(filename) = path.file_name() - { - return filename.to_string_lossy() == "Makefile"; - } - - false - }) - { + let mut removed_count = 0; + for entry in makefile_dir_entries { fs::remove_file(entry.path())?; removed_count += 1; } From 2e695e4c0b7d4ce5450280b90fd7f484b20b05e1 Mon Sep 17 00:00:00 2001 From: Colin Casey Date: Fri, 20 Feb 2026 17:21:50 -0400 Subject: [PATCH 8/8] Updated snapshots --- ...les_are_recompiled_even_on_cache_restore.snap | 16 +++++++++------- tests/snapshots/test_pnpm_10_workspace.snap | 3 ++- ...ycle_scripts_are_present_in_root_project.snap | 3 ++- ...scripts_are_present_in_workspace_project.snap | 3 ++- 4 files changed, 15 insertions(+), 10 deletions(-) diff --git a/tests/snapshots/test_npm_engine_native_modules_are_recompiled_even_on_cache_restore.snap b/tests/snapshots/test_npm_engine_native_modules_are_recompiled_even_on_cache_restore.snap index e66867232..0211b2867 100644 --- a/tests/snapshots/test_npm_engine_native_modules_are_recompiled_even_on_cache_restore.snap +++ b/tests/snapshots/test_npm_engine_native_modules_are_recompiled_even_on_cache_restore.snap @@ -24,13 +24,13 @@ heroku/nodejs - Determining npm package information - Found `engines.npm` version `11.x` declared in `package.json` - GET https://registry.npmjs.org/npm ... () - - Resolved npm version `11.x` to `11.8.0` + - Resolved npm version `11.x` to `11.10.1` - Installing npm - - GET https://registry.npmjs.org/npm/-/npm-11.8.0.tgz ... () + - GET https://registry.npmjs.org/npm/-/npm-11.10.1.tgz ... () - Extracting ... () - - Successfully installed `npm@11.8.0` + - Successfully installed `npm@11.10.1` - Installing node modules - - Using npm version `11.8.0` + - Using npm version `11.10.1` - Creating npm cache - Configuring npm cache directory - Running `npm ci` @@ -66,6 +66,7 @@ Adding layer 'heroku/nodejs:dist' Adding layer 'heroku/nodejs:npm_engine' Adding layer 'heroku/nodejs:npm_runtime_config' Adding layer 'heroku/nodejs:web_env' +Adding layer 'heroku/nodejs:z_node_module_bins' Adding layer 'buildpacksio/lifecycle:launch.sbom' Added 1/1 app layer(s) Adding layer 'buildpacksio/lifecycle:launcher' @@ -103,12 +104,12 @@ heroku/nodejs - Found `engines.npm` version `11.x` declared in `package.json` - GET https://registry.npmjs.org/npm ... (Not Modified) - Using cached packument for npm - - Resolved npm version `11.x` to `11.8.0` + - Resolved npm version `11.x` to `11.10.1` - Installing npm - Using cached version of npm - - Successfully installed `npm@11.8.0` + - Successfully installed `npm@11.10.1` - Installing node modules - - Using npm version `11.8.0` + - Using npm version `11.10.1` - Restoring npm cache - Configuring npm cache directory - Running `npm ci` @@ -144,6 +145,7 @@ Reusing layer 'heroku/nodejs:dist' Reusing layer 'heroku/nodejs:npm_engine' Reusing layer 'heroku/nodejs:npm_runtime_config' Reusing layer 'heroku/nodejs:web_env' +Reusing layer 'heroku/nodejs:z_node_module_bins' Reusing layer 'buildpacksio/lifecycle:launch.sbom' Reused 1/1 app layer(s) Reusing layer 'buildpacksio/lifecycle:launcher' diff --git a/tests/snapshots/test_pnpm_10_workspace.snap b/tests/snapshots/test_pnpm_10_workspace.snap index 1a377f402..83744a24e 100644 --- a/tests/snapshots/test_pnpm_10_workspace.snap +++ b/tests/snapshots/test_pnpm_10_workspace.snap @@ -69,12 +69,13 @@ heroku/nodejs Done in - Done () +- Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' Adding layer 'heroku/nodejs:dist' Adding layer 'heroku/nodejs:pnpm' - layer 'heroku/nodejs:virtual' +Adding layer 'heroku/nodejs:virtual' Adding layer 'heroku/nodejs:web_env' Adding layer 'heroku/nodejs:z_node_module_bins' Adding layer 'buildpacksio/lifecycle:launch.sbom' diff --git a/tests/snapshots/test_pnpm_workspace_prune_skipped_if_lifecycle_scripts_are_present_in_root_project.snap b/tests/snapshots/test_pnpm_workspace_prune_skipped_if_lifecycle_scripts_are_present_in_root_project.snap index 03407b962..c15ee9276 100644 --- a/tests/snapshots/test_pnpm_workspace_prune_skipped_if_lifecycle_scripts_are_present_in_root_project.snap +++ b/tests/snapshots/test_pnpm_workspace_prune_skipped_if_lifecycle_scripts_are_present_in_root_project.snap @@ -72,12 +72,13 @@ heroku/nodejs ! ! Since pruning can't be done safely for your build, it will be skipped. +- Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' Adding layer 'heroku/nodejs:dist' Adding layer 'heroku/nodejs:pnpm' - layer 'heroku/nodejs:virtual' +Adding layer 'heroku/nodejs:virtual' Adding layer 'heroku/nodejs:web_env' Adding layer 'heroku/nodejs:z_node_module_bins' Adding layer 'buildpacksio/lifecycle:launch.sbom' diff --git a/tests/snapshots/test_pnpm_workspace_prune_skipped_if_lifecycle_scripts_are_present_in_workspace_project.snap b/tests/snapshots/test_pnpm_workspace_prune_skipped_if_lifecycle_scripts_are_present_in_workspace_project.snap index bcb85d171..866dd8638 100644 --- a/tests/snapshots/test_pnpm_workspace_prune_skipped_if_lifecycle_scripts_are_present_in_workspace_project.snap +++ b/tests/snapshots/test_pnpm_workspace_prune_skipped_if_lifecycle_scripts_are_present_in_workspace_project.snap @@ -72,12 +72,13 @@ heroku/nodejs ! ! Since pruning can't be done safely for your build, it will be skipped. +- Cleaning up pnpm virtual store layer - Done (finished in ) ===> EXPORTING Adding layer 'heroku/nodejs:available_parallelism' Adding layer 'heroku/nodejs:dist' Adding layer 'heroku/nodejs:pnpm' - layer 'heroku/nodejs:virtual' +Adding layer 'heroku/nodejs:virtual' Adding layer 'heroku/nodejs:web_env' Adding layer 'heroku/nodejs:z_node_module_bins' Adding layer 'buildpacksio/lifecycle:launch.sbom'