Skip to content

Improve buggy stdlib bundling on ScalaJS#1386

Open
jiribenes wants to merge 8 commits intomainfrom
jiribenes/fix/build
Open

Improve buggy stdlib bundling on ScalaJS#1386
jiribenes wants to merge 8 commits intomainfrom
jiribenes/fix/build

Conversation

@jiribenes
Copy link
Copy Markdown
Contributor

@jiribenes jiribenes commented Apr 24, 2026

ScalaJS' pickler has some limit on string sizes which the recent stdlib additions changes overshoot by a lot on my computer.
This was preventing me from building the project locally.

The fix is horrible: we just chunk the string instead. Perhaps there's a better fix.

We generate a stdlib-resources.json file which is then loaded by JS, thus removing the !!!MULTILINEMARKER!!! hack and generating huge Scala files altogether. :)

This ought to:


[info] compiling 111 Scala sources to /nix/var/nix/builds/nix-63932-2468348747/effekt-src/effekt/js/target/scala-3.3.6/classes ...

  unhandled exception while running pickler on /nix/var/nix/builds/nix-63932-2468348747/effekt-src/effekt/js/target/scala-3.3.6/src_managed/main/Resour>

  An unhandled exception was thrown in the compiler.
  Please file a crash report here:
  https://github.com/scala/scala3/issues/new/choose
  For non-enriched exceptions, compile with -Xno-enrich-error-messages.


     while compiling: /nix/var/nix/builds/nix-63932-2468348747/effekt-src/effekt/js/target/scala-3.3.6/src_managed/main/Resources.scala
        during phase: pickler
                mode: Mode(ImplicitsEnabled,ReadPositions)
     library version: version 2.13.16
    compiler version: version 3.3.6
            settings: -Werror true -Xsemanticdb true -bootclasspath /nix/var/nix/builds/nix-63932-2468348747/tmp.aM6MNA3Y1w/project/.coursier/https/rep>

[error] ## Exception when compiling 111 sources to /nix/var/nix/builds/nix-63932-2468348747/effekt-src/effekt/js/target/scala-3.3.6/classes
[error] java.lang.ArrayIndexOutOfBoundsException: arraycopy: last destination index 3955373 out of bounds for char[2097152]
[error] java.base/java.lang.System.arraycopy(Native Method)
[error] scala.Array$.copy(Array.scala:110)

@jiribenes jiribenes added the bug Something isn't working label Apr 24, 2026
@jiribenes
Copy link
Copy Markdown
Contributor Author

I might have a better solution: generate a JSON { ..."<path>": "<file contents>" } and then import it from JavaScript.

@jiribenes jiribenes changed the title Split huge string in generated Resources.scala Improve buggy stdlib bundling on ScalaJS Apr 24, 2026
Comment thread build.sbt
Comment on lines 279 to 285
assembleJS := {
(Compile / clean).value
(Compile / compile).value
val jsFile = (Compile / fullOptJS).value.data
(Compile / fullLinkJS).value
val jsFile = (Compile / fullLinkJS / scalaJSLinkerOutputDirectory).value / "main.js"
val outputFile = (ThisBuild / baseDirectory).value / "out" / "effekt.js"
IO.copyFile(jsFile, outputFile)
},
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this doesn't look like it actually embeds the stdlib-resources.json, we might need a bundler or also copy that file onto the website...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant