Skip to content

GH-46600: [C++][CI] Add job with ARROW_LARGE_MEMORY_TESTS enabled #49490

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
raulcd wants to merge 15 commits into
base: main
Choose a base branch
from
+18 −4
Open

GH-46600: [C++][CI] Add job with ARROW_LARGE_MEMORY_TESTS enabled #49490

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
2f90e15
GH-46600: [C++][CI] Add job with ARROW_LARGE_MEMORY_TESTS enabled
raulcd Mar 10, 2026
c972717
Try removing /spot=capacity-optimized to see if quota issues are solved
raulcd Mar 10, 2026
78a6b22
Force spot=false
raulcd Mar 10, 2026
e8c640f
Try with 2xlarge to see if tests finish without timing out
raulcd Mar 11, 2026
e316fb5
Try setting build type to Release and ctest timeout to 15 minutes
raulcd Mar 11, 2026
12b83e0
Try single line just to validate
raulcd Mar 11, 2026
581f506
That worked but what about removing the backslash
raulcd Mar 11, 2026
b0e1b36
Increase timeout even more
raulcd Mar 11, 2026
530cc0d
Add an absurdely high timeout
raulcd Mar 11, 2026
9359393
Use ARROW_C_FLAGS_DEBUG and ARROW_CXX_FLAGS_DEBUG -O1 and BUILD_WARNI…
raulcd Mar 11, 2026
b927560
Increase volume size to avoid out of disk space
raulcd Mar 12, 2026
4960ed2
Increase timeout for parquet-arrow-reader-writer-test
raulcd Mar 12, 2026
0bdc786
Reduce timeout it took 1522 seconds and add ARROW_GANDIVA=OFF
raulcd Mar 12, 2026
70f036a
Failed with 1800
raulcd Mar 12, 2026
bf0f9ab
GH-49502: [C++][Parquet] For those large memory tests we want huge pages
raulcd Mar 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/workflows/cpp_extra.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,16 @@ jobs:
- image: alpine-linux-cpp
runs-on: ubuntu-latest
title: AMD64 Alpine Linux
- image: ubuntu-cpp
run-options: >-
-e ARROW_CTEST_TIMEOUT=2000
-e ARROW_C_FLAGS_DEBUG="-O1"
-e ARROW_CXX_FLAGS_DEBUG="-O1"
-e ARROW_GANDIVA=OFF
-e ARROW_LARGE_MEMORY_TESTS=ON
-e BUILD_WARNING_LEVEL=PRODUCTION
runs-on: "runs-on=${{ github.run_id }}/family=x8i.2xlarge/volume=80gb/spot=capacity-optimized"
title: AMD64 Ubuntu Large Memory Tests
- image: conda-cpp
run-options: >-
-e ARROW_USE_MESON=ON
Expand Down
12 changes: 8 additions & 4 deletions cpp/src/parquet/column_writer_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1053,8 +1053,10 @@ TEST(TestColumnWriter, LARGE_MEMORY_TEST(WriteLargeDictEncodedPage)) {
{
PrimitiveNode::Make("item", Repetition::REQUIRED, Type::INT32),
}));
auto properties =
WriterProperties::Builder().data_pagesize(1024 * 1024 * 1024)->build();
auto properties = WriterProperties::Builder()
.data_pagesize(1024 * 1024 * 1024)
->max_rows_per_page(std::numeric_limits<int64_t>::max())
->build();
auto file_writer = ParquetFileWriter::Open(sink, schema, properties);
auto rg_writer = file_writer->AppendRowGroup();

Expand Down Expand Up @@ -1124,8 +1126,10 @@ TEST(TestColumnWriter, LARGE_MEMORY_TEST(ThrowsOnDictIndicesTooLarge)) {
{
PrimitiveNode::Make("item", Repetition::REQUIRED, Type::INT32),
}));
auto properties =
WriterProperties::Builder().data_pagesize(4 * 1024LL * 1024 * 1024)->build();
auto properties = WriterProperties::Builder()
.data_pagesize(4 * 1024LL * 1024 * 1024)
->max_rows_per_page(std::numeric_limits<int64_t>::max())
->build();
auto file_writer = ParquetFileWriter::Open(sink, schema, properties);
auto rg_writer = file_writer->AppendRowGroup();

Expand Down
Loading