Skip to content

Feature: reject partition fullscan as gpcontrib extension#1713

Open
lqriu wants to merge 2 commits intoapache:mainfrom
lqriu:feature/reject-partition-fullscan-extension
Open

Feature: reject partition fullscan as gpcontrib extension#1713
lqriu wants to merge 2 commits intoapache:mainfrom
lqriu:feature/reject-partition-fullscan-extension

Conversation

@lqriu
Copy link
Copy Markdown

@lqriu lqriu commented May 2, 2026

What does this PR do?

Add a planner_hook-based extension under gpcontrib/ that rejects queries
on partitioned tables when no effective partition pruning occurs. This prevents
unintended full partition scans in production environments.

This replaces the previous PR #1711, which modified core optimizer files.
Per reviewer feedback, the implementation is now fully self-contained as an
extension with zero core file changes (only gpcontrib/Makefile is touched
to register the extension in the build).

Architecture

The extension installs a planner_hook that wraps standard_planner().
After the planner (including ORCA) produces a PlannedStmt, the hook walks
the plan tree using three complementary detection strategies:

Check 1 — PartitionPruneInfo check (Planner + ORCA JOIN path):
For nodes with part_prune_info != NULL, compare present_parts vs nparts.
Exempt nodes with initial_pruning_steps or exec_pruning_steps (runtime
pruning capable, e.g., prepared statements).

Check 2 — Append heuristic (Planner path, no useful pruning quals):
When the standard Planner processes queries with no WHERE, WHERE 1=1, or
WHERE on non-partition-key columns, it does not generate PartitionPruneInfo.
We detect these by checking if Append.apprelids references a partitioned
table RTE (relkind='p', inh=true).

Check 3 — DynamicScan check (ORCA path):
ORCA never sets part_prune_info on DynamicSeqScan nodes. We detect full
scans by comparing list_length(partOids) against the total partition count
from pg_inherits. Nodes with join_prune_paramids are skipped (JOIN
dynamic pruning).

GUC parameters

  • reject_partition_fullscan (bool, default ON) — enable/disable
  • partition_fullscan_threshold (int, default 0) — max partitions allowed
    after pruning; 0 means reject only true fullscans

Exemptions

Scenario Behavior
reject_partition_fullscan = off Allowed
enable_partition_pruning = off Allowed
Single-partition table (nparts <= 1) Allowed
Runtime pruning steps present (prepared stmt) Allowed
JOIN dynamic pruning (PartitionSelector / join_prune_paramids) Allowed
No WHERE / WHERE 1=1 Rejected
WHERE on non-partition-key column Rejected

Error message

ERROR: partitioned table "schema.table" full partition scan is not
       allowed, N partitions would be scanned
HINT:  Add a WHERE clause on the partition key to enable partition pruning.

Usage

-- Via shared_preload_libraries (recommended for production)
-- postgresql.conf: shared_preload_libraries = 'reject_partition_fullscan'

-- Or per-session via LOAD
LOAD 'reject_partition_fullscan';

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update

Breaking Changes

When loaded, queries that previously performed full partition scans will
be rejected by default. Users can disable with:
SET reject_partition_fullscan = off;

Test Plan

  • New regression test: sql/partition_fullscan_reject.sql (12 scenarios)
    • Basic rejection (no WHERE, WHERE 1=1, non-partition-key WHERE)
    • Pruning passes (WHERE on partition key)
    • GUC on/off toggle
    • enable_partition_pruning=off exemption
    • Single-partition table exemption
    • Threshold mode (threshold=2)
    • Prepared statement Param exemption
    • UPDATE/DELETE rejection
    • Subquery propagation
  • Compiled with -Werror — zero warnings, zero errors
  • Manual functional testing on utility-mode instance
  • make installcheck (full cluster)

Impact

  • Performance: negligible — one lightweight plan_tree_walker after
    planning, added to the existing ~11 post-planning traversals
  • Core files changed: zero (only gpcontrib/Makefile modified)
  • User-facing: new ERROR for unfiltered partition queries; two new GUCs

Compliance Checklist

  • I have read the contribution guide
  • My code follows PostgreSQL coding conventions
  • New functionality includes tests
  • I have reviewed my changes for security implications
  • I have requested appropriate reviewers

Files Changed

File Change
gpcontrib/Makefile Register extension in build (+4/-2)
gpcontrib/reject_partition_fullscan/reject_partition_fullscan.c Extension source (+470)
gpcontrib/reject_partition_fullscan/Makefile Build config (+18)
gpcontrib/reject_partition_fullscan/reject_partition_fullscan.control Extension metadata (+4)
gpcontrib/reject_partition_fullscan/reject_partition_fullscan--1.0.sql SQL install script (+8)
gpcontrib/reject_partition_fullscan/sql/partition_fullscan_reject.sql Regression test (+113)

Add a planner_hook-based extension under gpcontrib/ that rejects
queries on partitioned tables when no effective partition pruning
occurs.  This keeps all core optimizer files untouched.

The extension uses three complementary detection strategies:
1. PartitionPruneInfo check: for Planner Append nodes with pruning
   conditions and ORCA PartitionSelector nodes.
2. Append heuristic: for Planner Append/MergeAppend without
   PartitionPruneInfo (no WHERE, WHERE 1=1, non-partition-key WHERE),
   detected by cross-referencing apprelids with partitioned-table RTEs.
3. DynamicScan check: for ORCA DynamicSeqScan and similar nodes whose
   part_prune_info is never set by ORCA, detected by comparing
   partOids count against catalog partition count.

New GUC parameters (registered via DefineCustomXxxVariable):
- reject_partition_fullscan (bool, default on)
- partition_fullscan_threshold (int, default 0)

Exemptions: enable_partition_pruning=off, single-partition tables,
runtime pruning steps (prepared statements), JOIN dynamic pruning.
Comment thread gpcontrib/Makefile Outdated
gp_toolkit \
pg_hint_plan
pg_hint_plan \
reject_partition_fullscan
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's set only enable_debug_extensions=yes, enable reject_partition_fullscan extension

Per reviewer feedback, restrict the extension to build only when
enable_debug_extensions=yes, rather than in all builds.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants