Query for less data on pagination count

[rhysyngsun]

What are the relevant tickets?

Description (What does it do?)

• Refactors our pagination to have just 2 main implementations.
• Sets pagination class default and opts-out noncompliant views explicitly to maintain their existing interfaces.
• Updates the default pagination implementation to only query for the pk by default (this can be overridden if needed by subclassing).

The effect of this to turn queries like this that DRF previously ran to determine count (this example is for the /api/v1/featured/ endpoint):

SELECT COUNT(*)
FROM (
  SELECT DISTINCT "learning_resources_learningresource"."id" AS "col1",
    "learning_resources_learningresource"."created_on" AS "col2",
    "learning_resources_learningresource"."updated_on" AS "col3",
    "learning_resources_learningresource"."readable_id" AS "col4",
    "learning_resources_learningresource"."title" AS "col5",
    "learning_resources_learningresource"."description" AS "col6",
    "learning_resources_learningresource"."full_description" AS "col7",
    "learning_resources_learningresource"."last_modified" AS "col8",
    "learning_resources_learningresource"."published" AS "col9",
    "learning_resources_learningresource"."languages" AS "col10",
    "learning_resources_learningresource"."url" AS "col11",
    "learning_resources_learningresource"."image_id" AS "col12",
    "learning_resources_learningresource"."platform_id" AS "col13",
    "learning_resources_learningresource"."certification" AS "col14",
    "learning_resources_learningresource"."certification_type" AS "col15",
    "learning_resources_learningresource"."resource_type" AS "col16",
    "learning_resources_learningresource"."resource_category" AS "col17",
    "learning_resources_learningresource"."ocw_topics" AS "col18",
    "learning_resources_learningresource"."offered_by_id" AS "col19",
    "learning_resources_learningresource"."etl_source" AS "col20",
    "learning_resources_learningresource"."professional" AS "col21",
    "learning_resources_learningresource"."next_start_date" AS "col22",
    "learning_resources_learningresource"."prices" AS "col23",
    "learning_resources_learningresource"."availability" AS "col24",
    "learning_resources_learningresource"."completeness" AS "col25",
    "learning_resources_learningresource"."delivery" AS "col26",
    "learning_resources_learningresource"."license_cc" AS "col27",
    "learning_resources_learningresource"."test_mode" AS "col28",
    "learning_resources_learningresource"."continuing_ed_credits" AS "col29",
    "learning_resources_learningresource"."pace" AS "col30",
    "learning_resources_learningresource"."format" AS "col31",
    "learning_resources_learningresource"."location" AS "col32",
    "learning_resources_learningresource"."duration" AS "col33",
    "learning_resources_learningresource"."min_weeks" AS "col34",
    "learning_resources_learningresource"."max_weeks" AS "col35",
    "learning_resources_learningresource"."time_commitment" AS "col36",
    "learning_resources_learningresource"."min_weekly_hours" AS "col37",
    "learning_resources_learningresource"."max_weekly_hours" AS "col38",
    "learning_resources_learningresource"."require_summaries" AS "col39", COUNT(
    "learning_resources_learningresourceviewevent"."id") AS "_views_count",
    "learning_resources_learningresourcerelationship"."position" AS "position"
  FROM "learning_resources_learningresource"
  LEFT OUTER JOIN "learning_resources_learningresourceviewevent" ON (
    "learning_resources_learningresource"."id" =
    "learning_resources_learningresourceviewevent"."learning_resource_id")
  INNER JOIN "learning_resources_learningresourcerelationship" ON (
    "learning_resources_learningresource"."id" =
    "learning_resources_learningresourcerelationship"."child_id")
  WHERE (
    "learning_resources_learningresourcerelationship"."parent_id" IN (
      SELECT U0."featured_list_id"
      FROM "channels_channel" U0
      WHERE U0."channel_type" = 'unit'
    ) AND "learning_resources_learningresource"."published"
  )

into a query like this:

SELECT COUNT(*) 
FROM 
  (
    SELECT 
      DISTINCT "learning_resources_learningresource"."id" AS "col1", 
      "learning_resources_learningresourcerelationship"."position" AS "position" 
    FROM "learning_resources_learningresource" INNER JOIN "learning_resources_learningresourcerelationship" ON (
        "learning_resources_learningresource"."id" = "learning_resources_learningresourcerelationship"."child_id"
      ) 
    WHERE 
      (
        "learning_resources_learningresourcerelationship"."parent_id" IN (
          SELECT 
            U0."featured_list_id" 
          FROM "channels_channel" U0 
          WHERE U0."channel_type" = 'unit'
        ) 
        AND "learning_resources_learningresource"."published"
      )
  ) subquery;

The difference in performance between these two specific queries is ~1000x faster, going from ~500ms for the version currently in main to 0.3ms for this branch. I don't expect to see exactly the same improvement across the board, but there should be some kind of improvement because the count query won't be going to disk as much for data and ideally just hits a few indices now.

How can this be tested?

The app should still function. You likely won't see a huge difference in performance locally unless you have production-scale data around learning resources.

[github-actions]

OpenAPI Changes

Show/hide No detectable change.

Unexpected changes? Ensure your branch is up-to-date with main (consider rebasing).