-
Notifications
You must be signed in to change notification settings - Fork 100
CLOUD-4245 improve-redis-search-docs #651
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 2 commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -16,8 +16,19 @@ Use bucket aggregations when you want segmented analytics (for example by catego | |
| | [`$dateHistogram`](./date-histogram) | Fixed date/time intervals | | ||
| | [`$facet`](./facet) | Hierarchical FACET paths | | ||
|
|
||
| ### Input Format | ||
|
|
||
| Every bucket operator takes an object with a `field` property and operator-specific parameters: | ||
|
|
||
| ```json | ||
| {"by_category": {"$terms": {"field": "category", "size": 10}}} | ||
| {"price_ranges": {"$range": {"field": "price", "ranges": [{"to": 50}, {"from": 50, "to": 100}, {"from": 100}]}}} | ||
| {"by_month": {"$histogram": {"field": "price", "interval": 10}}} | ||
|
||
| {"by_date": {"$dateHistogram": {"field": "createdAt", "calendarInterval": "month"}}} | ||
| ``` | ||
|
|
||
| ### Behavior Notes | ||
|
|
||
| - Bucket operators can contain nested `$aggs`. | ||
| - Bucket operators can contain nested `$aggs` for per-bucket metrics. | ||
| - `$terms`, `$range`, `$histogram`, and `$dateHistogram` support nested `$aggs`. | ||
| - `$facet` does not support nested `$aggs` and cannot be used as a sub-aggregation. | ||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -20,9 +20,21 @@ Use metrics when you want one value (or stats object), not grouped buckets. | |||||
| | [`$extendedStats`](./extended-stats) | `$stats` + variance and std deviation metrics | | ||||||
| | [`$percentiles`](./percentiles) | Distribution percent points | | ||||||
|
|
||||||
| ### Input Format | ||||||
|
|
||||||
| Every metric operator takes an object with at least a `field` property: | ||||||
|
|
||||||
| ```json | ||||||
| {"alias_name": {"$avg": {"field": "price"}}} | ||||||
| {"alias_name": {"$avg": {"field": "price", "missing": 0}}} | ||||||
| ``` | ||||||
|
|
||||||
| The `field` value must be a string pointing to a FAST field in your schema. | ||||||
| Do **not** pass a bare string — it must be an object with `{"field": "..."}`. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| ### Behavior Notes | ||||||
|
|
||||||
| - Metric operators require a `field`. | ||||||
| - The field must be `FAST` in your schema. | ||||||
| - Metric operators require a `field` — you will get `missing required 'field' property` if it's omitted. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| - The field must be `FAST` in your schema — otherwise you will get an error: `operator '$avg' requires field '<field>' to be FAST`. See [FAST Fields](/redis/search/schema-definition#fast-fields). | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| - Metric operators do not support nested `$aggs`. | ||||||
| - For many metric operators, `missing` lets you provide a fallback value for documents where the field does not exist. | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -47,6 +47,68 @@ Aggregation requests have two phases: | |
|
|
||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ⚪ INFO — Response field naming differs between SDK examples without explicit explanation In Recommendation: Consider adding a brief note (e.g., in a [architecture] confidence: 65% |
||
| Each aggregation is defined with an **alias** (the key you choose for the result) and an **operator** that specifies what to compute. | ||
|
|
||
| ### Response Format | ||
|
|
||
| <Tabs> | ||
|
|
||
| <Tab title="TypeScript"> | ||
| ```ts | ||
| const result = await index.aggregate({ | ||
| aggregations: { | ||
| avg_price: { $avg: { field: "price" } }, | ||
| by_category: { $terms: { field: "category", size: 5 } }, | ||
| }, | ||
| }); | ||
|
|
||
| // result is an object keyed by alias: | ||
| // { | ||
| // avg_price: 49.99, | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. is this output correct? the backend returns something like {"avg_price":{"value":49.99}} |
||
| // by_category: [ | ||
| // { key: "electronics", docCount: 42 }, | ||
| // { key: "clothing", docCount: 31 }, | ||
| // ] | ||
| // } | ||
| ``` | ||
| </Tab> | ||
|
|
||
| <Tab title="Python"> | ||
| ```python | ||
| result = index.aggregate( | ||
| aggregations={ | ||
| "avg_price": {"$avg": {"field": "price"}}, | ||
| "by_category": {"$terms": {"field": "category", "size": 5}}, | ||
| }, | ||
| ) | ||
|
|
||
| # result is a dict keyed by alias: | ||
| # { | ||
| # "avg_price": 49.99, | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same as above |
||
| # "by_category": [ | ||
| # {"key": "electronics", "doc_count": 42}, | ||
| # {"key": "clothing", "doc_count": 31}, | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. are we doing anything special on the python sdk? if not, server sends docCount not doc_count |
||
| # ] | ||
| # } | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔵 LOW — Inconsistent response key casing between TypeScript and Python examples In the aggregations response format, the TypeScript example uses Recommendation: Add a brief note clarifying that each SDK normalizes response keys to its language's conventions (camelCase for TS, snake_case for Python), or verify the actual SDK behavior and correct whichever is wrong. [dx] confidence: 72% |
||
| ``` | ||
| </Tab> | ||
|
|
||
| <Tab title="Redis CLI"> | ||
| ```bash | ||
| SEARCH.AGGREGATE products '{}' '{"avg_price": {"$avg": {"field": "price"}}, "by_category": {"$terms": {"field": "category", "size": 5}}}' | ||
|
|
||
| # Response is a flat array of alternating alias-value pairs: | ||
| # ["avg_price", 49.99, "by_category", [["electronics", ["doc_count", 42]], ["clothing", ["doc_count", 31]]]] | ||
| ``` | ||
| </Tab> | ||
|
|
||
| </Tabs> | ||
|
|
||
| <Note> | ||
| All metric aggregation operators require the target field to be marked as `FAST` in your schema. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this is already mentioned. i would just remove this warning |
||
| If the field is not FAST, you will get an error like: | ||
| `Aggregation '<name>' operator '$avg' requires field '<field>' to be FAST`. | ||
| See [FAST Fields](/redis/search/schema-definition#fast-fields) for details. | ||
| </Note> | ||
|
|
||
| ## Filtering | ||
|
|
||
| Use `filter` to restrict which documents participate in the aggregation. | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,275 @@ | ||
| --- | ||
| title: Command Reference | ||
| --- | ||
|
|
||
| A complete reference of all Upstash Redis Search commands, their syntax, and return values. | ||
|
|
||
| <Note> | ||
| Upstash Redis Search uses `SEARCH.*` commands. These are **not** the same as the `FT.*` commands | ||
| from the open-source RediSearch module. The two are completely separate implementations and are | ||
| not compatible with each other. | ||
| </Note> | ||
|
|
||
| ## Index Commands | ||
|
|
||
| ### SEARCH.CREATE | ||
|
|
||
| Creates a new search index. | ||
|
|
||
| ```bash | ||
| SEARCH.CREATE <name> ON <JSON|HASH|STRING> | ||
| PREFIX <count> <prefix> [<prefix> ...] | ||
| [LANGUAGE <language>] | ||
| [SKIPINITIALSCAN] | ||
| [EXISTSOK] | ||
| SCHEMA <field> <type> [FAST] [NOSTEM] [NOTOKENIZE] [FROM <source_field>] ... | ||
| ``` | ||
|
|
||
| **Returns:** `1` on success. Returns an error if the index already exists (unless `EXISTSOK` is used). | ||
|
|
||
| <Tabs> | ||
| <Tab title="Redis CLI"> | ||
| ```bash | ||
| SEARCH.CREATE products ON JSON PREFIX 1 product: SCHEMA name TEXT price F64 FAST inStock BOOL | ||
| ``` | ||
| </Tab> | ||
| <Tab title="curl"> | ||
| ```bash | ||
| curl -X POST https://YOUR_ENDPOINT.upstash.io \ | ||
| -H "Authorization: Bearer YOUR_TOKEN" \ | ||
|
||
| -d '["SEARCH.CREATE", "products", "ON", "JSON", "PREFIX", "1", "product:", "SCHEMA", "name", "TEXT", "price", "F64", "FAST", "inStock", "BOOL"]' | ||
| ``` | ||
| </Tab> | ||
| </Tabs> | ||
|
|
||
| --- | ||
|
|
||
| ### SEARCH.DROP | ||
|
|
||
| Removes an index. The underlying Redis keys are **not** deleted. | ||
|
|
||
| ```bash | ||
| SEARCH.DROP <name> | ||
| ``` | ||
|
|
||
| **Returns:** `1` if dropped, `0` if the index was not found. | ||
|
|
||
| --- | ||
|
|
||
| ### SEARCH.DESCRIBE | ||
|
|
||
| Returns metadata about an index. | ||
|
|
||
| ```bash | ||
| SEARCH.DESCRIBE <name> | ||
| ``` | ||
|
|
||
| **Returns:** Index metadata including name, data type, prefixes, language, and schema definition. Returns `null` if the index does not exist. | ||
|
|
||
| --- | ||
|
|
||
| ### SEARCH.WAITINDEXING | ||
|
|
||
| Blocks until all pending index updates are processed and visible to queries. | ||
|
|
||
| ```bash | ||
| SEARCH.WAITINDEXING <name> | ||
| ``` | ||
|
|
||
| **Returns:** `1` when indexing is complete, `0` if the index was not found. | ||
|
|
||
| <Warning> | ||
| Do not call `SEARCH.WAITINDEXING` after every write. Batch updates are necessary for | ||
| optimal indexing performance. Use this command only when you need to ensure queries | ||
| reflect recent changes, such as in tests or CI pipelines. | ||
| </Warning> | ||
|
|
||
| --- | ||
|
|
||
| ## Query Commands | ||
|
|
||
| ### SEARCH.QUERY | ||
|
|
||
| Searches for documents matching a JSON filter. | ||
|
|
||
| ```bash | ||
| SEARCH.QUERY <name> '<json_filter>' | ||
| [LIMIT <count>] | ||
| [OFFSET <offset>] | ||
| [ORDERBY <field> <ASC|DESC>] | ||
| [SELECT <count> <field> [<field> ...]] | ||
| [NOCONTENT] | ||
| [HIGHLIGHT FIELDS <count> <field> [<field> ...] [TAGS <open> <close>]] | ||
| [SCOREFUNC ...] | ||
| ``` | ||
|
|
||
| **Parameters:** | ||
|
|
||
| | Parameter | Description | Default | | ||
| |-----------|-------------|---------| | ||
| | `LIMIT` | Maximum number of results to return. Must be between 1 and 1000. | 10 | | ||
| | `OFFSET` | Number of results to skip (for pagination). | 0 | | ||
| | `ORDERBY` | Sort by a FAST field in `ASC` or `DESC` order. | Sort by relevance score (descending) | | ||
| | `SELECT` | Return only specific fields. Prefix with count of fields. | All fields | | ||
| | `NOCONTENT` | Return only keys and scores, no document content. | Disabled | | ||
| | `HIGHLIGHT` | Wrap matching terms in tags. Default tags: `<em>` / `</em>`. | Disabled | | ||
| | `SCOREFUNC` | Adjust relevance scores using numeric field values. | Disabled | | ||
|
|
||
| **Response format (Redis CLI):** | ||
|
|
||
| ``` | ||
| [ | ||
| ["key1", "score1", [["$", "{\"name\": \"...\", ...}"]]], | ||
| ["key2", "score2", [["$", "{\"name\": \"...\", ...}"]]] | ||
| ] | ||
| ``` | ||
|
|
||
| Each result is an array of `[key, score, content]` where: | ||
| - `key` — the Redis key of the matching document | ||
| - `score` — relevance score (float as string) | ||
| - `content` — array of field-value pairs (for JSON: `[["$", "<json_string>"]]`) | ||
|
|
||
| When `NOCONTENT` is used, the content element is omitted. | ||
| When `SELECT` is used, only the selected fields appear in the content. | ||
|
|
||
| <Tabs> | ||
| <Tab title="Redis CLI"> | ||
| ```bash | ||
| SEARCH.QUERY products '{"name": "wireless"}' LIMIT 10 OFFSET 0 | ||
| ``` | ||
| </Tab> | ||
| <Tab title="curl"> | ||
| ```bash | ||
| curl -X POST https://YOUR_ENDPOINT.upstash.io \ | ||
| -H "Authorization: Bearer YOUR_TOKEN" \ | ||
|
||
| -d '["SEARCH.QUERY", "products", "{\"name\": \"wireless\"}", "LIMIT", "10", "OFFSET", "0"]' | ||
| ``` | ||
| </Tab> | ||
| </Tabs> | ||
|
|
||
| --- | ||
|
|
||
| ### SEARCH.COUNT | ||
|
|
||
| Returns the number of documents matching a query without retrieving them. | ||
|
|
||
| ```bash | ||
| SEARCH.COUNT <name> '<json_filter>' | ||
| ``` | ||
|
|
||
| **Returns:** An integer — the count of matching documents. | ||
|
|
||
| <Tabs> | ||
| <Tab title="Redis CLI"> | ||
| ```bash | ||
| SEARCH.COUNT products '{"inStock": true}' | ||
| ``` | ||
| </Tab> | ||
| <Tab title="curl"> | ||
| ```bash | ||
| curl -X POST https://YOUR_ENDPOINT.upstash.io \ | ||
| -H "Authorization: Bearer YOUR_TOKEN" \ | ||
|
||
| -d '["SEARCH.COUNT", "products", "{\"inStock\": true}"]' | ||
| ``` | ||
| </Tab> | ||
| </Tabs> | ||
|
|
||
| --- | ||
|
|
||
| ### SEARCH.AGGREGATE | ||
|
|
||
| Computes analytics (metrics and buckets) over matching documents. | ||
|
|
||
| ```bash | ||
| SEARCH.AGGREGATE <name> '<json_filter>' '<json_aggregations>' | ||
| ``` | ||
|
|
||
| **Response format (Redis CLI):** | ||
|
|
||
| The response is a flat array of alternating alias-value pairs: | ||
|
|
||
| ``` | ||
| ["alias1", <value1>, "alias2", <value2>, ...] | ||
| ``` | ||
|
|
||
| Where each value depends on the aggregation type: | ||
| - **Metric operators** (`$avg`, `$sum`, etc.) — a single number or stats object | ||
| - **Bucket operators** (`$terms`, `$range`, etc.) — an array of bucket objects | ||
|
||
|
|
||
| <Tabs> | ||
| <Tab title="Redis CLI"> | ||
| ```bash | ||
| SEARCH.AGGREGATE products '{}' '{"avg_price": {"$avg": {"field": "price"}}}' | ||
| ``` | ||
| </Tab> | ||
| <Tab title="curl"> | ||
| ```bash | ||
| curl -X POST https://YOUR_ENDPOINT.upstash.io \ | ||
| -H "Authorization: Bearer YOUR_TOKEN" \ | ||
|
||
| -d '["SEARCH.AGGREGATE", "products", "{}", "{\"avg_price\": {\"$avg\": {\"field\": \"price\"}}}"]' | ||
| ``` | ||
| </Tab> | ||
| </Tabs> | ||
|
|
||
| --- | ||
|
|
||
| ## Alias Commands | ||
|
|
||
| ### SEARCH.ALIASADD | ||
|
|
||
| Creates or updates an alias pointing to an index. | ||
|
|
||
| ```bash | ||
| SEARCH.ALIASADD <alias> <index_name> | ||
| ``` | ||
|
|
||
| **Returns:** `1` if a new alias was created, `2` if an existing alias was updated. | ||
|
|
||
| --- | ||
|
|
||
| ### SEARCH.ALIASDEL | ||
|
|
||
| Removes an alias. | ||
|
|
||
| ```bash | ||
| SEARCH.ALIASDEL <alias> | ||
| ``` | ||
|
|
||
| **Returns:** `1` if deleted, `0` if the alias was not found. | ||
|
|
||
| --- | ||
|
|
||
| ### SEARCH.LISTALIASES | ||
|
|
||
| Returns all aliases and the indices they point to. | ||
|
|
||
| ```bash | ||
| SEARCH.LISTALIASES | ||
| ``` | ||
|
|
||
| **Returns:** An array of `[alias, index_name]` pairs. | ||
|
|
||
| --- | ||
|
|
||
| ## REST API Usage | ||
|
|
||
| All search commands can be sent via the Upstash REST API using a JSON array POST body. | ||
| Each element of the array corresponds to a token in the command. | ||
|
|
||
| ```bash | ||
| curl -X POST https://YOUR_ENDPOINT.upstash.io \ | ||
| -H "Authorization: Bearer YOUR_TOKEN" \ | ||
|
||
| -d '["COMMAND", "arg1", "arg2", ...]' | ||
| ``` | ||
|
|
||
| The JSON filter and aggregation arguments are passed as string values within the array (not as nested JSON objects): | ||
|
|
||
| ```bash | ||
| # Correct — filter is a string inside the array | ||
| ["SEARCH.QUERY", "products", "{\"name\": \"wireless\"}", "LIMIT", "10"] | ||
|
|
||
| # Incorrect — filter is a nested object | ||
| ["SEARCH.QUERY", "products", {"name": "wireless"}, "LIMIT", "10"] | ||
| ``` | ||
|
|
||
| Search commands also work through the `/pipeline` endpoint for batching multiple commands in a single HTTP request. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
⚪ INFO — Bucket aggregation 'Input Format' block is not valid JSON (multiple root objects)
The Input Format example in the bucket aggregations overview shows four separate JSON objects on consecutive lines inside a single
```jsonblock. This is not valid JSON (a JSON document must have a single root value). While each line is individually valid, the block as a whole could confuse readers or tools.Recommendation: Either separate each example into its own code block, or use JSONL / plain text syntax highlighting, or wrap them in an array. A common pattern is to show each operator as a separate mini-example with a label.
[dx] confidence: 75%