Files
supabase/apps/docs/content/guides/database/full-text-search.mdx
T
2025-02-21 14:13:00 -05:00

1062 lines
24 KiB
Plaintext

---
id: 'full-text-search'
title: 'Full Text Search'
description: 'How to use full text search in PostgreSQL.'
subtitle: 'How to use full text search in PostgreSQL.'
tocVideo: 'b-mgca_2Oe4'
---
Postgres has built-in functions to handle `Full Text Search` queries. This is like a "search engine" within Postgres.
<div className="video-container">
<iframe
src="https://www.youtube-nocookie.com/embed/b-mgca_2Oe4"
frameBorder="1"
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
allowFullScreen
></iframe>
</div>
## Preparation
For this guide we'll use the following example data:
<Tabs
scrollable
size="small"
type="underlined"
defaultActiveId="data"
queryGroup="example-view"
>
<TabPanel id="data" label="Data">
{/* supa-mdx-lint-disable Rule003Spelling */}
| id | title | author | description |
| --- | ----------------------------------- | ---------------------- | ------------------------------------------------------------------ |
| 1 | The Poky Little Puppy | Janette Sebring Lowrey | Puppy is slower than other, bigger animals. |
| 2 | The Tale of Peter Rabbit | Beatrix Potter | Rabbit eats some vegetables. |
| 3 | Tootle | Gertrude Crampton | Little toy train has big dreams. |
| 4 | Green Eggs and Ham | Dr. Seuss | Sam has changing food preferences and eats unusually colored food. |
| 5 | Harry Potter and the Goblet of Fire | J.K. Rowling | Fourth year of school starts, big drama ensues. |
{/* supa-mdx-lint-enable Rule003Spelling */}
</TabPanel>
<TabPanel id="sql" label="SQL">
```sql
create table books (
id serial primary key,
title text,
author text,
description text
);
insert into books
(title, author, description)
values
(
'The Poky Little Puppy',
'Janette Sebring Lowrey',
'Puppy is slower than other, bigger animals.'
),
('The Tale of Peter Rabbit', 'Beatrix Potter', 'Rabbit eats some vegetables.'),
('Tootle', 'Gertrude Crampton', 'Little toy train has big dreams.'),
(
'Green Eggs and Ham',
'Dr. Seuss',
'Sam has changing food preferences and eats unusually colored food.'
),
(
'Harry Potter and the Goblet of Fire',
'J.K. Rowling',
'Fourth year of school starts, big drama ensues.'
);
```
</TabPanel>
</Tabs>
## Usage
The functions we'll cover in this guide are:
### `to_tsvector()` [#to-tsvector]
Converts your data into searchable tokens. `to_tsvector()` stands for "to text search vector." For example:
```sql
select to_tsvector('green eggs and ham');
-- Returns 'egg':2 'green':1 'ham':4
```
Collectively these tokens are called a "document" which Postgres can use for comparisons.
### `to_tsquery()` [#to-tsquery]
Converts a query string into tokens to match. `to_tsquery()` stands for "to text search query."
This conversion step is important because we will want to "fuzzy match" on keywords.
For example if a user searches for `eggs`, and a column has the value `egg`, we probably still want to return a match.
### Match: `@@` [#match]
The `@@` symbol is the "match" symbol for Full Text Search. It returns any matches between a `to_tsvector` result and a `to_tsquery` result.
Take the following example:
<Tabs
scrollable
size="small"
type="underlined"
defaultActiveId="sql"
queryGroup="language"
>
<TabPanel id="sql" label="SQL">
```sql
select *
from books
where title = 'Harry';
```
</TabPanel>
<TabPanel id="js" label="JavaScript">
```js
const { data, error } = await supabase.from('books').select().eq('title', 'Harry')
```
</TabPanel>
<TabPanel id="dart" label="Dart">
```dart
final result = await client
.from('books')
.select()
.eq('title', 'Harry');
```
</TabPanel>
<TabPanel id="swift" label="Swift">
```swift
let response = try await supabase.from("books")
.select()
.eq("title", value: "Harry")
.execute()
```
</TabPanel>
<TabPanel id="kotlin" label="Kotlin">
```kotlin
val data = supabase.from("books").select {
filter {
eq("title", "Harry")
}
}
```
</TabPanel>
<TabPanel id="python" label="Python">
```python
data = supabase.from_('books').select().eq('title', 'Harry').execute()
```
</TabPanel>
</Tabs>
The equality symbol above (`=`) is very "strict" on what it matches. In a full text search context, we might want to find all "Harry Potter" books and so we can rewrite the
example above:
<Tabs
scrollable
size="small"
type="underlined"
defaultActiveId="sql"
queryGroup="language"
>
<TabPanel id="sql" label="SQL">
```sql
select *
from books
where to_tsvector(title) @@ to_tsquery('Harry');
```
</TabPanel>
<TabPanel id="js" label="JavaScript">
```js
const { data, error } = await supabase.from('books').select().textSearch('title', `'Harry'`)
```
</TabPanel>
<TabPanel id="dart" label="Dart">
```dart
final result = await client
.from('books')
.select()
.textSearch('title', "'Harry'");
```
</TabPanel>
<TabPanel id="swift" label="Swift">
```swift
let response = try await supabase.from("books")
.select()
.textSearch("title", value: "'Harry'")
```
</TabPanel>
<TabPanel id="kotlin" label="Kotlin">
```kotlin
val data = supabase.from("books").select {
filter {
textSearch("title", "'Harry'", TextSearchType.NONE)
}
}
```
</TabPanel>
</Tabs>
## Basic full text queries
### Search a single column
To find all `books` where the `description` contain the word `big`:
<Tabs
scrollable
size="small"
type="underlined"
defaultActiveId="sql"
queryGroup="language"
>
<TabPanel id="sql" label="SQL">
```sql
select
*
from
books
where
to_tsvector(description)
@@ to_tsquery('big');
```
</TabPanel>
<TabPanel id="js" label="JavaScript">
```js
const { data, error } = await supabase.from('books').select().textSearch('description', `'big'`)
```
</TabPanel>
<TabPanel id="dart" label="Dart">
```dart
final result = await client
.from('books')
.select()
.textSearch('description', "'big'");
```
</TabPanel>
<TabPanel id="swift" label="Swift">
```swift
let response = await client.from("books")
.select()
.textSearch("description", value: "'big'")
.execute()
```
</TabPanel>
<TabPanel id="kotlin" label="Kotlin">
```kotlin
val data = supabase.from("books").select {
filter {
textSearch("description", "'big'", TextSearchType.NONE)
}
}
```
</TabPanel>
<TabPanel id="python" label="Python">
```python
data = supabase.from_('books').select().text_search('description', "'big'").execute()
```
</TabPanel>
<TabPanel id="data" label="Data">
{/* supa-mdx-lint-disable Rule003Spelling */}
| id | title | author | description |
| --- | ----------------------------------- | ----------------- | ----------------------------------------------- |
| 3 | Tootle | Gertrude Crampton | Little toy train has big dreams. |
| 5 | Harry Potter and the Goblet of Fire | J.K. Rowling | Fourth year of school starts, big drama ensues. |
{/* supa-mdx-lint-enable Rule003Spelling */}
</TabPanel>
</Tabs>
### Search multiple columns
Right now there is no direct way to use JavaScript or Dart to search through multiple columns but you can do it by creating [computed columns](https://postgrest.org/en/stable/api.html#computed-virtual-columns) on the database.
To find all `books` where `description` or `title` contain the word `little`:
<Tabs
scrollable
size="small"
type="underlined"
defaultActiveId="sql"
queryGroup="language"
>
<TabPanel id="sql" label="SQL">
```sql
select
*
from
books
where
to_tsvector(description || ' ' || title) -- concat columns, but be sure to include a space to separate them!
@@ to_tsquery('little');
```
</TabPanel>
<TabPanel id="js" label="JavaScript">
```sql
create function title_description(books) returns text as $$
select $1.title || ' ' || $1.description;
$$ language sql immutable;
```
```js
const { data, error } = await supabase
.from('books')
.select()
.textSearch('title_description', `little`)
```
</TabPanel>
<TabPanel id="dart" label="Dart">
```sql
create function title_description(books) returns text as $$
select $1.title || ' ' || $1.description;
$$ language sql immutable;
```
```dart
final result = await client
.from('books')
.select()
.textSearch('title_description', "little")
```
</TabPanel>
<TabPanel id="swift" label="Swift">
```sql
create function title_description(books) returns text as $$
select $1.title || ' ' || $1.description;
$$ language sql immutable;
```
```swift
let response = try await client
.from("books")
.select()
.textSearch("title_description", value: "little")
.execute()
```
</TabPanel>
<TabPanel id="kotlin" label="Kotlin">
```sql
create function title_description(books) returns text as $$
select $1.title || ' ' || $1.description;
$$ language sql immutable;
```
```kotlin
val data = supabase.from("books").select {
filter {
textSearch("title_description", "title", TextSearchType.NONE)
}
}
```
</TabPanel>
<TabPanel id="python" label="Python">
```sql
create function title_description(books) returns text as $$
select $1.title || ' ' || $1.description;
$$ language sql immutable;
```
```python
data = supabase.from_('books').select().text_search('title_description', "little").execute()
```
</TabPanel>
<TabPanel id="data" label="Data">
{/* supa-mdx-lint-disable Rule003Spelling */}
| id | title | author | description |
| --- | --------------------- | ---------------------- | ------------------------------------------- |
| 1 | The Poky Little Puppy | Janette Sebring Lowrey | Puppy is slower than other, bigger animals. |
| 3 | Tootle | Gertrude Crampton | Little toy train has big dreams. |
{/* supa-mdx-lint-enable Rule003Spelling */}
</TabPanel>
</Tabs>
### Match all search words
To find all `books` where `description` contains BOTH of the words `little` and `big`, we can use the `&` symbol:
<Tabs
scrollable
size="small"
type="underlined"
defaultActiveId="sql"
queryGroup="language"
>
<TabPanel id="sql" label="SQL">
```sql
select
*
from
books
where
to_tsvector(description)
@@ to_tsquery('little & big'); -- use & for AND in the search query
```
</TabPanel>
<TabPanel id="js" label="JavaScript">
```js
const { data, error } = await supabase
.from('books')
.select()
.textSearch('description', `'little' & 'big'`)
```
</TabPanel>
<TabPanel id="dart" label="Dart">
```dart
final result = await client
.from('books')
.select()
.textSearch('description', "'little' & 'big'");
```
</TabPanel>
<TabPanel id="swift" label="Swift">
```swift
let response = try await client
.from("books")
.select()
.textSearch("description", value: "'little' & 'big'");
.execute()
```
</TabPanel>
<TabPanel id="kotlin" label="Kotlin">
```kotlin
val data = supabase.from("books").select {
filter {
textSearch("description", "'title' & 'big'", TextSearchType.NONE)
}
}
```
</TabPanel>
<TabPanel id="python" label="Python">
```python
data = supabase.from_('books').select().text_search('description', "'little' & 'big'").execute()
```
</TabPanel>
<TabPanel id="data" label="Data">
{/* supa-mdx-lint-disable Rule003Spelling */}
| id | title | author | description |
| --- | ------ | ----------------- | -------------------------------- |
| 3 | Tootle | Gertrude Crampton | Little toy train has big dreams. |
{/* supa-mdx-lint-enable Rule003Spelling */}
</TabPanel>
</Tabs>
### Match any search words
To find all `books` where `description` contain ANY of the words `little` or `big`, use the `|` symbol:
<Tabs
scrollable
size="small"
type="underlined"
defaultActiveId="sql"
queryGroup="language"
>
<TabPanel id="sql" label="SQL">
```sql
select
*
from
books
where
to_tsvector(description)
@@ to_tsquery('little | big'); -- use | for OR in the search query
```
</TabPanel>
<TabPanel id="js" label="JavaScript">
```js
const { data, error } = await supabase
.from('books')
.select()
.textSearch('description', `'little' | 'big'`)
```
</TabPanel>
<TabPanel id="dart" label="Dart">
```dart
final result = await client
.from('books')
.select()
.textSearch('description', "'little' | 'big'");
```
</TabPanel>
<TabPanel id="swift" label="Swift">
```swift
let response = try await client
.from("books")
.select()
.textSearch("description", value: "'little' | 'big'")
.execute()
```
</TabPanel>
<TabPanel id="kotlin" label="Kotlin">
```kotlin
val data = supabase.from("books").select {
filter {
textSearch("description", "'title' | 'big'", TextSearchType.NONE)
}
}
```
</TabPanel>
<TabPanel id="python" label="Python">
```python
response = client.from_('books').select().text_search('description', "'little' | 'big'").execute()
```
</TabPanel>
<TabPanel id="data" label="Data">
{/* supa-mdx-lint-disable Rule003Spelling */}
| id | title | author | description |
| --- | --------------------- | ---------------------- | ------------------------------------------- |
| 1 | The Poky Little Puppy | Janette Sebring Lowrey | Puppy is slower than other, bigger animals. |
| 3 | Tootle | Gertrude Crampton | Little toy train has big dreams. |
{/* supa-mdx-lint-enable Rule003Spelling */}
</TabPanel>
</Tabs>
Notice how searching for `big` includes results with the word `bigger` (or `biggest`, etc).
## Partial search
Partial search is particularly useful when you want to find matches on substrings within your data.
### Implementing partial search
You can use the `:*` syntax with `to_tsquery()`. Here's an example that searches for any book titles beginning with "Lit":
```sql
select title from books where to_tsvector(title) @@ to_tsquery('Lit:*');
```
### Extending functionality with RPC
To make the partial search functionality accessible through the API, you can wrap the search logic in a stored procedure.
After creating this function, you can invoke it from your application using the SDK for your platform. Here's an example:
<Tabs
scrollable
size="small"
type="underlined"
defaultActiveId="sql"
queryGroup="language"
>
<TabPanel id="sql" label="SQL">
```sql
create or replace function search_books_by_title_prefix(prefix text)
returns setof books AS $$
begin
return query
select * from books where to_tsvector('english', title) @@ to_tsquery(prefix || ':*');
end;
$$ language plpgsql;
```
</TabPanel>
<TabPanel id="js" label="JavaScript">
```js
const { data, error } = await supabase.rpc('search_books_by_title_prefix', { prefix: 'Lit' })
```
</TabPanel>
<TabPanel id="dart" label="Dart">
```dart
final data = await supabase.rpc('search_books_by_title_prefix', params: { 'prefix': 'Lit' });
```
</TabPanel>
<TabPanel id="swift" label="Swift">
```swift
let response = try await supabase.rpc(
"search_books_by_title_prefix",
params: ["prefix": "Lit"]
)
.execute()
```
</TabPanel>
<TabPanel id="kotlin" label="Kotlin">
```kotlin
val rpcParams = mapOf("prefix" to "Lit")
val result = supabase.postgrest.rpc("search_books_by_title_prefix", rpcParams)
```
</TabPanel>
<TabPanel id="python" label="Python">
```python
data = client.rpc('search_books_by_title_prefix', { 'prefix': 'Lit' }).execute()
```
</TabPanel>
</Tabs>
This function takes a prefix parameter and returns all books where the title contains a word starting with that prefix. The `:*` operator is used to denote a prefix match in the `to_tsquery()` function.
## Handling spaces in queries
When you want the search term to include a phrase or multiple words, you can concatenate words using a `+` as a placeholder for space:
```sql
select * from search_books_by_title_prefix('Little+Puppy');
```
## Creating indexes
Now that we have Full Text Search working, let's create an `index`. This will allow Postgres to "build" the documents preemptively so that they
don't need to be created at the time we execute the query. This will make our queries much faster.
### Searchable columns
Let's create a new column `fts` inside the `books` table to store the searchable index of the `title` and `description` columns.
We can use a special feature of Postgres called
[Generated Columns](https://www.postgresql.org/docs/current/ddl-generated-columns.html)
to ensure that the index is updated any time the values in the `title` and `description` columns change.
<Tabs
scrollable
size="small"
type="underlined"
defaultActiveId="sql"
queryGroup="example-view"
>
<TabPanel id="sql" label="SQL">
```sql
alter table
books
add column
fts tsvector generated always as (to_tsvector('english', description || ' ' || title)) stored;
create index books_fts on books using gin (fts); -- generate the index
select id, fts
from books;
```
</TabPanel>
<TabPanel id="data" label="Data">
```
| id | fts |
| --- | --------------------------------------------------------------------------------------------------------------- |
| 1 | 'anim':7 'bigger':6 'littl':10 'poki':9 'puppi':1,11 'slower':3 |
| 2 | 'eat':2 'peter':8 'rabbit':1,9 'tale':6 'veget':4 |
| 3 | 'big':5 'dream':6 'littl':1 'tootl':7 'toy':2 'train':3 |
| 4 | 'chang':3 'color':9 'eat':7 'egg':12 'food':4,10 'green':11 'ham':14 'prefer':5 'sam':1 'unus':8 |
| 5 | 'big':6 'drama':7 'ensu':8 'fire':15 'fourth':1 'goblet':13 'harri':9 'potter':10 'school':4 'start':5 'year':2 |
```
</TabPanel>
</Tabs>
### Search using the new column
Now that we've created and populated our index, we can search it using the same techniques as before:
<Tabs
scrollable
size="small"
type="underlined"
defaultActiveId="sql"
queryGroup="language"
>
<TabPanel id="sql" label="SQL">
```sql
select
*
from
books
where
fts @@ to_tsquery('little & big');
```
</TabPanel>
<TabPanel id="js" label="JavaScript">
```js
const { data, error } = await supabase.from('books').select().textSearch('fts', `'little' & 'big'`)
```
</TabPanel>
<TabPanel id="dart" label="Dart">
```dart
final result = await client
.from('books')
.select()
.textSearch('fts', "'little' & 'big'");
```
</TabPanel>
<TabPanel id="swift" label="Swift">
```swift
let response = try await client
.from("books")
.select()
.textSearch("fts", value: "'little' & 'big'")
.execute()
```
</TabPanel>
<TabPanel id="kotlin" label="Kotlin">
```kotlin
val data = supabase.from("books").select {
filter {
textSearch("fts", "'title' & 'big'", TextSearchType.NONE)
}
}
```
</TabPanel>
<TabPanel id="python" label="Python">
```python
data = client.from_('books').select().text_search('fts', "'little' & 'big'").execute()
```
</TabPanel>
<TabPanel id="data" label="Data">
{/* supa-mdx-lint-disable Rule003Spelling */}
| id | title | author | description | fts |
| --- | ------ | ----------------- | -------------------------------- | ------------------------------------------------------- |
| 3 | Tootle | Gertrude Crampton | Little toy train has big dreams. | 'big':5 'dream':6 'littl':1 'tootl':7 'toy':2 'train':3 |
{/* supa-mdx-lint-enable Rule003Spelling */}
</TabPanel>
</Tabs>
## Query operators
Visit [Postgres: Text Search Functions and Operators](https://www.postgresql.org/docs/current/functions-textsearch.html)
to learn about additional query operators you can use to do more advanced `full text queries`, such as:
### Proximity: `<->` [#proximity]
The proximity symbol is useful for searching for terms that are a certain "distance" apart.
For example, to find the phrase `big dreams`, where the a match for "big" is followed immediately by a match for "dreams":
<Tabs
scrollable
size="small"
type="underlined"
defaultActiveId="sql"
queryGroup="language"
>
<TabPanel id="sql" label="SQL">
```sql
select
*
from
books
where
to_tsvector(description) @@ to_tsquery('big <-> dreams');
```
</TabPanel>
<TabPanel id="js" label="JavaScript">
```js
const { data, error } = await supabase
.from('books')
.select()
.textSearch('description', `'big' <-> 'dreams'`)
```
</TabPanel>
<TabPanel id="dart" label="Dart">
```dart
final result = await client
.from('books')
.select()
.textSearch('description', "'big' <-> 'dreams'");
```
</TabPanel>
<TabPanel id="swift" label="Swift">
```swift
let response = try await client
.from("books")
.select()
.textSearch("description", value: "'big' <-> 'dreams'")
.execute()
```
</TabPanel>
<TabPanel id="kotlin" label="Kotlin">
```kotlin
val data = supabase.from("books").select {
filter {
textSearch("description", "'big' <-> 'dreams'", TextSearchType.NONE)
}
}
```
</TabPanel>
<TabPanel id="python" label="Python">
```python
data = client.from_('books').select().text_search('description', "'big' <-> 'dreams'").execute()
```
</TabPanel>
</Tabs>
We can also use the `<->` to find words within a certain distance of each other. For example to find `year` and `school` within 2 words of each other:
<Tabs
scrollable
size="small"
type="underlined"
defaultActiveId="sql"
queryGroup="language"
>
<TabPanel id="sql" label="SQL">
```sql
select
*
from
books
where
to_tsvector(description) @@ to_tsquery('year <2> school');
```
</TabPanel>
<TabPanel id="js" label="JavaScript">
```js
const { data, error } = await supabase
.from('books')
.select()
.textSearch('description', `'year' <2> 'school'`)
```
</TabPanel>
<TabPanel id="dart" label="Dart">
```dart
final result = await client
.from('books')
.select()
.textSearch('description', "'year' <2> 'school'");
```
</TabPanel>
<TabPanel id="swift" label="Swift">
```swift
let response = try await supabase
.from("books")
.select()
.textSearch("description", value: "'year' <2> 'school'")
.execute()
```
</TabPanel>
<TabPanel id="kotlin" label="Kotlin">
```kotlin
val data = supabase.from("books").select {
filter {
textSearch("description", "'year' <2> 'school'", TextSearchType.NONE)
}
}
```
</TabPanel>
<TabPanel id="python" label="Python">
```python
data = client.from_('books').select().text_search('description', "'year' <2> 'school'").execute()
```
</TabPanel>
</Tabs>
### Negation: `!` [#negation]
The negation symbol can be used to find phrases which _don't_ contain a search term.
For example, to find records that have the word `big` but not `little`:
<Tabs
scrollable
size="small"
type="underlined"
defaultActiveId="sql"
queryGroup="language"
>
<TabPanel id="sql" label="SQL">
```sql
select
*
from
books
where
to_tsvector(description) @@ to_tsquery('big & !little');
```
</TabPanel>
<TabPanel id="js" label="JavaScript">
```js
const { data, error } = await supabase
.from('books')
.select()
.textSearch('description', `'big' & !'little'`)
```
</TabPanel>
<TabPanel id="dart" label="Dart">
```dart
final result = await client
.from('books')
.select()
.textSearch('description', "'big' & !'little'");
```
</TabPanel>
<TabPanel id="swift" label="Swift">
```swift
let response = try await client
.from("books")
.select()
.textSearch("description", value: "'big' & !'little'")
.execute()
```
</TabPanel>
<TabPanel id="kotlin" label="Kotlin">
```kotlin
val data = supabase.from("books").select {
filter {
textSearch("description", "'big' & !'little'", TextSearchType.NONE)
}
}
```
</TabPanel>
<TabPanel id="python" label="Python">
```python
data = client.from_('books').select().text_search('description', "'big' & !'little'").execute()
```
</TabPanel>
</Tabs>
## Resources
- [Postgres: Text Search Functions and Operators](https://www.postgresql.org/docs/12/functions-textsearch.html)