--- id: 'full-text-search' title: 'Full Text Search' description: 'How to use full text search in PostgreSQL.' subtitle: 'How to use full text search in PostgreSQL.' tocVideo: 'b-mgca_2Oe4' --- Postgres has built-in functions to handle `Full Text Search` queries. This is like a "search engine" within Postgres.
## Preparation For this guide we'll use the following example data: {/* supa-mdx-lint-disable Rule003Spelling */} | id | title | author | description | | --- | ----------------------------------- | ---------------------- | ------------------------------------------------------------------ | | 1 | The Poky Little Puppy | Janette Sebring Lowrey | Puppy is slower than other, bigger animals. | | 2 | The Tale of Peter Rabbit | Beatrix Potter | Rabbit eats some vegetables. | | 3 | Tootle | Gertrude Crampton | Little toy train has big dreams. | | 4 | Green Eggs and Ham | Dr. Seuss | Sam has changing food preferences and eats unusually colored food. | | 5 | Harry Potter and the Goblet of Fire | J.K. Rowling | Fourth year of school starts, big drama ensues. | {/* supa-mdx-lint-enable Rule003Spelling */} ```sql create table books ( id serial primary key, title text, author text, description text ); insert into books (title, author, description) values ( 'The Poky Little Puppy', 'Janette Sebring Lowrey', 'Puppy is slower than other, bigger animals.' ), ('The Tale of Peter Rabbit', 'Beatrix Potter', 'Rabbit eats some vegetables.'), ('Tootle', 'Gertrude Crampton', 'Little toy train has big dreams.'), ( 'Green Eggs and Ham', 'Dr. Seuss', 'Sam has changing food preferences and eats unusually colored food.' ), ( 'Harry Potter and the Goblet of Fire', 'J.K. Rowling', 'Fourth year of school starts, big drama ensues.' ); ``` ## Usage The functions we'll cover in this guide are: ### `to_tsvector()` [#to-tsvector] Converts your data into searchable tokens. `to_tsvector()` stands for "to text search vector." For example: ```sql select to_tsvector('green eggs and ham'); -- Returns 'egg':2 'green':1 'ham':4 ``` Collectively these tokens are called a "document" which Postgres can use for comparisons. ### `to_tsquery()` [#to-tsquery] Converts a query string into tokens to match. `to_tsquery()` stands for "to text search query." This conversion step is important because we will want to "fuzzy match" on keywords. For example if a user searches for `eggs`, and a column has the value `egg`, we probably still want to return a match. ### Match: `@@` [#match] The `@@` symbol is the "match" symbol for Full Text Search. It returns any matches between a `to_tsvector` result and a `to_tsquery` result. Take the following example: ```sql select * from books where title = 'Harry'; ``` ```js const { data, error } = await supabase.from('books').select().eq('title', 'Harry') ``` ```dart final result = await client .from('books') .select() .eq('title', 'Harry'); ``` ```swift let response = try await supabase.from("books") .select() .eq("title", value: "Harry") .execute() ``` ```kotlin val data = supabase.from("books").select { filter { eq("title", "Harry") } } ``` ```python data = supabase.from_('books').select().eq('title', 'Harry').execute() ``` The equality symbol above (`=`) is very "strict" on what it matches. In a full text search context, we might want to find all "Harry Potter" books and so we can rewrite the example above: ```sql select * from books where to_tsvector(title) @@ to_tsquery('Harry'); ``` ```js const { data, error } = await supabase.from('books').select().textSearch('title', `'Harry'`) ``` ```dart final result = await client .from('books') .select() .textSearch('title', "'Harry'"); ``` ```swift let response = try await supabase.from("books") .select() .textSearch("title", value: "'Harry'") ``` ```kotlin val data = supabase.from("books").select { filter { textSearch("title", "'Harry'", TextSearchType.NONE) } } ``` ## Basic full text queries ### Search a single column To find all `books` where the `description` contain the word `big`: ```sql select * from books where to_tsvector(description) @@ to_tsquery('big'); ``` ```js const { data, error } = await supabase.from('books').select().textSearch('description', `'big'`) ``` ```dart final result = await client .from('books') .select() .textSearch('description', "'big'"); ``` ```swift let response = await client.from("books") .select() .textSearch("description", value: "'big'") .execute() ``` ```kotlin val data = supabase.from("books").select { filter { textSearch("description", "'big'", TextSearchType.NONE) } } ``` ```python data = supabase.from_('books').select().text_search('description', "'big'").execute() ``` {/* supa-mdx-lint-disable Rule003Spelling */} | id | title | author | description | | --- | ----------------------------------- | ----------------- | ----------------------------------------------- | | 3 | Tootle | Gertrude Crampton | Little toy train has big dreams. | | 5 | Harry Potter and the Goblet of Fire | J.K. Rowling | Fourth year of school starts, big drama ensues. | {/* supa-mdx-lint-enable Rule003Spelling */} ### Search multiple columns Right now there is no direct way to use JavaScript or Dart to search through multiple columns but you can do it by creating [computed columns](https://postgrest.org/en/stable/api.html#computed-virtual-columns) on the database. To find all `books` where `description` or `title` contain the word `little`: ```sql select * from books where to_tsvector(description || ' ' || title) -- concat columns, but be sure to include a space to separate them! @@ to_tsquery('little'); ``` ```sql create function title_description(books) returns text as $$ select $1.title || ' ' || $1.description; $$ language sql immutable; ``` ```js const { data, error } = await supabase .from('books') .select() .textSearch('title_description', `little`) ``` ```sql create function title_description(books) returns text as $$ select $1.title || ' ' || $1.description; $$ language sql immutable; ``` ```dart final result = await client .from('books') .select() .textSearch('title_description', "little") ``` ```sql create function title_description(books) returns text as $$ select $1.title || ' ' || $1.description; $$ language sql immutable; ``` ```swift let response = try await client .from("books") .select() .textSearch("title_description", value: "little") .execute() ``` ```sql create function title_description(books) returns text as $$ select $1.title || ' ' || $1.description; $$ language sql immutable; ``` ```kotlin val data = supabase.from("books").select { filter { textSearch("title_description", "title", TextSearchType.NONE) } } ``` ```sql create function title_description(books) returns text as $$ select $1.title || ' ' || $1.description; $$ language sql immutable; ``` ```python data = supabase.from_('books').select().text_search('title_description', "little").execute() ``` {/* supa-mdx-lint-disable Rule003Spelling */} | id | title | author | description | | --- | --------------------- | ---------------------- | ------------------------------------------- | | 1 | The Poky Little Puppy | Janette Sebring Lowrey | Puppy is slower than other, bigger animals. | | 3 | Tootle | Gertrude Crampton | Little toy train has big dreams. | {/* supa-mdx-lint-enable Rule003Spelling */} ### Match all search words To find all `books` where `description` contains BOTH of the words `little` and `big`, we can use the `&` symbol: ```sql select * from books where to_tsvector(description) @@ to_tsquery('little & big'); -- use & for AND in the search query ``` ```js const { data, error } = await supabase .from('books') .select() .textSearch('description', `'little' & 'big'`) ``` ```dart final result = await client .from('books') .select() .textSearch('description', "'little' & 'big'"); ``` ```swift let response = try await client .from("books") .select() .textSearch("description", value: "'little' & 'big'"); .execute() ``` ```kotlin val data = supabase.from("books").select { filter { textSearch("description", "'title' & 'big'", TextSearchType.NONE) } } ``` ```python data = supabase.from_('books').select().text_search('description', "'little' & 'big'").execute() ``` {/* supa-mdx-lint-disable Rule003Spelling */} | id | title | author | description | | --- | ------ | ----------------- | -------------------------------- | | 3 | Tootle | Gertrude Crampton | Little toy train has big dreams. | {/* supa-mdx-lint-enable Rule003Spelling */} ### Match any search words To find all `books` where `description` contain ANY of the words `little` or `big`, use the `|` symbol: ```sql select * from books where to_tsvector(description) @@ to_tsquery('little | big'); -- use | for OR in the search query ``` ```js const { data, error } = await supabase .from('books') .select() .textSearch('description', `'little' | 'big'`) ``` ```dart final result = await client .from('books') .select() .textSearch('description', "'little' | 'big'"); ``` ```swift let response = try await client .from("books") .select() .textSearch("description", value: "'little' | 'big'") .execute() ``` ```kotlin val data = supabase.from("books").select { filter { textSearch("description", "'title' | 'big'", TextSearchType.NONE) } } ``` ```python response = client.from_('books').select().text_search('description', "'little' | 'big'").execute() ``` {/* supa-mdx-lint-disable Rule003Spelling */} | id | title | author | description | | --- | --------------------- | ---------------------- | ------------------------------------------- | | 1 | The Poky Little Puppy | Janette Sebring Lowrey | Puppy is slower than other, bigger animals. | | 3 | Tootle | Gertrude Crampton | Little toy train has big dreams. | {/* supa-mdx-lint-enable Rule003Spelling */} Notice how searching for `big` includes results with the word `bigger` (or `biggest`, etc). ## Partial search Partial search is particularly useful when you want to find matches on substrings within your data. ### Implementing partial search You can use the `:*` syntax with `to_tsquery()`. Here's an example that searches for any book titles beginning with "Lit": ```sql select title from books where to_tsvector(title) @@ to_tsquery('Lit:*'); ``` ### Extending functionality with RPC To make the partial search functionality accessible through the API, you can wrap the search logic in a stored procedure. After creating this function, you can invoke it from your application using the SDK for your platform. Here's an example: ```sql create or replace function search_books_by_title_prefix(prefix text) returns setof books AS $$ begin return query select * from books where to_tsvector('english', title) @@ to_tsquery(prefix || ':*'); end; $$ language plpgsql; ``` ```js const { data, error } = await supabase.rpc('search_books_by_title_prefix', { prefix: 'Lit' }) ``` ```dart final data = await supabase.rpc('search_books_by_title_prefix', params: { 'prefix': 'Lit' }); ``` ```swift let response = try await supabase.rpc( "search_books_by_title_prefix", params: ["prefix": "Lit"] ) .execute() ``` ```kotlin val rpcParams = mapOf("prefix" to "Lit") val result = supabase.postgrest.rpc("search_books_by_title_prefix", rpcParams) ``` ```python data = client.rpc('search_books_by_title_prefix', { 'prefix': 'Lit' }).execute() ``` This function takes a prefix parameter and returns all books where the title contains a word starting with that prefix. The `:*` operator is used to denote a prefix match in the `to_tsquery()` function. ## Handling spaces in queries When you want the search term to include a phrase or multiple words, you can concatenate words using a `+` as a placeholder for space: ```sql select * from search_books_by_title_prefix('Little+Puppy'); ``` ## Creating indexes Now that we have Full Text Search working, let's create an `index`. This will allow Postgres to "build" the documents preemptively so that they don't need to be created at the time we execute the query. This will make our queries much faster. ### Searchable columns Let's create a new column `fts` inside the `books` table to store the searchable index of the `title` and `description` columns. We can use a special feature of Postgres called [Generated Columns](https://www.postgresql.org/docs/current/ddl-generated-columns.html) to ensure that the index is updated any time the values in the `title` and `description` columns change. ```sql alter table books add column fts tsvector generated always as (to_tsvector('english', description || ' ' || title)) stored; create index books_fts on books using gin (fts); -- generate the index select id, fts from books; ``` ``` | id | fts | | --- | --------------------------------------------------------------------------------------------------------------- | | 1 | 'anim':7 'bigger':6 'littl':10 'poki':9 'puppi':1,11 'slower':3 | | 2 | 'eat':2 'peter':8 'rabbit':1,9 'tale':6 'veget':4 | | 3 | 'big':5 'dream':6 'littl':1 'tootl':7 'toy':2 'train':3 | | 4 | 'chang':3 'color':9 'eat':7 'egg':12 'food':4,10 'green':11 'ham':14 'prefer':5 'sam':1 'unus':8 | | 5 | 'big':6 'drama':7 'ensu':8 'fire':15 'fourth':1 'goblet':13 'harri':9 'potter':10 'school':4 'start':5 'year':2 | ``` ### Search using the new column Now that we've created and populated our index, we can search it using the same techniques as before: ```sql select * from books where fts @@ to_tsquery('little & big'); ``` ```js const { data, error } = await supabase.from('books').select().textSearch('fts', `'little' & 'big'`) ``` ```dart final result = await client .from('books') .select() .textSearch('fts', "'little' & 'big'"); ``` ```swift let response = try await client .from("books") .select() .textSearch("fts", value: "'little' & 'big'") .execute() ``` ```kotlin val data = supabase.from("books").select { filter { textSearch("fts", "'title' & 'big'", TextSearchType.NONE) } } ``` ```python data = client.from_('books').select().text_search('fts', "'little' & 'big'").execute() ``` {/* supa-mdx-lint-disable Rule003Spelling */} | id | title | author | description | fts | | --- | ------ | ----------------- | -------------------------------- | ------------------------------------------------------- | | 3 | Tootle | Gertrude Crampton | Little toy train has big dreams. | 'big':5 'dream':6 'littl':1 'tootl':7 'toy':2 'train':3 | {/* supa-mdx-lint-enable Rule003Spelling */} ## Query operators Visit [Postgres: Text Search Functions and Operators](https://www.postgresql.org/docs/current/functions-textsearch.html) to learn about additional query operators you can use to do more advanced `full text queries`, such as: ### Proximity: `<->` [#proximity] The proximity symbol is useful for searching for terms that are a certain "distance" apart. For example, to find the phrase `big dreams`, where the a match for "big" is followed immediately by a match for "dreams": ```sql select * from books where to_tsvector(description) @@ to_tsquery('big <-> dreams'); ``` ```js const { data, error } = await supabase .from('books') .select() .textSearch('description', `'big' <-> 'dreams'`) ``` ```dart final result = await client .from('books') .select() .textSearch('description', "'big' <-> 'dreams'"); ``` ```swift let response = try await client .from("books") .select() .textSearch("description", value: "'big' <-> 'dreams'") .execute() ``` ```kotlin val data = supabase.from("books").select { filter { textSearch("description", "'big' <-> 'dreams'", TextSearchType.NONE) } } ``` ```python data = client.from_('books').select().text_search('description', "'big' <-> 'dreams'").execute() ``` We can also use the `<->` to find words within a certain distance of each other. For example to find `year` and `school` within 2 words of each other: ```sql select * from books where to_tsvector(description) @@ to_tsquery('year <2> school'); ``` ```js const { data, error } = await supabase .from('books') .select() .textSearch('description', `'year' <2> 'school'`) ``` ```dart final result = await client .from('books') .select() .textSearch('description', "'year' <2> 'school'"); ``` ```swift let response = try await supabase .from("books") .select() .textSearch("description", value: "'year' <2> 'school'") .execute() ``` ```kotlin val data = supabase.from("books").select { filter { textSearch("description", "'year' <2> 'school'", TextSearchType.NONE) } } ``` ```python data = client.from_('books').select().text_search('description', "'year' <2> 'school'").execute() ``` ### Negation: `!` [#negation] The negation symbol can be used to find phrases which _don't_ contain a search term. For example, to find records that have the word `big` but not `little`: ```sql select * from books where to_tsvector(description) @@ to_tsquery('big & !little'); ``` ```js const { data, error } = await supabase .from('books') .select() .textSearch('description', `'big' & !'little'`) ``` ```dart final result = await client .from('books') .select() .textSearch('description', "'big' & !'little'"); ``` ```swift let response = try await client .from("books") .select() .textSearch("description", value: "'big' & !'little'") .execute() ``` ```kotlin val data = supabase.from("books").select { filter { textSearch("description", "'big' & !'little'", TextSearchType.NONE) } } ``` ```python data = client.from_('books').select().text_search('description', "'big' & !'little'").execute() ``` ## Resources - [Postgres: Text Search Functions and Operators](https://www.postgresql.org/docs/12/functions-textsearch.html)