mirror of
https://github.com/supabase/supabase.git
synced 2026-10-05 09:25:06 +03:00
Blog: Add Timescale to pg_partman migration guide (#40037)
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Blog post on using pg_partman instead of TimescaleDB to prepare for the upcoming deprecation ## What is the current behavior? ## What is the new behavior? Blog post to include migration information for those using Timescale ## Additional context Not to be merged until pg_partman is released in 15 and 17 images <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added a comprehensive pg_partman guide covering setup, time- and integer-based partitioning, maintenance, automation, and resources. * Added a migration guide for moving from TimescaleDB hypertables to native PostgreSQL partitioning using pg_partman. * Updated TimescaleDB docs with migration notes and support guidance. * **New Features** * Listed pg_partman in the public extensions reference and added navigation entries linking to the pg_partman guide and migration guide. <sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub> <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This commit is contained in:
1 parent
732180608a
commit
81415c6053
5 files changed
+229
No files matched your search
@@ -1049,6 +1049,10 @@ export const database: NavMenuConstant = {
|
||||
name: 'Partitioning your tables',
|
||||
url: '/guides/database/partitions' as `/${string}`,
|
||||
},
|
||||
{
|
||||
name: 'Migrating to pg_partman',
|
||||
url: '/guides/database/migrating-to-pg-partman' as `/${string}`,
|
||||
},
|
||||
{
|
||||
name: 'Managing connections',
|
||||
url: '/guides/database/connection-management' as `/${string}`,
|
||||
@@ -1249,6 +1253,10 @@ export const database: NavMenuConstant = {
|
||||
name: 'pg_net: Async Networking',
|
||||
url: '/guides/database/extensions/pg_net' as `/${string}`,
|
||||
},
|
||||
{
|
||||
name: 'pg_partman: Partition management',
|
||||
url: '/guides/database/extensions/pg_partman' as `/${string}`,
|
||||
},
|
||||
{
|
||||
name: 'pg_plan_filter: Restrict Total Cost',
|
||||
url: '/guides/database/extensions/pg_plan_filter' as `/${string}`,
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
---
|
||||
id: 'pg_partman'
|
||||
title: 'pg_partman: partition management'
|
||||
description: 'Automated partition management'
|
||||
---
|
||||
|
||||
[`pg_partman`](https://github.com/pgpartman/pg_partman) is a Postgres extension that automates the creation and maintenance of partitions for tables using Postgres native partitioning.
|
||||
|
||||
## Enable the extension
|
||||
|
||||
To enable `pg_partman`, create a dedicated schema for it and enable the extension there.
|
||||
|
||||
{/* prettier-ignore */}
|
||||
```sql
|
||||
create schema if not exists partman;
|
||||
create extension if not exists pg_partman with schema partman;
|
||||
```
|
||||
|
||||
## Create a partitioned table
|
||||
|
||||
`pg_partman` requires your parent table to already be declared as a partitioned table.
|
||||
|
||||
{/* prettier-ignore */}
|
||||
```sql
|
||||
create table public.messages (
|
||||
id bigint generated by default as identity,
|
||||
sent_at timestamptz not null,
|
||||
sender_id uuid,
|
||||
recipient_id uuid,
|
||||
body text,
|
||||
primary key (sent_at, id)
|
||||
)
|
||||
partition by range (sent_at);
|
||||
```
|
||||
|
||||
## Set up partitioning
|
||||
|
||||
You configure the parent table using `partman.create_parent()`. The function takes an `ACCESS EXCLUSIVE` lock briefly while it creates the initial partitions.
|
||||
|
||||
### Time-based partitions
|
||||
|
||||
{/* prettier-ignore */}
|
||||
```sql
|
||||
select partman.create_parent(
|
||||
p_parent_table := 'public.messages',
|
||||
p_control := 'sent_at',
|
||||
p_type := 'range',
|
||||
p_interval := '7 days',
|
||||
p_premake := 7,
|
||||
p_start_partition := '2025-01-01 00:00:00'
|
||||
);
|
||||
```
|
||||
|
||||
### Integer-based partitions
|
||||
|
||||
{/* prettier-ignore */}
|
||||
```sql
|
||||
create table public.events (
|
||||
id bigint generated by default as identity,
|
||||
inserted_at timestamptz not null default now(),
|
||||
payload jsonb,
|
||||
primary key (id)
|
||||
)
|
||||
partition by range (id);
|
||||
|
||||
select partman.create_parent(
|
||||
p_parent_table := 'public.events',
|
||||
p_control := 'id',
|
||||
p_type := 'range',
|
||||
p_interval := '100000'
|
||||
);
|
||||
```
|
||||
|
||||
## Running maintenance
|
||||
|
||||
It’s important to call `pg_partman` maintenance regularly so future partitions are pre-created and retention policies are applied.
|
||||
|
||||
{/* prettier-ignore */}
|
||||
```sql
|
||||
call partman.run_maintenance_proc();
|
||||
```
|
||||
|
||||
To automate this, schedule it using `pg_cron`.
|
||||
|
||||
{/* prettier-ignore */}
|
||||
```sql
|
||||
create extension if not exists pg_cron;
|
||||
|
||||
select
|
||||
cron.schedule('@hourly', $$call partman.run_maintenance_proc()$$);
|
||||
```
|
||||
|
||||
## Resources
|
||||
|
||||
- Official [pg_partman documentation](https://github.com/pgpartman/pg_partman/blob/development/doc/pg_partman.md)
|
||||
@@ -8,6 +8,10 @@ description: 'Scalable time-series data storage and analysis'
|
||||
|
||||
The `timescaledb` extension is deprecated in projects using Postgres 17. It continues to be supported in projects using Postgres 15, but will need to dropped before those projects are upgraded to Postgres 17. See the [Upgrading to Postgres 17 notes](/docs/guides/platform/upgrading#upgrading-to-postgres-17) for more information.
|
||||
|
||||
If you are using hypertables, follow the [migration guide](/docs/guides/database/migrating-to-pg-partman) to convert to native partitioning managed by `pg_partman`.
|
||||
|
||||
For additional support, contact our Success team by creating a support ticket in the Supabase Dashboard.
|
||||
|
||||
</Admonition>
|
||||
|
||||
[`timescaledb`](https://docs.timescale.com/timescaledb/latest/) is a Postgres extension designed for improved handling of time-series data. It provides a scalable, high-performance solution for storing and querying time-series data on top of a standard Postgres database.
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
---
|
||||
id: 'migrating-from-timescaledb-to-pg-partman'
|
||||
title: 'Migrate from TimescaleDB to pg_partman'
|
||||
description: 'Convert TimescaleDB hypertables to Postgres native partitions managed by pg_partman.'
|
||||
---
|
||||
|
||||
Starting from Postgres 17, Supabase projects do not have the `timescaledb` extension available. If your project relies on TimescaleDB hypertables, you will need to migrate to standard Postgres tables before upgrading.
|
||||
|
||||
This guide shows one approach to migrate a hypertable to a native Postgres partitioned table and optionally configure `pg_partman` to automate ongoing partition maintenance.
|
||||
The approach outlined in this guide can also be used for traditional partitioned tables.
|
||||
|
||||
## Before you begin
|
||||
|
||||
- Test the migration path in a staging environment (for example by creating a copy of your production project or using branching).
|
||||
- Review your application for TimescaleDB-specific SQL usage (for example `time_bucket()`, compression policies). Those features are not provided by `pg_partman`.
|
||||
|
||||
## Migration overview
|
||||
|
||||
1. Create a new partitioned table.
|
||||
2. Copy data from the hypertable to the new table.
|
||||
3. Swap over and drop the hypertable.
|
||||
4. Configure `pg_partman` (optional) and schedule maintenance.
|
||||
|
||||
## Example: Migrate `messages` from hypertable to native partitions
|
||||
|
||||
This example assumes a `messages` hypertable partitioned by `sent_at`.
|
||||
|
||||
### 1. Rename the existing hypertable
|
||||
|
||||
This keeps the original data in place while you create a new partitioned table with the original name.
|
||||
|
||||
{/* prettier-ignore */}
|
||||
```sql
|
||||
alter table public.messages rename to ht_messages;
|
||||
```
|
||||
|
||||
### 2. Create a new partitioned table
|
||||
|
||||
When using native partitioning, the partitioning column must be included in any unique index (including the primary key).
|
||||
|
||||
{/* prettier-ignore */}
|
||||
```sql
|
||||
create table public.messages (
|
||||
like public.ht_messages including all,
|
||||
primary key (sent_at, id)
|
||||
)
|
||||
partition by range (sent_at);
|
||||
```
|
||||
|
||||
### 3. Copy data into the new table
|
||||
|
||||
For large tables, consider copying in batches (for example by time range) during a maintenance window.
|
||||
|
||||
{/* prettier-ignore */}
|
||||
```sql
|
||||
insert into public.messages
|
||||
select *
|
||||
from public.ht_messages;
|
||||
```
|
||||
|
||||
### 4. Drop the old hypertable (and TimescaleDB)
|
||||
|
||||
Only drop the extension once you’ve migrated all hypertables and no other objects depend on it.
|
||||
|
||||
{/* prettier-ignore */}
|
||||
```sql
|
||||
drop table public.ht_messages;
|
||||
|
||||
drop extension if exists timescaledb;
|
||||
```
|
||||
|
||||
### 5. Configure `pg_partman` (optional)
|
||||
|
||||
Enable `pg_partman` and register your table so partitions are created ahead of time.
|
||||
|
||||
{/* prettier-ignore */}
|
||||
```sql
|
||||
create schema if not exists partman;
|
||||
create extension if not exists pg_partman with schema partman;
|
||||
|
||||
select partman.create_parent(
|
||||
p_parent_table := 'public.messages',
|
||||
p_control := 'sent_at',
|
||||
p_type := 'range',
|
||||
p_interval := '7 days',
|
||||
p_premake := 7,
|
||||
p_start_partition := '2025-01-01 00:00:00'
|
||||
);
|
||||
```
|
||||
|
||||
## Keep partitions up to date
|
||||
|
||||
`pg_partman` requires running maintenance to pre-make partitions and apply retention policies.
|
||||
|
||||
{/* prettier-ignore */}
|
||||
```sql
|
||||
call partman.run_maintenance_proc();
|
||||
```
|
||||
|
||||
To automate this, schedule it with `pg_cron`.
|
||||
|
||||
{/* prettier-ignore */}
|
||||
```sql
|
||||
create extension if not exists pg_cron;
|
||||
|
||||
select cron.schedule('@daily', $$call partman.run_maintenance_proc()$$);
|
||||
```
|
||||
|
||||
## Additional resources
|
||||
|
||||
- [Partitioning your tables](/docs/guides/database/partitions).
|
||||
- [`pg_partman` documentation](/docs/guides/database/extensions/pg_partman)
|
||||
- [`pg_partman` migration guides](https://github.com/pgpartman/pg_partman/blob/development/doc/migrate_to_partman.md)
|
||||
@@ -278,6 +278,15 @@
|
||||
"product": "Database Webhooks",
|
||||
"product_url": "/project/{ref}/integrations/webhooks"
|
||||
},
|
||||
{
|
||||
"name": "pg_partman",
|
||||
"comment": "Automated partition management",
|
||||
"tags": ["Utility"],
|
||||
"link": "/guides/database/extensions/pg_partman",
|
||||
"github_url": "https://github.com/pgpartman/pg_partman",
|
||||
"product": null,
|
||||
"product_url": null
|
||||
},
|
||||
{
|
||||
"name": "pg_prewarm",
|
||||
"comment": "prewarm relation data",
|
||||
|
||||
Reference in new issue
Block a user