diff --git a/README.md b/README.md index 0f75fbb3c36..2c19e93ffed 100644 --- a/README.md +++ b/README.md @@ -54,7 +54,7 @@ You can also [self-host](https://supabase.com/docs/guides/self-hosting) and [dev ![Architecture](https://supabase.com/docs/assets/images/supabase-architecture-9050a7317e9ec7efb7807f5194122e48.png) - [PostgreSQL](https://www.postgresql.org/) is an object-relational database system with over 30 years of active development that has earned it a strong reputation for reliability, feature robustness, and performance. -- [Realtime](https://github.com/supabase/realtime) is an Elixir server that allows you to listen to PostgreSQL inserts, updates, and deletes using websockets. Supabase listens to Postgres' built-in replication functionality, converts the replication byte stream into JSON, then broadcasts the JSON over websockets. +- [Realtime](https://github.com/supabase/realtime) is an Elixir server that allows you to listen to PostgreSQL inserts, updates, and deletes using websockets. Realtime polls Postgres' built-in replication functionality for database changes, converts changes to JSON, then broadcasts the JSON over websockets to authorized clients. - [PostgREST](http://postgrest.org/) is a web server that turns your PostgreSQL database directly into a RESTful API - [Storage](https://github.com/supabase/storage-api) provides a RESTful interface for managing Files stored in S3, using Postgres to manage permissions. - [postgres-meta](https://github.com/supabase/postgres-meta) is a RESTful API for managing your Postgres, allowing you to fetch tables, add roles, and run queries, etc. diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml index ac54c922e0a..53f36bc21a0 100644 --- a/docker/docker-compose.yml +++ b/docker/docker-compose.yml @@ -87,7 +87,7 @@ services: realtime: container_name: supabase-realtime - image: supabase/realtime:v0.15.0 + image: supabase/realtime:v0.19.0 depends_on: - db restart: unless-stopped @@ -97,10 +97,17 @@ services: DB_NAME: postgres DB_USER: postgres DB_PASSWORD: ${POSTGRES_PASSWORD} - SLOT_NAME: supabase_realtime + DB_SSL: "false" PORT: 4000 - SECURE_CHANNELS: "true" JWT_SECRET: ${JWT_SECRET} + REPLICATION_MODE: RLS + REPLICATION_POLL_INTERVAL: 100 + SECURE_CHANNELS: "true" + SLOT_NAME: supabase_realtime_rls + TEMPORARY_SLOT: "true" + command: > + bash -c "./prod/rel/realtime/bin/realtime eval Realtime.Release.migrate + && ./prod/rel/realtime/bin/realtime start" storage: container_name: supabase-storage diff --git a/docker/volumes/db/init/00-initial-schema.sql b/docker/volumes/db/init/00-initial-schema.sql index f3e63542a92..81a2ed4fc40 100644 --- a/docker/volumes/db/init/00-initial-schema.sql +++ b/docker/volumes/db/init/00-initial-schema.sql @@ -1,4 +1,5 @@ --- Set up reatime +-- Set up realtime +create schema if not exists realtime; -- create publication supabase_realtime; -- defaults to empty publication create publication supabase_realtime; diff --git a/examples/nextjs-auth-tailwind/pages/index.js b/examples/nextjs-auth-tailwind/pages/index.js index abe68990d10..287a39a1cdb 100644 --- a/examples/nextjs-auth-tailwind/pages/index.js +++ b/examples/nextjs-auth-tailwind/pages/index.js @@ -15,7 +15,7 @@ const Index = () => {

Create a backend in less than 2 minutes. Start your project with a Postgres Database, - Authentication, instant APIs, realtime subscriptions and Storage. + Authentication, instant APIs, Realtime subscriptions and Storage.

diff --git a/studio/components/to-be-cleaned/Docs/Pages/Authentication.js b/studio/components/to-be-cleaned/Docs/Pages/Authentication.js index 4ed087b5b06..94313c915bd 100644 --- a/studio/components/to-be-cleaned/Docs/Pages/Authentication.js +++ b/studio/components/to-be-cleaned/Docs/Pages/Authentication.js @@ -42,18 +42,19 @@ export default function Authentication({ autoApiService, selectedLang }) {

Realtime Security

- The realtime server doesn't provide per-user security. Until we build a more robust auth - system for websockets, you shouldn't use realtime on any client which is exposed to - users. + Realtime server broadcasts database changes to authorized users depending on your Row Level Security (RLS) policies. + We recommend that you enable row level security and set row security policies on tables that you add to the publication. + However, you may choose to disable RLS on a table and have changes broadcast to all connected clients.

- For security purposes, you can run{' '} + You can get started by running{' '} begin; drop publication if exists supabase_realtime; create publication supabase_realtime; commit; . This creates a publication which is not subscribed to any table and completely - disables realtime updates on the Supabase client. + disables Realtime on the Supabase client. Then, you can add any table from your `public` schema and + changes will be broadcast accordingly.

diff --git a/studio/pages/project/[ref]/api/index.tsx b/studio/pages/project/[ref]/api/index.tsx index 8f77c40f869..38c3914be07 100644 --- a/studio/pages/project/[ref]/api/index.tsx +++ b/studio/pages/project/[ref]/api/index.tsx @@ -533,10 +533,9 @@ const ResourceContent = ({

- Supabase provides realtime functionality. Right now we recommend only using this on a - server-side application (see our notes in the Authentication section). + Supabase provides realtime functionality and broadcasts database changes to authorized users depending on Row Level + Security (RLS) policies.

-

We are building advanced auth so that you can use realtime streams from anywhere.

Learn more. diff --git a/web/docs/going-into-prod.mdx b/web/docs/going-into-prod.mdx index c4fe7e50438..5ea25588196 100644 --- a/web/docs/going-into-prod.mdx +++ b/web/docs/going-into-prod.mdx @@ -15,10 +15,9 @@ After developing your project, and deciding it's time to Go Live With Real Users - Ensure RLS is enabled - Tables that do not have RLS enabled with reasonable policies allow any client to access and modify their data. This is unlikely to be what you want in the majority of cases. - [Learn more about RLS](/docs/guides/auth/row-level-security) -- Ensure replication is disabled for tables containing sensitive data: - - Today, Realtime does not respect RLS policies and any client with your anon key can listen to changes on tables where replication is enabled. - - Go to the Database > Replication page in the Supabase Dashboard to manage these settings - - We are launching [RLS for realtime](https://github.com/supabase/walrus) soon, but in the meantime you should only use it for tables containing public data (scoreboards, blog posts, etc.) +- Enable replication on tables containing sensitive data by enabling Row Level Security (RLS) and setting row security policies: + - Go to the Authentication > Policies page in the Supabase Dashboard to enable RLS and create security policies + - Go to the Database > Replication page in the Supabase Dashboard to manage replication tables - Enable 2FA on Github - Since your Github account gives you administrative rights to your Supabase project, you should protect it with a strong password and 2FA using a U2F key or a TOTP app. - Ensure email confirmations are enabled in the Auth > Settings page diff --git a/web/docs/guides/auth.mdx b/web/docs/guides/auth.mdx index 624b0e8210f..0ad779fd151 100644 --- a/web/docs/guides/auth.mdx +++ b/web/docs/guides/auth.mdx @@ -106,14 +106,17 @@ Supabase provides a special function in Postgres, `auth.uid()`, which extracts t ## Tips -#### Disable realtime for private tables +#### Enable Realtime for database tables + +Realtime server broadcasts database changes to authorized users depending on your Row Level Security (RLS) policies. +We recommend that you enable row level security and set row security policies on tables that you add to the publication. +However, you may choose to disable RLS on a table and have changes broadcast to all connected clients. -Our realtime server doesn't provide per-user security. Until we build a more robust auth system for WebSockets, you can disable realtime functionality for any private tables. To do this, you can manage the underlying Postgres replication publication: ```sql /** * REALTIME SUBSCRIPTIONS - * Only allow realtime listening on public tables. + * Realtime enables listening to any table in your public schema. */ begin; @@ -131,8 +134,6 @@ alter publication supabase_realtime add table products; alter publication supabase_realtime add table posts; ``` -We're in the process of building [enhanced realtime security](https://github.com/supabase/walrus). - ## Next Steps - Read more about Auth in the [Guides](/docs/guides/auth/intro). diff --git a/web/docs/guides/auth/row-level-security.mdx b/web/docs/guides/auth/row-level-security.mdx index 5ee6a1a7ff0..3291924c4ef 100644 --- a/web/docs/guides/auth/row-level-security.mdx +++ b/web/docs/guides/auth/row-level-security.mdx @@ -270,14 +270,16 @@ using ( ## Tips -### Disable realtime for private tables +### Enable Realtime for database tables -Our realtime server doesn't provide per-user security. Until we build a more robust auth system for WebSockets, you can disable realtime functionality for any private tables. To do this, you can manage the underlying Postgres replication publication: +Realtime server broadcasts database changes to authorized users depending on your Row Level Security (RLS) policies. +We recommend that you enable row level security and set row security policies on tables that you add to the publication. +However, you may choose to disable RLS on a table and have changes broadcast to all connected clients. ```sql /** * REALTIME SUBSCRIPTIONS - * Only allow realtime listening on public tables. + * Realtime enables listening to any table in your public schema. */ begin; @@ -295,8 +297,6 @@ alter publication supabase_realtime add table products; alter publication supabase_realtime add table posts; ``` -We're in the process of building [enhanced realtime security](https://github.com/supabase/walrus). - ### You don't have to use policies You can also put your authorization rules in your middleware, similar to how you would create security rules with any other `backend <-> middleware <-> frontend` architecture. diff --git a/web/docs/guides/database.mdx b/web/docs/guides/database.mdx index 3890adcaa01..703cd695322 100644 --- a/web/docs/guides/database.mdx +++ b/web/docs/guides/database.mdx @@ -70,7 +70,12 @@ You can enable Postgres extensions with the click of a button within the Supabas ### Realtime Supabase provides a realtime engine on top of Postgres, so that you can listen to changes as they happen. -Our realtime engine uses the built-in replication functionality of Postgres. +Our realtime engine uses the built-in replication functionality of Postgres. + +Realtime server broadcasts database changes to authorized users depending on your Row Level Security (RLS) policies. +We recommend that you enable row level security and set row security policies on tables that you add to the publication. +However, you may choose to disable RLS on a table and have changes broadcast to all connected clients. + You can manage the realtime system, simply by [updating](/docs/guides/database/replication) the `supabase_realtime` publication. @@ -92,10 +97,6 @@ alter publication supabase_realtime add table products; alter publication supabase_realtime add table posts; ``` -| SECURITY WARNING | -| - | -| Please check [Disable realtime for private tables](/docs/guides/auth#disable-realtime-for-private-tables) for the current state of Realtime security. | - By default only "new" values are sent, but if you want to receive the old record (previous values) whenever you `update` or `delete` a record, you can update the replica identity of your tables, setting it to `full`: diff --git a/web/docs/guides/hosting/overview.mdx b/web/docs/guides/hosting/overview.mdx index eab6c17b5e3..9794838b8cf 100644 --- a/web/docs/guides/hosting/overview.mdx +++ b/web/docs/guides/hosting/overview.mdx @@ -24,7 +24,7 @@ If the tool doesn't exist, we build and open source it ourselves. - [PostgreSQL](https://www.postgresql.org/) is an object-relational database system with over 30 years of active development that has earned it a strong reputation for reliability, feature robustness, and performance. -- [Realtime](https://github.com/supabase/realtime) is an Elixir server that allows you to listen to PostgreSQL inserts, updates, and deletes using websockets. Supabase listens to Postgres' built-in replication functionality, converts the replication byte stream into JSON, then broadcasts the JSON over websockets. +- [Realtime](https://github.com/supabase/realtime) is an Elixir server that allows you to listen to PostgreSQL inserts, updates, and deletes using websockets. Realtime polls Postgres' built-in replication functionality for database changes, converts changes to JSON, then broadcasts the JSON over websockets to authorized clients. - [PostgREST](http://postgrest.org/) is a web server that turns your PostgreSQL database directly into a RESTful API - [Storage](https://github.com/supabase/storage-api) provides a RESTful interface for managing Files stored in S3, using Postgres to manage permissions. - [postgres-meta](https://github.com/supabase/postgres-meta) is a RESTful API for managing your Postgres, allowing you to fetch tables, add roles, and run queries, etc. diff --git a/www/_blog/2021-11-30-supabase-studio.mdx b/www/_blog/2021-11-30-supabase-studio.mdx index 9fbd05d62c5..257f07f347e 100644 --- a/www/_blog/2021-11-30-supabase-studio.mdx +++ b/www/_blog/2021-11-30-supabase-studio.mdx @@ -4,7 +4,7 @@ description: "The same Dashboard that you're using on our Platform is now availa author: paul_copplestone author_url: https://github.com/kiwicopple author_image_url: https://github.com/kiwicopple.png -authorURL: https://github.com/thorwebdev +authorURL: https://github.com/kiwicopple image: launch-week-three/studio/open-source-studio-og.png thumb: launch-week-three/studio/open-source-studio-thumb.png tags: diff --git a/www/_blog/2021-12-01-realtime-row-level-security-in-postgresql.mdx b/www/_blog/2021-12-01-realtime-row-level-security-in-postgresql.mdx new file mode 100644 index 00000000000..142b3e4e55c --- /dev/null +++ b/www/_blog/2021-12-01-realtime-row-level-security-in-postgresql.mdx @@ -0,0 +1,182 @@ +--- +title: "Realtime Postgres RLS now available on Supabase" +description: "Realtime database changes are now broadcast to authenticated users, respecting the same PostgreSQL policies that you use for Row Level Security." +author: oli_rice +author_url: https://github.com/olirice +author_image_url: https://github.com/olirice.png +authorURL: https://github.com/olirice +image: launch-week-three/realtime-row-level-security-in-postgresql/realtime-row-level-security-in-postgresql-og.png +thumb: launch-week-three/realtime-row-level-security-in-postgresql/realtime-row-level-security-in-postgresql-thumb.png +tags: + - launch-week + - realtime + - securitu +date: '2021-12-02' +toc_depth: 3 +--- + +Realtime is a server that listens to changes in your PostgreSQL database and broadcasts the changes to clients through a websocket connection. + +Today, we're announcing security improvements to Realtime, where database changes will be broadcast to authenticated users, respecting the same PostgreSQL policies that you use for Row Level Security. + +## Demo + + + + +## Overview + +Since the first commit of Realtime server back in [September 2019](https://github.com/supabase/realtime/commit/175f649784147af80acfc9ff5be9d160285c76ea), +we've worked hard to improve its usability and scalability. + +Until now, Realtime did not adhere to RLS policies and instead broadcast all database changes to all clients. +The unsafe nature of this behavior is the reason why Realtime has been an opt-in feature, and a key reason why we are still in Beta. + +As more developers rely on Realtime to receive and send database changes in their apps and services, +security has become a primary concern for us and others in the community who wish to build secure systems with Realtime. + +Supabase projects have supported Row Level Security (RLS) for API authorization since our [Auth launch](https://supabase.io/blog/2020/08/05/supabase-auth). +In that time, it has quickly become the recommended way to implement authorization. + +As we were evaluating possible solutions to improve Realtime security, we looked to our Auth implementation as inspiration for a cohesive security system. + +Today, we're updating Realtime to respect PostgreSQL RLS policies, so you can define your security rules once and have them automatically apply everywhere! + +Before diving deeper into our Realtime RLS implementation, let's briefly cover how RLS works in PostgreSQL. + +## Row Level Security Primer + +When you need to control access to individual rows of data, PostgreSQL has you covered with [Row Level Security (RLS) policies](https://www.postgresql.org/docs/current/ddl-rowsecurity.html). +An RLS policy is a snippet of SQL filtering which users have the authority to create/read/update/delete rows in a table. + +For example, the following policy would allow users to select their own rows in a todos table: + +```sql hideCopy +create policy todo_select_policy + on todos for select + using ( auth.uid() = user_id ); +``` + +which is equivalent to adding + +```sql hideCopy +select * +from todos +where auth.uid() = todos.user_id; -- Policy is implicitly added. +``` + +to queries. + +Check out the [Row Level Security guide](https://supabase.com/docs/guides/auth/row-level-security) for more info on how to use RLS with your project. + +## Realtime Design + +Our Realtime server receives and decodes binary changes from PostgreSQL logical replication, converts those changes to JSON, and broadcasts them to all connected clients. + +### Challenge for RLS + +The challenge, when applying row level security to the replication stream is that the visibility of a row may be different for each user subscribed to a database table. + +We recognized that to fully secure Realtime in accordance with row level security, a row's visibility must be checked separately for each user on every change. +However, this quickly becomes a performance bottleneck when the number of changes, or number of subscribers, is large. + +Since we can't control the number of subscribers or the number of changed records, we instead focused on making the security check for each user on every change as fast as possible. + +### Implementation Overview + +With these challenges in mind, we upstreamed the security responsibility to the database. Write Ahead Log Realtime Unified Security (WALRUS) exposes a PostgreSQL +function that Realtime server invokes with database changes. + +### WALRUS Implementation + +[WALRUS](https://github.com/supabase/walrus) inspects each record in the replication change to: + +- Identify the source table (e.g. `public.notes`). +- Identify the change's action (INSERT/UPDATE/DELETE/TRUNCATE*). +- Query the `subscription` table to determine the connected users who are actively subscribed to the source table. The `subscription` table is kept up to date by the Realtime server and tracks all connected users and the tables they are currently subscribed to. +- For each subscriber: + - Assume the identity of the subscriber. + - Query the source table to see if the record is visible to that subscriber. +- Report the list of subscribers who are authorized to view the record back to Realtime server. + +*Realtime server does not broadcast TRUNCATE changes + +**Efficiently Query to Check Access** + +To maximize throughput, the query used to evaluate if a row is visible to a subscriber always queries using the tables primary key. + +For example: + +```sql hideCopy +select exists(select 1 from some_table where id = 806); +``` + +When more than one subscriber exists, the query is wrapped in a [prepared statement](https://www.postgresql.org/docs/13/sql-prepare.html) to remove the cost of the PostgreSQL +query planner on subsequent calls. The query planner time is frequently 2-3x execution time for simple queries, so this immediately multiplies throughput in the most common cases! + +```sql hideCopy +"Planning Time: 0.099 ms" +"Execution Time: 0.051 ms" +``` + +**Colocation** + +Colocating the security engine with subscriber data inside PostgreSQL allows us to avoid significant overhead when applying RLS policies. +Namely, network round-trip latency and I/O bottlenecks are removed while connection overhead is reduced relative to testing each record's visibility by polling the database from a separate process. +Instead, the SQL function only consumes a single connection and performs no network I/O. + +### Performance + +The throughput performance of the database server is best measured in terms of record processing time. As the number of subscribers to a table grows, the time required to process each record, +and the resultant processing time also grows. + +![supabase-realtime-processing-per-subscription](/images/blog/launch-week-three/realtime-row-level-security-in-postgresql/supabase-realtime-processing-per-subscription.png) + +| Subscribers | 1 | 5 | 10 | 25 | 50 | 100 | 250 | 500 | 1,000 | 2,000 | 5,000 | 10,000 | +|------------------------|------|------|------|------|------|------|------|------|-------|-------|-------|--------| +| Processing Time (ms) | 11.2 | 12.5 | 14.2 | 16.7 | 18.8 | 24.5 | 27.8 | 29.1 | 64.7 | 75.5 | 158.4 | 303.8 | + +## Best Practices for Performance + +To get the most out of Realtime row level security, follow these guidelines: + +### Disable for public tables + +If your data is insensitive or publicly available, such as stock prices listed under NASDAQ, then don't enable row level security! + +The fastest security policy is one that doesn't exist :) + +### Optimize your policies + +If you do need row level security, make sure that your policies are fast. + +Remember that your policy is executed *each* time a query touches the table that the policy is applied to. If your policy is slow, all access to that table will be slow. +Avoid joins within RLS policies when you can, and make sure all filter conditions use an index. + +Additionally, keep in mind that if you use joins within an RLS policy, any RLS policies on the tables you're joining to will also be executed in turn, adding to the overall overhead. + +### Small primary keys + +Keep your primary keys small and efficient. + +Use single column primary keys with a fixed field size (integer, uuid, etc.) over text or multi-column indexes. + +## Next Steps + +Realtime RLS is available today on all existing and new Supabase projects. To get started, upgrade your Supabase JavaScript client to version v1.23.0 +and launch your new PostgreSQL database today: [database.new](https://database.new) + +## Credits + +Authored by: + +- [Oliver Rice](https://github.com/olirice) +- [Wen Bo Xie](https://github.com/w3b6x9) \ No newline at end of file diff --git a/www/components/Hero.tsx b/www/components/Hero.tsx index a1ceafdab3e..8486d875537 100644 --- a/www/components/Hero.tsx +++ b/www/components/Hero.tsx @@ -22,7 +22,7 @@ const Hero = () => {

Create a backend in less than 2 minutes. Start your project with a Postgres - Database, Authentication, instant APIs, realtime subscriptions and Storage. + Database, Authentication, instant APIs, Realtime subscriptions and Storage.

Serverless functions coming soon

diff --git a/www/data/Announcement.json b/www/data/Announcement.json index 88a0ed0026e..88fd688b4cf 100644 --- a/www/data/Announcement.json +++ b/www/data/Announcement.json @@ -1,6 +1,6 @@ { "show": true, "text": "Supabase Launch Week III", - "cta": "Supabase Studio is now Open Source", - "link": "https://www.producthunt.com/posts/open-source-postgresql-dashboard" + "cta": "Realtime Postgres RLS now available", + "link": "/blog/2021/12/01/realtime-row-level-security-in-postgresql" } diff --git a/www/lib/authors.json b/www/lib/authors.json index 80f65287107..db0b667bdec 100644 --- a/www/lib/authors.json +++ b/www/lib/authors.json @@ -89,5 +89,19 @@ "position": "Engineering", "username": "gurjeet", "author_image_url": "https://github.com/gurjeet.png" + }, + "wenbo_xie": { + "author": "Wen Bo Xie", + "author_url": "https://github.com/w3b6x9", + "position": "Engineering", + "username": "w3b6x9", + "author_image_url": "https://github.com/w3b6x9.png" + }, + "oli_rice": { + "author": "Oliver Rice", + "author_url": "https://github.com/olirice", + "position": "Engineering", + "username": "olirice", + "author_image_url": "https://github.com/olirice.png" } } diff --git a/www/public/images/blog/launch-week-three/realtime-row-level-security-in-postgresql/realtime-row-level-security-in-postgresql-og.png b/www/public/images/blog/launch-week-three/realtime-row-level-security-in-postgresql/realtime-row-level-security-in-postgresql-og.png new file mode 100644 index 00000000000..bc11f5552fd Binary files /dev/null and b/www/public/images/blog/launch-week-three/realtime-row-level-security-in-postgresql/realtime-row-level-security-in-postgresql-og.png differ diff --git a/www/public/images/blog/launch-week-three/realtime-row-level-security-in-postgresql/realtime-row-level-security-in-postgresql-thumb.png b/www/public/images/blog/launch-week-three/realtime-row-level-security-in-postgresql/realtime-row-level-security-in-postgresql-thumb.png new file mode 100644 index 00000000000..bc11f5552fd Binary files /dev/null and b/www/public/images/blog/launch-week-three/realtime-row-level-security-in-postgresql/realtime-row-level-security-in-postgresql-thumb.png differ diff --git a/www/public/images/blog/launch-week-three/realtime-row-level-security-in-postgresql/supabase-realtime-processing-per-subscription.png b/www/public/images/blog/launch-week-three/realtime-row-level-security-in-postgresql/supabase-realtime-processing-per-subscription.png new file mode 100644 index 00000000000..ab5ca1e6488 Binary files /dev/null and b/www/public/images/blog/launch-week-three/realtime-row-level-security-in-postgresql/supabase-realtime-processing-per-subscription.png differ diff --git a/www/public/rss.xml b/www/public/rss.xml index e8cc43a4429..124fa2d82eb 100644 --- a/www/public/rss.xml +++ b/www/public/rss.xml @@ -5,9 +5,17 @@ https://supabase.com Latest news from Supabase en - Mon, 29 Nov 2021 16:00:00 GMT + Wed, 01 Dec 2021 16:00:00 GMT + + https://supabase.com/blog/2021/12/01/realtime-row-level-security-in-postgresql + Realtime Postgres RLS now available on Supabase + https://supabase.com/blog/2021/12/01/realtime-row-level-security-in-postgresql + Realtime database changes are now broadcast to authenticated users, respecting the same PostgreSQL policies that you use for Row Level Security. + Wed, 01 Dec 2021 16:00:00 GMT + + https://supabase.com/blog/2021/11/30/supabase-studio Supabase Studio