Merge pull request #4183 from supabase/docs/realtime-security

docs: add Realtime security docs
This commit is contained in:
Copple authored and GitHub committed 2021-12-01 22:57:43 +08:00
commit f1de7a1d3a
20 files changed
+252 -39

No files matched your search

+1 -1
View File
@@ -54,7 +54,7 @@ You can also [self-host](https://supabase.com/docs/guides/self-hosting) and [dev
![Architecture](https://supabase.com/docs/assets/images/supabase-architecture-9050a7317e9ec7efb7807f5194122e48.png)
- [PostgreSQL](https://www.postgresql.org/) is an object-relational database system with over 30 years of active development that has earned it a strong reputation for reliability, feature robustness, and performance.
- [Realtime](https://github.com/supabase/realtime) is an Elixir server that allows you to listen to PostgreSQL inserts, updates, and deletes using websockets. Supabase listens to Postgres' built-in replication functionality, converts the replication byte stream into JSON, then broadcasts the JSON over websockets.
- [Realtime](https://github.com/supabase/realtime) is an Elixir server that allows you to listen to PostgreSQL inserts, updates, and deletes using websockets. Realtime polls Postgres' built-in replication functionality for database changes, converts changes to JSON, then broadcasts the JSON over websockets to authorized clients.
- [PostgREST](http://postgrest.org/) is a web server that turns your PostgreSQL database directly into a RESTful API
- [Storage](https://github.com/supabase/storage-api) provides a RESTful interface for managing Files stored in S3, using Postgres to manage permissions.
- [postgres-meta](https://github.com/supabase/postgres-meta) is a RESTful API for managing your Postgres, allowing you to fetch tables, add roles, and run queries, etc.
+10 -3
View File
@@ -87,7 +87,7 @@ services:
realtime:
container_name: supabase-realtime
image: supabase/realtime:v0.15.0
image: supabase/realtime:v0.19.0
depends_on:
- db
restart: unless-stopped
@@ -97,10 +97,17 @@ services:
DB_NAME: postgres
DB_USER: postgres
DB_PASSWORD: ${POSTGRES_PASSWORD}
SLOT_NAME: supabase_realtime
DB_SSL: "false"
PORT: 4000
SECURE_CHANNELS: "true"
JWT_SECRET: ${JWT_SECRET}
REPLICATION_MODE: RLS
REPLICATION_POLL_INTERVAL: 100
SECURE_CHANNELS: "true"
SLOT_NAME: supabase_realtime_rls
TEMPORARY_SLOT: "true"
command: >
bash -c "./prod/rel/realtime/bin/realtime eval Realtime.Release.migrate
&& ./prod/rel/realtime/bin/realtime start"
storage:
container_name: supabase-storage
+2 -1
View File
@@ -1,4 +1,5 @@
-- Set up reatime
-- Set up realtime
create schema if not exists realtime;
-- create publication supabase_realtime; -- defaults to empty publication
create publication supabase_realtime;
+1 -1
View File
@@ -15,7 +15,7 @@ const Index = () => {
</h1>
<p className="mt-3 text-base text-white sm:mt-5 sm:text-lg sm:max-w-xl sm:mx-auto md:mt-5 md:text-xl lg:mx-0">
Create a backend in less than 2 minutes. Start your project with a Postgres Database,
Authentication, instant APIs, realtime subscriptions and Storage.
Authentication, instant APIs, Realtime subscriptions and Storage.
</p>
<div className="mt-5 sm:mt-8 sm:flex sm:justify-center lg:justify-start">
<div className="rounded-md shadow">
@@ -42,18 +42,19 @@ export default function Authentication({ autoApiService, selectedLang }) {
</p>
<h4 className="mt-8">Realtime Security</h4>
<p>
The realtime server doesn't provide per-user security. Until we build a more robust auth
system for websockets, you shouldn't use realtime on any client which is exposed to
users.
Realtime server broadcasts database changes to authorized users depending on your Row Level Security (RLS) policies.
We recommend that you enable row level security and set row security policies on tables that you add to the publication.
However, you may choose to disable RLS on a table and have changes broadcast to all connected clients.
</p>
<p>
For security purposes, you can run{' '}
You can get started by running{' '}
<code>
begin; drop publication if exists supabase_realtime; create publication
supabase_realtime; commit;
</code>
. This creates a publication which is not subscribed to any table and completely
disables realtime updates on the Supabase client.
disables Realtime on the Supabase client. Then, you can add any table from your `public` schema and
changes will be broadcast accordingly.
</p>
</article>
<article className="code">
+2 -3
View File
@@ -533,10 +533,9 @@ const ResourceContent = ({
<div className="doc-section">
<article className="text ">
<p>
Supabase provides realtime functionality. Right now we recommend only using this on a
server-side application (see our notes in the Authentication section).
Supabase provides realtime functionality and broadcasts database changes to authorized users depending on Row Level
Security (RLS) policies.
</p>
<p>We are building advanced auth so that you can use realtime streams from anywhere.</p>
<p>
<a href="https://supabase.com/docs/client/subscribe" target="_blank">
Learn more.
+3 -4
View File
@@ -15,10 +15,9 @@ After developing your project, and deciding it's time to Go Live With Real Users
- Ensure RLS is enabled
- Tables that do not have RLS enabled with reasonable policies allow any client to access and modify their data. This is unlikely to be what you want in the majority of cases.
- [Learn more about RLS](/docs/guides/auth/row-level-security)
- Ensure replication is disabled for tables containing sensitive data:
- Today, Realtime does not respect RLS policies and any client with your anon key can listen to changes on tables where replication is enabled.
- Go to the Database > Replication page in the Supabase Dashboard to manage these settings
- We are launching [RLS for realtime](https://github.com/supabase/walrus) soon, but in the meantime you should only use it for tables containing public data (scoreboards, blog posts, etc.)
- Enable replication on tables containing sensitive data by enabling Row Level Security (RLS) and setting row security policies:
- Go to the Authentication > Policies page in the Supabase Dashboard to enable RLS and create security policies
- Go to the Database > Replication page in the Supabase Dashboard to manage replication tables
- Enable 2FA on Github
- Since your Github account gives you administrative rights to your Supabase project, you should protect it with a strong password and 2FA using a U2F key or a TOTP app.
- Ensure email confirmations are enabled in the Auth > Settings page
+6 -5
View File
@@ -106,14 +106,17 @@ Supabase provides a special function in Postgres, `auth.uid()`, which extracts t
## Tips
#### Disable realtime for private tables
#### Enable Realtime for database tables
Realtime server broadcasts database changes to authorized users depending on your Row Level Security (RLS) policies.
We recommend that you enable row level security and set row security policies on tables that you add to the publication.
However, you may choose to disable RLS on a table and have changes broadcast to all connected clients.
Our realtime server doesn't provide per-user security. Until we build a more robust auth system for WebSockets, you can disable realtime functionality for any private tables. To do this, you can manage the underlying Postgres replication publication:
```sql
/**
* REALTIME SUBSCRIPTIONS
* Only allow realtime listening on public tables.
* Realtime enables listening to any table in your public schema.
*/
begin;
@@ -131,8 +134,6 @@ alter publication supabase_realtime add table products;
alter publication supabase_realtime add table posts;
```
We're in the process of building [enhanced realtime security](https://github.com/supabase/walrus).
## Next Steps
- Read more about Auth in the [Guides](/docs/guides/auth/intro).
+5 -5
View File
@@ -270,14 +270,16 @@ using (
## Tips
### Disable realtime for private tables
### Enable Realtime for database tables
Our realtime server doesn't provide per-user security. Until we build a more robust auth system for WebSockets, you can disable realtime functionality for any private tables. To do this, you can manage the underlying Postgres replication publication:
Realtime server broadcasts database changes to authorized users depending on your Row Level Security (RLS) policies.
We recommend that you enable row level security and set row security policies on tables that you add to the publication.
However, you may choose to disable RLS on a table and have changes broadcast to all connected clients.
```sql
/**
* REALTIME SUBSCRIPTIONS
* Only allow realtime listening on public tables.
* Realtime enables listening to any table in your public schema.
*/
begin;
@@ -295,8 +297,6 @@ alter publication supabase_realtime add table products;
alter publication supabase_realtime add table posts;
```
We're in the process of building [enhanced realtime security](https://github.com/supabase/walrus).
### You don't have to use policies
You can also put your authorization rules in your middleware, similar to how you would create security rules with any other `backend <-> middleware <-> frontend` architecture.
+6 -5
View File
@@ -70,7 +70,12 @@ You can enable Postgres extensions with the click of a button within the Supabas
### Realtime
Supabase provides a realtime engine on top of Postgres, so that you can listen to changes as they happen.
Our realtime engine uses the built-in replication functionality of Postgres.
Our realtime engine uses the built-in replication functionality of Postgres.
Realtime server broadcasts database changes to authorized users depending on your Row Level Security (RLS) policies.
We recommend that you enable row level security and set row security policies on tables that you add to the publication.
However, you may choose to disable RLS on a table and have changes broadcast to all connected clients.
You can manage the realtime system, simply by
[updating](/docs/guides/database/replication) the `supabase_realtime` publication.
@@ -92,10 +97,6 @@ alter publication supabase_realtime add table products;
alter publication supabase_realtime add table posts;
```
| SECURITY WARNING |
| - |
| Please check [Disable realtime for private tables](/docs/guides/auth#disable-realtime-for-private-tables) for the current state of Realtime security. |
By default only "new" values are sent, but if you want to receive the old record (previous values) whenever you `update` or `delete` a record,
you can update the replica identity of your tables, setting it to `full`:
+1 -1
View File
@@ -24,7 +24,7 @@ If the tool doesn't exist, we build and open source it ourselves.
- [PostgreSQL](https://www.postgresql.org/) is an object-relational database system with over 30 years of active development that has earned it a strong reputation for reliability, feature robustness, and performance.
- [Realtime](https://github.com/supabase/realtime) is an Elixir server that allows you to listen to PostgreSQL inserts, updates, and deletes using websockets. Supabase listens to Postgres' built-in replication functionality, converts the replication byte stream into JSON, then broadcasts the JSON over websockets.
- [Realtime](https://github.com/supabase/realtime) is an Elixir server that allows you to listen to PostgreSQL inserts, updates, and deletes using websockets. Realtime polls Postgres' built-in replication functionality for database changes, converts changes to JSON, then broadcasts the JSON over websockets to authorized clients.
- [PostgREST](http://postgrest.org/) is a web server that turns your PostgreSQL database directly into a RESTful API
- [Storage](https://github.com/supabase/storage-api) provides a RESTful interface for managing Files stored in S3, using Postgres to manage permissions.
- [postgres-meta](https://github.com/supabase/postgres-meta) is a RESTful API for managing your Postgres, allowing you to fetch tables, add roles, and run queries, etc.
+1 -1
View File
@@ -4,7 +4,7 @@ description: "The same Dashboard that you're using on our Platform is now availa
author: paul_copplestone
author_url: https://github.com/kiwicopple
author_image_url: https://github.com/kiwicopple.png
authorURL: https://github.com/thorwebdev
authorURL: https://github.com/kiwicopple
image: launch-week-three/studio/open-source-studio-og.png
thumb: launch-week-three/studio/open-source-studio-thumb.png
tags:
@@ -0,0 +1,182 @@
---
title: "Realtime Postgres RLS now available on Supabase"
description: "Realtime database changes are now broadcast to authenticated users, respecting the same PostgreSQL policies that you use for Row Level Security."
author: oli_rice
author_url: https://github.com/olirice
author_image_url: https://github.com/olirice.png
authorURL: https://github.com/olirice
image: launch-week-three/realtime-row-level-security-in-postgresql/realtime-row-level-security-in-postgresql-og.png
thumb: launch-week-three/realtime-row-level-security-in-postgresql/realtime-row-level-security-in-postgresql-thumb.png
tags:
- launch-week
- realtime
- securitu
date: '2021-12-02'
toc_depth: 3
---
Realtime is a server that listens to changes in your PostgreSQL database and broadcasts the changes to clients through a websocket connection.
Today, we're announcing security improvements to Realtime, where database changes will be broadcast to authenticated users, respecting the same PostgreSQL policies that you use for Row Level Security.
## Demo
<iframe
className="w-full video-with-border"
width="640"
height="385"
src="https://www.youtube-nocookie.com/embed/zHvatf2wySI"
frameBorder="1"
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
allowFullScreen
></iframe>
## Overview
Since the first commit of Realtime server back in [September 2019](https://github.com/supabase/realtime/commit/175f649784147af80acfc9ff5be9d160285c76ea),
we've worked hard to improve its usability and scalability.
Until now, Realtime did not adhere to RLS policies and instead broadcast all database changes to all clients.
The unsafe nature of this behavior is the reason why Realtime has been an opt-in feature, and a key reason why we are still in Beta.
As more developers rely on Realtime to receive and send database changes in their apps and services,
security has become a primary concern for us and others in the community who wish to build secure systems with Realtime.
Supabase projects have supported Row Level Security (RLS) for API authorization since our [Auth launch](https://supabase.io/blog/2020/08/05/supabase-auth).
In that time, it has quickly become the recommended way to implement authorization.
As we were evaluating possible solutions to improve Realtime security, we looked to our Auth implementation as inspiration for a cohesive security system.
Today, we're updating Realtime to respect PostgreSQL RLS policies, so you can define your security rules once and have them automatically apply everywhere!
Before diving deeper into our Realtime RLS implementation, let's briefly cover how RLS works in PostgreSQL.
## Row Level Security Primer
When you need to control access to individual rows of data, PostgreSQL has you covered with [Row Level Security (RLS) policies](https://www.postgresql.org/docs/current/ddl-rowsecurity.html).
An RLS policy is a snippet of SQL filtering which users have the authority to create/read/update/delete rows in a table.
For example, the following policy would allow users to select their own rows in a todos table:
```sql hideCopy
create policy todo_select_policy
on todos for select
using ( auth.uid() = user_id );
```
which is equivalent to adding
```sql hideCopy
select *
from todos
where auth.uid() = todos.user_id; -- Policy is implicitly added.
```
to queries.
Check out the [Row Level Security guide](https://supabase.com/docs/guides/auth/row-level-security) for more info on how to use RLS with your project.
## Realtime Design
Our Realtime server receives and decodes binary changes from PostgreSQL logical replication, converts those changes to JSON, and broadcasts them to all connected clients.
### Challenge for RLS
The challenge, when applying row level security to the replication stream is that the visibility of a row may be different for each user subscribed to a database table.
We recognized that to fully secure Realtime in accordance with row level security, a row's visibility must be checked separately for each user on every change.
However, this quickly becomes a performance bottleneck when the number of changes, or number of subscribers, is large.
Since we can't control the number of subscribers or the number of changed records, we instead focused on making the security check for each user on every change as fast as possible.
### Implementation Overview
With these challenges in mind, we upstreamed the security responsibility to the database. Write Ahead Log Realtime Unified Security (WALRUS) exposes a PostgreSQL
function that Realtime server invokes with database changes.
### WALRUS Implementation
[WALRUS](https://github.com/supabase/walrus) inspects each record in the replication change to:
- Identify the source table (e.g. `public.notes`).
- Identify the change's action (INSERT/UPDATE/DELETE/TRUNCATE*).
- Query the `subscription` table to determine the connected users who are actively subscribed to the source table. The `subscription` table is kept up to date by the Realtime server and tracks all connected users and the tables they are currently subscribed to.
- For each subscriber:
- Assume the identity of the subscriber.
- Query the source table to see if the record is visible to that subscriber.
- Report the list of subscribers who are authorized to view the record back to Realtime server.
<small>*Realtime server does not broadcast TRUNCATE changes</small>
**Efficiently Query to Check Access**
To maximize throughput, the query used to evaluate if a row is visible to a subscriber always queries using the tables primary key.
For example:
```sql hideCopy
select exists(select 1 from some_table where id = 806);
```
When more than one subscriber exists, the query is wrapped in a [prepared statement](https://www.postgresql.org/docs/13/sql-prepare.html) to remove the cost of the PostgreSQL
query planner on subsequent calls. The query planner time is frequently 2-3x execution time for simple queries, so this immediately multiplies throughput in the most common cases!
```sql hideCopy
"Planning Time: 0.099 ms"
"Execution Time: 0.051 ms"
```
**Colocation**
Colocating the security engine with subscriber data inside PostgreSQL allows us to avoid significant overhead when applying RLS policies.
Namely, network round-trip latency and I/O bottlenecks are removed while connection overhead is reduced relative to testing each record's visibility by polling the database from a separate process.
Instead, the SQL function only consumes a single connection and performs no network I/O.
### Performance
The throughput performance of the database server is best measured in terms of record processing time. As the number of subscribers to a table grows, the time required to process each record,
and the resultant processing time also grows.
![supabase-realtime-processing-per-subscription](/images/blog/launch-week-three/realtime-row-level-security-in-postgresql/supabase-realtime-processing-per-subscription.png)
| Subscribers | 1 | 5 | 10 | 25 | 50 | 100 | 250 | 500 | 1,000 | 2,000 | 5,000 | 10,000 |
|------------------------|------|------|------|------|------|------|------|------|-------|-------|-------|--------|
| Processing Time (ms) | 11.2 | 12.5 | 14.2 | 16.7 | 18.8 | 24.5 | 27.8 | 29.1 | 64.7 | 75.5 | 158.4 | 303.8 |
## Best Practices for Performance
To get the most out of Realtime row level security, follow these guidelines:
### Disable for public tables
If your data is insensitive or publicly available, such as stock prices listed under NASDAQ, then don't enable row level security!
The fastest security policy is one that doesn't exist :)
### Optimize your policies
If you do need row level security, make sure that your policies are fast.
Remember that your policy is executed *each* time a query touches the table that the policy is applied to. If your policy is slow, all access to that table will be slow.
Avoid joins within RLS policies when you can, and make sure all filter conditions use an index.
Additionally, keep in mind that if you use joins within an RLS policy, any RLS policies on the tables you're joining to will also be executed in turn, adding to the overall overhead.
### Small primary keys
Keep your primary keys small and efficient.
Use single column primary keys with a fixed field size (integer, uuid, etc.) over text or multi-column indexes.
## Next Steps
Realtime RLS is available today on all existing and new Supabase projects. To get started, upgrade your Supabase JavaScript client to version v1.23.0
and launch your new PostgreSQL database today: [database.new](https://database.new)
## Credits
Authored by:
- [Oliver Rice](https://github.com/olirice)
- [Wen Bo Xie](https://github.com/w3b6x9)
+1 -1
View File
@@ -22,7 +22,7 @@ const Hero = () => {
<Typography.Text>
<p className="mt-5 text-base sm:mt-5 lg:text-lg ">
Create a backend in less than 2 minutes. Start your project with a Postgres
Database, Authentication, instant APIs, realtime subscriptions and Storage.
Database, Authentication, instant APIs, Realtime subscriptions and Storage.
</p>
<p className="mt-3 text-base">Serverless functions coming soon</p>
</Typography.Text>
+2 -2
View File
@@ -1,6 +1,6 @@
{
"show": true,
"text": "Supabase Launch Week III",
"cta": "Supabase Studio is now Open Source",
"link": "https://www.producthunt.com/posts/open-source-postgresql-dashboard"
"cta": "Realtime Postgres RLS now available",
"link": "/blog/2021/12/01/realtime-row-level-security-in-postgresql"
}
+14
View File
@@ -89,5 +89,19 @@
"position": "Engineering",
"username": "gurjeet",
"author_image_url": "https://github.com/gurjeet.png"
},
"wenbo_xie": {
"author": "Wen Bo Xie",
"author_url": "https://github.com/w3b6x9",
"position": "Engineering",
"username": "w3b6x9",
"author_image_url": "https://github.com/w3b6x9.png"
},
"oli_rice": {
"author": "Oliver Rice",
"author_url": "https://github.com/olirice",
"position": "Engineering",
"username": "olirice",
"author_image_url": "https://github.com/olirice.png"
}
}
+9 -1
View File
@@ -5,9 +5,17 @@
<link>https://supabase.com</link>
<description>Latest news from Supabase</description>
<language>en</language>
<lastBuildDate>Mon, 29 Nov 2021 16:00:00 GMT</lastBuildDate>
<lastBuildDate>Wed, 01 Dec 2021 16:00:00 GMT</lastBuildDate>
<atom:link href="https://supabase.com/blog/rss.xml" rel="self" type="application/rss+xml"/>
<item>
<guid>https://supabase.com/blog/2021/12/01/realtime-row-level-security-in-postgresql</guid>
<title>Realtime Postgres RLS now available on Supabase</title>
<link>https://supabase.com/blog/2021/12/01/realtime-row-level-security-in-postgresql</link>
<description>Realtime database changes are now broadcast to authenticated users, respecting the same PostgreSQL policies that you use for Row Level Security.</description>
<pubDate>Wed, 01 Dec 2021 16:00:00 GMT</pubDate>
</item>
<item>
<guid>https://supabase.com/blog/2021/11/30/supabase-studio</guid>
<title>Supabase Studio</title>