Exporting all rows (in CSV, SQL, or JSON format) currently uses offset pagination, which can cause performance problems if the table is large. There is also a correctness problem if the table is being actively updated as the export happens, because the relative row offsets could shift between queries.
Now that composite filters are available in postgres-meta, we can change to using cursor pagination on the primary key (or any non-null unique keys) wherever possible. Where this is not possible, the user will be shown a confirmation dialog explaining the possible performance impact.
---------
Co-authored-by: Ali Waseem <waseema393@gmail.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
There is an edge case interaction between the Postgres query parser and
MSSQL foreign tables, where the query parser may drop sort clauses that
are redundant with applied filters. This leads to invalid MSSQL syntax,
because the resulting query has a `limit` but no `sort`, and the user
sees a confusing error message.
This PR detects this edge case on MSSQL foreign tables. There are three
cases:
1. The user filters by a column, but there are still other columns
available for sorting. The default search for a sorting column will
leave out the filtered column.
2. The user filters by a column/several columns, and there are no more
columns that can be used for sorting. We stop the query and show an
admonition.
3. The user filters by a column, then tries to sort by the same column.
We stop the query and show an admonition.
* Add custom types for queries, mutations and infinite queries.
* Migrate all queries to use the new type.
* Migrate all infinite queries to useCustomInfiniteQueryOptions.
* Migrate all mutations to use useCustomMutationOptions.
* Add type to all imports in `types` folder.
* Migrate all uses of invalidateQueries to use object syntax.
* Migrate the remainder of useInfiniteQuery.
* Migrate all setQueriesData.
* Migrate all fetchQuery uses.
* Migrate some leftover functions from RQ.
* Fix issues found by Charis.
Increasing this because of reports that CSV uploads are hitting 429s. (I
checked and there are no Retry-After headers, so we should be falling
back to the default exponential backoff.)
Also brings the implementation in line with the comment :P Before, the
comment claimed we were starting at 1s but we were starting at 500ms.
* feat: optimized users page
* Update UI
* Reinstate footer with count if mode is freeform
* Simplify disabled sort in performance mode
* Clean
* Small fix
* Final fixes
* Shift users SQL query to packages/pg-meta
* Nit unrelated: Clear query params from useLogsUrlState when going to logs tab of a selected user
* Shift user count SQL and get count estimate SQL into packages/pg-meta
* Fix
* Nit
* Nit
* Minor nits
* Refactor UX for searching
---------
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
Add telemetry tracking for activation-related table operations
- Implement SQL event parser to detect table creation, data insertion, and RLS enablement
- Add telemetry tracking for these operations in table editor as well
- Add test coverage for SQL event parser
* Update Supabase docs URLs to use env variable
Co-authored-by: a <a@alaisteryoung.com>
* Refactor: Use DOCS_URL constant for documentation links
This change centralizes documentation links using a new DOCS_URL constant, improving maintainability and consistency.
Co-authored-by: a <a@alaisteryoung.com>
* Refactor: Use DOCS_URL constant for all documentation links
This change replaces hardcoded documentation URLs with a centralized constant, improving maintainability and consistency.
Co-authored-by: a <a@alaisteryoung.com>
* replace more instances
* ci: Autofix updates from GitHub workflow
* remaining instances
* fix duplicate useRouter
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: alaister <10985857+alaister@users.noreply.github.com>
fix: handle undefined count when fetchint table rows
count can be undefined in the return from fetching table rows (it uses
executeSqlQuery under the hood which has no type safety) so we need to
deal with that case
* fix: estimate number of users when count is large
In the Auth Users table, we always fetch an exact count of users. This
can be a problem for projects with many (>50K) users as the count(*)
might cause performance issues on the database. We already have logic on
the Table Editor to only run automatic count estimates (fetching the
exact count only if usr requests it), this change ports the same logic
over to Auth Users.
* Nit refactor
---------
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
* chore(studio): move Query to pgMeta add tests
- Move the Query builder from studio to pgMeta
- Add e2e tests over the generated sql to ensure syntax and runtime
result over pg database
- fix bug with orde by for table with undefined column
* chore: add table-row-query to pgMeta and tests
* chore: fix query import path
* chore: reduce maxArraySize
* chore: use pg-meta getTableRowsSql implementation in studio
* chore: add truncation on large array fields
* chore: set ES target for lint
* chore: update comment
* chore: reduce test size for CI
* chore(studio): move Query to pgMeta add tests
- Move the Query builder from studio to pgMeta
- Add e2e tests over the generated sql to ensure syntax and runtime
result over pg database
- fix bug with orde by for table with undefined column
* chore: fix query import path
* chore: set ES target for lint
* chore: add github action for pg-meta test package
* chore: add tsconfig to sparse checkout
* chore: remove useExecuteSqlQuery part 1
* fix invalidation
* fixes
* more fixes
* I fixed it, but tested on preview instead of local 🤦🏻
* only refetch table rows and not count on row updates
* removed unneeded invalidations and prefetched new table after create
* Update the design of the sonner toasts. Add the close button by default.
* Migrate studio and www apps to use the SonnerToaster.
* Migrate all toasts from studio.
* Migrate all leftover toasts in studio.
* Add a new toast component with progress. Use it in studio.
* Migrate the design-system app.
* Refactor the consent toast to use sonner.
* Switch docs to use the new sonner toasts.
* Remove toast examples from the design-system app.
* Remove all toast-related components and old code.
* Fix the progress bar in the toast progress component. Also make the bottom components vertically centered.
* Fix the width of the toast progress.
* Use text-foreground-lighter instead of muted for ToastProgress text
* Rename ToastProgress to SonnerProgress.
* Shorten the text in sonner progress.
* Use the correct classes for the close button. Add a const var for the default toast duration. Remove the custom width class from sonner.
* Set the position for all progress toasts to bottom right. Set the duration for all toasts to the default (when reusing a toast id from loading/progress toast, the duration is set to infinity).
* Fix the playwright tests.
* Refactor imports to use ui instead of @ui.
* Change all imports of react-hot-toast with sonner. These components were merged since the last commit to this branch.
* Remove react-hot-toast lib.
---------
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
Co-authored-by: Jonathan Summers-Muir <MildTomato@users.noreply.github.com>
* Update default sort order to use all primary keys in the case of composite primary keys to prevent row resorting after updating a row (still within threshold of applying only when max 50k rows or less)
* Fix updating column fk value when there are multiple columns whose target schema, table and column are the same
* Fix updating composite foreign key from withn the grid editor itself
* Set the primary key as the default sort if no other sort exists
* Rename
* Add contextual toast messages when trying to update/delete row on table with no PKs
---------
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
* Support rendering large text/jsons in grid
* Support rendering large json in side panel, and large text in grid
* Support rendering large text in side panel
* Fix
* Update comment
* Fix the editValue type for json fields.
* Fix
* Update blur value
* Fix
* Feex
* Padding bump
* Small refactor
---------
Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com>
Co-authored-by: Terry Sutton <saltcod@gmail.com>
