diff --git a/apps/docs/docs/ref/self-hosting-analytics/introduction.mdx b/apps/docs/docs/ref/self-hosting-analytics/introduction.mdx index 5d6efe41eda..ea4297b6c87 100644 --- a/apps/docs/docs/ref/self-hosting-analytics/introduction.mdx +++ b/apps/docs/docs/ref/self-hosting-analytics/introduction.mdx @@ -26,8 +26,57 @@ However, it's important to note that certain [differences](#differences) may ari ## Getting Started +### Pre-requisites + +Logflare currently requires BigQuery usage. You will need to create a Google Cloud project with billing enabled. + +The requirements are as follows after creating the project: + +- Project ID +- Project number +- A service account key + +#### Setting up BigQuery Service Account + +To ensure that you have sufficient permissions to insert into your Google Cloud BigQuery, ensure that you have created a service account with either: + +- BigQuery Admin role; or +- The following permissions: + - bigquery.datasets.create + - bigquery.datasets.get + - bigquery.datasets.getIamPolicy + - bigquery.datasets.update + - bigquery.jobs.create + - bigquery.routines.create + - bigquery.routines.update + - bigquery.tables.create + - bigquery.tables.delete + - bigquery.tables.get + - bigquery.tables.getData + - bigquery.tables.update + - bigquery.tables.updateData + +You can create the service account via the web console or `gcloud`, as per the [Google Cloud documentation](https://cloud.google.com/iam/docs/keys-create-delete). In the web console, you can create the key by navigating to IAM > Service Accounts > Actions (dropdown) > Manage Keys + +We recommend setting the BigQuery Admin role, as it simplifies permissions setup. + +#### Downloading the Service Account key + +After the service account is created, you will need to create a key for the service account. This key will sign the JWTs for API requests that the Analytics server makes with BigQuery. + +### Docker Compose + Using the example [self-hosting stack based on docker-compose](https://github.com/supabase/supabase/tree/master/docker), you include the logging related services using the following command +You will first need to update the `.env.example` file with the necessary environment variables. + +- `GOOGLE_PROJECT_ID` +- `GOOGLE_PROJECT_NUMBER` + +You will need to place your Service Account key in your present working directory with the filename `gcloud.json`. + +Thereafter, you can run the docker-compose commands and include the logging-specific compose file. + ```bash # assuming you clone the supabase/supabase repo. cd docker @@ -41,6 +90,18 @@ This would include two additional docker services to your compose stack: Logflar server. +#### Vector Usage + +In the Docker Compose example, we utilize vector to coordinate the logging pipeline between the services. However, if you peer into the [vector configuration file](https://github.com/supabase/supabase/blob/master/docker/volumes/logs/vector.yml), you will be able to see that Vector sends logs to the Analytics ingestion endpoint. + +If you need to customize the logging pipeline for your own needs, you must ensure that the payloads matches the expected event schema structure. Without the correct structure, it would cause the Studio Logs UI features to break. + +### Standalone Docker Container + +If desired, you can utilize the standalone docker-container. Please refer to the [docker-compose file](https://github.com/supabase/supabase/tree/master/docker/docker-compose-logging.yml) for required docker configuration. + +Additional supplementary technical documentation on self-hosting and using the `supabase/logflare` image for a full Logflare experience is available at the [official Logflare documentation](https://docs.logflare.app/self-hosting/). + ## Differences API logs rely on Kong instead of the Supabase Cloud API Gateway. Logs from Kong are not enriched with platform-only data. diff --git a/apps/docs/pages/guides/cli/local-development.mdx b/apps/docs/pages/guides/cli/local-development.mdx index f6fe7c6c637..ca38587e786 100644 --- a/apps/docs/pages/guides/cli/local-development.mdx +++ b/apps/docs/pages/guides/cli/local-development.mdx @@ -412,14 +412,15 @@ supabase functions deploy Local logs rely on the Supabase Analytics Server. This can be enabled via the CLI configuration, and requires a Google Cloud project and BigQuery access. -The Google Cloud project must have billing enabled. + The Google Cloud project must have billing enabled. Read more about this requirement + [here](https://supabase.com/docs/reference/self-hosting-analytics/introduction#bigquery). Requirements: -1. Google Cloud project number (must be paid) +1. Google Cloud project number 2. Google Cloud project ID -3. Google Cloud JWT, obtained through [Google Cloud IAM dashboard](https://console.cloud.google.com/iam-admin/iam) +3. Google Cloud Service Account Key, obtained through [Google Cloud IAM dashboard](https://console.cloud.google.com/iam-admin/iam), with BigQuery Admin role assigned to the service account. Once you have these 3 items, follow these steps: @@ -433,7 +434,7 @@ gcp_project_id = "my-project-id" gcp_jwt_path = "supabase/gcloud.json" ``` -2. Place your JWT file at the corresponding path and ensure that it is correctly named. +2. Place your service account key (a JSON file) at the corresponding path and ensure that it is correctly named. 3. Start your local stack using `supabase start`