Merge pull request #13497 from supabase/docs/analytics-getting-started-pre-requisites

docs: add IAM and service account  information
This commit is contained in:
Ziinc authored and GitHub committed 2023-04-07 16:56:36 +08:00
commit edd67aaa9d
2 files changed
+66 -4

No files matched your search

@@ -26,8 +26,57 @@ However, it's important to note that certain [differences](#differences) may ari
## Getting Started
### Pre-requisites
Logflare currently requires BigQuery usage. You will need to create a Google Cloud project with billing enabled.
The requirements are as follows after creating the project:
- Project ID
- Project number
- A service account key
#### Setting up BigQuery Service Account
To ensure that you have sufficient permissions to insert into your Google Cloud BigQuery, ensure that you have created a service account with either:
- BigQuery Admin role; or
- The following permissions:
- bigquery.datasets.create
- bigquery.datasets.get
- bigquery.datasets.getIamPolicy
- bigquery.datasets.update
- bigquery.jobs.create
- bigquery.routines.create
- bigquery.routines.update
- bigquery.tables.create
- bigquery.tables.delete
- bigquery.tables.get
- bigquery.tables.getData
- bigquery.tables.update
- bigquery.tables.updateData
You can create the service account via the web console or `gcloud`, as per the [Google Cloud documentation](https://cloud.google.com/iam/docs/keys-create-delete). In the web console, you can create the key by navigating to IAM > Service Accounts > Actions (dropdown) > Manage Keys
We recommend setting the BigQuery Admin role, as it simplifies permissions setup.
#### Downloading the Service Account key
After the service account is created, you will need to create a key for the service account. This key will sign the JWTs for API requests that the Analytics server makes with BigQuery.
### Docker Compose
Using the example [self-hosting stack based on docker-compose](https://github.com/supabase/supabase/tree/master/docker), you include the logging related services using the following command
You will first need to update the `.env.example` file with the necessary environment variables.
- `GOOGLE_PROJECT_ID`
- `GOOGLE_PROJECT_NUMBER`
You will need to place your Service Account key in your present working directory with the filename `gcloud.json`.
Thereafter, you can run the docker-compose commands and include the logging-specific compose file.
```bash
# assuming you clone the supabase/supabase repo.
cd docker
@@ -41,6 +90,18 @@ This would include two additional docker services to your compose stack: Logflar
server.
</Admonition>
#### Vector Usage
In the Docker Compose example, we utilize vector to coordinate the logging pipeline between the services. However, if you peer into the [vector configuration file](https://github.com/supabase/supabase/blob/master/docker/volumes/logs/vector.yml), you will be able to see that Vector sends logs to the Analytics ingestion endpoint.
If you need to customize the logging pipeline for your own needs, you must ensure that the payloads matches the expected event schema structure. Without the correct structure, it would cause the Studio Logs UI features to break.
### Standalone Docker Container
If desired, you can utilize the standalone docker-container. Please refer to the [docker-compose file](https://github.com/supabase/supabase/tree/master/docker/docker-compose-logging.yml) for required docker configuration.
Additional supplementary technical documentation on self-hosting and using the `supabase/logflare` image for a full Logflare experience is available at the [official Logflare documentation](https://docs.logflare.app/self-hosting/).
## Differences
API logs rely on Kong instead of the Supabase Cloud API Gateway. Logs from Kong are not enriched with platform-only data.
@@ -412,14 +412,15 @@ supabase functions deploy <function_name>
Local logs rely on the Supabase Analytics Server. This can be enabled via the CLI configuration, and requires a Google Cloud project and BigQuery access.
<Admonition type="note">
The Google Cloud project must have billing enabled.
The Google Cloud project must have billing enabled. Read more about this requirement
[here](https://supabase.com/docs/reference/self-hosting-analytics/introduction#bigquery).
</Admonition>
Requirements:
1. Google Cloud project number (must be paid)
1. Google Cloud project number
2. Google Cloud project ID
3. Google Cloud JWT, obtained through [Google Cloud IAM dashboard](https://console.cloud.google.com/iam-admin/iam)
3. Google Cloud Service Account Key, obtained through [Google Cloud IAM dashboard](https://console.cloud.google.com/iam-admin/iam), with BigQuery Admin role assigned to the service account.
Once you have these 3 items, follow these steps:
@@ -433,7 +434,7 @@ gcp_project_id = "my-project-id"
gcp_jwt_path = "supabase/gcloud.json"
```
2. Place your JWT file at the corresponding path and ensure that it is correctly named.
2. Place your service account key (a JSON file) at the corresponding path and ensure that it is correctly named.
3. Start your local stack using `supabase start`