lw15 d2 (#37148)
* d2 init * LW15 Analytics Buckets Blog Post (#37131) * lw-15 analytics bucket blog post * polish * og and thumb --------- Co-authored-by: Francesco Sansalvadore <f.sansalvadore@gmail.com> * add nav to mobile main stage slider * d2 data * fix nit * add video id * add author * rss --------- Co-authored-by: Oliver Rice <github@oliverrice.com>
No files matched your search
@@ -0,0 +1,154 @@
|
||||
---
|
||||
title: 'Supabase Analytics Buckets with Iceberg Support'
|
||||
description: 'Analytics buckets optimized for large-scale data analysis with Apache Iceberg support.'
|
||||
categories:
|
||||
- product
|
||||
- launch-week
|
||||
tags:
|
||||
- launch-week
|
||||
- storage
|
||||
date: '2025-07-15:10:00'
|
||||
toc_depth: 3
|
||||
author: oli_rice,fabrizio
|
||||
image: launch-week-15/day-2-analytics-buckets/og.jpg
|
||||
thumb: launch-week-15/day-2-analytics-buckets/thumb.png
|
||||
launchweek: 15
|
||||
---
|
||||
|
||||
Today we're launching **Supabase Analytics Buckets** in private alpha. These are a new kind of storage bucket optimized for analytics, with built-in support for the [Apache Iceberg](https://iceberg.apache.org/) table format.
|
||||
|
||||
Analytics buckets are integrated into Supabase Studio, power table-level views instead of raw files, and can be queried using the new **Supabase Iceberg Wrapper**, also launching in alpha.
|
||||
|
||||
<div className="video-container mb-8">
|
||||
<iframe
|
||||
className="w-full"
|
||||
src="https://www.youtube-nocookie.com/embed/BigtFoFCVBk"
|
||||
title="Supabase Analytics Buckets with Iceberg Support"
|
||||
allow="accelerometer; autoplay; clipboard-write; encrypted-media; fullscreen; gyroscope; picture-in-picture; web-share"
|
||||
allowfullscreen
|
||||
/>
|
||||
</div>
|
||||
|
||||
## Why Iceberg
|
||||
|
||||
Apache Iceberg is a high-performance, open table format for large-scale analytics on object storage. It brings the performance and features of a database to the flexibility of flat files.
|
||||
|
||||
We chose Iceberg for its bottomless data model (append-only, immutable history), built-in snapshotting and versioning (time travel), and support for schema evolution. Iceberg is also an open standard widely supported across the ecosystem. Supabase is committed to [**open standards and portability**](https://supabase.com/blog/open-data-standards-postgres-otel-iceberg), and Iceberg aligns with that goal by enabling users to move data in and out without being locked into proprietary formats.
|
||||
|
||||
## Setting up Analytics Buckets
|
||||
|
||||
Once your project has been accepted into the alpha release program, Analytics buckets can be created via Studio and the API. To create an analytics bucket, visit `Storage > New bucket` in Studio.
|
||||
|
||||
<Img
|
||||
alt="Creating a new analytics bucket"
|
||||
src="/images/blog/launch-week-15/day-2-analytics-buckets/img1.png"
|
||||
/>
|
||||
|
||||
Analytics buckets are a separate bucket type from standard Supabase Storage buckets. You can't mix file types between the two.
|
||||
|
||||
They're stored in a new system table: `storage.buckets_iceberg`. These buckets are not included in the `storage.buckets` table and objects inside them are not shown in `storage.objects`. However, the `listBuckets()` endpoint returns a merged list of standard and analytics buckets for consistency with Studio and API consumers.
|
||||
|
||||
<Img
|
||||
alt="Analytics buckets in Studio"
|
||||
src="/images/blog/launch-week-15/day-2-analytics-buckets/img4.png"
|
||||
/>
|
||||
|
||||
After creating the bucket, we're met with connection details. Copy the `WAREHOUSE`, `VAULT_TOKEN`, and `CATALOG_URI` values and and create an Iceberg namespace and table using your preferred method. The example below uses pyiceberg to create a namespace `market` with table `prices`:
|
||||
|
||||
```python
|
||||
import datetime
|
||||
import pyarrow as pa
|
||||
from pyiceberg.catalog.rest import RestCatalog
|
||||
from pyiceberg.exceptions import NamespaceAlreadyExistsError, TableAlreadyExistsError
|
||||
|
||||
# Define catalog connection details (replace variables)
|
||||
WAREHOUSE= ...
|
||||
VAULT_TOKEN = ...
|
||||
CATALOG_URI= ...
|
||||
|
||||
# Connect to Supabase Data Catalog
|
||||
catalog = RestCatalog(
|
||||
name="catalog",
|
||||
warehouse=WAREHOUSE,
|
||||
uri=CATALOG_URI,
|
||||
token=VAULT_TOKEN,
|
||||
)
|
||||
|
||||
# Schema and Table Names
|
||||
namespace_name = "market"
|
||||
table_name = "prices"
|
||||
|
||||
# Create default namespace
|
||||
catalog.create_namespace(namespace_name)
|
||||
|
||||
df = pa.table({
|
||||
"tenant_id": pa.array([], type=pa.string()),
|
||||
"store_id": pa.array([], type=pa.string()),
|
||||
"item_id": pa.array([], type=pa.string()),
|
||||
"price": pa.array([], type=pa.float64()),
|
||||
"timestamp": pa.array([], type=pa.int64()),
|
||||
})
|
||||
|
||||
# Create an Iceberg table
|
||||
table = catalog.create_table(
|
||||
(namespace_name, table_name),
|
||||
schema=df.schema,
|
||||
)
|
||||
```
|
||||
|
||||
Back in Studio, we can see the newly created our newly created Namespace with `0/1 connected tables`
|
||||
|
||||
<Img
|
||||
alt="Iceberg namespace in Studio"
|
||||
src="/images/blog/launch-week-15/day-2-analytics-buckets/img5.png"
|
||||
/>
|
||||
|
||||
Click connect and select a `Target Schema` to map the Iceberg tables into. It is reccomended to create a standalone schema for your tables. Do not use the `public` schema because that would expose your table over the project's REST API.
|
||||
|
||||
<Img
|
||||
alt="Connecting Iceberg tables to schema"
|
||||
src="/images/blog/launch-week-15/day-2-analytics-buckets/img2.png"
|
||||
/>
|
||||
|
||||
## Querying Analytics Buckets
|
||||
|
||||
Viewing an analytics bucket in Supabase Studio redirects you to the Table Editor. Instead of exposing raw Parquet files, the system shows a table explorer, powered by the [**Supabase Iceberg Wrapper**](https://fdw.dev/catalog/).
|
||||
|
||||
The wrapper exposes Iceberg tables through a SQL interface, so you can inspect and query your data using Studio, or any SQL IDE. This makes analytical data feel like a native part of your Supabase project.
|
||||
|
||||
<Img
|
||||
alt="Querying Iceberg tables in Studio"
|
||||
src="/images/blog/launch-week-15/day-2-analytics-buckets/img3.png"
|
||||
/>
|
||||
|
||||
In this case the corresponding SQL query to access the data would be
|
||||
|
||||
```sql
|
||||
select
|
||||
*
|
||||
from market_analytics.prices;
|
||||
```
|
||||
|
||||
## Writing to Analytics Buckets
|
||||
|
||||
Writing is a work in progress. We're actively building [**Supabase ETL**](https://github.com/supabase/etl), which will allow you to write directly from Postgres into Iceberg-backed buckets. We'll also add write capability to the Supabase Iceberg Wrapper as soon as write support lands in the upstream [iceberg-rust client library](https://github.com/apache/iceberg-rust). This will complete the workflow of **write → store → query**, all inside Supabase.
|
||||
|
||||
Once live, that enables bottomless Postgres storage through shifting records into Analytics Buckets, all using open formats. As a bonus, Iceberg gets us time travel for free.
|
||||
|
||||
## Alpha Launch Limits
|
||||
|
||||
Analytics Buckets are launching in private alpha with the following constraints:
|
||||
|
||||
- Two analytics buckets per project
|
||||
- Up to five namespaces per bucket
|
||||
- Ten tables per namespace
|
||||
- Pricing will be announced in a few weeks
|
||||
- You cannot store standard objects in analytics buckets
|
||||
|
||||
## Roadmap and What's Next
|
||||
|
||||
This launch marks the first step toward full analytical capabilities in Supabase. Over the next few months, we'll introduce SQL catalog support so you can explore Iceberg table metadata directly from the database. Studio will also gain deeper integration for schema inspection, column-level filtering, and time travel queries. Our goal is to make Supabase a full-featured HTAP backend, where you can write, store, and query analytical data seamlessly.
|
||||
|
||||
## Try It Out
|
||||
|
||||
[Join the waitlist here](https://forms.supabase.com/analytics-buckets) to get early access and start working with bottomless, time-travel-capable analytics data inside Supabase.
|
||||
@@ -19,7 +19,7 @@ const Hero = () => {
|
||||
<AnnouncementBadge
|
||||
url="/launch-week#main-stage"
|
||||
badge="LW15"
|
||||
announcement={announcement.launch}
|
||||
announcement={`Day 2: ${announcement.launch}`}
|
||||
className="lg:-mt-8"
|
||||
hasArrow
|
||||
/>
|
||||
|
||||
@@ -113,7 +113,7 @@ const LW15Heading = () => {
|
||||
<div
|
||||
data-animate
|
||||
data-animate-delay={300}
|
||||
className="hidden md:inline-block opacity-0 col-span-3 lg:col-span-2 text-xs overflow-hidden h-fit max-w-[400px]"
|
||||
className="opacity-0 col-span-3 lg:col-span-2 text-xs overflow-hidden h-fit max-w-[400px]"
|
||||
>
|
||||
Five days of launches to supercharge your development.
|
||||
</div>
|
||||
|
||||
@@ -1,20 +1,27 @@
|
||||
import 'swiper/css'
|
||||
import 'swiper/css/a11y'
|
||||
import 'swiper/css/navigation'
|
||||
import 'swiper/css/controller'
|
||||
import React, { useEffect, useRef, useState } from 'react'
|
||||
import Image from 'next/image'
|
||||
import Link from 'next/link'
|
||||
import { Button, cn } from 'ui'
|
||||
import { useTheme } from 'next-themes'
|
||||
import { Swiper, SwiperSlide } from 'swiper/react'
|
||||
import { Swiper, SwiperClass, SwiperRef, SwiperSlide } from 'swiper/react'
|
||||
import { Controller, Navigation, A11y } from 'swiper/modules'
|
||||
|
||||
import SectionContainer from 'components/Layouts/SectionContainer'
|
||||
import { mainDays, WeekDayProps } from './data'
|
||||
import { useWindowSize } from 'react-use'
|
||||
import { useBreakpoint } from 'common'
|
||||
import { DayLink } from './lw15.components'
|
||||
import { ChevronLeft, ChevronRight } from 'lucide-react'
|
||||
|
||||
const LW15MainStage = ({ className }: { className?: string }) => {
|
||||
const { resolvedTheme } = useTheme()
|
||||
const isDark = resolvedTheme?.includes('dark')
|
||||
const swiperRef = useRef<SwiperRef>(null)
|
||||
const [controlledSwiper, setControlledSwiper] = useState<SwiperClass | null>(null)
|
||||
const days = mainDays(isDark!)
|
||||
|
||||
return (
|
||||
@@ -26,14 +33,36 @@ const LW15MainStage = ({ className }: { className?: string }) => {
|
||||
)}
|
||||
id="main-stage"
|
||||
>
|
||||
<h3 className="text-2xl lg:text-3xl">Main Stage</h3>
|
||||
<div className="flex justify-between items-center">
|
||||
<h3 className="text-2xl lg:text-3xl">Main Stage</h3>
|
||||
<div className="flex xl:hidden items-center gap-2 text-foreground-muted">
|
||||
<button
|
||||
onClick={() => controlledSwiper?.slidePrev()}
|
||||
className="p-2 rounded-full hover:text-foreground border hover:border-foreground transition-colors"
|
||||
>
|
||||
<ChevronLeft className="w-4 h-4 -translate-x-px text-current" />
|
||||
</button>
|
||||
<button
|
||||
onClick={() => controlledSwiper?.slideNext()}
|
||||
className="p-2 rounded-full hover:text-foreground border hover:border-foreground transition-colors"
|
||||
>
|
||||
<ChevronRight className="w-4 h-4 translate-x-px text-current" />
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<div className="hidden xl:flex flex-nowrap justify-between gap-2">
|
||||
{days.map((day) => (
|
||||
<DayCard day={day} key={day.dd} />
|
||||
))}
|
||||
</div>
|
||||
</SectionContainer>
|
||||
<CardsSlider slides={days} className="xl:hidden" />
|
||||
<CardsSlider
|
||||
slides={days}
|
||||
className="xl:hidden"
|
||||
swiperRef={swiperRef}
|
||||
setControlledSwiper={setControlledSwiper}
|
||||
controlledSwiper={controlledSwiper}
|
||||
/>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -49,7 +78,7 @@ const DayCard = ({ day }: { day: WeekDayProps }) =>
|
||||
>
|
||||
<div className="w-full h-full relative z-10 flex flex-col justify-between gap-4">
|
||||
<div></div>
|
||||
<div className="flex flex-col gap-2 p-4">
|
||||
<div className="flex flex-col gap-2 p-4 pt-0">
|
||||
<span className="text-xl text-foreground-lighter">{day.date}</span>
|
||||
<span className="text-base leading-snug text-foreground-muted">
|
||||
[ Access locked ]
|
||||
@@ -82,7 +111,7 @@ const DayCardShipped = ({ day }: { day: WeekDayProps }) => {
|
||||
>
|
||||
<CardBG day={day} />
|
||||
<div className="w-full h-full relative z-10 flex flex-col justify-between gap-4 overflow-hidden">
|
||||
<ul className="flex flex-col gap-1 p-4">
|
||||
<ul className="flex flex-col gap-1 p-4 pb-0 lg:opacity-0 lg:blur-lg duration-300 group-hover/main:lg:blur-none transition-all group-hover/main:lg:opacity-100">
|
||||
{day.links?.map((link) => (
|
||||
<li key={link.href}>
|
||||
<DayLink
|
||||
@@ -93,7 +122,7 @@ const DayCardShipped = ({ day }: { day: WeekDayProps }) => {
|
||||
))}
|
||||
</ul>
|
||||
<div
|
||||
className="flex flex-col p-4 gap-2 relative group-hover/main:!bottom-0 !ease-[.25,.25,0,1] duration-300"
|
||||
className="flex flex-col p-4 pt-0 gap-2 relative group-hover/main:!bottom-0 !ease-[.25,.25,0,1] duration-300"
|
||||
style={{
|
||||
bottom: isTablet ? 0 : -hiddenHeight + 'px',
|
||||
}}
|
||||
@@ -152,16 +181,31 @@ const CardBG = ({ day }: { day: WeekDayProps }) => (
|
||||
interface Props {
|
||||
className?: string
|
||||
slides: WeekDayProps[]
|
||||
swiperRef: React.RefObject<SwiperRef>
|
||||
setControlledSwiper: (swiper: SwiperClass) => void
|
||||
controlledSwiper: SwiperClass | null
|
||||
}
|
||||
|
||||
const CardsSlider: React.FC<Props> = ({ slides, className }) => (
|
||||
const CardsSlider: React.FC<Props> = ({
|
||||
slides,
|
||||
className,
|
||||
swiperRef,
|
||||
setControlledSwiper,
|
||||
controlledSwiper,
|
||||
}) => (
|
||||
<div className={cn('relative lg:container mx-auto px-6 lg:px-16', className)}>
|
||||
<Swiper
|
||||
initialSlide={0}
|
||||
ref={swiperRef}
|
||||
onSwiper={setControlledSwiper}
|
||||
modules={[Controller, Navigation, A11y]}
|
||||
initialSlide={1}
|
||||
spaceBetween={8}
|
||||
slidesPerView={1.5}
|
||||
breakpoints={{
|
||||
540: {
|
||||
520: {
|
||||
slidesPerView: 1.9,
|
||||
},
|
||||
640: {
|
||||
slidesPerView: 2.5,
|
||||
},
|
||||
720: {
|
||||
@@ -174,6 +218,7 @@ const CardsSlider: React.FC<Props> = ({ slides, className }) => (
|
||||
speed={400}
|
||||
watchOverflow
|
||||
threshold={2}
|
||||
controller={{ control: controlledSwiper }}
|
||||
updateOnWindowResize
|
||||
allowTouchMove
|
||||
className="!w-full !overflow-visible"
|
||||
|
||||
@@ -62,7 +62,7 @@ const days: (isDark?: boolean) => WeekDayProps[] = (isDark = true) => [
|
||||
d: 1,
|
||||
dd: 'Mon',
|
||||
shipped: true,
|
||||
isToday: true,
|
||||
isToday: false,
|
||||
hasCountdown: false,
|
||||
blog: '/blog/jwt-signing-keys',
|
||||
date: 'Monday',
|
||||
@@ -92,22 +92,18 @@ const days: (isDark?: boolean) => WeekDayProps[] = (isDark = true) => [
|
||||
id: 'day-2',
|
||||
d: 2,
|
||||
dd: 'Tue',
|
||||
shipped: false,
|
||||
isToday: false,
|
||||
shipped: true,
|
||||
isToday: true,
|
||||
hasCountdown: false,
|
||||
blog: '/blog/',
|
||||
blog: '/blog/analytics-buckets',
|
||||
date: 'Tuesday',
|
||||
published_at: '2025-04-01T07:00:00.000-07:00',
|
||||
title: 'Lorem ipsum dolor sit amet',
|
||||
description: <></>,
|
||||
title: 'Introducing Supabase Analytics Buckets with Iceberg Support',
|
||||
description: '',
|
||||
links: [
|
||||
{
|
||||
type: 'video',
|
||||
href: '',
|
||||
},
|
||||
{
|
||||
type: 'xSpace',
|
||||
href: 'https://twitter.com/i/spaces/',
|
||||
href: 'BigtFoFCVBk',
|
||||
},
|
||||
],
|
||||
steps: [
|
||||
@@ -133,17 +129,13 @@ const days: (isDark?: boolean) => WeekDayProps[] = (isDark = true) => [
|
||||
blog: '/blog/',
|
||||
date: 'Wednesday',
|
||||
published_at: '2025-04-02T07:00:00.000-07:00',
|
||||
title: 'Lorem ipsum dolor sit amet',
|
||||
description: <></>,
|
||||
title: '',
|
||||
description: '',
|
||||
links: [
|
||||
{
|
||||
type: 'video',
|
||||
href: '',
|
||||
},
|
||||
{
|
||||
type: 'xSpace',
|
||||
href: 'https://twitter.com/i/spaces/',
|
||||
},
|
||||
],
|
||||
steps: [
|
||||
{
|
||||
@@ -168,17 +160,13 @@ const days: (isDark?: boolean) => WeekDayProps[] = (isDark = true) => [
|
||||
blog: '/blog/',
|
||||
date: 'Thursday',
|
||||
published_at: '2025-04-03T07:00:00.000-07:00',
|
||||
title: 'Lorem ipsum dolor sit amet',
|
||||
description: <></>,
|
||||
title: '',
|
||||
description: '',
|
||||
links: [
|
||||
{
|
||||
type: 'video',
|
||||
href: '',
|
||||
},
|
||||
{
|
||||
type: 'xSpace',
|
||||
href: 'https://twitter.com/i/spaces/',
|
||||
},
|
||||
],
|
||||
steps: [
|
||||
{
|
||||
@@ -203,17 +191,17 @@ const days: (isDark?: boolean) => WeekDayProps[] = (isDark = true) => [
|
||||
blog: '/blog/',
|
||||
date: 'Friday',
|
||||
published_at: '2025-04-04T07:00:00.000-07:00',
|
||||
title: 'Lorem ipsum dolor sit amet',
|
||||
title: '',
|
||||
description: '',
|
||||
links: [
|
||||
{
|
||||
type: 'video',
|
||||
href: '',
|
||||
},
|
||||
{
|
||||
type: 'xSpace',
|
||||
href: 'https://twitter.com/i/spaces/',
|
||||
},
|
||||
// {
|
||||
// type: 'xSpace',
|
||||
// href: 'https://twitter.com/i/spaces/',
|
||||
// },
|
||||
],
|
||||
steps: [
|
||||
{
|
||||
|
||||
|
After Width: | Height: | Size: 189 KiB |
|
After Width: | Height: | Size: 101 KiB |
|
After Width: | Height: | Size: 111 KiB |
|
After Width: | Height: | Size: 144 KiB |
|
After Width: | Height: | Size: 148 KiB |
|
After Width: | Height: | Size: 183 KiB |
|
After Width: | Height: | Size: 91 KiB |
@@ -8,6 +8,13 @@
|
||||
<lastBuildDate>Tue, 15 Jul 2025 00:00:00 -0700</lastBuildDate>
|
||||
<atom:link href="https://supabase.com/rss.xml" rel="self" type="application/rss+xml"/>
|
||||
<item>
|
||||
<guid>https://supabase.com/blog/analytics-buckets</guid>
|
||||
<title>Supabase Analytics Buckets with Iceberg Support</title>
|
||||
<link>https://supabase.com/blog/analytics-buckets</link>
|
||||
<description>Analytics buckets optimized for large-scale data analysis with Apache Iceberg support.</description>
|
||||
<pubDate>Tue, 15 Jul 2025 00:00:00 -0700</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<guid>https://supabase.com/blog/figma-make-support-for-supabase</guid>
|
||||
<title>Create a Supabase backend using Figma Make</title>
|
||||
<link>https://supabase.com/blog/figma-make-support-for-supabase</link>
|
||||
|
||||
@@ -6,7 +6,7 @@ export const announcement = announcementJSON
|
||||
|
||||
export const AnnouncementBanner = () => {
|
||||
return (
|
||||
<Announcement show={true} announcementKey="announcement_sos25">
|
||||
<Announcement show={true} announcementKey="announcement_lw15_d2">
|
||||
<LW15Banner />
|
||||
</Announcement>
|
||||
)
|
||||
|
||||
@@ -55,7 +55,7 @@ export function LW15Banner() {
|
||||
<p className="flex gap-1.5 items-center font-mono uppercase tracking-widest text-sm">
|
||||
{announcement.text}
|
||||
</p>
|
||||
<p className="text-sm">{announcement.launch}</p>
|
||||
<p className="text-sm hidden sm:block">{announcement.launch}</p>
|
||||
<Button size="tiny" type="default" className="px-2 !leading-none text-xs" asChild>
|
||||
<Link href={announcement.link}>{announcement.cta}</Link>
|
||||
</Button>
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"text": "LW15: Day 1",
|
||||
"launch": "JWT Signing Keys",
|
||||
"launchDate": "2025-07-14T08:00:00.000-07:00",
|
||||
"text": "LW15: Day 2",
|
||||
"launch": "Supabase Analytics Buckets",
|
||||
"launchDate": "2025-07-15T08:00:00.000-07:00",
|
||||
"link": "/launch-week#main-stage",
|
||||
"cta": "Learn more"
|
||||
}
|
||||