PostHog details scaling failures with DuckDB, announces migration to custom Trino
1 Nov 2 · 9d ago · 1 article · 3 posts · 16 comments · 3 sources · development 1 of 1
PostHog published a detailed postmortem explaining why DuckDB failed at scale. Single-node architecture required oversized instances for large workloads; the DuckLake lakehouse extension suffered from poor write performance (3+ minute commits due to full catalog re-reads); and maintaining a fork of DuckLake prevented upstream contribution. The company is migrating to a Trino derivative with HogLake, its own open table format.
“We love DuckDB. It's probably the best single-node tool for analyzing heaps of data… The same features that are great locally gave us problems in the cloud.”
PostHogPostHog Analytics SaaS companyDuckDB Open-source OLAP database
The whole story articlespostscomments the bright band is this development · numbered dots are the others · click one to jump
Reported in the same hours no headline names this development itself — these 1 claim were published in its stretch
-
first by HN Frontpage, 9d ago
What people said 16 voices · best of 17 · verbatim
-
Sometimes a company just isn't able to manage a data warehouse; I've worked for a few.I even phone screened with this one but couldn't cover their crazy quilt of requirements; at that time they were also looking for someone to rewrite the clickhouse optimizer, and they were stuck trying to reshard their cluster, which should take an afternoon.I…
-
H
Moving our warehouse off DuckDB Link: https:// posthog.com/blog/moving-off-du ckdb Discussion: https:// news.ycombinator.com/item?id=5 0338747
-
Things like this are a part of why I push back on people telling me we can do important things later. I don’t want to swap the engine out of a machine that’s moving that fast. There’s no time to learn about how it behaves with real versus synthetic traffic and what the failure modes are when you’re doing many thousands of transactions per…
-
> Five months ago, we rebuilt our warehouse on DuckDB instead of ClickHoueThey switched from ClickHouse to DuckDB five months ago, then switched from DuckDB to their own Trino derivative?The article then ends before saying if the new setup solved their problems.It’s great that we can now make these migrations with little downtime (I assume?) and…
-
>The article then ends before saying if the new setup solved their problems.This reads to me like a recruiting post; thats why they are open if it solved their problems. I'm not a fan of the way they framed it, it seems they have made their architecture decision and now need help implementing this.I'd imagine if you are the kind of person that…
-
Listen, lad. I built this database up from nothing. When I started here, all there was was swamp. Other kings said I was daft to build a database on a swamp, but I built it all the same, just to show 'em. It sank into the swamp. So, I built a second one. That sank into the swamp. So, I built a third one. That burned down, fell over, then sank into…
-
I wonder if they have considered moving back to ClickHouse. ClickHouse Iceberg support has improved a lot over the past few months. You can easily run stateless ClickHouse on top of Iceberg. This solves the tenancy issue, and it can easily handle petabytes scale [1].[1]
-
> Five months ago, we rebuilt our warehouse [1] on DuckDB instead of ClickHouse.Why did they write this post if they had not even allowed things to settle. It sounds like someone wanted to claim victory (and promo?)
-
The key insight I am getting from this is that as migrations get easier to do with agents, there will be a lot more of them.But more importantly a lot of them will happen because we will all get more trigger happy to start one. On the other hand I also don't know if at scales like PostHog, a successful migration can be tested without doing all of…
-
Now they have their "own flavor" of Trino and a brand new open table format called "HogLake". This gives the impression of making excuses with little introspection.It would be great to see their ducklake fork, their patches and github issues so the community can learn.
-
Am I missing something here, or is the only technical detail in the post "a trillion rows is a lot of data, maybe trying trinio will work this time"?It feels like a lot of churn and not a lot of measurements or engineering.
-
Founder here -- this is about our warehouse product which is separate from our main analytics DB. Our DuckDB warehouse was used by a handful of pilot customers. Our clickhouse database is serving all of our analytical queries.
-
> The article then ends before saying if the new setup solved their problems.They got to put ClickHouse, DuckDB and Trino on their resume & LinkedIn. Problem solved!
-
It reminds of people who use ORMs, never pop the hood to analyze the queries, indexes, etc - then decide it's the database that's the problem.
-
>We rebuilt everything around our own flavor of Trino and created our own open table format, HogLake.Ah yes, making the same mistake again.
-
OK, but I think I would have tested "will it scale?" before switching to DuckDB in the first place.
All 1 developments of PostHog abandons DuckDB warehouse after five months, builds… →
MastodonHacker NewsNewswiresLobsters