Article

x

Two Point O

A static data layer for headless websites

Filip Verswijver
Composable
Cloud platform
Technology

Introduction

We have been chasing content on the edge for a while at Two Point O, and so has the wider community. We want to render pages close to the visitor, personalise them there, and do it from content that marketers change without a developer in the loop.

Edge side includes were an early attempt. The page is split into containers of content and the edge stitches them together on the way out. Edge middleware came later. A statically generated page is rewritten in transit to fit the visitor it is going to. We have built both. Both are complicated to get right, and neither has stuck. The framework side has its own answers, ISR and now PPR in Next.js.

All of these are attempts at the same two demands: content changes should land quickly, and the site should stay up whatever happens upstream. The static data layer, SDL for short, is how we meet those two demands on Cloudflare, and it is where our earlier attempts ended up.

What a headless site usually does

Most headless sites we come across do not render from the CMS at request time. The CMS API is a network hop away from wherever the page is built, and it is not fast or available enough to sit in the path of every visitor. So teams generate pages ahead of time, with static site generation or incremental regeneration, and rebuild them when content changes.

That works, and it moves the problem rather than removing it. Someone has to work out which pages a piece of content appears on, which of those to regenerate, and which caches to purge. Purging everything is too expensive, and rebuilding the whole site for one text change is what everyone was trying to get away from. The dependency graph between content and pages is code that has to be written and maintained, and it is where the occasional stale page comes from.

What SDL does instead

SDL moves the data rather than the pages. When content changes in the CMS, a PIM or any other source system, a webhook triggers an ETL flow. We extract the changed entry and whatever the page needs around it, transform it into the shape the page expects, and store it in Cloudflare Workers KV. KV replicates that data to every Cloudflare location. A Worker renders the page on each request from the local copy, read through a binding, without a network call to the source system. A cron-based full sync catches anything a webhook missed.

A publish in the source system triggers the ETL flow. KV replicates the result to every Cloudflare location. Visitor traffic never travels this path.

The read is fast because it happens in the same data centre as the render. Cloudflare's own figures from September 2024 put the hottest KV keys at under a millisecond, served from an in-memory cache, and the 90th percentile of all KV reads under 12 milliseconds.

On an SDL site every request at every location reads the same set of keys, so those keys are hot everywhere and the page never pays for a cold read to central storage or a round trip to a CMS in another region. What is left is the distance between the visitor and the nearest Cloudflare location, and no architecture removes that.

With the CMS in the request path every visitor waits on a cross-region call. With SDL the Worker reads a local copy in the same data centre.

Static in the name is meant literally. The layer holds data that changes on the scale of minutes or slower and is read from everywhere: pages, navigation, product descriptions, translations. Nothing in the request path ever falls through to the CMS, so this is a different thing from a cache in front of it. The layer holds the complete dataset the site needs, kept current by the source systems pushing changes rather than by the site pulling them.

This is not an argument against Next.js, or against Vercel, Azure or AWS as a host. The framework can stay. The render moves to Cloudflare Workers, and the data moves with it, so it is already there when the request arrives.

No invalidation logic

A content change results in one ETL run and a write to KV. There is no graph of affected pages to compute and no cache to purge, because the page reads the data at render time and the data is simply newer.

KV is eventually consistent, and that has a concrete meaning here. A Cloudflare location can keep serving the previous value for as long as the read cache TTL, which is 60 seconds by default and can be set as low as 30. A pre-rendered site with a 60-second revalidate gives you the same window. What SDL removes is the decision about what to rebuild, not the minute of delay.

It also removes the first slow request. On a pre-rendered site, every page in every region is first rendered because a visitor asked for it, whether that render happens while they wait or in the background after a stale copy is served. With SDL the data is written when content is published, so every location has it before anyone asks, and the first visitor to a page gets the same render as the thousandth. There is nothing to warm up.

The page stays up

The previous value stays in KV until the new one replaces it. If the CMS is down or a sync fails, including a transformation that throws on unexpected input, the site keeps serving the last good content. The failure mode is content that is a few minutes old, not an error page.

A CMS outage stops the sync, not the site. The last good content stays in KV and the new version lands when the sync recovers.

This changes the availability calculation. With the CMS in the request path, the site's availability is the product of Cloudflare's availability and the CMS vendor's. With SDL it is Cloudflare's availability alone, and the CMS can be unavailable for an afternoon without anyone outside the marketing team noticing. It also changes what editors can do. They can publish and restructure pages on their own, because nothing they do can take the site down.

The CMS bill stops tracking traffic

CMS API calls scale with edits, not with visitors. Each edit pulls the entry and its dependencies once. A campaign that triples traffic for a week adds zero calls to the CMS, so API overage tiers stop being a planning concern.

Illustrative shape, not measured data. Traffic during a campaign week against the CMS calls made by the sync, which happen once per edit.

The cost moves to Cloudflare usage: KV reads, Worker requests, Queues and Workflows for the sync. We would rather pay for KV reads than pay a developer to maintain invalidation code and the monitoring around it, and KV reads are cheap.

Personalisation without a cache problem

Because every request renders from local data, a personalised variant of a page costs the same to serve as the default. This removes the usual objection, which is that personalisation destroys the cache hit ratio of a pre-rendered site.

The approach is architectural. A bounded set of variants per page lives in KV. At request time the Worker picks which keys to read based on whatever signal is available: Cloudflare's request properties such as country and language, a cookie, a segment from a CDP. The variants themselves are content, authored in the CMS and synced through the same ETL as everything else. Per-user data does not belong in the layer, and we do not put it there.

Preview for editors

Preview is the usual weak spot of headless setups, and SDL handles it inside the same architecture. When an editor opens the visual editor or preview mode, the page code and the transformers are unchanged. Only the data source switches. For a highly available source such as the CMS, we read the draft data directly at request time and run it through the same transformers.

For a source that is not built for that kind of load, a PIM for instance, we keep our own cached snapshot of its data and combine it with the CMS drafts. The editor sees the page as it will be published, and the PIM never sees preview traffic.

What it is not for

Real-time data does not fit, and inventory is the standard example. It is too dynamic to be propagated globally and read from a copy. Search and faceted filtering stay with a search engine. The ETL can sync the same data to one, but SDL is a key lookup, not a query engine, and we do not intend it to replace one. Data that changes faster than it is read, or that differs per user, is not what the layer is built for.

Side by side

Where to draw the line

Put data in the static layer if it changes on the scale of minutes or slower and is read from everywhere. Keep it at the source if it changes on the scale of seconds or differs per user. Outside commerce, most of the sites we see fit entirely in the first category. Commerce sites end up with both: catalogue content in the layer, stock and pricing fetched at request time.

Book a call with us