Engineering a site at the speed of breaking news

Many Vercel and Next.js users deal with large swaths of data. But few wrangle data in the way The Washington Post Elections Engineering team does.

Knowing their platform must be fast and visually compelling—all while handling constant updates from thousands of federal, state, and local elections—The Post moved to Next.js and Vercel for the 2022 US midterm elections.

The unique challenges of a newsroom engineer

Tyler Fisher is a Principal Software Engineer on the Election Engineering team. He elevates and innovates how The Post uses political and election data. “We do that by building data pipelines, predictive models, and compelling visuals for readers,” Fisher says. “Essentially, we build infrastructure for all election results pages.”

A close partner of The Post’s newsroom, the team processes the constant stream of data from thousands of elections nationwide on November 8—US Election Day—while handling traffic spikes from engaged users. 

The engineers need to "match the speed of The Post's formidable newsroom," according to Anthony Pesce, Engineering Manager of the Election Engineering team. They stay constantly attuned to reporters’ needs and the stories they want to tell on election night—which are always in flux.

Before Vercel, The Post’s rendering layer was deployed on AWS using an ECS cluster. In order to fine-tune the data updates needed for such a heavy workload, the team orchestrated an intricate revalidation system leveraging SQS and EC2.

“It was complex and there were onboarding hurdles for new developers and fine tune performance. And it was expensive. Vercel took a lot of that load off of us,” says Fisher. 

Moving to Vercel for fast iteration and smooth collaboration

Above all, The Post team knows that they have a hard deadline of November 8. “There’s no pushing a release by a couple days or weeks,” says Fisher. “It’s an immovable deadline.”

Prior to the 2022 elections season, the Election Engineering team’s previous solution was already Next.js-based. They had small a window between end of primary season and November general election where they had time to go all-in on Vercel, with four goals in mind:

1. Migrate to Next.js 12, to take full advantage of Incremental Static Regeneration

2. Host development and production environments to streamline stakeholder review and ease the burden of infrastructure management

3. Rewrite their frontend in TypeScript to catch errors before they happen

4. Improve the quality of test coverage and documentation

Vercel’s collaboration tooling is integral to the team’s success as well. With stakeholders like political editors, reporters, and graphics teams all working together, they “have to review, test, and deploy rapidly, because every second matters,” says Fisher. “We love the ability to integrate our stakeholders into the process.”

Spiky traffic, streaming data

For The Post’s Election Engineering team, Vercel proved to be a perfect fit for their platform. Fast iteration via seamless collaboration is key for them, as is the platform’s unique ability to handle hefty data and aggressive traffic spikes. 

“To keep up with both our incredibly spiky traffic and the sheer amount of data streaming in, we need a solution that has the primary benefit of static sites—easy and instant scalability—with the benefit of dynamic sites—responsive to quickly changing data—without rebuilding the entire project,” says Fisher.

This resulted in a flawless election night presentation of over 1,000 individual elections spread across 52 different results pages—all hosted on Vercel. 

A huge election night win

Midterm election night went off without a hitch for Fisher’s team at The Washington Post, even with a rewrite that he risked implementing at the last moment.

“Our highest data load page was for the US House of Representatives, which has roughly 400 races on one page. That’s a lot of data. We had been using a websocket solution to update the page while the user was on it, and it was lagging like crazy. So, I rewrote the solution at the last second. It held up and worked really well!”

“Our night was a lot simpler because of Vercel and Next.js. It was a huge win for us,” Fisher says.

Pesce adds that on Vercel, “it was the smoothest election night anyone could remember.”