How we moved from batch, asynchronous processing to a real-time, full-circle process

By Dion Bailey and Louise Story

illustration of blue fibre cables with ones and zeros passing through inside

It was election night, audiences were flocking to our platforms, and — — beyond all the great journalism going on — we had a big technology breakthrough.

We were collecting close to the order of a million data points every few minutes and processing it all in real-time. It was a remarkable feat compared to what we experienced just months earlier during coronavirus traffic spikes before our internal systems were built.

Behind this remarkable feat was our new data pipeline, where we…

Should we make stuff or test stuff?

Increasingly, as we have moved into a more iterative model of product and content development, it’s the latter. To accelerate this process, we are hiring for 13 new roles in our Experimentation team (see below for details).

Isometric illustration of a/b testing two web designs on mobile

We are thinking about our projects differently — looking for opportunities to test hypotheses in front of our audience, launching in phases so we embed audience feedback into each phase of development of any new experience, and measuring performance objectively with the knowledge that many new features will fail.

At the heart of this is a…

By John Schimmel and Louise Story

Imagine if drivers on a highway needed to understand the motor of every other car around them in order to change lanes? Even worse, imagine if a break-down in one motor affected another car 1,000 feet away?

Sounds impossible, right? Yet, for a long time, that’s how web pages worked.

Fixing that metaphorical highway system is why The Wall Street Journal moved to a component-based framework across our web platforms.

By Mike Finkel and Louise Story

Screenshot of a Kanban board used by The WSJ
WSJ’s web team decided to switch from Scrum to Kanban

A lot of things have changed since we all started working from home, but one thing we have worked hard to preserve is our culture of empowering engineering teams to be leaders in decision-making.

One recent decision by WSJ’s web team reflects our collaborative approach to decision-making and has turned out to help us a lot in the new world of ‘Work From Home’.

It was the middle of the summer when we started a dialogue around the way our web product development team works. Like some of our other teams, the web team…

I like to think about rock climbing when I face cross-roads in my work. You look a few feet up and find the next route forward, focusing on that pitch with an eye of appreciation for what’s ahead and where we’ve already traveled. There’s often multiple routes forward, and I find the one that works for me and the project at hand. I am always grateful for teamwork and I like to hold the rope for others, knowing they’ll hold the rope for me.

So many people in our industry want to know what lies ahead and how to navigate their careers in a changing environment. My answer is to keep learning and growing, and your path ahead will naturally develop.

A few of my colleagues suggested I share this podcast episode about my career and the work I’ve done helping media companies change and grow. It was produced by M.B.A. students at Yale and can be listened to here:

Here’s the transcript:

Welcome to Career Conversations, a podcast from the Yale School of Management. Each episode of Career Conversations is a candid…

People working at The Wall Street Journal newsroom in New York
The Wall Street Journal newsroom in New York

We are expanding! Come join our talented and fun teams making awesome content experiences at The Wall Street Journal.

We are creating a team focused on engagement, and expanding our mobile experiences and storytelling teams.

These particular jobs are in product design and in engineering. They sit within the WSJ’s Digital Experience & Strategy unit (DXS), a cross-disciplinary department that includes editors, reporters, community specialists, developers, data scientists, product managers, designers and strategists. We all work together towards our goals of helping and engaging our audiences.

The jobs are listed here. Check them out and spread the word!

Lead iOS Engineer

Senior iOS Engineer

Senior iOS Engineer, Storytelling

Lead Software Engineer

Principal Engineer, Engagement

Senior Web Engineer, Engagement

Manager, Operations and Initiatives

UX Designer, Inclusive Product Design

Product Manager, Audience Touchpoints

Technical Product Manager

SEO Optimization Analyst (Contract)

Standing in other people’s shoes to work together is essential as we bring different ways of working together and focus on our audiences. Image: Diversity quilt by Oregon Department of Transport

Focus on the MACU.

That’s one of the most important things I tell our teams at The Wall Street Journal, and it’s a mantra that leads to positive results.

The MACU refers to: Members, Audiences, Customers and Users.

You see, media companies have decades-old divides in the organizational structure, meant to protect the news operations from commercial pressures. They often do, in a good way.

But sometimes organizational divides get in the way of conversations around common goals and ways of working.

Take the MACU, for instance. In marketing organizations, you’ve got entire disciplines that have developed subscription models around…

Talk2020 user interface

Innovative new ideas can have many twists and turns in the development process, and I want to share a great tale of one of our newest features, Talk2020.

Talk2020 is a tool that helps our audiences look up past statements by the presidential candidates and their vice presidential partners. The value proposition: Look up what they said, when they said it and analyze every word for yourself.

It has been a hit on debate nights. People watching the debate want real-time accuracy checks. This is a tool that lets them do it for themselves.

But Talk2020 actually started life as…

Louise Story

Journalism leader with a background in product, technology, investigative reporting and masthead-level editing. Cares about future of news industry.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store