Weather data pipeline

Today we staged our weather processing pipeline for internal testing. Our pipeline converts any live weather data into maps that update in real-time. From NOAA to Forecast.io to The Weather Channel, our pipeline takes streams of weather data and turns them into highly optimized vector tiles rendered on a map. Developers control everything, from the data sources, to styling the weather data, to the design of labels on top of the data.

Working with a diverse array of datasets in a single pipeline means that we're dealing with a broad scope of resolutions, both geographically and in update frequency. Some datasets are refreshed once a day while others are updated hourly. Our on-the-fly transformations synthesize these datasets into a comprehensible data source that can be styled with Mapbox Studio Classic and queried with the Surface API. Rather than storing separate geometries for an infinite number of possible layers - temperature, predicted temperature, precipitation, snow depth, and so forth - our pipeline generates a "lattice" of triangles that exposes all datasets simultaneously. With this live streaming approach, designers and developers can create and style weather maps with our cartographic tools while the weather data updates seamlessly behind the scenes.

The map below live streams updates from NOAA's Global Forecast System (GFS) every six hours. I've composited the GFS live world temperature data together with Mapbox Streets and Mapbox Terrain sources, added a very subtle texture of wind gusts, and styled it all with Mapbox Studio Classic.

Weather data vector tiles open up a whole new world of design control. The map below uses the exact same live-updating vector tiles as the map above but is styled to accentuate the wind gust data of the incoming dataset.

Global wind gusts from NOAA GFS data. The gusty Roaring Forties wrap around the southern part of the globe at latitudes of -40° to -50°.

High winds buffet the Tierra del Fuego on the southern tip of South America.

We're launching the pipeline publicly in the fall and testing now with select partners. Once publicly released, anyone will be able to design and query custom maps with weather data that automatically update as live data steams in.

Hit me or Ian up on twitter to talk about processing and styling weather data.