​​DC Data Engineering Meetup: Open Geodata Workshop

​​bit.ly/dc-data-3​

  • ​​60 minute hands on workshop to extract raw spatial data from OSM, filter, transform and visualize it on the browser
​​
 
DC Data Engineering Meetup 3: OpenStreetMap ETL Pipeline
Wed, Dec 18, 2019, 6:00 PM: This event is sponsored by Silectis.In this special *workshop* edition of the DC Data Engineering meetup, Arun Ganesh of Mapbox will lead a hands on workshop building an ET
www.meetup.com
DC Data Engineering Meetup 3: OpenStreetMap ETL Pipeline • www.meetup.com
​​
​​

​​OpenStreetMap www.osm.org​

  • ​​Globally crowdsourced street map of the world
  • ​​5+ billion points
  • ​​Data license: ODbl (CC-by-sa like)
​​
​​
 
User-uploaded image: new2.gif
​​
​​
​​Edits animation: https://osmlab.github.io/show-me-the-way/
​​Change explorer: https://osmcha.mapbox.com
​​
​​

​​OpenStreetMap Data ​📦​ 

​​
​​Dumps: https://planet.openstreetmap.org/
  • ​​All objects are versioned, full history: 
​​
​​Formats: protobuf, XML
​​
​​Mirrors and smaller extracts: https://wiki.openstreetmap.org/wiki/Planet.osm#Country_and_area_extracts
  • ​​Geographic extracts
  • ​​Filtered data may not have all the OSM objects
​​
​​Formats: GeoJSON, Shapefiles
​​
​​Building an upto date OSM data mirror
  1. ​​Get latest weekly planet
  1. ​​Apply latest day, hour, minute diffs since the last weekly dump
​​https://wiki.openstreetmap.org/wiki/Planet.osm/diffs
​​
​​
​​

​​OSM data primitives

​​http://wiki.openstreetmap.org/wiki/Elements

​​Geometry:Node

​​
 
User-uploaded image: node-example1.png
​​
​​https://labs.mapbox.com/mapping/images/node-example1.png
​​

​​Geometry:Way

​​
​​Unlike traditional GIS data both lines and areas are treated as way geometries in OSM.
​​