How does OpenStreetMap save its data

File formats

Reviewed on 2016-09-05

As with other data, there are several ways in which geographic data can be stored on a computer. They can be stored in databases, specialized systems for storing and retrieving data, and of course there are databases specially designed for storing geographic data. They can also be stored in traditional computer files, and there are many different file formats for geographic data.

In this chapter we will look at some ways of storing geographic data, how they work, and how they are typically used.

.OSM files

The .osm File format is from OpenStreetMap. You won't find it anywhere else. If you have ever downloaded data and saved it to a file using JOSM, you may have noticed that the file with the extension .osm was saved. If you are a GIS user, you may have noticed that these files cannot be easily opened with software like QGIS.

So why do you save OSM data in a file format that no one else is using? The answer is that many geographic file formats emerged before the Internet and were designed for quick access and querying, such as database queries. OSM data, on the other hand, was developed in a standardized form for easy exchange over the Internet. Hence are .osm Files written in XML and contain geographic data in a structured, ordered format. A simple one .osm The file looks like this in the text editor:

Data in .osm Getting format is easy - you do it every time you download data into JOSM, but using those files for analysis or mapping is not easy. It is better to convert the data into another format or have the data converted by a service.

Raw OSM data is usually in .osm Files saved, but you can also see files with .bz2 and .pbf Endings. These are essentially compressed .osm Files to save space, which is very useful when working with large files.

Shapefiles

The Shapefile is a widely used format for storing geographic vector data. It was developed by ESRI, a company that develops ArcGIS, a well-known collection of GIS applications.

Shapefiles are actually a collection of different files. A shapefile with building files could e.g. have files with the following extensions:

  • buildings.shp
  • buildings.shx
  • buildings.dbf

Shapefiles often have additional files that contain other information.

A shapefile can only contain one type of map element (points, lines or polygons) and each map element has its attributes in a table. In contrast to OpenStreetMap, where each object can have an unlimited number of tags, map elements attributes in a shapefile must correspond to the table structure defined in the shapefile, which can look like this:

OpenStreetmap data can be converted into shapefiles. Numerous websites offer converted OSM data as shapefiles. These are discussed in the next chapter.

Databases

Many types of information are stored in database systems, which provide a logical way of organizing and accessing data. Geographical data is no different, although special spatial data databases are necessary to deal with complex functions for querying geographical data.

OpenStreetMap data is often stored in PostgreSQl databases with PostGIS extension. This type of database provides fast data access and can easily be used in conjunction with Mapnik, software that creates the map tiles in Web Slippy Maps. There are numerous tools for importing raw OSM data into a PostgreSQL database.

Another database is SQLite, which offers similar functions to a PostgreSQL database, but everything is stored in a single file and does not require running database software. They are a little harder to create yourself, but you can work with them more easily when you have little data.

Summary

In the following chapters we will see how to download data in different formats from the Internet and how to use numerous tools to manipulate raw OSM data yourself.