Importing Geospatial Data into Postgres/Django
Notes on different approaches to import data including for GeoDjango
Introduction
This article discusses a few of the ways that geospatial data can be brought into Postgres. At this stage the document is an overview summary of steps which I may flesh out later. The focus is primarily focused on importing into Django as the existing notes on other avenues are well documented already.
PostGIS Shapefile Loader (shp2pgsql)
There is guidance on the PostGIS site regarding how to import data. This included some options which are command line tools (ogr2ogr) and shap2pgsql. The latter specifically imports data from a shapefile. This also has a nice GUI version that works on Windows. Details from the PostGIS site can be found here.
The shap2pgsql gui tool is by far the easiest approach if you are working on a local machine. The command line options better if you are importing into a remote server. The gui tool is bundled with PostGIS, you just need to make sure it is selected in the Windows installer.
QGIS Database Manager
Another super simple way to import data is via QGIS. In the Database -> DB Manager from the top menu, simply select import later. Note that the shapefile you are wanting to import needs to have been already loaded as a layer into QGIS. The UI is straighforward and it's possible to apply transforms at the time of import. I have used this on a remote connection to a cloud server for small shapefiles (e.g. a city boundary or 10-20 simple features) without issue although it's a little slow. This fails for more complex ( sizeable shapefiles)
It's also possible to define a model in Django and then use this approach to import the data. Works fine locally but not suitable for a remote db. For
Django you need to select the option to overwrite the existing table.
One thing to keep an eye on, if importing to an existing table, it to check the suggested output table and select from the dropdown rather than
accepting the default suggested (this does not follow Django naming convention and can end up creating a new table by mistake).
Importing into Django (GeoDjando)
GeoDjango provides some tools to help import data using the LayerMapping tool. I provide links below for the Django documentation and a nice tutorial on the
same from youtube.
This is a pretty straight-forward process. The "ogrinspect" function will output a django model defintion based on a supplied shapefile together with a
mapping dictionary. The instructions (together with the available tutorials) mean this is relateive straight-forward. The format for this command is
as follows:
python manage.py ogrinspect [options] [data_source] [model_name] [options]
An example:
python manage.py ogrinspect /xfer/buildings/buildings.shp EnschedeBuildings --srid=28992 --mapping --multi-geom --null True
An example output of the command is provided below for a shapefile/model called EnschedeBuiltBoundary
enschedebuiltboundary_mapping = {
'objectid': 'OBJECTID',
'boundaryid': 'BoundaryID',
'year': 'Year',
'shape_leng': 'SHAPE_Leng',
'shape_area': 'SHAPE_Area',
'geom': 'POLYGON',
}
class EnschedeBuiltBoundary(models.Model):
objectid = models.FloatField(null=True)
boundaryid = models.BigIntegerField(null=True)
year = models.BigIntegerField(null=True)
shape_leng = models.FloatField(null=True)
shape_area = models.FloatField(null=True)
geom = models.PolygonField(srid=26918)
This information can be used to create the Django model and then a management command that uses the LayerMapping to import the data. I shall not go into further detail here as this is documented in the links below and is pretty straight-forward. I've only done this with vector data so far and will update these notes when I try with a raster.
