The first official week of my ‘summer’ of code was a succes. I managed to import the NASA SRTM3 data into Postgres. That is, the import is running at about 1 tile per minute while I am writing this. The result is available through Subversion and Git.
I think I am getting the hang of Test Driven Development. It would be great if someone can point me to a Postgres Python test tool. For now, I created a test database and wrote my own functions to populate it with test data for each test and clean up afterwards. In a month I will attend the Continuous Integration and Testing Conference in Melbourne (an OpenSpace event).
I mostly use PyGreSQL to connect to Postgres, but for large inserts I wanted to use the efficient Postgres COPY function. Psycopg2 supports this with its copy_from(table, file) function. However, this function mysteriously freezes, so I had to work around that.
Next week I am going to figure out how my application wants to receive a route. My original plan was that my application receives an xml request, but I just read the announcement of OpenRouteService so I will have a look at how they do it.
Tonight I am flying to Melbourne so I have some time to read The Summer of Code Mystery Book.
Just making a reference to OpenStreetMap’s Wiki: http://wiki.openstreetmap.org/index.php/Contours#Importing_the_SRTM3_data.
@Stefan : thanks for the link.
The main difference between that approach and what I am doing, is that
I am not using contours: I use the original grid of data points. I am
thinking about calculating local contours on the fly, or use some form
of interpolating the data points. That way I should be able to safe
storage space and it may also be easier to combine it with altitude
data from other sources at some point in the future.
Also, my script adds the (not very spectacular) service of downloading
and verifying the zip files.
I already discovered that my approach eats way to much hard disk
space, so I am changing a few things:
* In stead of storing the coordinates as two double precision floats
(2 * 8 bytes), I will create a bigint index (1 * 8 bytes). Also I
changed the altitude from double (8 bytes) to smallint (2 bytes),
since the SRTM data was already to rounded to the nearest meter.
* In stead of the whole continent I will stick to a smaller part of
Australia for my laptop. I’ll see how far I push my home computer
later.