navigation on old maps
Introduction
I am a full stack developer at the cultural and historical IT portal Konigsland, which successfully started its work about a month ago. This resource is dedicated to culture and history East Prussia and is a kind of chronicle of times, which most of all resembles a virtual museum, where you can get quite complete information about the history of this great region, and this information is replenished as I have free time.
The pages of this chronicle lift the veil of secrecy and allow those who are fond of antiquity to benefit from modern technology.
How my GPS was stolen
The idea to implement such mechanics came to me a long time ago, since my first pet project. Then I lacked knowledge, and it took a huge amount of time to implement it. Nevertheless, to achieve the goal and do something plus or minus working, it turned out even then. At that time, as an alternative hobby to programming, I chose fishing, for this reason cartography was used to aggregate data about the reservoirs of the region. Now I have switched to searching for treasures and, accordingly, completely different locations have taken priority.
At one of the fishing competitions, my partner and I took the largest fish, and received a GPS navigator as a gift, on which I later loaded maps of the area in 1893, and used it to navigate the terrain.
And now this navigator is gone. Of course, he can be tracked by GPS, but all access to the accounts is lost, and in general I have already come to terms with the fact that after 8 years of our strong friendship with him, he was stolen from me. This brought me mental anguish amid increased stress – I even installed a mobile application with paid cards in order to once again be able to control my location in space and time. But it was not very convenient, and paying money is not the time now.
Tile server
To be honest, I did this part at the very end (the whole platform as a whole has a rather large infrastructure in which the tile server is a small microservice), because it turned out to be the least transparent to understand. Although it would seem – what can be difficult in the coordinate system – there is a point, it has two coordinates – nothing is simpler. Despite detailed documentation here hereit turned out to be a little more difficult than I thought, and now I will try to tell you why.
Initially, I tried to find open tile servers with old maps, similar to servers google And arcgis but only that they render the tiles of the old map. There are many of these maps in the public domain, but data formats can sometimes be misleading. When my attempts were unsuccessful, I came to the logical conclusion that I needed my own tile server.
Deploying my apache2 tile server was no problem, but there were problems with the cards – I could not find a way to convert the cards to the correct format. The example uses the utility osm2pgsqlwhich allows you to convert maps from the format pbf and put them in postgres. And everything worked fine with the maps from the documentation osm. But where can I find the necessary old maps (East Prussia) in this format? It turned out to be very difficult.
However, it turned out to be easy to find maps in the format of a ready-made sqlite3 database – there are a lot of these maps on the net and they are available for almost any region. The maps of the entire region I needed were divided into two databases – the central part and the eastern part. In the databases, in fact, there was one tiles plate that I needed. This plate in which there was no id, but there were indexed columns: x, y, z. The last fourth column contained my cherished tile with a piece of the old map in Blob format. If x (longitude) and y (latitude) were clear, then with z (zoom) awareness came a little later.
Let’s now see what the component instance looks like TileLayer from react-leaflet libraries:
import {TileLayer} from "react-leaflet";
<TileLayer
minZoom={2}
maxZoom={20}
attribution='© <a href="http://osm.org/copyright">OpenStreetMap</a> contributors'
url="https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png"
/>In this fragment, we see that the component uses the same parameters in the request to the tile server that we have in the database, and after processing them, we can get the tile we need and return it to the client upon request. To do this, we don’t even need apache2 – just process the http request parameters and make the correct call to the database using them. Wow, now it does not seem so complicated, because the matter is small.
In order not to waste time on configuration and preparation, we use django:
urls.py
from django.contrib import admin
from django.urls import path
from app.views import IndexView
urlpatterns = [
path('admin/', admin.site.urls),
path(
'tile/<int:x>/<int:y>/<int:z>.jpeg', IndexView.as_view(), name="endpoint"
),
] views.py
import io
from PIL import Image
from django.http import HttpResponse
from django.views import generic
from app.models import Tiles
class IndexView(generic.View):
def get(self, request, x, y):
if Tiles.objects.filter(x=x, y=y).first():
response = HttpResponse(content_type="image/jpeg")
Image.open(io.BytesIO(tile.image)).save(response, "JPEG")
else:
response = HttpResponse(200)
return response
models.py
import base64
from django.db import models
class BlobField(models.Field):
description = "Blob"
def db_type(self, connection):
return 'blob'
class Tiles(models.Model):
x = models.IntegerField(primary_key=True)
y = models.IntegerField()
z = models.IntegerField()
s = models.IntegerField()
image = BlobField(db_column="https://habr.com/ru/post/654281/image", blank=True)
def set_data(self, data):
self._data = base64.encodestring(data)
def get_data(self):
return base64.decodestring(self._data)
data = property(get_data, set_data)
class Meta:
db_table="tiles"
unique_together = (('x', 'y', 'z', 's'),)If in human language, then we take the latitude and longitude from the request, find the bytecode of the picture from them, convert it to HttpResponse from {content-type: image/jpeg} and give to our client. Here we need our favorite library for working with pictures: Pillow
In order to merge two sqlite3 databases into one, it is enough to add settings for both, and write a simple script for migrating data from one database to another:
settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': BASE_DIR / 'db.sqlite3',
},
'second_db_name': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': BASE_DIR / 'db2.sqlite3',
},
}from app.models import *
second_db_tiles = Tiles.objects.using('second_db_name').all()
i=0
for sdt in second_db_tiles:
try:
sdt.save(using='default', force_insert=True)
except Exception as e:
print(e.__class__.__name__)
i+=1
nginx.conf
location /tile/ {
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://127.0.0.1:8080/tiles/;
}
The next step is to set up the ALLOWED_HOSTS directive to restrict access to our server to our domain only. Then just run gunicorn --daemon and specify the correct url (https://yourdomain/tile/{x}/{y}/{z}.jpeg) for your brand new tile server in the client application. And as if by magic, you can see the outlines of ancient streets and houses from centuries-old documents.
Conclusion
I’m not very good at cartography, but it seems like I did something useful – I myself actively use this application, which allows me to replenish the site database with the most interesting materials that are available on the interactive map. By the way, any user who has passed authentication can add information to the map that he would like to share and contribute to the history of East Prussia.
Let someone else use this information so that the percentage of benefit to the time that I spent on all this increased.
