We'll try to post most of the major changes here.
- We earlier stated that we planned to import a new planet.osm every 6th week. We have instead decided to import the first planet.osm that's released each month, so once per month instead. It has been like this for half a year now.
- We also stated that we would keep ~5 databases in hot storage. But time has shown us that the cold storage databases are quite fast and that using our cold storage hasn't been an issue. Since that storage is cheaper for us we'll only keep the latest 2-4 databases in "hot storage".
- Late April this year we added the export format OSMPOLY. It was a requested format and a solution was found. However, it was very slow to compile them in the way we did. Today we are releasing a much faster version, which should produce identical files.
- We have noticed that the site haven't been caching compiled files as it should. For now we have increased disk space for cached files, we might have to increase it even further in the near future though.
- Since the site's database are growing with at least 1 TB per year we have decided to introduce Hot storage and Cold storage as a feature. The concept is simple, we will with time move the older databases into cheaper and slower storage to be able to add more databases without removing any existing data. Databases on cold storage might be significantly slower to use, but they should still serve the data within reasonable time. The exact strategy hasn't been decided yet, but it looks like we will keep the latest ~5 database in "Hot storage".
- The plan is to import a new OpenStreetMap dump (planet.osm) every ~6th week. Next import should be planet-211004.osm, which hasn't been released yet, at the time of writing this post.
- KML exports has been temporary disabled. For unknown reasons they sometimes get stuck in the SQL server, increasing the load more and more. We will have to figure out why before we can enable them again.
- Rounding all polygon vertices to five decimals in the display layer (WGS84). Saves 10-15% memory and makes it a bit easier on the client, while it shouldn't be noticable on the rendered result.
- Map can now be right clicked to bring up a context menu. This far there are only two functions added.
- Log coordinates (DD.DDDDD DDD.DDDDDD) to the developer console in the web browser.
- Select all polygons that exists at that very position.
- Added web-ui options to choose download format. Available formats are GeoJSON, EWKT, EWKB and KML.
- Corrected suggested filenames for EWKT and EWKB.
- Added option between SRID 4326 and 3857 in the download form.
- Database 2021-04-12 has been partially available for a while, but is now also fully available. Hoping to get a few really old OSM databases imported soon.
- A quite large rewrite of things again. It's mostly backend code though, so ultimately nothing should be noticable.
- Export format EWKB has been added as well. It's not possible to choose it in the UI though, but the url can be updated to use it. The advantage is that it's lossless, other formats might have rounding issues.
- We have a known issue where the site sometimes doesn't respond. We will need to reconfigure and restart the database server to fix it. However, currently there is an SQL query running that has been running for ~2 weeks. The estimated time is 21-23 days, so we don't want to abort it right now. We are waiting.
- We are planning to import some older OSM extracts as well, hopefully one extract per year since 2006. No promises though, if there are compatibility issues we might scratch the idea. We have started to prepare some data for it, with some luck the databases will start showing in a week or two.
- The plan is to import one new database per month in the future. This does however mean that we will have to drop some of the existing databases as well. We have not yet decided how this selection will be made, but we know that we do not have the hardware resources to keep them all online.
- This is a major release. All in all the site had ~7,000 lines of code before this release. The diff vs the master branch before merging was ~3,000 lines of code.
- Database 2021-02-01 has been added.
- Database 2021-03-08 is now partially ready, which is a new feature in itself. Sadly China and India is broken in this OSM snapshot, so there are a lot of items in the root of the tree that doesn't belong there. Hopefully it's fixed in OSM until the next database import.
- Database 2021-03-15 has also been added as partially ready.
- The download queue system has been rewritten more or less from scratch, the system that was in place was never intended to be a permanent solution. The new system uses the same workers as do other background processing.
- Starting with database 2021-03-08 all polygons which have the tag boundary set will be imported, regardless if the value is administrative or not.
- A database can now be seen as Partially ready. What it means is that the import itself is completed. It can be viewed on the web and generally polygons can be downloaded. The land-only polygons is still being computed in the background though. If you try to download the land-only polygons you might get an error saying that your download job has been postponed. The needed polygons have then moved up in priority in the background processing queue, but it might still take a long time until they are ready. This can be anything from minutes until three weeks depending on the complexity (Chile requires three weeks).
- The server side download filename is now prefixed with OSMB-, also the hashes has been changed from MD5 (32 characters) to SHA1 (40 characters). An example filename: OSMB-4396c685b195158e7c76a34285e40953c727d4de.geojson.gz.
- All downloads should check the HTTP code as well. If you did not receive code 200 there was an error. In case of errors the request headers will suggest the filename OSMB-error.txt. Feel free to retry the download after five minutes.
- Now color coding some of the select boxes (settings) below the map, green represents the suggested values.
- Added a Status page.
- Simplify now uses ST_SimplifyPreserveTopology instead of ST_Simplify. However, each polygon is simplified for itself, so topology between them isn't taken into consideration (yet).
- Database 2021-01-04 finally added, took a few more days than expected.
- Future databases (after 2021-01-04) will also show polygons where boundary=historic.
- In the tree view, polygons that do not have boundary=administrative now uses a different color (brownish).
- Added a selection box to choose if the name in the tree view should prioritize field name or name:en. The tree isn't re-ordered, it's always based on the default name.
- Added link Show all tags to the map tooltip.
- Added option to show land-only-polygons directly on the map. Be aware that they are more complex, larger, and slower.
- Added database 2020-12-28.
- Database 2021-01-04 will be added in a few days as well, currently processing in the background.
- The backend now supports parameter srid which can be either 4326 (default) or 3857.
- Plenty of backend rewriting regarding:
- Import new data.
- Processing SRID 3857.
- Calculating land-only-polygons.
- A database currently requires ~2 weeks of processing before we can put it live. In the future we would like to be able to put it live before the land-only-polygons have been processed, without those an import would only require around a day.
- Storing local data in SRID 4326 instead of 3857. This makes the imports for us a lot slower, but it also increases the quality of the data.
- Added database 2020-09-07.
- Added Antarctica and territory parts of Antarctica.
- Added polygons for boundary=disputed.
- Added polygon for United States Minor Outlying Islands with a manual select (since we were in need of it ourselves).
- Updated icons in the tree.
- Polygon colors adjusted again since we now have polygons without any admin_level available.
- A tool to import new database is ~30% done. More progress will be made upon the next database import.
- Added context menu option to copy all selected IDs.
- Added the ID to the map's tooltips.
- Updated the documentation explaining what parts of OpenStreetMap we are importing.
- Fixed a bug where the oldest database sometime was used instead of latest when Latest was selected.
- Updated colors of the polygons, they should now be more distinguished from each other, and most important, admin level 10 isn't gray anymore.
- Fixed z-index of polygons, higher admin level is now always above the lower ones.
- Added a new database, OSM 2020-08-10. It seems that Thailand as a country is missing from the OSM data for some reason, it seems to be out of our control. If you need Thailand (administrative level 2), use another database with the database selector below the map.
- Administrative boundaries which didn't have neither parents or children hasn't been available in the tree view, due to a design error in our end. This release should fix that.
- Due to major changes in the data structure, the older databases will be temporary disabled for 1-2 days while we rebuild some data.
- There are now a new(er) version of OSM data available. The extract is from August the 3rd.
- There is a select box below the map where you can select which data-set you wish to use. Currently two available, May and August.
- We added a Feedback button linking to UserEcho. Questions, feature requests and bug reports are most welcome there.
- We now have both a Live and a Development environment. Which was about time. No more live editing the site.
- Site moved from dedicated hardware to a virtual machine. This is actually an improvement in the sense of performance. The dedicated hardware was used as a temporary solution under the initial development.
- Add external links to OpenStreetMap for OSM relations.
- Added oAuth to OpenStreetMap.org.