I recently finished off a tutorial covering steps required to get real world data into Arma 3 (RV4 Engine) using only open source tools:
https://github.com/rossoe/Arma3_QGIS
Whilst working out how to manage heightmap and satellite imagery within QGIS in preparation for loading into RV4 Engine, I discovered some interesting things along the way.
The first major issue reared it’s head following a GDAL update affecting the gdalwarp command, which broke a step I had been relying on – ‘Clip to raster by mask layer‘ I asked the question on gdal’s github page, and it was confirmed that I had to switch to use full extents, instead of the easier shapefile mask layer. See below original pipeline which mainly relied upon QGIS GUI config.
This lead me to invest more time learning the construct of the GDAL commands within QGIS’s python console. There is of course a lot more flexibility within command line than using the GUI.
Summary of the key discoveries:
1. Added -wo SOURCE_EXTRA=1000
This removed corruption of data during CRS reprojection step.
NASA’s Robert Simmon had some very useful tips re constructing GDAL commands, and noted the benefit of using SOURCE_EXTRA, he also mentioned compression which feeds into my second point
https://medium.com/planet-stories/a-gentle-introduction-to-gdal-part-2-map-projections-gdalwarp-e05173bd710a
“-r lanczos defines the resampling method, with a few more options than gdal_translate. Lanczos is slow but high quality. -wo SOURCE_EXTRA=1000 is an example of a warp option—advanced parameters that determine how the reprojection is calculated. SOURCE_EXTRA adds a buffer of pixels around the map as it is reprojected, which helps prevent gaps in the output. Not all reprojections require it, but it doesn’t hurt to add the option to be on the safe side.”
2. Added compression using -r cubic
An alternative -r lanczos was noted in article above, but it resulted in degradation of raster.
3. Used -te instead of -crop_to_cutline
Although this required acquisition of precise extents rather than just selecting a shapefile to cut by, it avoided the incorrect resolution output due to the GDAL update for gdalwarp command.
I initially managed to reduce 3 tasks – reproject CRS, set cell size & cut to extents into 2 commands (see below), but I could not escape slight degradation with the raster output. I had hoped that by adding -r cubic to both of the following commands, it would resolve the issue, but same result.
Two gdalwarp commands to reproject CRS & then set cell size + cut to extents:
import os
os.system(r'''gdalwarp -t_srs EPSG:32631 -wo SOURCE_EXTRA=1000 -tr 5.0 5.0 -r cubic -of GTiff "E:/Heightmaps Opentopo/output_srtm.asc" D:/QGIS/crm4.tif''')
import os
os.system(r'''gdalwarp -tr 5.0 5.0 -r cubic -te 174425.817977 5600556.828797 215386.817977 5641517.828797 D:/Arma/QGIS/crm3.tif D:/QGIS/cut12.tif''')It was only when I compressed the above into a single command that I got a perfect output raster, I suspect this was due to too much processing on the raster.
Single gdalwarp command to reproject CRS, Set cell size & Cut to extents:
import os
os.system(r'''gdalwarp -t_srs EPSG:32631 -wo SOURCE_EXTRA=1000 -tr 5.0 5.0 -r cubic -ot Float32 -of GTiff -te 173421.201222 5600629.600063 214381.201222 5641589.600063 "E:/Heightmaps Opentopo/output_srtm.asc" E:/QGIS/converted9.tif''')