# Mapmaking Discussion & Philosophy (WIP/Critique) > Regional/World Mapping >  [WIP] Atlas Maps of Valmere | From Real-World Height Data to Vector Map

## morne

Hi All,

I am starting work on a new continent from scratch, so this thread is going to be a sort WIP/guide/rundown of my process, starting with a continent landmask, already projected to equidistant conic.

Here is the landmass I will be working with:



I'll need to finish up the coastline detail before I can continue with it. In the meantime, the next few posts will detail how and where I source my DEM data, and working with projections in GDAL from the command line.

Bonus: How I create the coastline in Adobe Illustrator:



(Fair warning, my coastlines are not the best. I'm still studying real-world coastlines to get better at it)

Tutorial Links

* Downloading real-world DEM data
* Managing a Fantasy World Map with GDAL

----------


## Adfor

It looks really good so far! The land mass looks natural. A lot of the jagged edges are indicative of rivers exiting to the sea, while the gulf/bays are smooth from erosion of circular water patterns.

Looking forward to seeing what you do with elevations!

IR

----------


## morne

This guide will show where to obtain high resolution DEM data and convert it using QGIS and command-line GDAL to a usable format for use in Photoshop. The data we will be downloading is best used for continental to global scale maps. The resolution is something like 40-230 meters per pixel, depending on the latitude. This data is what I use the create the heightmaps of my landmasses.

For this tutorial, we will assemble a full heightmap of Australia.

Downloading the Data from USGS

First, create an account at https://earthexplorer.usgs.gov/. It is also very useful to download and install the Bulk Downloader application https://lta.cr.usgs.gov/EEHelp/DownloadBulkApplication. At the time of writing this, USGS seems to be having issues with providing a download link to the bda application, but hopefully that will be fixed in the near future.

After logging in, return to the main page.

Begin a search by specifying the search limits. I like to navigate to where I want to search in the map view, then click the "Use Map" feature to define the search area.

Next, navigate to the Data sets tab. Select Digital Elevation -> GMTED2010.

At this point, you can skip to the results tab to initiate the search and show results. The results controls drop down is useful for showing the footprints of each data set in the map view.



Note: This gif turned out really bad, oops. This link is a little better.

Select the data sets you want, and either download them individually, or use the bulk downloader to download multiple sets at once. You'll want the 7.5-arcsecond data, as that is the highest resolution.

Merging the Data in QGIS

If you have not already done so, install QGIS: https://qgis.org/en/site/forusers/download.html

Unzip all the data sets, then open QGIS.

Go to Layer -> Add Layer -> Add Raster Layer (or use Ctrl + Shift + R)

Click the ellipses to browse and navigate to the data sets. Select and add each *__gmted_mea075.tif file.

At this point, you should have something that looks like:



(Hey look, New Zealand is in our map!)

The first problem we need to solve is that each data set tile is not properly level set to its neighbors. Fortunately there is a QGIS tool for that!

Run Raster -> Miscellaneous -> Merge. Use the ellipses to select all layers in the project, then click run. It may take a few minutes, but the end result should look like this:



OPTIONAL STEP: As an optional step, you can reproject the data before saving and exiting QGIS. The original data, I believe, is in equirectangular format.

To do this, run Raster -> Projections -> Warp (Reproject). Under the Target CRS option, select the icon to the right to choose from pre-defined CRS's (If checked, uncheck the "No CRS" option). I searched in the Filter box for "Australia" and found GDA94 / Australian Albers---works for me! (disclaimer: I am *not* a GIS/mapping expert). Select run to process the reprojection command. This will almost certainly take a long time.

To export from QGIS, right-click on the layer that you want to export (this layer will probably be called "merged" or "Reproject"), go to Export -> Save As. Enter the file name you want to save it as and click okay. I called mine "australia_merged.tif"

Converting the data using GDAL

At this point, you may be tempted to open up the tif in Photoshop and be off to the races. More likely than not, photoshop will greet you with an entirely black image if you do this. That is because the data in the tif is scaled differently than what photoshop expects.

For this last step, I use GDAL from the command line: https://gdal.org/. You can install GDAL using Conda forge: https://anaconda.org/conda-forge/gdal, or if you are a masochist like me you can build and install it from source  :Smile: 

I'm sure there are ways to do these same things in QGIS (which is just using GDAL commands behind the scenes anyways), but I find it easier to use the command line.

Open a command line prompt where you saved the tif export from QGIS.

Run the command:



```
gdalinfo -mm australia_merged.tif
```

You should see output that looks something like:



```
Driver: GTiff/GeoTIFF
Files: australia_merged.tif
Size is 43200, 19200
Coordinate System is:
GEOGCRS["WGS 84",
    DATUM["World Geodetic System 1984",
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Origin = (89.999861111000001,-10.000138889000000)
Pixel Size = (0.002083333333333,-0.002083333333333)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (  89.9998611, -10.0001389) ( 89d59'59.50"E, 10d 0' 0.50"S)
Lower Left  (  89.9998611, -50.0001389) ( 89d59'59.50"E, 50d 0' 0.50"S)
Upper Right ( 179.9998611, -10.0001389) (179d59'59.50"E, 10d 0' 0.50"S)
Lower Right ( 179.9998611, -50.0001389) (179d59'59.50"E, 50d 0' 0.50"S)
Center      ( 134.9998611, -30.0001389) (134d59'59.50"E, 30d 0' 0.50"S)
Band 1 Block=43200x1 Type=Float32, ColorInterp=Gray

    Computed Min/Max=-61.000,3159.000
```

The computed Min/Max is what we are after.

For a 16-bit grayscale image, the min/max values should be 0 (black) and 65535 (white). To change the min/max values of our tif, we can run:



```
gdal_translate -ot UInt16 -scale -61.000 3159.000 0 65535 australia_merged.tif australia_merged.tif
```

Congratulations! You now have a tif that you can open in Photoshop (along with Gimp, or Affinity Photo, I presume). You can repeat this process for other continents. I have North and South America, Eurasia, and Africa also completed using this process to give me lots of resources to pull from when creating custom heightmaps.



Let me know if any steps need further clarifying.

----------


## Michi il Disperso

> Computed Min/Max=-61.000,3159.000[/CODE]
> 
> The computed Min/Max is what we are after.
> 
> For a 16-bit grayscale image, the min/max values should be 0 (black) and 65535 (white). To change the min/max values of our tif, we can run:


Wait, what did you do there?
The original -61,3159 was altitude in meters, right?
With that passage didn't you off-setted that?
I try to explain: if you do that passage did the image show "max white" at 3159m and black at -61m?

If i'm too forward just stop me ^^

----------


## morne

> Wait, what did you do there?
> The original -61,3159 was altitude in meters, right?
> With that passage didn't you off-setted that?
> I try to explain: if you do that passage did the image show "max white" at 3159m and black at -61m?
> 
> If i'm too forward just stop me ^^


Yes I believe -61.000/3159.000 is the minimum/maximum value in meters of the original tif. Photoshop does not know how to interpret that data as meters, so we are remapping the values to be the full range of 16-bit grayscale, which is 0 to 65535. This does mean that the finalized tif will show full black as being -61 meters, and full white as being 3159m, which changes the image from what you originally see in QGIS. But when I am using this data to create heightmaps, I end up adding adjustment layers and other things so much that I don't care about retaining any sort of accurate scale, I'd rather have as much fidelity as possible.

If you wanted it to be closer to the original, instead of mapping to the full range of values, you would just map it to something less than 65535 instead. Hope this makes sense.

----------


## Michi il Disperso

Yes! Understood! Thank you!

----------


## Impesio

> Hi All,
> 
> I am starting work on a new continent from scratch, so this thread is going to be a sort WIP/guide/rundown of my process, starting with a continent landmask, already projected to equidistant conic.
> 
> Here is the landmass I will be working with:
> 
> 
> 
> I'll need to finish up the coastline detail before I can continue with it. In the meantime, the next few posts will detail how and where I source my DEM data, and working with projections in GDAL from the command line.
> ...


I did not really understand what happened in the video. But thank you for this insight! Do you plan to continue in this tutorial?

----------


## morne

> I did not really understand what happened in the video. But thank you for this insight! Do you plan to continue in this tutorial?


Yes, today I'm going to post about how I use GDAL to manage projections of my maps. I'll likely not touch on working out the coastlines in illustrator again, but I mostly just follow the technique that Artifexian uses in this tutorial: https://youtu.be/glt_aMlqFsc

----------


## Impesio

> Y but I mostly just follow the technique that Artifexian uses in this tutorial: https://youtu.be/glt_aMlqFsc


 Ahhh, It's just that in the video the coastline seems to pop out from nothing xD




> Yes, today I'm going to post about how I use GDAL to manage projections of my maps.


Fantastic, Thank You for sharing this!

----------


## morne

> Ahhh, It's just that in the video the coastline seems to pop out from nothing xD


Yeah, the GIF quality kind of destroyed it, but if you zoom in you can see the thin blue pencil line as I'm drawing haha.

----------


## Skalimoi

Thank you for the tutorial! I use a similar approach to make realistic terrain (mostly downloading SRTM data and then merging it in QGIS to use it in Photoshop), but I don't understand why do you need to use GDAL to convert the TIF to a readable file?

What I do is to simply right-click the merged layer, Export -> Save As -> and then change the Output mode from "raw data" to "Rendered image". This exports the full heightmap and it's perfectly readable by Photoshop. Maybe your method takes into account things that I haven't, so let me know if I've missed something  :Smile:

----------


## morne

> Thank you for the tutorial! I use a similar approach to make realistic terrain (mostly downloading SRTM data and then merging it in QGIS to use it in Photoshop), but I don't understand why do you need to use GDAL to convert the TIF to a readable file?
> 
> What I do is to simply right-click the merged layer, Export -> Save As -> and then change the Output mode from "raw data" to "Rendered image". This exports the full heightmap and it's perfectly readable by Photoshop. Maybe your method takes into account things that I haven't, so let me know if I've missed something


Oh wow, just tried it. That's brilliant! Much easier and better than my method of manual re-scaling, since it actually retains the correct grayscale values. I'm going to update the tutorial to use this method instead.

EDIT: Actually, I realize now a fundamental difference between the two methods. The rendered image is only 8-bit. My method ensures that the final tif is 16-bit, which is much higher fidelity (65536 values of gray vs. 256 values). This becomes especially important when looking at detail at lower elevations.

----------


## Skalimoi

> Oh wow, just tried it. That's brilliant! Much easier and better than my method of manual re-scaling, since it actually retains the correct grayscale values. I'm going to update the tutorial to use this method instead.


Glad I helped!

----------


## morne

> Glad I helped!


Sorry, see my edit. The "rendered image" output is only 8-bit, which is unfortunate.

----------


## Skalimoi

> Sorry, see my edit. The "rendered image" output is only 8-bit, which is unfortunate.


Ohhhh, I see, no worries. Maybe I should try working with 16-bit images then, I've had some problems with lower elevations that maybe get fixed by doing that.

----------


## morne

> Ohhhh, I see, no worries. Maybe I should try working with 16-bit images then, I've had some problems with lower elevations that maybe get fixed by doing that.


It will only help. The difference is _really_ hard to see with grayscale. Here's a comparison with a gradient filter on:

8-bit:


16-bit:


That stepping is definitely going to cause problems if you are trying to post-process your terrain in wilbur or similar terrain software.

----------


## Skalimoi

> It will only help. The difference is _really_ hard to see with grayscale. Here's a comparison with a gradient filter on:
> 
> 8-bit:
> 
> 
> 16-bit:
> 
> 
> That stepping is definitely going to cause problems if you are trying to post-process your terrain in wilbur or similar terrain software.


I see! Thank you. I thought it was just a resolution problem, something like the files not being high-quality enough and that affecting lower altitudes, but it seems that isn't the case.

----------


## Michi il Disperso

Wow.. that files are near 1Gb EACH!

EDIT: ... but they are damn beautiful!

----------


## Rwhyte

This is great! Thanks for documenting. And the Q&A along the way is great too

If it's ok to interject one more option...




> ...
> At this point, you may be tempted to open up the tif in Photoshop and be off to the races. More likely than not, Photoshop will greet you with an entirely black image if you do this. That is because the data in the tif is scaled differently than what photoshop expects.


Yes, as you open these Tifs from QGIS (32-bit images) in Photoshop, you are greeted by a black or white image, but, there are some options... (At least, with Photoshop CS5? and later). 
Under Image --> Mode --> You can convert from 32 to 16 bit:


And in that process, you're presented with options on how to re-map the 32-bit values to the 16-bit scale, defaulting to Local Adaption. 


There are other options as well, Exposure adjustment would be like a linear offset. Or, Histogram Equalize stretches the image contrast to the 16-bit scale.
If it's just the elevation gradient as an image that's important, this is great. This is less ideal if the original elevation gradients need to be maintained, since this can shift in non-linear ways


One more caveat. While this is convenient, Photoshop is not equipped to handle signed (negative) values. I believe these are essentially truncated to 0 in the process. So, if the tif elevations contain negative values, and those are important, re-scaling like you've done in GDAL would be best. Here's a tif example with negative, below sea level elevations, truncated to zero in Photoshop, and lost forever, below the waves. 
However, there may be many cases where negative values can safely be treated as zero, without loosing real information. But it does depend on the source data and location (eg Death Valley, bathymetric data, etc)

----------


## morne

> This is great! Thanks for documenting. And the Q&A along the way is great too
> 
> If it's ok to interject one more option...


It is always okay to interject more opinions! There is _always_ another way to do things. Doing the conversion in Photoshop is something I had not thought of, and could definitely be a more convenient way, especially if you don't want to work with the command line.  Good note about the negative values, I think with the GMTED2010 data you wouldn't be truncating anything important, save for land depressions.

----------


## morne

Managing a fantasy world map with GDAL


The benefit of using GDAL to manage a world map with GDAL is that you are not limited by resolution like you would be with G.Projector. You can make your map as detailed as possible, and GDAL should be able to handle it, as long as your computer can. 

Disclaimer: Everything I do with GDAL on the command line you can absolutely do with QGIS. I'm a software engineer by trade, though, so I'm at home on the command line.

Disclaimer 2: I am *not* a GIS expert, so if anything I say in this post sounds completely wrong, please don't hesitate to let me know  :Smile: 

GDAL Version

I use GDAL version 3.2. You need at least GDAL version 3.1 to run the commands I run in this tutorial, since the  -a_ulurll command is only available in 3.1+

Equirectangular Map

Your "source of truth" is your equirectangular map. My process starts with an landmask equirectangular world map, white=land and black=ocean.

The plan is to reproject our equirectangular map in different ways to reduce distortion for finalizing the coastlines and building the heightmap. Working with as little distortion as possible makes it easier to create the heightmap, figure out rivers, and more. Getting an entire landmass relatively distortion-free becomes much harder if your landmass is huge, and this is personally something I have not tackled yet. My main world is half the radius of earth, and probably 85+% water, so my landmasses are relatively small and easy to capture in a single projection.

For this tutorial, I'll be using a new world map:



I'll work with the lower left continent for now.

Adding Geo Referencing to the Equirectangular World Map

First, we need to add geo-referencing information to our base map. To do this, run the command:



```
gdal_edit.py -a_srs EPSG:4326 -a_ullr -180 90 180 -90 map.tif
```

EPSG:4326 is basically the "plate caree" or equirectangular projection. the numbers define the extents of the projection.

Figuring out projection parameters

The easiest way to figure out what exactly you want to project to is to use G.Projector. I will create a lower resolution version of my equirectangular map, usually 6000x3000 or so, in png format.

Loading that up into G.Projector and playing with the settings, I end up with a equidistant conic projection that looks like this:



Reprojecting to Equidistant Conic

This web page: http://geotiff.maptools.org/proj_list/ has a master list of the project codes for pretty much all projections that GDAL can do.

To reproject our base map, we want to use the gdal_warp command. Setting up our equidistant projection looks like this:



```
gdalwarp -t_srs "+proj=eqdc +lat_1=-30 +lat_2=-60 +lon_0=-70" -r mode map.tif equidistant_conic.tif
```

Depending on the resolution of your map, this will take awhile.

Once complete, you can run gdalinfo equidistant_conic.tif to see the new georeferencing information:



```
Driver: GTiff/GeoTIFF
Files: equidistant_conic.tif
Size is 8760, 6947
Coordinate System is:
PROJCRS["unknown",
    BASEGEOGCRS["WGS 84",
        DATUM["World Geodetic System 1984",
            ELLIPSOID["WGS 84",6378137,298.257223563,
                LENGTHUNIT["metre",1]]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433]],
        ID["EPSG",4326]],
    CONVERSION["Equidistant Conic",
        METHOD["Equidistant Conic"],
        PARAMETER["Latitude of natural origin",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",-70,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
        PARAMETER["Latitude of 1st standard parallel",-30,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8823]],
        PARAMETER["Latitude of 2nd standard parallel",-60,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8824]],
        PARAMETER["False easting",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8806]],
        PARAMETER["False northing",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8807]]],
    CS[Cartesian,2],
        AXIS["easting",east,
            ORDER[1],
            LENGTHUNIT["metre",1,
                ID["EPSG",9001]]],
        AXIS["northing",north,
            ORDER[2],
            LENGTHUNIT["metre",1,
                ID["EPSG",9001]]]]
Data axis to CRS axis mapping: 1,2
Origin = (-21227652.971526119858027,10001965.729313574731350)
Pixel Size = (4846.117353881320014,-4846.117353881320014)
Metadata:
  AREA_OR_POINT=Area
  TIFFTAG_DATETIME=2020:07:23 11:35:30
  TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)
  TIFFTAG_SOFTWARE=Adobe Photoshop 21.0 (Windows)
  TIFFTAG_XRESOLUTION=72
  TIFFTAG_YRESOLUTION=72
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  (-21227652.972,10001965.729) (134d21'23.00"W,169d 3'49.28"N)
Lower Left  (-21227652.972,-23664011.528) (117d51'16.66"E,120d14'19.24"N)
Upper Right (21224335.048,10001965.729) (  5d39' 0.05"W,169d 2'32.92"N)
Lower Right (21224335.048,-23664011.528) (102d 9' 3.46"E,120d12'46.74"N)
Center      (   -1658.962,-6831022.899) ( 70d 1'51.34"W, 61d35'16.99"S)
Band 1 Block=8760x1 Type=Byte, ColorInterp=Red
Band 2 Block=8760x1 Type=Byte, ColorInterp=Green
Band 3 Block=8760x1 Type=Byte, ColorInterp=Blue
```

Reprojecting back to Equirectangular

Inevitably, you will be editing this equidistant conic map in Photoshop or other photo editing/terrain editing software. Throughout this process, your tif file will lose the georeferencing information, since when you save in these programs, they don't know to retain that metadata. What I would recommend is to, when making edits, save to a new file. For instance, equidistant_conic_heightmap.tif. That way, you can go back and reference your original georeferencing data in equidistant_conic.tif.




**Note: This post is not covering heightmap creation, yet. I just pasted a bit of the australia map from the last post. The scale is not even close. I will have a feature post covering my franken-heightmap process.

Once we have a new map ready to reproject back to equirectangular, we will need to add back the georeferencing information. We need to set the type of projection and associated information, along with the corner coordinates, which defines the extents of the map. You can find this information using gdalinfo like we did in the last section.

For the a_ulurll* command, we are giving "upper left", "upper right", "lower left". We need to use this parameter instead of the standard -a_ullr command since the equidistant conic projection is not square (it's probably more complicated then that, but that's my rudimentary understanding). p

*this is the option that is only available in 3.1+, and we need it!



```
gdal_edit.py -a_srs "+proj=eqdc +lat_1=-30 +lat_2=-60 +lon_0=-70" -a_ulurll -21227652.972 10001965.729 21224335.048 10001965.729 -21227652.972 -23664011.528 equidistant_conic_heightmap.tif
```

With the georeferencing data added back in, we can reproject back to equirectangular:



```
gdalwarp -t_srs EPSG:4326 -r mode equidistant_conic_heightmap.tif equirectangular_final.tif
```




The plan is to follow a similar process for each continent, then combine each equirectangular map into a single master file. The great thing about this process is that you can do it at as high of a resolution as you want. Fair warning, though, gdalwarp operations can take _a long_ time at really high resolutions, especially with a 16-bit image.

----------


## Michi il Disperso

One question (before i even try with Gdal or qgis): from wich latitude you think distortion is too high to directly use equirectangular? Or you reproject in every case?

----------


## morne

> One question (before i even try with Gdal or qgis): from wich latitude you think distortion is too high to directly use equirectangular? Or you reproject in every case?


Not sure I'm entirely qualified to answer this, but I would say 50/60 degrees latitude is where the distortion starts to become pretty noticeable. The other thing you _could_ do is only use terrain from the same latitudes when creating your heightmap, since I'm pretty sure the GMTED2010 data is equirectangular. But using projections means you can get things on a more "even playing field" so to speak.

----------


## MrBragg

> One question (before i even try with Gdal or qgis): from wich latitude you think distortion is too high to directly use equirectangular? Or you reproject in every case?


This figure from Wikipedia (https://en.wikipedia.org/wiki/Equire...lar_projection) gives a pretty good sense of how the distortion increases toward the poles.  By the time you get to 60 degrees N/S, you have a factor of 2x stretching in the horizontal (vertical remains undistorted), so morne's answer is probably a pretty good rule of thumb for where the distortion will start to get hard to handle.

----------


## morne

Here we go....



Tutorial coming soon.

----------


## Michi il Disperso

Wow.. Just wow.. although i feel it will be hard XD

----------


## morne

Finally got around to throwing together a super simple blog: http://blog.aratheum.com/

I retroactively added the two tutorials I've written so far. I will still include a full write-up in a post here, though, for each tutorial I write.

----------


## morne

Figured something out today that pleasantly surprised me: a method of doing vector patterns in Affinity Designer. This is one of the big things I was really missing coming from Illustrator, so I was really happy to figure this out. This isn't a full tutorial or anything, but I wrote about my method here: http://blog.aratheum.com/cartography...-designer.html

----------


## Michi il Disperso

Hi! I have a question: how big is you basemap? How you manage to use it?
I currently use a Inkscape vector map 40K px x 20K px; it doesn't even try to export it as a png.
I have to resize it? How much?
It's convenient to do continent per continent?
I don't even know if gimp is able to open a map so big XD

----------


## morne

> Hi! I have a question: how big is you basemap? How you manage to use it?
> I currently use a Inkscape vector map 40K px x 20K px; it doesn't even try to export it as a png.
> I have to resize it? How much?
> It's convenient to do continent per continent?
> I don't even know if gimp is able to open a map so big XD


My master Equirectangular map is going to be 80k x 40k pixels  :Very Happy: . Photoshop can handle it, but I don’t do any actual work on the full map. All of my heightmap creation/editing happens on smaller projected maps per continent, usually maxing out at like 12k x 12k pixels. 

The only thing that’s been annoying so far is reprojecting from the master map takes a long time now, usually 8 hours or so.

----------


## Michi il Disperso

> usually 8 hours or so.


eight WTF HOURS?

lol i'm gonna find a faster method, or i'll finish my map by the time of retirement XD

----------


## morne

> eight WTF HOURS?
> 
> lol i'm gonna find a faster method, or i'll finish my map by the time of retirement XD


Sounds like a lot but I really only do it once a continent is finished, to reproject back to Equirectangular. So its really not that bad.

----------


## Michi il Disperso

Oh, youre right! The inverse process: from equirectangular to conic is faster? Im still struggling to have a big map exported

----------


## morne

Hey everyone, sorry this thread's been so quiet as of late. 

I kind of went off on a major tangent these past few days. I've been messing with and learning a lot about QGIS, and I think my goals for this project have just become more lofty... I finally figured out how to take the vectors layers of my roads, cities, topography, etc. etc. and import it into QGIS all with proper georeferencing. Here's my current road data overlayed with a basic topographic hillshade:



And the real kicker, with two clicks of a button I can reproject everything on the fly (equirectangular in this case):



This is really exciting for me because it means I can retain _all_ of my data globally, not just the heightmaps, which is what my previous plan was. I'm still probably going to create the vector data in Affinity, simply cause the drawing tools are so much easier and flexible to work with, but once the data is in my "master" QGIS project, it'll be a cinch to reproject the data however I want to create all sorts of other maps, even exporting back out to svg to work with in Affinity designer again.

The biggest help this will have currently is the little overview map in my atlas maps that shows the nation's footprint relative to the continental scale, which is in a different projection. I was reprojecting a tif image of the borders and redrawing them, but now I don't have to...

----------


## morne

Minor update:

Still working on getting the map situated in QGIS. Definitely learning a lot. Also, was unhappy with a few sections of the DEM being a little too "round," so I added some erosion with Gaea. In the meantime, here's a quicky map showing major lakes on the continent of Kador, calculated using QGIS (because why not):

----------


## morne

Another quick update here. Still messing with QGIS. I've started to experiment with GRASS tools to calculate optimal routes through terrain to give me some decent looking automated roads. I think this method works less if you are trying to get highways, railways, and the like, since this basically tries to avoid steep terrain at all costs.

Anyways, here you can see a small portion of my map.



First thing I did was generate a "travel cost" map, basically representing how "hard" it is to travel at certain points. For this, I used r.slope.aspect on my base DEM, and combined that with a rivers, lakes, and ocean mask. This travel cost map looks like:



Next, I used r.walk, passing in my DEM and travel cost map, to generate a cumulative cost map from the city Azherk to Roktoum. This ends up looking like this:



Lastly, I used r.drain, passing in my DEM and directional map from r.walk, to generate the actual road, starting at Roktoum and "draining" to the black area of the cumulative cost map. After some generous smoothing, I end up with this:



Not too shabby, if I do say so myself. For this to be a viable solution, I think I need to create an input that defines which cities are "neighbors" of other cities, and write a script that will follow this process for each unique pair of neighbors. If I had to do each pair manually, like I have been, I may as well just draw roads by hand. There's probably an opportunity to use Dijkstra's algorithm in there somewhere, too. Another issue is overlapping roads where they should converge, like in my example. Honestly, the easiest solution there is manual editing.

If anyone is actually reading this, let me know what you think!

----------


## Cookiegod

I'm very impressed by your high level of skill and your playing around. Also the map size... Wow. I thought the map I'm working on is a monster, at 17k x 12k px, and it pretty much manages to reduce my progress to a crawl when I'm not focused on small bits of it. How you manage working with 80k x 40k pixels, I can't fathom. But also I'm wondering what your goal is exactly?
Don't get me wrong, it's impressive in every way, it just seems like a massive time investment.

Are you world building for something specific, or just trying to find out what's possible?

To answer your question about the roads: They look quite realistic, except of course where they lay on top of one another. But that's something you can smooth out yourself.

----------


## morne

> I'm very impressed by your high level of skill and your playing around. Also the map size... Wow. I thought the map I'm working on is a monster, at 17k x 12k px, and it pretty much manages to reduce my progress to a crawl when I'm not focused on small bits of it. How you manage working with 80k x 40k pixels, I can't fathom. But also I'm wondering what your goal is exactly?
> Don't get me wrong, it's impressive in every way, it just seems like a massive time investment.
> 
> Are you world building for something specific, or just trying to find out what's possible?


Thanks for the kind words!

To be fair, I'm almost never working on the full size world map all at once. I'm doing each continent piecemeal. The smaller continent I've been working on is only ~1000km wide or so, so its only 6k×6k resolution. The larger continent I've started is 16k² pixels. 

I used to run a homebrew DnD campaign in this world, and I have plans to write 4 novels or so taking place in this world (I'm ~30k words into the first novel). Of course, the level of detail I've been going into with these new maps is way more than I would ever need, so i guess I am just doing it cause its fun, haha. I'm hugely inspired by vorropohaiah; following his maps have motivated me to create something of similar scale.

----------


## morne

Finally finished the next leg in the "tutorial" series:

http://blog.aratheum.com/cartography...heightmap.html

Creating Fantasy Heightmaps from Real World DEMs


At the heart of all of my maps is a grayscale heightmap that I've created for my fantasy world. This heightmap is created by cutting, pasting, blending, and modifying real world DEM data. In this post, I'll walk through how I properly scale my continental maps, set up my layers in photoshop, and how I actually blend terrain together.

I'll be starting with this continent mask:



Scaling the continent


The first thing you'll want to do is scale the continent to match the source DEMs you'll be using to create the heightmap. To do this, you'll need to know how big your continent is. There are a ton of different ways to measure it if you have an equirectangular map (http://maptoglobe.com/, gplates, QGIS, etc.). I like to use the Alps for this, but you could just as easily use any other earthly feature that is easily identifiable from your source DEMs.

I know, from looking at Google Earth, that the alps are roughly ~900 km long (in a straight line). In gplates, I'll measure a line roughly that long between two points, like so:



In Photoshop, I'll cut the Alps from my eurasia source DEM and paste it into my heightmap, lining it up with the reference points I layed out in gplates. Remember, this is a rough estimate. I'm not trying to *perfectly* scale this. Within a few hundred km's should be good enough. Change the resolution of your base heightmap until the Alps are roughly the same length as the distance you measured in gplates or similar:



For my continent, it ends up being 14551 x 14550 pixels.

Photoshop layers setup


Before actually adding any data to the heightmap, I'd like to talk about how I have my layers set up in Photoshop to help aid in building the heightmap:



The top group, labelled *Topography*, contains all of the peices that make up my heightmap. I have applied a mask to it, which is the overall land mask of the continent, slightly blurred (usually only a few pixels). This ensures that I keep my continent shape when placing peices of DEMs. The Gradient Map and Posterize are to aid in creation of the map. The gradient makes it much easier to see the changes in elevation, especially at lower values. You can download my gradients here. 

Below that is a layer called adjustments. This layer is one way I modify the underlying data. By adding either small amounts of black and white to the adjustments layer, usually done at 5-20% opacity, I can modify the resulting elevation. I use this layer a lot to lower the elevation along the coastlines by adding layers of black. 

Beneath the adjustments layer is my actual DEM data that makes up the overall heightmap. I usually apply adjustment masks to these to clip the data to however much I want.

Beginning to build the heightmap


I'll be honest, there isn't really any sort of secret to doing this. A lot of it is gut feel and trying different things to see what works. But what I'll outline below is a few of the techniques that I've started to develop while building these heightmaps. Something that would probably help is to establish rough areas of mountains, valleys, etc. before beginning to place things. If you are a tectonics aficionado, you may have already worked out exactly where your orogenies and the like are, and if so, great! Use those as a guide for where to place things. If you aren't, like me, then https://maps-for-free.com/ is your friend. By studying this map, you can get a feel for what sorts of topography looks natural, and the distribution of elevations that are common. Really, though, I don't try very hard to make things look natural beyond knowing where major plate boundaries are. Everything else is just a lot of experimentation and studying the real world to make something that looks plausible "enough".

The only thing left to do at this point is to start cutting and pasting! This is more of an art than a science, so just have at it.

Adjusting for coastlines


One of the first things you'll want to do is adjust for coastlines. Take this bit that I posted into my map:



Notice that much of the coastline is rather high in elevation, all the way up to the ocean. This obviously shouldn't happen in real life very often unless you have sheer cliffs. In order to fix this, I apply some black, usually at 15% opacity or so, to the adjustments layer, until my coastlines look a little more natural:



Merging bits together


In order to make things look seamless, you'll want to "meld" all of your DEM bits together, creating one seamless heightmap. Take, for instance, this bit of data I've just pasted in to extend a coastal mountain range:



Note that the way I have arranged the new data, it overlaps with the existing data a bit. This will be important for blending.

I've already flipped and rotated it so that it is roughly in the position that I want, but it doesn't seamlessly blend together with the data that's already there! There are two things we can do to make this look better.

Add a levels adjustment, clipped to the new layer, and adjust them so that the height values roughly match the surrounding data:





Next, add a layer mask to the new data. Using a low opacity black, very gradually mask the edges of the data to blend it in with the existing terrain:



General Guidance


Everything I do to create my heightmaps is some combination of layering, subtle masking, clipping, adjustment layers, and levelling. To build up an entire continent, it just takes doing those things over and over until you get something that looks plausible and complex.

Don't be afraid to delete parts and start over if it just isn't working. 

When copying data over, try to line up valleys and ridges of mountain ranges. Remember, though, you can also always adjust the values of certain pieces to more seamlessly blend bits together.

Overlapping is your friend. The more you overlap different pieces, the more you can use masking to gradually transition between different bits of data.

The larger of pieces you cut and paste, the less blending work you have to do. On the other hand, using smaller pieces will better allow you to conform to your coastlines and general plan for terrain. I recommend using larger bits to start with, then filling in gaps with smaller bits of data to change the overall shape of the terrain until you are satisfied.

Another use of the adjustments layer is to fill "holes" in the terrain. In general, with very few exceptions, your topography should not have any "depressions" in it, where water flows would pool (endorheic basins exist, but they are pretty rare). In general, you want your terrain to flow all the way to the coast, which means you shouldn't have random points of low elevation in the center of your continent. If you do end up with this, either try new bits of data to fill in the holes, or use adjustments to "match" the surrounding terrain. The posterize layer is good for this, as it will show depressions pretty clearly.

You can also repeat this same process for bathymetric data to create a heightmap of your oceans. This is honestly much easier, since your heightmap does not need to follow any sort of erosion "rules".

Here is my current progress so far on this continent:



Once I've finished adding everything, the last stage of the heightmap process will be filling any basins I may have missed, and adding on some terrain to parts of the coastline that might look unnatural.

Thanks for reading!

----------


## morne

Starting to experiment with city map insets (ignore the unfinished borders):



Base city was generated using http://fantasycities.watabou.ru/. Not sure how I feel about it honestly. I would likely label more of the city to fill it out, but something about it feels less put together than the regional map.

Thoughts?

----------


## Redrobes

Love the style and cleanliness of the main map. The settings that you have used and the choice of colours are excellent. The inserted map does look a bit auto-generated. Its fine but having plazas within a city not quite square is odd and the uniformity of the buildings whilst random but on the other hand of a uniform type of randomness - perhaps I should say its narrow band limited noise - it looks odd. Now generating cities is very hard for anyone and nobody has done a better job but they do look a bit out of place.

Overall tho the map is looking really good.

----------


## morne

Thanks for the great feedback!




> The inserted map does look a bit auto-generated. Its fine but having plazas within a city not quite square is odd and the uniformity of the buildings whilst random but on the other hand of a uniform type of randomness - perhaps I should say its narrow band limited noise - it looks odd. Now generating cities is very hard for anyone and nobody has done a better job but they do look a bit out of place.


This is exactly it, I was trying to pinpoint exactly what looked off about it, and it's just so obvious it was procedurally generated. The generator is fantastic for sure, but us humans are way too good at detected that sort of thing, even subconsciously. I am trying to emulate the style of city mapping used in Pergamon's world atlas. These maps don't even show detail at a city level, not even sure if it's at a block level. I think the solution is going to be going with the hand-drawn approach.

----------


## morne

Here's my second attempt at the embedded city-map:



I generated large-scale topography for the area around the city and used that to hand-draw main roads in the city, forming, roughly, city blocks, instead of individual buildings. The population of the city is intended to be ~265k people.

Looking for all kinds of feedback on this, don't be afraid to be too critical! How's the city map from a realism perspective? How does the style of the city map work with the style of the main national map? Should I just give up and leave the city maps out? Any tidbit of critic is greatly appreciated!

----------


## Robulous

Looks amazing. I really should get into something like QGIS but seems quite a steep learning curve for the non-technical  :Wink:

----------


## Skalimoi

The city looks great compared to the last one. You definitely improved it a lot, but I do think that maybe there's still some room for improvement in the shapes. They look kind of floaty? Like they don't really belong there, mostly with the smaller urban core on the right. Hopefully I'm explaining myself. Really great work overall, though!

----------


## morne

> Looks amazing. I really should get into something like QGIS but seems quite a steep learning curve for the non-technical


Thanks! QGIS really isn't all that bad---I also highly recommend it. Using actual mapmaking software is seriously a game-changer.




> The city looks great compared to the last one. You definitely improved it a lot, but I do think that maybe there's still some room for improvement in the shapes. They look kind of floaty? Like they don't really belong there, mostly with the smaller urban core on the right. Hopefully I'm explaining myself. Really great work overall, though!


Thanks for the feedback! You're definitely on point that the shapes looked "floaty". For one, I think I made my roads too "squiggly". Most roads, especially on relatively flat elevation, should be much straighter. I also didn't have enough roads/blocks outside the city walls.

Here's what will probably be my last WIP update for this:



All that's left is to import the city data into ArcGIS Pro for labelling.

----------


## Ryan Pourchot

Very nice work here

Sent from my SM-G988U using Tapatalk

----------


## CredePendrel

Hey Morne! 

Amazing work this is mind blowing. I have some GIS experience (with ESRI software, not QGIS) and wanted to ask about the workflow. After you have the custom heightmap are you just generating contours from the elevation data and then applying your colour scheme to them? I did some tests with a height map of iceland and generated coloured contour lines at different intervals (20m, 100m, 500m). Its not anywhere close to actually looking good yet, but it has potential. I think I need a different test area...Iceland has some prety steep elevation changes that make some parts look a bit funny....




 I will try importing a more topographic like colour scheme later on but I got a bit of a start on it.

----------


## morne

> After you have the custom heightmap are you just generating contours from the elevation data and then applying your colour scheme to them?


Yup, this is exactly what I'm doing. I used to create the vector contours in Photoshop using work paths, save as AI files, DXW, then import into QGIS to convert to shapefile. I would not recommend this process. Very time consuming. 

I now use the contour tools in ArcGIS Pro and the process is very quick. The only other deviation from before is that I'll rescale my heightmap data from 0-65535 to "true" elevation values using gdal, deciding on the min and max values more or less by winging it. I'll generate contours using a few different intervals (I wish the contour tools would allow staggered intervals, but alas) then merge them, cherry picking the levels that I want for my final topography. I use this same method to generate bathymetry contours as well. Then I'll apply a unique value symbology based on contour minimum elevation. I sometimes need to manually fix layer draw order, but it's all pretty straightforward. 

Btw, I think it's looking great so far!

----------


## CredePendrel

Ah ok thank you for your reply. I am glad I am on the right track here. This iceland image was created with the filled contour option in ArcPro. I still tend to use ArcMap for most of my day to day work.  

Thanks for the tip. I was wondering how you were able to get such no-standard contour intervals into your maps, but creating multiple intervals and cherry-picking only the ones you need and merging them together makes perfect sense. Now that I understand how the workflow operates I will get back to making my custom height-map first. Iceland turned out ok for a test, but it didn't have types of mountains I was after.

----------


## CredePendrel

While I still haven't completed my world's custom height maps yet I wanted to go through the whole mapping process with some data I did have available. The colour scheme is the default one from ArcPro but will make a custom ramp later on. I did very little data clean up which is obvious as you zoom and see in the one image. 

https://drive.google.com/drive/folde...H7?usp=sharing

----------


## CredePendrel

I started creating my own custom heightmap based on this tutorial and others scattered around the forums and finding it much harder than I thought. I figured it would be easy to recreate the workflow I used for the images in the previous post, which also uses real world data, but no. Some of the relief (generated with Contour-Polygons) looks ok but most of the contours themselves make so sense. 

Heightmap in Wilbur


Contour Polygons


Some real ugly contour lines at the coast

----------


## morne

> I started creating my own custom heightmap based on this tutorial and others scattered around the forums and finding it much harder than I thought. I figured it would be easy to recreate the workflow I used for the images in the previous post, which also uses real world data, but no. Some of the relief (generated with Contour-Polygons) looks ok but most of the contours themselves make so sense. 
> 
> Heightmap in Wilbur
> 
> 
> Contour Polygons
> 
> 
> Some real ugly contour lines at the coast


The main issue here is adjusting for the coastline. You should be deciding what your "sea level" is in grayscale, and making sure that you are tapering off your coasts. It looks like you are doing that a little bit, but it's a very uniform and dramatic taper. If you look at real-world coastlines, it's more varied than that. If you _do_ want certain parts of the coast to be steep/cliff-like, I've also found that doing targeted erosion in those areas can really help make it look more realistic and match the coastline a little bit better.

I'm working on a map right now where I did just that:



The above map is a mountainous region with fjords similar to Norway. When I first put DEM data in, the terrain didn't really match the fjords of the coastline. So I took it into an erosion tool and targeted erosion around the coastline to make it match up

You can create masks to erode in Wilbur. You don't have a scale in your maps, but the colors suggest you have pretty high elevations. Remember, 90% of Earth's topography is below 3000 meters (as a _rough_ estimate. Parts of high elevation should be carefully placed.

One other thing that might help is taking your time and using smaller bits of data. In your example above, it looks like you've used entire chunks of data that represent your entire island/continent (Even if you didn't do this, the following advice is still useful). I would recommend being more selective and cutting, rotating, and blending data that conforms more closely to the plan you have for your terrain/coastline. Working out a rough tectonic layout (doesn't need to be detailed by any means) can be extremely helpful for determining where to place mountainous regions, instead of placing them haphazardly.

----------


## CredePendrel

Thank you 'morne' for your response. 

I was definitely rushing too fast with the heightmap so I could get something into ArcPro and keep testing. I WAS really happy how the images in the google drive link turned out and wanted to see how easy it would be to replicate it...turns out not so easy lol. I understand your comment about the coastline and why that is contributing to the dense contours there, thank you. Using targeted erosion sounds cool. Is that a tutorial or documentation about creating masks to erode in Wilbur?

Near the end of the evening I was getting frustrated by the results of the contour-polygons and absolutely starting pasting in larger chunks of data. I was getting very sharp contours and weird square geometric patterns and was trying to find the root cause. I will go back and start with much smaller chunks of data and trying blending more.

This morning I went back to the Madagascar data I used in the Google Drive link and tried a different technique to modify the landform with some mild success. I loaded the HeightMap into ArcPro and then started adjusting the water levels bit by bit essentially flooding the landscape. This produced some interesting new shapes and made a new landmass based on Madagascar but different. This could work really well, especially after rotating the shapes a bit.




In the example map you showed above: are the rivers and lakes derived from spatial analysis?

----------


## morne

> Thank you 'morne' for your response. 
> 
> I was definitely rushing too fast with the heightmap so I could get something into ArcPro and keep testing. I WAS really happy how the images in the google drive link turned out and wanted to see how easy it would be to replicate it...turns out not so easy lol. I understand your comment about the coastline and why that is contributing to the dense contours there, thank you. Using targeted erosion sounds cool. Is that a tutorial or documentation about create masks to erode in Wilbur?
> 
> Near the end of the evening I was getting frustrated by the results of the contour-polygons and absolutely starting pasting in larger chunks of data. I was getting very sharp contours and weird square geometric patterns and was trying to find the root cause. I will go back and start with much smaller chunks of data and trying blending more.
> 
> This morning I went back to the Madagascar data I used in the Google Drive link and tried a different technique to modify the landform with some mild success. I loaded the HeightMap into ArcPro and then started adjusting the water levels bit by bit essentially flooding the landscape. This produced some interesting new shapes and made a new landmass based on Madagascar but different. This could work really well, especially after rotating the shapes a bit.
> 
> 
> ...


No tutorial, but I would create a b/w mask and, in Wilbur, do Selection -> Load Selection. Then perform erosion steps as normal. I actually have started using Genbrush (https://mavichist.itch.io/genbrush) for targeted erosion and that works pretty well. Be aware, though, that I have to manually convert the raw data to tif to get my results out, since the current application only supports exporting 8-bit.

I've tried using the Arcgis Pro workflow for deriving rivers and lakes, but it ends up being difficult because of the necessity to perform a fill step. If there are too many basins that need to be filled you end up with a lot of straight rivers that pass through the filled basins which looks bad. Currently, I use World Machine to derive rivers and lakes, which for whatever reason their "find water" algorithms are more forgiving and gives better results. I take the raster output rivers from World Machine, then use the Raster -> Polyline in Arcgis Pro (Raster -> Polygon for lakes). Then I work on cleaning up errors and combining polylines to form "full" rivers. I export that into a vector program like Affinity, and create the tapered effect using variable stroke, adjusting all the rivers manually. I expand the stroke out and import back into ArcGIS Pro as polygons. It's a frustratingly manual process but it's better than hand-drawing rivers. The only thing this method lacks is any way to get river deltas.

----------


## CredePendrel

Wow I thought I was beginning to get the hang of it but the workflow gets more complex lol. I love World Machine and have Alpine Lakes but am not very good at it. I am much better at creating terrains and textures in Gaea but it lacks some key features WM does right now, like an actual river system. I haven't tried a heightmap produced in Gaea in ArcPro yet, but sounds like something I can try tonight.

Your process to get the rivers back into Pro sounds good albeit, fully manual like you said. I found this old blog post for ArcMap about tapering streams/rivers but it should still work in Pro. 
https://www.esri.com/arcgis-blog/pro...m-to-your-map/


Some Random Gaea Maps
https://drive.google.com/file/d/1inZ...ew?usp=sharing
https://drive.google.com/file/d/14Jm...ew?usp=sharing
https://drive.google.com/file/d/1X1T...ew?usp=sharing

----------


## cyanide_baby

> No tutorial, but I would create a b/w mask and, in Wilbur, do Selection -> Load Selection. Then perform erosion steps as normal. I actually have started using Genbrush (https://mavichist.itch.io/genbrush) for targeted erosion and that works pretty well. Be aware, though, that I have to manually convert the raw data to tif to get my results out, since the current application only supports exporting 8-bit.
> 
> I've tried using the Arcgis Pro workflow for deriving rivers and lakes, but it ends up being difficult because of the necessity to perform a fill step. If there are too many basins that need to be filled you end up with a lot of straight rivers that pass through the filled basins which looks bad. Currently, I use World Machine to derive rivers and lakes, which for whatever reason their "find water" algorithms are more forgiving and gives better results. I take the raster output rivers from World Machine, then use the Raster -> Polyline in Arcgis Pro (Raster -> Polygon for lakes). Then I work on cleaning up errors and combining polylines to form "full" rivers. I export that into a vector program like Affinity, and create the tapered effect using variable stroke, adjusting all the rivers manually. I expand the stroke out and import back into ArcGIS Pro as polygons. It's a frustratingly manual process but it's better than hand-drawing rivers. The only thing this method lacks is any way to get river deltas.


Hello,

Sorry for reviving this thread, but I'm very curious about your river tapering technique using Affinity Designer.
Tapering indeed works very well when dealing with a single, no-branch, river, but it can't seem to work when dealing with a multi branch river.

Any tips?

Note: my river network is generated from my fantasy DEM. I import the un-tapered vectorized river network in Affinity Designer.

----------

