How To Create Garmin Topo Maps - Part 8 - Combining Data

Author: Dan Blomberg | Last updated January 18th, 2016 at 02:11pm

Introduction

The 8th part of this tutorial details how we will be combining all the data we downloaded and processed into 100k quad sized segment .mp files which we will then compile into .img files in the next, and final, lesson.

Extra Data

Before combining the data I would like to make you aware of the fact that additional data exists on the internet.  This includes National Forest trails and waypoints.  If you have any trails they should be have MP_TYPE = 0x16 and the name as NAME.  A discussion of how to get trail and FS data is occurring in our forum's, check it out here.

Data Check

In this step we will make sure that you have all the required data. If something is missing go back to the step where the file was created.

Data Global Mapper File QGIS File
Contour Data Option 1: N/A (see QGIS)
Option 2: A series of gmp files		 Option 1: contoursfinal.shp
Option 2: N/A
Water Data nhdareawaterbodyfinal.shp
nhdflowlinefinal.shp		 nhdareafinal.shp
nhdflowlinefinal.shp
nhdwaterbodyfinal.shp
Transportation mergededges.shp mergededges.shp
Points of Interest gnis.shp gnis.shp
Federal Lands reservations.shp
reserves.shp		 reservations.shp
reserves.shp
Borders StateBorder.shp
CountyBorders.shp		 StateBorder.shp
CountyBorders.shp
Other Data Named as you created Named as you created

 

Combining The Data

We will combine the data into sets of files and then export them into approximately 100k grids.  The process is very different between Global Mapper and QGIS. Global Mapper is the easiest, QGIS is more in-depth but automated. We've removed GPSMapEdit from the list of options since it is more intensive. Read on for Global Mapper or click here for QGIS.

Global Mapper

We will be creating a new map catalog with Global Mapper with all of our files. Global Mapper can export directly to Polish Text (MP files) so this process is relatively straight forward.

  1. Click File>Create New Map Catalog
  2. Save the file in your main topo folder, name it map_catalog
  3. Click "Add Files..." and open the "borders" folder and add both StateBorder.shp and CountyBorders.shp
  4. Click "Add Files..." and open the "dem" folder
    1. If you did contour option 1 then: add the contoursfinal.shp file
    2. If you did contour option 2 then: click the top dem file then hold shift and select the bottom file.  This will select all the dem files created previously.
  5. Click "Add Files..." and open the federal_land folder and add reservations.shp and reserves.shp
  6. Click "Add Files..." and open the gnis folder and add the gnis.shp file.
  7. Click "Add Files..." and open the nhd folder.  Add nhdareawaterbodyfinal.shp and nhdflowlinefinal.shp
  8. Click "Add Files..." and open the random_trails folder and add any random trail data you created.
  9. Click "Add Files..." and open the transportation folder.  Add mergededges.shp.
  10. Click File>Export Vector Data>Export Polish MP (cgpsmapper) File...
  11. Name the map (mine is Missippi Topo).
  12. I use a header file that includes a dictionary for zoom level.  If you would like to use it download it here and then extract it to your map folder and set it as the Template File as shown here:
  13. Click the gridding tab.
  14. Set it to split by "Specify Individual Grid Cell Width and Height"
  15. Grid width: 1
  16. Grid height: 0.5
  17. Leave everything else alone unless you are continuing a stopped set, in that case check the "Skip Existing Files" box.
  18. Click Ok.
  19. Save the files to your "complete" folder.  I called mine MSTopo.  This means each map will be called MSTopo_XX.mp where XX are a letter and number for the grid.
  20. It will begin exporting the files.  This may take several hours.

You will now have a series of maps for each 100k quad grid. Click here to head to the bottom to move to compiling the map.

QGIS & GPSFDshp2mp

QGIS does not have the ability to export directly to Polish Text (.mp) files so first we combine as many shapefiles together as possible. Then we split them into a grid so they're not too big for the GPS or GPSFDshp2mp to handle. Finally,we process them through GPSFDshp2mp in order to have the final .mp files.

  1. Open QGIS
  2. Add all of the data we processed into QGIS by going to Layer > Add Layer > Add Vector Layer... and adding: contoursfinal.shp, nhdareafinal.shp, nhdflowlinefinal.shp, nhdwaterbodyfinal.shp, mergededges.shp, gnis.shp, reservations.shp, reserves.shp, StateBorder.shp, CountyBorders.shp, and any other random data you acquired. Your layers panel should look similar to this (names may be slightly different since processes changed over time):
  3. Since QGIS doesn't have a built in export everything on screen as a merged shapefile we have to do some extra steps. First we will reduce the data in all the shapefiles by removing all the fields except the NAME and MP_TYPE since that is the only information the MP files need (also because the merge can fail if two datasets have the same name as a different type). Since Table Manager backs up the database we can easily recover it if required.
  4. For each layer in the layer panel click the layer and then click Table Manager
  5. Select all the rows that aren't NAME or MP_TYPE.
  6. Click Delete
  7. Click Save
  8. Click Yes
  9. Repeat steps 4 - 8 for all the layers.
  10. Now we will merge all the Lines together. Select MMQGIS > Combine > Merge Layers
  11. In the Merge Layers box select only the lines. You'll know if its a line because in the layers panel the symbol will be a line rather than a circle or a square.
  12. For Output Shapefile make a new folder called QGIS in your main map folder; go in there and call the file Lines_multi.shp
  13. Click OK (in the lower left of the main QGIS window you can see the progress as it goes; however, the status froze on me and it was still working so just give it time)
  14. Now we will merge all the Polygons together. Select MMQGIS > Combine > Merge Layers
  15. In the Merge Layers box select only the polygons. You'll know if its a polygon because in the layers panel the symbol will be a square rather than a circle or a line.
  16. For Output Shapefile call the file Polygons_multi.shp and put in in your QGIS folder from step 12.
  17. Click OK
  18. Now we will merge all the Points together. Select MMQGIS > Combine > Merge Layers
  19. In the Merge Layers box select only the points. You'll know if its a point because in the layers panel the symbol will be a circle rather than a square or a line. (It is possible GNIS will be the only points)
  20. For Output Shapefile call the file Points.shp and put in in your QGIS folder from step 12.
  21. Click OK
  22. Now we will convert the lines shapefile to a singlepart file so that GPSFDshp2mp can process it. Go to Vector > Geometry Tools > Multipart to singleparts.
  23. Pick the lines_multi layer for the input and then export a new shapefile called lines.
  24. Now we will convert the polygons shapefile to a singlepart file so that GPSFDshp2mp can process it. Go to Vector > Geometry Tools > Multipart to singleparts.
  25. Pick the polygons_multi layer for the input and then export a new shapefile called polygons.
  26. Now remove all the layers except for Lines.shp, Polygons.shp, and Points.shp
  27. At this point we are going to split the files into their final grid. This will be done using the GridSplitter plugin. Go to Plugins > Grid Splitter > Cut Layers Into Pieces
  28. In the parameters tab change the input layer to Lines.shp; set the method to tile size with the X axis and Y axis at 1. (if you think the files will be too big you can use X axis of 0.5 and Y axis of 1)
  29. Set the output base directory to a new folder called Split inside the QGIS folder
  30. Move to the additional options tab and select all in one folder and make the prefix Lines_
  31. Click OK and let it work; this can take a long time and will not display a status update.
    1. NOTE: If it says it will create an insane number of tiles click Abort. The map units are not arc degrees.  Now you can just pick a logical number (if your state is 5 degrees wide and you want 1 degree segments then do number of tiles as 5).
  32. Now we will split the polygons. Go to Plugins > Grid Splitter > Cut Layers Into Pieces
  33. In the parameters tab change the input layer to Polygons.shp; set the method to tile size with the X axis and Y axis at 1. (if you think the files will be too big you can use X axis of 0.5 and Y axis of 1)
  34. Set the output base directory to the Split folder inside the QGIS folder
  35. Move to the additional options tab and select all in one folder and make the prefix Polygons_
  36. Click OK and let it work
  37. FInally we can split the points. Go to Plugins > Grid Splitter > Cut Layers Into Pieces
  38. In the parameters tab change the input layer to Points.shp; set the method to tile size with the X axis and Y axis at 1.
  39. Set the output base directory to the Split folder inside the QGIS folder
  40. Move to the additional options tab and select all in one folder and make the prefix Points_
  41. Click OK and let it work
  42. Close QGIS; we'll be doing the processing into MP files with GPSFDshp2mp and some fancy command work.

Convert to .MP

For this tutorial we've gone ahead and created some helped bat files that make processing massive amounts of data easy. For this to work you must have followed the steps above, especially about naming the files Lines_, Polygons_, and Points_.

  1. Download the bat files that will be used for this step. Make sure you have already downloaded and extracted GPSFDshp2mp
  2. Extract them into the root folder for your map (i.e. mississippitopo)
  3. Right click on ConvertSplit2mp and click edit (should open in Notepad).
  4. Modify the following information as required:
    • Change the splitfolder data to the folder where you put the split files from QGIS (step 12 above)
    • Change gpsfdshp2mpfolder to the folder containing GPSFDshp2mp.exe (downloaded in Setup)
    • Change dictionary_file to the dictionary file location (a sample one is included in the bat files zip file)
    • Change dictionary_blank to the blank dictionary file (its empty) location (one is included in the bat files zip file)
    • Change mapname to the name of your map, such as "Arizona Topo"
    • Change initialmapid to an initial map ID number; these should be unique for each tile so it will automatically increment them. This also must be an 8 digit number that does not start with zero. Also, do not pick a really high number such as 99999990 since the 10th image will become a 9 digit number.
  5. Save the bat file and close Notepad.
  6. Now right click on CombineSplitMps and click edit (should open in Notepad).
  7. Modify the following information as required:
    • Change splitfolder to the folder containing all the .mp files (should be the same split folder from the previous bat file)
    • Change finalfolder to the folder where the final .mp files should end up; normally this is a complete folder. (note: this cannot be the split folder or the compile in the next part won't work)
    • If you have different prefixes change the prefix variables
  8. Now double click on ConvertSplit2mp and let it run.
    • Now your split folder will contain a .mp file for each .shp file
  9. Now double click on CombineSplitMps and let it run
    • Now your complete folder will have a single .mp file for each grid
  10. Go to your "final" folder and check and make sure you have a series of Final_ files

You will now have a series of maps split into a grid that are ready for compiling.

What's Next?

The final step is next.  We will compile the data using cgpsmapper and then create a setup file so other people can easily use it.

<-- Previous - Borders | Table Of Contents | Next - Compiling -->