Guantanamo Theater
-
Thanks LS!
It seems i was doing it worng all this time then i figure out what exactly those numbers mean in the RR conf file! LOL and also had an flash of the read me…
Everything now seems perfect!Cheers!
-
Well, I am no yet dead… Neither is this project.
I have been trying to solve the “NoName” issue that is give to all my objectives Cuban places.e00.
I have also reworked some things to include part on Florida (Key West for sure) and Bahamas.
Still WIP
-
Well, I am no yet dead… Neither is this project.
I have been trying to solve the “NoName” issue that is give to all my objectives Cuban places.e00.
I have also reworked some things to include part on Florida (Key West for sure) and Bahamas.
Still WIP
no name as???
e00 can be loaded in QGIS.
U can load a ton of GIS data in qgis.
I’m like 99% sure that your points will be the same as let’s say OSM data…
so for those overlapping get the OSM names, and please please don’t do it by hand, one by one… -
I finally solved It. I Will explain how in another post.
The thing about using other sources is two problems: Dem2Terrain converst all objectives to XY tile coordinates and also removes all objectives that are out of the frame of the theater but part of the country/U.S state.
So yes. Names are there (OSM and others) but will require such conversion from lat-long coordinAtes to XY tiles, and later on a removal.
Also Dem2Terrain generates the factories and probably other stuff out of the labels.e00 files which only has the names of cities and Villages…
cellphone
-
I finally solved It. I Will explain how in another post.
The thing about using other sources is two problems: Dem2Terrain converst all objectives to XY tile coordinates and also removes all objectives that are out of the frame of the theater but part of the country/U.S state.
So yes. Names are there (OSM and others) but will require such conversion from lat-long coordinAtes to XY tiles, and later on a removal.
Also Dem2Terrain generates the factories and probably other stuff out of the labels.e00 files which only has the names of cities and Villages…
cellphone
well if we could only had the CRS (coordinates reference system) that DEM2Terrain uses things could be waaaaay different.
I tried with a dirty logic to do so for Caucasus, I got very close but it’s not spot on… needs time to decipher the process between Lambert conformal conic projection and falcon theater… which I assume it must be different per area…
If we could have this then things would be a piece of cake with QGIS.
I’m still slowly investigating it but I’m not GIS expert… If someone is and knows the exact way old Falcon dem2terrain made the calculations it would be very much appreciated… -
I will try to be brief, and If I can explain further later.
My two problems:
- The ADF database does not contain all the names for the populated points (At least how I have consulted importing it to ILWIS), therefore the .e00 gerenated does not have all the names either, there are missing maybe 30% or so, but I think the missing one are all from very lightly populated points (small villages).
2)Furthermore, DEM2TERRAIN fails to export the names that ARE in the .e00 probably because some parsing errors I failed to locate to solve.
The evident solution, since the names are reachable from the .e00 beaig a plian text file… to get the missing ones from the .e00 (In my case, missing Florida and Cuba villages, but Bahamas are fine)
Problem with this… NOT ALL POINTS ARE EXPORTED with DEM2TERRAIN since the theater CUTS some of this territories (CUBA and FLORIDA) out, so It is hard to tell without the XY coordinates which belong to my theater and which are left out.
SOLUTION: I generated a bigger version of my 64 segments theater to 128 segmenst using the same center coordinate. This way I have a generated by DEM2TERRAIN .csv with all the objectives that are inside the e00 label files of each region. and with the coordinates… BUT my coordinates are not the same but still I can use them to eliminate the pbjectives that don’t belong.
I exported this 128seg .csv to excel and filtered our any X smaller that 511 or bigger than 1535 and erase them, and also any Ysmaller that 511 or bigger than 1535(my 64 seg theater is 1024 tiles long but the 128 is 2024 tiles long so the objectives that don’t belong to my theater are under 511 or above 1535. First tile is 0,0, thats why 511 instead of 512 which is 1024/2)
I also exported the 64sg csv to excel and compared the “trimmed out” 128 excel to the 64 seg ones and they almost matched perfectly but for 2 factories DEM2TERRAIN had generated in BAHAMAS. So I coppied all the names of Cuba and FLorida and left out the BAHAMAS ones since they were fine and to leave out the Factories issue.
- The ADF database does not contain all the names for the populated points (At least how I have consulted importing it to ILWIS), therefore the .e00 gerenated does not have all the names either, there are missing maybe 30% or so, but I think the missing one are all from very lightly populated points (small villages).
-
I noticed this as well in ny theater,many Indian cities seem to be missing…gonna try your method via 128.csv
-
https://i.postimg.cc/Jh4PR1NR/MC-new-Guantanamo.jpg
I will try to be brief, and If I can explain further later.
My two problems:
- The ADF database does not contain all the names for the populated points (At least how I have consulted importing it to ILWIS), therefore the .e00 gerenated does not have all the names either, there are missing maybe 30% or so, but I think the missing one are all from very lightly populated points (small villages).
2)Furthermore, DEM2TERRAIN fails to export the names that ARE in the .e00 probably because some parsing errors I failed to locate to solve.
The evident solution, since the names are reachable from the .e00 beaig a plian text file… to get the missing ones from the .e00 (In my case, missing Florida and Cuba villages, but Bahamas are fine)
Problem with this… NOT ALL POINTS ARE EXPORTED with DEM2TERRAIN since the theater CUTS some of this territories (CUBA and FLORIDA) out, so It is hard to tell without the XY coordinates which belong to my theater and which are left out.
SOLUTION: I generated a bigger version of my 64 segments theater to 128 segmenst using the same center coordinate. This way I have a generated by DEM2TERRAIN .csv with all the objectives that are inside the e00 label files of each region. and with the coordinates… BUT my coordinates are not the same but still I can use them to eliminate the pbjectives that don’t belong.
I exported this 128seg .csv to excel and filtered our any X smaller that 511 or bigger than 1535 and erase them, and also any Ysmaller that 511 or bigger than 1535(my 64 seg theater is 1024 tiles long but the 128 is 2024 tiles long so the objectives that don’t belong to my theater are under 511 or above 1535. First tile is 0,0, thats why 511 instead of 512 which is 1024/2)
I also exported the 64sg csv to excel and compared the “trimmed out” 128 excel to the 64 seg ones and they almost matched perfectly but for 2 factories DEM2TERRAIN had generated in BAHAMAS. So I coppied all the names of Cuba and FLorida and left out the BAHAMAS ones since they were fine and to leave out the Factories issue.
Nice will check further as well!
Therefore there is another solution but again there’s no X,Y unles im not seing them XD (Arty might help here) Importing the pppoint file in qgis to extract the names (you can export in csv also each e00) but as you said in your method i didnt see any X/Y And for sure the problem is when a coutnry is splitted from the boundaries of the theater and partially cut but of course if everything is within the theater then you dont have a problem. I hope Arty can help a bit in this but i will invastigate as well further. - The ADF database does not contain all the names for the populated points (At least how I have consulted importing it to ILWIS), therefore the .e00 gerenated does not have all the names either, there are missing maybe 30% or so, but I think the missing one are all from very lightly populated points (small villages).
-
I have given up on trying to read the CATE manual on the screen. Had to print it
I have also done some cleanup on my 1st post.
-
guys u want points of data like cities factories etc if I understood correctly?
So u do what u do in CATE and DEM2Terrain (Sorry almost never got in to those, and didn’t understood what and how they do it).
Now if you want to get GIS data in to your theater and not do it the classic way one by one with TE or MC.
The following knowledge was ignited mostly from other BMS members and then I started digging more in as I already had some knowledge on QGIS application.
The thing is that we must work on modern tools and with modern data one of which is QGIS and OSM.
First you-we must make a compromise (as always in Falcon). The Falcon theater projection is “unknown”. I made a test with Manos but we didn’t verify it to adjust the parameters.Now converting the 3D ellipsoid surface of the earth to 2D perfect square is called a projection and CRS in QGIS application Coordinates Reference System. Now if we new which is in Falcon we could just apply from the crs list or create a custom one.
So in the tests with Manos I tried to mimic a custom crs for the Caucasus area to have a perfect square area and set the 0.0 point of the theater to be the bottom left corner.
It’s not spot on but it’s close, further more it’s kinda the same situation in Falcon, as you go far from the 0.0 point the more offset to real coordinates u have.This means from now on you have to work on a tool on a base that by default has a fault factor in projection, meaning some tiny areas might be off the map.
we can kinda over come cause as I understand u get a theater map from D2T or Cate?
if so we can georeference this image in QGIS and use it as reference for the custom crs for your theater. Edit: or calculate an offset and use it as a parameter to better place data - objectives.Since those r done the rest are easy and just the typical theater work BUT with a million times factor on speed, cause you work on big data… you have tons of data and points to work and filter to end up to those 8K objectives.
To get the Falcon XY, you reproject your data to your theater custom crs then you add 2 fields X and Y in the data table and then you set as values for those $x and $y. those will be your Falcon coordinates.
From QGIS you can export all point layers data in CSV or XLS.
I can create a small guide if you like with screenshots and details if u like.
QGIS is a way powerful app with an enormous potential to do things like never before.
Then you can even go in features depth if you like with the same logic, but that will be just for reference and naming as there is no way to mass import such in falcon.It’s a bit late right now to go in more details, but please don’t hesitate to ask for specific details or how I do this how that etc.
Just to clear out I’m self taught on GIS and QGIS and thanx to some BMS members that got me started on this subject…
-
So guys to test the CRS thing in QGIS please send me some theater data.
Corner and center coordinates.
I believe u get those from dem2terrain?Now regarding the rest of the process to get data from GIS to Falcon with the use of QGIS I don’t know if we should keep doing it here or if we should start a new thread as QGIS and what and how we can do with it it’s a whole project by it’s self.
So… I will answer any questions and in the mean time I’ll try to form up a guide for you with the basic staff which we will enrich further more in the future by our endeavors on the subject and create a new thread so that everything will be there.So what I missed from my previous post at 04:33 ( lol ) yesterday is where and how you get the data and some tricks on the process and the QGIS manipulation.
by manipulation I mean at some point and specially if you work on a team with many ppl u might want to share your work or send it to someone. As the files structure will be different on the project u might get stack and loose time on this.
So create a folder and in there place all your sources and layers u create in your QGIS project. Also as the usual file format is .shp it’s better to use and save as geopackages the layers.
This way u can take specific layers and projects transfer them wherever and the project will be working and not stack cause it doesn’t find c:\users*usarename*\mydocuments\gis\my_project_name\my_specific_sub_project
In the other computer when u transfer it you must have the exact folder structure for the project to work.Now where u get the layers.
there is https://overpass-turbo.eu/
create a bbox meaning bounding box which will include all your theater area.
then use wizard to build the query.
Important set the timeout to something larger like 250 cause if you ask for a big dataset it will timeout continuously and frustrate u.
If you want more details the trick is keys and values for those queries and understand a bit the way OSM data work from here:
https://wiki.openstreetmap.org/wiki/Map_featuresthen press run and you will se the results on the map.
then select export and select download/copy as GeoJSON
name and save the layer to something understandable.Now u can drag and drop this saved file to QGIS.
Another source but with the caveat that provides only local names is geofabrik
http://download.geofabrik.de/caveat also is that you get country by country the data and you have to merge them. Not a big deal but it’s one step more.
Now since we want the names you can get the OSM data for the same categories and then grab the names from them, geofabrik has the OSMid so logically we can get name in English from the OSM layers by doing a join between geofabrik and OSM. Also have in mind OSM data are so rich that can be used for filtering the data better while with geofabrik data set you don’t get that info. Yeap you will need it cause you will end up to some million points and you must have just 8.000 objectives in your theater.
So you gather all you can and then start the rip off to end up with what will go in Falcon. Also a big pain are bridges… Those are zillion per country and you must perform a procedure to reduce them down to like bellow a thousand…Another valuable source but it’s HELL to get the data is Wikimapia.
http://wikimapia.org/#lang=en⪫=37.966700&lon=23.716700&z=12&m=wwikimapia has very good data for military data. Example:
Do a search for military in your theater area and you will get what I mean.
They have an API ( http://wikimapia.org/api/ ) to get data but gives only 100-200 results…
another tool I found for wikimapia is
https://github.com/martinedoesgis/wikimapia-exportdownload it extract it in a folder and open index.html to your browser. First create an API key in wikimapia.
with this tool you can go area by area and get the data… which is :crazy: for a 1024^2 theater area.I know some might consider it as not reliable military data-info, BUT it’s this or nothing. So you get a data set and then talk to your sources and alter remove add whatever u want.
-
So guys to test the CRS thing in QGIS please send me some theater data.
Corner and center coordinates.
I believe u get those from dem2terrain?Now regarding the rest of the process to get data from GIS to Falcon with the use of QGIS I don’t know if we should keep doing it here or if we should start a new thread as QGIS and what and how we can do with it it’s a whole project by it’s self.
So… I will answer any questions and in the mean time I’ll try to form up a guide for you with the basic staff which we will enrich further more in the future by our endeavors on the subject and create a new thread so that everything will be there.It is very interesting, please do open the new thread and copy this two posts and I will join the conversation for sure.
On last thing I see after some more digging into CATE, this tool uses the .tdf files generated by DEM2TERRAIN for some tiling, specially rivers and roads. But this data seems fine in the ADF/e00 database we have available.
Another thing, DEM2TERRAIN uses the center coordinate only, but does provide coordinates for all corners. And then we know that the distance of the whole theater is 1024 km wide and 1024 km long if you change the resolution in D2T from miles to km and 0,25 insteand of 0,29983892or something like that that returns a 1023.something km² theater.
Mi theater has
Size (L2 segments): 64 x 64Size (L2 tiles): 1024 x 1024 Size (Kilometers): 1024 x 1024 Resolution: 0.25 Kilometers Centered on: Lat 23.3 Long -78.3 NW: Lat 27.9045 Long -83.5103 NE: Lat 27.9045 Long -73.0897 SW: Lat 18.6955 Long -83.161 SE: Lat 18.6955 Long -73.439 Highest point: 5838ft Lowest point: 0ft
-
It is very interesting, please do open the new thread and copy this two posts and I will join the conversation for sure.
On last thing I see after some more digging into CATE, this tool uses the .tdf files generated by DEM2TERRAIN for some tiling, specially rivers and roads. But this data seems fine in the ADF/e00 database we have available.
Another thing, DEM2TERRAIN uses the center coordinate only, but does provide coordinates for all corners. And then we know that the distance of the whole theater is 1024 km wide and 1024 km long if you change the resolution in D2T from miles to km and 0,25 insteand of 0,29983892or something like that that returns a 1023.something km² theater.
Mi theater has
Size (L2 segments): 64 x 64Size (L2 tiles): 1024 x 1024 Size (Kilometers): 1024 x 1024 Resolution: 0.25 Kilometers Centered on: Lat 23.3 Long -78.3 NW: Lat 27.9045 Long -83.5103 NE: Lat 27.9045 Long -73.0897 SW: Lat 18.6955 Long -83.161 SE: Lat 18.6955 Long -73.439 Highest point: 5838ft Lowest point: 0ft
yeap exactly all those “something like that” is the issue and something like that I try to create a custom “something like that” crs in QGIS in order GIS data to be close to were the rest of the theater features (rivers roads etc u get from d2T and cate).
That’s the compromise and the after u go this way forget going back to cate and d2t… or start from scratch.ok guys give me some time to gather my pile of info I can provide u, in the mean time keep asking, just a side note I just fight with a failed hdd with valuable falcon data and many many many other falcon things that I work on so please be gentle to this old slow minion, will work hard for banana… :lol:
-
I have given up on trying to read the CATE manual on the screen. Had to print it
Keep also in mind that the CATE manual is written with the texture numbers
of the old originally Falcon4 texture.bin.So you’ll need to use the texture numbers of your .bin in the scripts.
And you need to do that in the correct order.For the correct order see especially for RR and transtion tiles texture sets in the manual.
Eventually you want to organize the textures within a set in the same order in your
texture.bin for an easier work with the scripts.Else you could use the originally old texture.bin and textures as a start
for CATE tiling, then replace the old textures with your textures finally.
Cheers, :yo:
LS -
Keep also in mind that the CATE manual is written with the texture numbers
of the old originally Falcon4 texture.bin.So you’ll need to use the texture numbers of your .bin in the scripts.
And you need to do that in the correct order.For the correct order see especially for RR and transtion tiles texture sets in the manual.
Eventually you want to organize the textures within a set in the same order in your
texture.bin for an easier work with the scripts.Else you could use the originally old texture.bin and textures as a start
for CATE tiling, then replace the old textures with your textures finally.
Cheers, :yo:
LSIts what i did for Georgia. Eventually i changed the original .conf files to what i wanted adding/deleting stuff and of course because im not an RTFM guy all by trial and error. I maybe retiled the whole theater from scratch more than 100 times
No seriously i did read a bit of the manual but just a bit -
Keep also in mind that the CATE manual is written with the texture numbers
of the old originally Falcon4 texture.bin.So you’ll need to use the texture numbers of your .bin in the scripts.
And you need to do that in the correct order.For the correct order see especially for RR and transtion tiles texture sets in the manual.
Eventually you want to organize the textures within a set in the same order in your
texture.bin for an easier work with the scripts.Else you could use the originally old texture.bin and textures as a start
for CATE tiling, then replace the old textures with your textures finally.
Cheers, :yo:
LSThose are just a script that colors and then splits to tiles? like that height brown, that height green, that area brown, that height grey etc?
If so something like this can be done in qgis and then split then by 1km to tiles.
we can merge rivers and roads for the visual of it…Oppps I shouldn’t have said that… yea I hear the rain from far coming with loud thunders!!!
// Arty runs and hides under qgis layers.Edit something like this??
-
Its what i did for Georgia. Eventually i changed the original .conf files to what i wanted adding/deleting stuff and of course because im not an RTFM guy all by trial and error. I maybe retiled the whole theater from scratch more than 100 times
No seriously i did read a bit of the manual but just a bitWell, it seems you like and need the PITA! :mrgreen:
But you did it and got nice results, and that’s important finally. :thumb:
Cheers, :yo:
LS -
Those are just a script that colors and then splits to tiles?
No Arty, differently.
Although CATE can tile according to elevations easily, using a script …
# Fixes tiles. Sets dry farm tiles up to 1260 feet, # forest tiles up to 4200 ft, and snow tiles above that. # Further variation will follow. UpdateOceanTiles=OK [Rules: Standard mode section] Color=-1,-1,-1 ForceFog=* * * TileRule=0 0 (1-23) (0) * * * * TileRule=1 1260 (1116,1220) (48) * * * * TileRule=1261 4650 (688,1116,1220) (882) * * * * TileRule=4651 99999 (*) (694) * * * * [EndRules] # EOF
… I’m mainly talking about the scripts for roads/rivers and transition tiles
and the CATE logic/ algorithm to tile them.See and old example of a CATE script to tile transitions here:
# Creates 2nd transition pass between various terrain types. # Processes dry farm, wet farm, forest, snow & ocean area types. DoTransitions=OK [TerrainFeatures: Farm] TerrainType=1 TerrainTiles=(32-46,48) TransitionDef=ToOcean 1 (0) (39,41,42,43,44,46,34,35,45,40,32,36,38,37,33) [EndTerrainFeatures Farm] [TerrainFeatures: Forest] TerrainType=2 TerrainTiles=(64-78,240-254,882,144-158) TransitionDef=ToFarm 2 (48) (72,73,76,66,75,77,71,64,78,67,74,65,69,70,68) TransitionDef=ToOcean 1 (0) (252,245,246,241,251,254,249,240,253,248,247,242,244,243,250) TransitionDef=ToWetFarm 2 (160) (153,154,157,145,150,158,152,146,156,151,155,144,148,149,147) [EndTerrainFeatures Farm] [TerrainFeatures: City] TerrainType=3 TerrainTiles=(321,1424-1438) TransitionDef=ToOcean 1 (0) (1429,1437,1428,1430,1436,1431,1427,1424,1438,1435,1432,1425,1433,1426,1434) [EndTerrainFeatures City] [TerrainFeatures: WetFarm] TerrainType=4 TerrainTiles=(1-14,160) TransitionDef=ToOcean 1 (0) (12,11,2,8,4,13,1,5,14,7,3,9,10,6) [EndTerrainFeatures WetFarm] [TerrainFeatures: Snow] TerrainType=5 TerrainTiles=(688-703) TransitionDef=ToForest 2 (882) (696,697,700,689,702,703,695,688,701,693,698,692,691,690,694) [EndTerrainFeatures Snow] # EOF
Each of those texture numbers must match the correct tile in the texture.bin,
else the results are wrong.
Cheers, :yo:
LS -
Those are just a script that colors and then splits to tiles? like that height brown, that height green, that area brown, that height grey etc?
If so something like this can be done in qgis and then split then by 1km to tiles.
we can merge rivers and roads for the visual of it…Oppps I shouldn’t have said that… yea I hear the rain from far coming with loud thunders!!!
// Arty runs and hides under qgis layers.Edit something like this??
https://i.ytimg.com/vi/GK9aBRVJdRo/maxresdefault.jpgNo, It is a program that by following some rules, or a RGB coloured .BMP or a combination of rules and .tdf files assigns the given in the rules tile textures to the tiles that meet the rule criteria…
Is kinda the other way around, you feed It the .BMP (or just the rules, or rules + .tdf) and It uses It to “auto tile”
cellphone
-
No, It is a program that by following some rules, or a RGB coloured .BMP or a combination of rules and .tdf files assigns the given in the rules tile textures to the tiles that meet the rule criteria…
Is kinda the other way around, you feed It the .BMP (or just the rules, or rules + .tdf) and It uses It to “auto tile”
cellphone
depends how you see it. as u describe it it’s the same thing for me, just different procedure.
I’m talking also about auto tile. It’s called Classification.@Lazy ok I kinda get it… let’s not wake the dead… :lol: