Guantanamo Theater
-
Unfortunelly they are all the Villages and cities.
cellphone
-
Ok, for what I have understood so far, in order for CATE to work I would have to have an image (.BMP I Guess) of my theater where each pixel represents a tile.
Sooo if my Theater is 64 segments wide and 64 height and each segment is also 16 tiles by 16 tiles….
64x16=1024 pixels wide.
Therefore for my theater I need to grnerate a 1024×1024 bitmap where each RGB color pixel represents a type of tile.
I would have to tell cate by script what to do when It founds color XYZ (RGB) etc…
Sounds easy… and I am sure It wont be.
cellphone
Using a .bmp is just one way to define the tiles.
The strength of CATE is to tile the terrain according to elevation basicly,
then according to features like roads, rivers (little buggy), airbases etc., including their
transition tiles.Multible differently scripts, executed in correct order can do much tiling work here.
For example, usally for terrain the sea level is at 0ft, so you can already tile Ocean and Land.
The first script use only those 2 plain tiles to tile Ocean at 0ft and Land for anything higher.
The second script place the transition tiles between Ocean and Land.Look at the CATE manual for howto write such scripts.
I’m sure you’ll figure it out.
Cheers, :yo:
LS -
Thanks for the tip!
What looks promising is also the variation thing…!
cellphone
-
The noname are all noname or some?
If they have parents and those parents are named then it’s a piece of cake to name them, depending on the category…
For example bridges that many theaters have them like noname or 0 it’s super easy to baptize them all at once automatically.Yup +1 on this.
What i did was to convert most of the noname to factories,refineries,junctions etc and then baptise them by their parent. Its a bit time consuming specially when they are many but its a good solution i believe -
Using a .bmp is just one way to define the tiles.
The strength of CATE is to tile the terrain according to elevation basicly,
then according to features like roads, rivers (little buggy), airbases etc., including their
transition tiles.Multible differently scripts, executed in correct order can do much tiling work here.
For example, usally for terrain the sea level is at 0ft, so you can already tile Ocean and Land.
The first script use only those 2 plain tiles to tile Ocean at 0ft and Land for anything higher.
The second script place the transition tiles between Ocean and Land.Look at the CATE manual for howto write such scripts.
I’m sure you’ll figure it out.
Cheers, :yo:
LSLazy i read that in bmp mode u can actually tell cate also make roads and rivers? did i read correctly or missunderstood?
Another lets say problem i have is when i run the features conf for roads and rivers it completely ignores the transitions and messes the city tiles by placing the roads. (with rivers its ok) Is there any solution to this?Thanks!
-
Yup +1 on this.
What i did was to convert most of the noname to factories,refineries,junctions etc and then baptise them by their parent. Its a bit time consuming specially when they are many but its a good solution i believeNo time consuming at all m8 if you do it in excel. just a few seconds.
Bellow in first column (RED) it was all bridges named as 0.First export from MC to csv, or if you have them as csv or excel.
add one column after the ID column.
On that empty column copy and paste the Name column, don’t erase the first Name column.
Filter the whole list, Data->filter.
Filter your selection to display noname or 0 or whatever you want to replace with the parent name.
in the name column on the first cell insert the formula:=CONCATENATE(VLOOKUP(V3;E:F;2;FALSE);" Bridge")
press enter and then drag the cell all the way down to fill all the raws in the column.
ready.
If you want the values, instead of the formula in the cells, just select the cells with the formula, right click copy, right click paste values.For the formula, important, depending on your locale it could be ; or , that separate the formula parts.
so the formula could be:=CONCATENATE(VLOOKUP(V3,E:F,2,FALSE)," Bridge")
This is a nested formula where we concatenate or merge if u like the result of a search match (vlookup) with the text " Bridge"
So vlookup will search based on the value of V3 which is the parent ID column inside E:F columns and always on the first column which is E, once found a result we tell it to return 2 which means the second column (F) from E:F set, which has the names, we add FALSE in vlookup at the end which dictates vlookup to find an exact match only. If you don’t add false u r in trouble as it will return anything familiar or close to the search value.I use such at work everyday for decades and I write those blindfolded and I recall first times was a pain to comprehend… don’t worry excel is very good guiding you once you press the fx button, and provides a form to input or select via mouse.
it literally takes a few seconds… so Manos sorry you had to do it one by one…One more reason I bark to ask in the public forum… don’t fight the beast alone… I know that there are new tools and tricks added to do one job but this is the monster and this is how we fight it…
Unfortunately we can’t avoid the reinvention of wheel in the process cause the spread of knowledge is vast, but we can help each other and no one knows all…Edit: in my picture I have different cell values in the formula cause I added 3 columns… V3 or Y3 in the pic is the column that has the parent ID, so type this column in your formula.
Edit2: also in the pic in column F you see already the nae with Bridge, this is because I used a formula =A1, so initially it was 0, but once I added the formula it returned the formula result… don’t get confused by this, just ignore it. Also on B column I just concatenated (merged) some columns to have them as reference and cross check the type mostly. -
No time consuming at all m8 if you do it in excel. just a few seconds.
Bellow in first column (RED) it was all bridges named as 0.https://i.postimg.cc/3wPxQYzf/unknown.png
First export from MC to csv, or if you have them as csv or excel.
add one column after the ID column.
On that empty column copy and paste the Name column, don’t erase the first Name column.
Filter the whole list, Data->filter.
Filter your selection to display noname or 0 or whatever you want to replace with the parent name.
in the name column on the first cell insert the formula:=CONCATENATE(VLOOKUP(V3;E:F;2;FALSE);" Bridge")
press enter and then drag the cell all the way down to fill all the raws in the column.
ready.
If you want the values, instead of the formula in the cells, just select the cells with the formula, right click copy, right click paste values.For the formula, important, depending on your locale it could be ; or , that separate the formula parts.
so the formula could be:=CONCATENATE(VLOOKUP(V3,E:F,2,FALSE)," Bridge")
This is a nested formula where we concatenate or merge if u like the result of a search match (vlookup) with the text " Bridge"
So vlookup will search based on the value of V3 which is the parent ID column inside E:F columns and always on the first column which is E, once found a result we tell it to return 2 which means the second column (F) from E:F set, which has the names, we add FALSE in vlookup at the end which dictates vlookup to find an exact match only. If you don’t add false u r in trouble as it will return anything familiar or close to the search value.I use such at work everyday for decades and I write those blindfolded and I recall first times was a pain to comprehend… don’t worry excel is very good guiding you once you press the fx button, and provides a form to input or select via mouse.
it literally takes a few seconds… so Manos sorry you had to do it one by one…One more reason I bark to ask in the public forum… don’t fight the beast alone… I know that there are new tools and tricks added to do one job but this is the monster and this is how we fight it…
Unfortunately we can’t avoid the reinvention of wheel in the process cause the spread of knowledge is vast, but we can help each other and no one knows all…Edit: in my picture I have different cell values in the formula cause I added 3 columns… V3 or Y3 in the pic is the column that has the parent ID, so type this column in your formula.
Edit2: also in the pic in column F you see already the nae with Bridge, this is because I used a formula =A1, so initially it was 0, but once I added the formula it returned the formula result… don’t get confused by this, just ignore it. Also on B column I just concatenated (merged) some columns to have them as reference and cross check the type mostly.+1 for the info!
Maybe u can help with CATE issue as well ? I found a way but still it “eats” some of the city transition files. Far more managable than spliting a city in half to lay road tiles but still it would be far more awesome if it wouldn’t happen at all!!!Cheers
-
Lazy i read that in bmp mode u can actually tell cate also make roads and rivers? did i read correctly or missunderstood?
Another lets say problem i have is when i run the features conf for roads and rivers it completely ignores the transitions and messes the city tiles by placing the roads. (with rivers its ok) Is there any solution to this?Thanks!
Beside that RGB xxx is tile number x, there is no tiling logic in BMP mode.
Tiling the roads/rivers (RR) has been always a bit messy and tricky with CATE,
and the more tiles comes into play, the worse it might get.If I would tile a terrain totay, I would tile plain ocean, land, and cities
without any transitions first. (to keep it simple for RR tiling algorithm)Then RR, without any transitions.
Next the transitions for RR, then the transitions for ocean to land
and city to land.Note: the RR “interact” with each other, so the transitions for the RR
needs separated scripts, one for rivers, one for roads.Then correct tiling errors using the replace function in Monsters TE.
This would be a good base for further tiling.
Forest for example could be painted then on the BMP and tiled on land tiles only,
Next change land RR and land cities to forest RR and forest city tiles.Then tile forest transitions.
… and go from there …
Cheers, :yo:
LS -
Thanks LS!
It seems i was doing it worng all this time then i figure out what exactly those numbers mean in the RR conf file! LOL and also had an flash of the read me…
Everything now seems perfect!Cheers!
-
Well, I am no yet dead… Neither is this project.
I have been trying to solve the “NoName” issue that is give to all my objectives Cuban places.e00.
I have also reworked some things to include part on Florida (Key West for sure) and Bahamas.
Still WIP
-
Well, I am no yet dead… Neither is this project.
I have been trying to solve the “NoName” issue that is give to all my objectives Cuban places.e00.
I have also reworked some things to include part on Florida (Key West for sure) and Bahamas.
Still WIP
no name as???
e00 can be loaded in QGIS.
U can load a ton of GIS data in qgis.
I’m like 99% sure that your points will be the same as let’s say OSM data…
so for those overlapping get the OSM names, and please please don’t do it by hand, one by one… -
I finally solved It. I Will explain how in another post.
The thing about using other sources is two problems: Dem2Terrain converst all objectives to XY tile coordinates and also removes all objectives that are out of the frame of the theater but part of the country/U.S state.
So yes. Names are there (OSM and others) but will require such conversion from lat-long coordinAtes to XY tiles, and later on a removal.
Also Dem2Terrain generates the factories and probably other stuff out of the labels.e00 files which only has the names of cities and Villages…
cellphone
-
I finally solved It. I Will explain how in another post.
The thing about using other sources is two problems: Dem2Terrain converst all objectives to XY tile coordinates and also removes all objectives that are out of the frame of the theater but part of the country/U.S state.
So yes. Names are there (OSM and others) but will require such conversion from lat-long coordinAtes to XY tiles, and later on a removal.
Also Dem2Terrain generates the factories and probably other stuff out of the labels.e00 files which only has the names of cities and Villages…
cellphone
well if we could only had the CRS (coordinates reference system) that DEM2Terrain uses things could be waaaaay different.
I tried with a dirty logic to do so for Caucasus, I got very close but it’s not spot on… needs time to decipher the process between Lambert conformal conic projection and falcon theater… which I assume it must be different per area…
If we could have this then things would be a piece of cake with QGIS.
I’m still slowly investigating it but I’m not GIS expert… If someone is and knows the exact way old Falcon dem2terrain made the calculations it would be very much appreciated… -
I will try to be brief, and If I can explain further later.
My two problems:
- The ADF database does not contain all the names for the populated points (At least how I have consulted importing it to ILWIS), therefore the .e00 gerenated does not have all the names either, there are missing maybe 30% or so, but I think the missing one are all from very lightly populated points (small villages).
2)Furthermore, DEM2TERRAIN fails to export the names that ARE in the .e00 probably because some parsing errors I failed to locate to solve.
The evident solution, since the names are reachable from the .e00 beaig a plian text file… to get the missing ones from the .e00 (In my case, missing Florida and Cuba villages, but Bahamas are fine)
Problem with this… NOT ALL POINTS ARE EXPORTED with DEM2TERRAIN since the theater CUTS some of this territories (CUBA and FLORIDA) out, so It is hard to tell without the XY coordinates which belong to my theater and which are left out.
SOLUTION: I generated a bigger version of my 64 segments theater to 128 segmenst using the same center coordinate. This way I have a generated by DEM2TERRAIN .csv with all the objectives that are inside the e00 label files of each region. and with the coordinates… BUT my coordinates are not the same but still I can use them to eliminate the pbjectives that don’t belong.
I exported this 128seg .csv to excel and filtered our any X smaller that 511 or bigger than 1535 and erase them, and also any Ysmaller that 511 or bigger than 1535(my 64 seg theater is 1024 tiles long but the 128 is 2024 tiles long so the objectives that don’t belong to my theater are under 511 or above 1535. First tile is 0,0, thats why 511 instead of 512 which is 1024/2)
I also exported the 64sg csv to excel and compared the “trimmed out” 128 excel to the 64 seg ones and they almost matched perfectly but for 2 factories DEM2TERRAIN had generated in BAHAMAS. So I coppied all the names of Cuba and FLorida and left out the BAHAMAS ones since they were fine and to leave out the Factories issue.
- The ADF database does not contain all the names for the populated points (At least how I have consulted importing it to ILWIS), therefore the .e00 gerenated does not have all the names either, there are missing maybe 30% or so, but I think the missing one are all from very lightly populated points (small villages).
-
I noticed this as well in ny theater,many Indian cities seem to be missing…gonna try your method via 128.csv
-
https://i.postimg.cc/Jh4PR1NR/MC-new-Guantanamo.jpg
I will try to be brief, and If I can explain further later.
My two problems:
- The ADF database does not contain all the names for the populated points (At least how I have consulted importing it to ILWIS), therefore the .e00 gerenated does not have all the names either, there are missing maybe 30% or so, but I think the missing one are all from very lightly populated points (small villages).
2)Furthermore, DEM2TERRAIN fails to export the names that ARE in the .e00 probably because some parsing errors I failed to locate to solve.
The evident solution, since the names are reachable from the .e00 beaig a plian text file… to get the missing ones from the .e00 (In my case, missing Florida and Cuba villages, but Bahamas are fine)
Problem with this… NOT ALL POINTS ARE EXPORTED with DEM2TERRAIN since the theater CUTS some of this territories (CUBA and FLORIDA) out, so It is hard to tell without the XY coordinates which belong to my theater and which are left out.
SOLUTION: I generated a bigger version of my 64 segments theater to 128 segmenst using the same center coordinate. This way I have a generated by DEM2TERRAIN .csv with all the objectives that are inside the e00 label files of each region. and with the coordinates… BUT my coordinates are not the same but still I can use them to eliminate the pbjectives that don’t belong.
I exported this 128seg .csv to excel and filtered our any X smaller that 511 or bigger than 1535 and erase them, and also any Ysmaller that 511 or bigger than 1535(my 64 seg theater is 1024 tiles long but the 128 is 2024 tiles long so the objectives that don’t belong to my theater are under 511 or above 1535. First tile is 0,0, thats why 511 instead of 512 which is 1024/2)
I also exported the 64sg csv to excel and compared the “trimmed out” 128 excel to the 64 seg ones and they almost matched perfectly but for 2 factories DEM2TERRAIN had generated in BAHAMAS. So I coppied all the names of Cuba and FLorida and left out the BAHAMAS ones since they were fine and to leave out the Factories issue.
Nice will check further as well!
Therefore there is another solution but again there’s no X,Y unles im not seing them XD (Arty might help here) Importing the pppoint file in qgis to extract the names (you can export in csv also each e00) but as you said in your method i didnt see any X/Y And for sure the problem is when a coutnry is splitted from the boundaries of the theater and partially cut but of course if everything is within the theater then you dont have a problem. I hope Arty can help a bit in this but i will invastigate as well further. - The ADF database does not contain all the names for the populated points (At least how I have consulted importing it to ILWIS), therefore the .e00 gerenated does not have all the names either, there are missing maybe 30% or so, but I think the missing one are all from very lightly populated points (small villages).
-
I have given up on trying to read the CATE manual on the screen. Had to print it
I have also done some cleanup on my 1st post.
-
guys u want points of data like cities factories etc if I understood correctly?
So u do what u do in CATE and DEM2Terrain (Sorry almost never got in to those, and didn’t understood what and how they do it).
Now if you want to get GIS data in to your theater and not do it the classic way one by one with TE or MC.
The following knowledge was ignited mostly from other BMS members and then I started digging more in as I already had some knowledge on QGIS application.
The thing is that we must work on modern tools and with modern data one of which is QGIS and OSM.
First you-we must make a compromise (as always in Falcon). The Falcon theater projection is “unknown”. I made a test with Manos but we didn’t verify it to adjust the parameters.Now converting the 3D ellipsoid surface of the earth to 2D perfect square is called a projection and CRS in QGIS application Coordinates Reference System. Now if we new which is in Falcon we could just apply from the crs list or create a custom one.
So in the tests with Manos I tried to mimic a custom crs for the Caucasus area to have a perfect square area and set the 0.0 point of the theater to be the bottom left corner.
It’s not spot on but it’s close, further more it’s kinda the same situation in Falcon, as you go far from the 0.0 point the more offset to real coordinates u have.This means from now on you have to work on a tool on a base that by default has a fault factor in projection, meaning some tiny areas might be off the map.
we can kinda over come cause as I understand u get a theater map from D2T or Cate?
if so we can georeference this image in QGIS and use it as reference for the custom crs for your theater. Edit: or calculate an offset and use it as a parameter to better place data - objectives.Since those r done the rest are easy and just the typical theater work BUT with a million times factor on speed, cause you work on big data… you have tons of data and points to work and filter to end up to those 8K objectives.
To get the Falcon XY, you reproject your data to your theater custom crs then you add 2 fields X and Y in the data table and then you set as values for those $x and $y. those will be your Falcon coordinates.
From QGIS you can export all point layers data in CSV or XLS.
I can create a small guide if you like with screenshots and details if u like.
QGIS is a way powerful app with an enormous potential to do things like never before.
Then you can even go in features depth if you like with the same logic, but that will be just for reference and naming as there is no way to mass import such in falcon.It’s a bit late right now to go in more details, but please don’t hesitate to ask for specific details or how I do this how that etc.
Just to clear out I’m self taught on GIS and QGIS and thanx to some BMS members that got me started on this subject…
-
So guys to test the CRS thing in QGIS please send me some theater data.
Corner and center coordinates.
I believe u get those from dem2terrain?Now regarding the rest of the process to get data from GIS to Falcon with the use of QGIS I don’t know if we should keep doing it here or if we should start a new thread as QGIS and what and how we can do with it it’s a whole project by it’s self.
So… I will answer any questions and in the mean time I’ll try to form up a guide for you with the basic staff which we will enrich further more in the future by our endeavors on the subject and create a new thread so that everything will be there.So what I missed from my previous post at 04:33 ( lol ) yesterday is where and how you get the data and some tricks on the process and the QGIS manipulation.
by manipulation I mean at some point and specially if you work on a team with many ppl u might want to share your work or send it to someone. As the files structure will be different on the project u might get stack and loose time on this.
So create a folder and in there place all your sources and layers u create in your QGIS project. Also as the usual file format is .shp it’s better to use and save as geopackages the layers.
This way u can take specific layers and projects transfer them wherever and the project will be working and not stack cause it doesn’t find c:\users*usarename*\mydocuments\gis\my_project_name\my_specific_sub_project
In the other computer when u transfer it you must have the exact folder structure for the project to work.Now where u get the layers.
there is https://overpass-turbo.eu/
create a bbox meaning bounding box which will include all your theater area.
then use wizard to build the query.
Important set the timeout to something larger like 250 cause if you ask for a big dataset it will timeout continuously and frustrate u.
If you want more details the trick is keys and values for those queries and understand a bit the way OSM data work from here:
https://wiki.openstreetmap.org/wiki/Map_featuresthen press run and you will se the results on the map.
then select export and select download/copy as GeoJSON
name and save the layer to something understandable.Now u can drag and drop this saved file to QGIS.
Another source but with the caveat that provides only local names is geofabrik
http://download.geofabrik.de/caveat also is that you get country by country the data and you have to merge them. Not a big deal but it’s one step more.
Now since we want the names you can get the OSM data for the same categories and then grab the names from them, geofabrik has the OSMid so logically we can get name in English from the OSM layers by doing a join between geofabrik and OSM. Also have in mind OSM data are so rich that can be used for filtering the data better while with geofabrik data set you don’t get that info. Yeap you will need it cause you will end up to some million points and you must have just 8.000 objectives in your theater.
So you gather all you can and then start the rip off to end up with what will go in Falcon. Also a big pain are bridges… Those are zillion per country and you must perform a procedure to reduce them down to like bellow a thousand…Another valuable source but it’s HELL to get the data is Wikimapia.
http://wikimapia.org/#lang=en⪫=37.966700&lon=23.716700&z=12&m=wwikimapia has very good data for military data. Example:
Do a search for military in your theater area and you will get what I mean.
They have an API ( http://wikimapia.org/api/ ) to get data but gives only 100-200 results…
another tool I found for wikimapia is
https://github.com/martinedoesgis/wikimapia-exportdownload it extract it in a folder and open index.html to your browser. First create an API key in wikimapia.
with this tool you can go area by area and get the data… which is :crazy: for a 1024^2 theater area.I know some might consider it as not reliable military data-info, BUT it’s this or nothing. So you get a data set and then talk to your sources and alter remove add whatever u want.
-
So guys to test the CRS thing in QGIS please send me some theater data.
Corner and center coordinates.
I believe u get those from dem2terrain?Now regarding the rest of the process to get data from GIS to Falcon with the use of QGIS I don’t know if we should keep doing it here or if we should start a new thread as QGIS and what and how we can do with it it’s a whole project by it’s self.
So… I will answer any questions and in the mean time I’ll try to form up a guide for you with the basic staff which we will enrich further more in the future by our endeavors on the subject and create a new thread so that everything will be there.It is very interesting, please do open the new thread and copy this two posts and I will join the conversation for sure.
On last thing I see after some more digging into CATE, this tool uses the .tdf files generated by DEM2TERRAIN for some tiling, specially rivers and roads. But this data seems fine in the ADF/e00 database we have available.
Another thing, DEM2TERRAIN uses the center coordinate only, but does provide coordinates for all corners. And then we know that the distance of the whole theater is 1024 km wide and 1024 km long if you change the resolution in D2T from miles to km and 0,25 insteand of 0,29983892or something like that that returns a 1023.something km² theater.
Mi theater has
Size (L2 segments): 64 x 64Size (L2 tiles): 1024 x 1024 Size (Kilometers): 1024 x 1024 Resolution: 0.25 Kilometers Centered on: Lat 23.3 Long -78.3 NW: Lat 27.9045 Long -83.5103 NE: Lat 27.9045 Long -73.0897 SW: Lat 18.6955 Long -83.161 SE: Lat 18.6955 Long -73.439 Highest point: 5838ft Lowest point: 0ft