Elevation Edits (Heightmap aka .raw)
-
It’s recently come to my attention that we are able to extract a heightmap aka .raw from Terrain editor to be manipulated in PS. Note: I am no expert in this area and am relying on the wealth of knowledge from the community to help guide me.
Steps to extract the Height map:
1. In Terrain Editor; Tools>ModTerrain>Extract Elv; open “.e2” as a .raw, re size to 4096 and overwrite the “.e0” for importing back into falcon.
or
2. may result in this though > https://www.benchmarksims.org/forum/showthread.php?15231-Q-Bert-rocks-the-pyramid
Image has exposure adjusted to show details in terrain.
Let me explain why I care to know how:
I’ve collected the http://visibleearth.nasa.gov/view.php?id=73934 data, coverted it to a .raw and have also converted it to a TIFF. I re sized this data to 65536x32768 for use in other projects that I work on.I believe it was Lukas that brought to my attention that our theaters use the Sinusoidal projection which creates a problem for my data set. I need to convert the projection into the mentioned one but have not figured out how I’m going to do it yet. I have QGIS and plan on looking up how to convert from wgs84 to sinusoidal; once I figure that out, I’ll be able to extract the srtm data from the TIFF I made in QGIS and will be able to put it into the raw for TE to use.
Once I get past that I’ll have another problem to overcome. Terrain flattening for AB’s, cities, smoothing for roads, water boundaries and I’m sure there’s probably more objectives/features that need flattened terrain. Again, I do not know since this is not my area of knowledge, all I know is buildings need flat terrain to sit on.
Well, I have an idea that may be able to quickly level or smooth out these areas. I showed a picture in my Korea overhaul thread -> http://i1298.photobucket.com/albums/ag52/King_Richard_III/Korea_2048_zps069237cd.png~original <- which is an image of the entire theater wearing it’s tiles. Original image is 26215x26215 but is dependent on TE cache settings. I also have a set of tiling textures shown in the pic that I use for tiling. I can use this image to mask the areas in the raw that need flattening or smoothing which would result in minimal hand edits within TE.
The other bonus. I’m tiling by hand so I could always load up a reference image of the height map to use in TE while tiling rivers and such and work with the elevations, thus minimizing the bulldozing. I’d prefer to start off this way but would need to figure out that sinusoidal conversion asap.
So, is my export method correct?
What’s each of these channels do?
And those of you whom are already editing .raw, would you please share your process entirely; things to look out for, etc… it’d be nice to start documenting some of these things more.
Thank you.
EDIT: updated the steps slightly. I’ll come back through and add pics at some point along with instructions to import back into falcon.
-
Richard… making the elevations took me 2 month !
Making a real terrain was my initial idea too, but with the only 4096 unique tile limitation of bms and a very bad elv-mesh of 1000m i just gave up on that idea and understood that i had to compromise with a good looking artificial one, artificial tiles and artificial elevation which match up.
In FSX things are easier …waaay easier to get real data into the sim (did that).
So, why dont you wait another 2 weeks and just take my work as it will be available for free for everyone…and put then your nice tiling job on it.
You dont have to…but maaaan…it killed alot of time to make that and i feel for those trying to do the same.Thats how it looks done in L0 (compare to stock). Dont save this jpg and try to convert to raw… too much loss in data.
TIP:
Extract terrain components with ModTerrain in TE.
Get the Theater.e2 file and rename it to Theater.raw.
Open that with PS.
Increase size to 4096 (L0). (use “preserve” in PS to maintain data at first, use “smooth” if you want to smoothen the edges while scaling)
Save it as Theater.raw again.
Rename it now to Theater.e0 !! not e2.
Vio la …you just have created the first (we have to repeat that) instance of L0 which you can load with TE.
All final L2 processed elevations must be derivatives of L0s considering the files created (like the .mea) upon that to ensure proper radar-terrain-masking,
terrain-following-features working, AI not crashing into hills and the “no lock bandits through mountain tips” - meaning you need always L0 at first, which you just created. This is why your stock L0 file does not match up with L2 at all i.e.But this is not everything…there are many “omgs” and “ahas” along the road…trying to make the best with the available options and tools
and Lakes, Rivers, Roads, Objectives…are unfortunatly a manual job in the FIRST L2 raw-by hand edit, where neigbouring tiles and their effect (tilt) on the main tile have to be considered aswell.
If you really decide to make all that …then i wish you much patience… or as i said…just take what i have done… and save 2 month -
King_Richard why in PS u declare it as 4096x4096? that way every one kilometer is one pixel only. This must go as high as your computer or u can handle. Or u could split the enormous file and work it down in pieces.
I believe something familiar did our Aegean theater elevations guru and the result is phenomenal in many areas our elevation difference is about 200metter from real. After that we fixed Areas of interest by hand which was way easy specially with the new addition by Monster.
Real Greece flyers or ppl that leave at some areas that they know them well say: “yes it is like that… just a bit higher in real” and as we examined we found the 200metters difference.
-
A.S.,
I’m well aware of the time involved in these types of projects, there’s nothing short term or quick when it comes to these types of things. Quickest editing I’ve seen for simulations is like you said in FSX, but that’s another beast. And yet it still doesn’t satisfy me like falcon does.
I don’t feel that the 4096 limit is an issue. We cannot create a million plus textures to cover the entire theater, nor would it look good in the sat imagery that’s freely available. But, the 1km resolution is a bit harsh and I do not know what an updated srtm data set installed in bms would look like with that limitation, however, I am curious.
I do plan on checking out your work once released and if everything is visually realistic then I will most definitely use it! Either way, at some point I will need to edit elevations since I have re tiled nearly all areas effected by water. I’ve also had to re route small portions of roads here and there.
Now when your editing the raw, are you editing each channel “alpha 1 and alpha 2” individually, or only one of them?
When you import the raw back in are you using the option in Terrain editor :: Tools>Mod Terrain>“import raw elevation data”.
I also did a comparison and the “make elv file from loaded terrain” gave me exactly what the .e2 file did except in 4096 resolution(res) already, no need to resize with that option!
The data set that I have is @500m, best resolution I could find at the time covering the entire globe. Our theater is only 1024x1024 tiles, hence the 1024x1024 res image from the L2, and a 4096x4096 res image for the L0 @ 250m. Sure I could blow up the image to 8192 or 16384 and comfortably work on it but what’s the point when everything is going to be scaled down in the L2 to 1024 in the end result. < For clarification, I’m not saying I’m going to scale it down to 1024, I see the note on making all changes on the L0!
-
Both, Alpha 1 and 2.
The “make elv file from loaded terrain” feature is bugged for me for some reason: https://www.benchmarksims.org/forum/showthread.php?15231-Q-Bert-rocks-the-pyramidYou can do what you like with my work Richard…and if it really helps you … we both are happy.
You seem to be confident in what you are doing and i like that. Let us see WIP shots asap -
I’ll have to experiment some time to see how the two Alpha’s effect the elevations because I’m not seeing exactly what Alpha 1 is doing. It appears to be a map of the average height of each tile or similar but I’m not exactly sure about that.
I did not try to import that make elv file back in with TE; so it could be fubared, it just visually appeared to be an exact replica of the L2 data once scaled so I don’t know. I’ll try to remember to check it out later on this week to see if it does the same for me.
WIP shot, lol, got some stuff to do before I can get in game to take some screens. But I’ll show ya the forest/dense farm transitions that I’m working on right now. I’ve scaled them down to 512 and only have 8 of them partially complete out of 64, lol. And yes the square farms is one tile repeated, easier to keep it like that I until I completely finish my variations.
-
Looks promising if that quality is maintained through the whole map. Keep it up
I assume alpha1 is for shaders over water and rivers, but i havent used water-tile-sets, instead plain-sets with water-areas within.
-
I think I can shed some light here about Alpha1 and 2.
The e2 (or e0 - elv) file contains the raw elevation data, each elevation value is 2 bytes (16 bits) long.
When you open the file as photoshop raw, you can determine the dimensions and the record length. This is achieved when selecting the menu (in photoshop) “Open As…”, selecting the e2 file (no need to rename it to raw) and in the Open As field select Photoshop raw. You are then presented with the following dialog:
The Channel area is determining how PS will read the data. If you select Depth 8 bits (1 byte) and interleaved, in order to match up the file size you need 2 layers (count value). In that configuration, PS will take the first byte and put it on Alpha 1, the second on Alpha 2, the third on Alpha 1 again etc. This way, you get two layers with data, one with the first eight bits of the actual elevation values and the other with the remaining 8 bits. IMHO this is not good for directly editing and saving back the file, because there’s no guarantee that changing separately the two bytes of a single value, you’ll get a usable value in return.
I think the best way is to open it in 16 bit depth. You can then experiment with the IBM PC or MAC byte order (defines which byte will calculate first and which second, to display the gray value). The advantage here is that each pixel has the exact elevation value the e2 file has, so whatever changes you make, should be making sense when converted to elevation.
-
Thanks Monster, I had just assumed that falcon was using 8bits and the two channels was just another one of those quirks within falcon.
These settings worked for me to view the e0 file on a windows/intel system.
-
I used the “Guess” option for Guam Alpha in CS5 and it worked for us.DUNNO how that may work for you??
demer
-
You can leave the apha 1 channel without touching it. Its irrelevant. Experimantal confirmations by reloading the differently created e0s advisable.
What is required instead is accurate information without loss, especially if you increase a L2 raw to a L0 raw (1024² to 4090²).
For that PS provides the “preserve” while scaling option, whereas the “smooth” option also has some nice sideeffects,
such as smoothening out the zick-zacks in your elevation work (in later processing steps).
Nevertheless, adding temporarily additional alpha channels can be handy aswell; this is how i gave the water a natural decline/flow i.e.Important is to generate the FIRST ACCURATE e0 !!, which you dont have as you work and manual-edits start in L2, and the one in the stock install (L0 file) is completly off from L2 data. Then you generate the theater from that basis (e0) and rest is fine-tuning.
Remember, regardless if you import real data or if you edit, your edits are in L2!!, unless you are insane enough to edit in L0 over a million meshes manually,
thus, if you want to create accurate .mea, map and On (n1, n2, etc) files and recalculate the radars later in tacedit …you need to have first a correct e0.One could generally choose pathes:
-
either importing real elv. data and trying to design elevation-appropiate tiles and graphics in respect (NUTS!, because of repetitions and really low quali mesh)
-
or using (creating) plain tiles and adjusting the elevation in regards to those (NUTS! too, but with hope for acceptable results, lol).
But carefull with this option aswell, as i have seen couple korea texture mods out there meanwhile where the rivers are not where the bridges are i.e :mrgreen:
-
-
The PS settings that I put up work. I just tested/verified with the original heights and modified heights. Lets take the guess work out of it.
I also got around to checking out the “make elv from loaded terrain” and had no problems with it when I imported it back into falcon.
I’m not sure if I’m importing the elevations correctly. Here’s how I’ve been doing it so far:
In the folder that I’m importing I’ve put all of the exported files into it. Had to change the e2 and e0 files in order to see the changes. I was under the assumption that you’s were creating the L2 from the L0 in Terrain Editor. I had to scale my 4096 image down to 1024 to create the L2 manually. So my modified folder has altered e0 and e2 files only. When scaling I used the preserve edges option.
I also tried the “import elv data from raw” option but that one would quit working on me every time I tried. Looked at monster’s help section but didn’t grasp the concept of how to do it from what I read, anyone have success building the elevations from the .elv file? I would like to use that option since it’d auto build the other L’s and it gives us a few more options.
Also learned to make multiple saves on important PS files, my transitions file was corrupted and now have to start them over, had 16 tiles done. I do periodic backups but have now added in the save copy option whenever I save something important or time consuming.
-
Can you please explain at which point ModTerrain failed and if there were any messages about it?
-
Stops and gives me the windows app crash window after I click on run;
Settings used;
Command prompt window appears;
Then app crash window pops up.
OS: Windows Vista 64
-
Yes I confirm this, I’m very sorry, it was just a little typo in th code that caused this.
I’ve made a new version (v1.3.1) that you can find in http://sakgiok.gr/downloads and must be working.
Just replace the files in TerrainEditor’s folder with the content of the zip file. The new version will be distributed with any future new version of TerrainEditor as well.
-
Have never used the import function, but thanks for the quick fix Monster.
-
I believe it was Lukas that brought to my attention that our theaters use the Sinusoidal projection which creates a problem for my data set. I need to convert the projection into the mentioned one but have not figured out how I’m going to do it yet. I have QGIS and plan on looking up how to convert from wgs84 to sinusoidal; once I figure that out, I’ll be able to extract the srtm data from the TIFF I made in QGIS and will be able to put it into the raw for TE to use.
The ‘dem2terrain projection’ was discussed years ago. It was written in these posts:
http://tactical.nekromantix.com/forum/viewtopic.php?f=12&t=20922&start=40 -
Thank you very much, will read through it when I get the time and report back.
The ‘dem2terrain projection’ was discussed years ago. It was written in these posts:
http://tactical.nekromantix.com/forum/viewtopic.php?f=12&t=20922&start=40 -
Spent my night reading/researching and only thing I came away with is that sinusoidal or similar projection is used with dem2terrain but the original Korea theater does not use this projection. Instead it’s mentioned that possibly Mercator is the projection type. There was talk of some tools being used/made too but nothing published and I doubt they were designed for working with “.raw”.
I see a Mercator projection in qgis, but I still have not taken the time to learn how to convert projections and extract them using that program. I keep putting that part on hold.
-
You are walking a path i did aswell… trying desperatly to import real elv data… and you cant imagine how hard i tried…till i understood the results will always suck this way and the only way is to create an artificial semi-realistic one. But i also know you are passionate - like i was - and that you have to find your own way.