AiTiles [Rev. 3] - AI Upscaled Terrain Tiles for Falcon BMS
-
I think it’s a really clever strategy to “let the AI do all the work”. I’m certainly aware of the fact that many person-moons were spent on the development of the original brain scan image enhancement network, but to “coax” this network into doing work on something completely different (insert funny Monty Python video here) like the BMS KTO tileset ist just genius
All the best,
Uwe
-
Amazing work. Very clever!!
-
Pls. make somebody comparison screenshots.
-
Actually, Arty converted KTO tiles into 1k*1k res already,
The change is minor compare to the size and demand from the GPU\CPU to process those in sim -
Thanks I mean that is mightily impressive work!
Anyone who has manually worked on Terrain tiles knows the time they take and the massive limitations of them due to that. Concepts using forms of AI are definitely the way forward.
-
I think it’s a really clever strategy to “let the AI do all the work”. I’m certainly aware of the fact that many person-moons were spent on the development of the original brain scan image enhancement network, but to “coax” this network into doing work on something completely different (insert funny Monty Python video here) like the BMS KTO tileset ist just genius
Oh, it was just one me-month for this. I started working on this to as an idea to save space (we have petabytes of images for structural connectomics and it’d be nice to not have to store everything at full resolution, or at least not have to fetch them at full resolution – storage is expensive), but then realised that the requirements for EM and BMS Tileset were similar:
- Dataset is very limited domain but there’s huge variability within that domain.
- Context awareness is crucial to getting the correct image.
- Good feature localisation that’s stable with respect to semantic segmentation at multiple scales.
- Off-the-shelf loss used by neural nets to supplement the adversarial loss with regularisation are likely to be irrelevant.
Actually, Arty converted KTO tiles into 1k*1k res already,
The change is minor compare to the size and demand from the GPU\CPU to process those in simI did come across that post when I was searching the forums, but I think he was using Topaz (which is for photographs) – I don’t know how well that would / did generalise to terrain textures, though. Do you have them somewhere?
Also, BMS isn’t that demanding even on modest hardware in 2021 (unless it’s in inclement weather or something) – there’s certainly diminishing returns, but I think this is still worth the increase in size (2k x 2k probably won’t be for most people, though, and definitely not for ITO).
Anyone who has manually worked on Terrain tiles knows the time they take and the massive limitations of them due to that. Concepts using forms of AI are definitely the way forward.
The reasonable things that were done to save time (e.g. blending cities and forests with masks, drawing two skinny grey rectangles for a freeway, upsampling 256x256 to 512x512 and working with that as the base and not paying attention to the blocks) are the things that end up causing issues with the net, though there’s ways I can think of around it.
I think it wouldn’t be super hard to make / teach another network to generate tiles based on some sort of semantic encoding (“here’s elevation data, where we want cities, towns, roads, freeways, forests, fields, farms, water – come up with tiles that look reasonable”) IF there are consistent tiles for the network to learn from AND good labelling, but I’m likely being way overoptimistic.
As an unrelated side note, the KTO will never be super realistic for me because I don’t think anybody would take the time to implement the rooftop GOPs (Not to mention the FA-50 Fighting Eagle, though that only entered service in 2012.) Many skyscrapers – including residential high-rises – have AAA batteries on the top. (https://namu.wiki/w/%EB%B9%8C%EB%94%A9%20GOP)
-
Hi Dodam,
How similar is your method to this one:
https://mrl.cs.nyu.edu/projects/image-analogies/ -
@tad:
Hi Dodam,
How similar is your method to this one:
https://mrl.cs.nyu.edu/projects/image-analogies/That’s an ancient website and an ancient method (20 years in machine learning is a verrrry long time) – the paper doesn’t use any convolutions, nonlinear activations, or pooling, which are all hallmarks of modern deep learning. (I don’t know what year these started becoming popular – I know that some of them have existed since the 90s, at least.)
It’s a precursor, but I think about the only similarity is “filters are learned” – this is what my method is ripping off, though I have made pretty significant changes: https://arxiv.org/abs/1809.00219; it’s capable of doing some amazing things for game textures in general.
-
It’s a precursor, but I think about the only similarity is “filters are learned”
No, I fully understand that the link in my post is “ancient”. I wanted just to confirm that I understood your method with some added clarity.
-
Pretty nice initiative this yours, Dodam.
Thanks for sharing and your commitment. As just seems, major changes will happen in theater making in the next future, but until then… oh, who cares, after then, we will see how to apply all what we learned in these years about.
I’m just curious now to see how things will go even with my RTX 2080Ti, “ancient” if compared with your powerful 3090 (good purchase, BTW, well done this too!)
With best compliments and regards.
-
@tad:
No, I fully understand that the link in my post is “ancient”. I wanted just to confirm that I understood your method with some added clarity.
The “analogy” in this sense would be (2m res aerial photography : 0.5m resolution aerial photography) :: (512px terrain tiles : ?), but with a lot more going into the sausage.
I’m doing what’s called a “zero-shot generalisation”, which is just to say that the network never sees the terrain tiles in learning until it’s asked to actually upscale it for production – this is obviously necessary since I don’t have high resolution terrain tiles I can give the network as “this is what it should look like in the ideal case”. This relies on the terrain tiles looking similar to actual aerial photography, though there are techniques to help with generalisation. I might add another network that just focuses on photorealism (“take this 2048x2048 terrain tile, and make it look like a 2048x2048 aerial photograph”) in the future – the way to do that is to use another GAN, though I would really need to sit down and think about what the best way to do that would be since there’s only so many tiles.
Something that’s cool is that the night tiles are also a zero-shot generalisation – the SPUN-GAN has never seen a night tile (high resolution orthoimagery isn’t available for night scenes), but it seems know what to do with night tiles without being explicitly taught.
-
Thanks for sharing and your commitment. As just seems, major changes will happen in theater making in the next future, but until then… oh, who cares, after then, we will see how to apply all what we learned in these years about.
Heh – if nothing else, I hope this can reduce the workload for theatrebuilders. How are they changing?
Also, if you need an accurate OOB for ROK with approximate locations at the battalion level, I’m happy to translate that from what’s available publicly on Korean wikis. There’s also a bunch of spelling errors / border inaccuracies in the KTO.
-
That’s just the conclusion I’d wish to hear, dear Dodam.
And, about your kind proposal, my huge thanks in advance if you don’t bother to do that… it would be simply great and one more valuable gift.
Even more valuable for a mission creating addict as, on my little own, I think I am.With best regards.
-
I’ll probably only be able to do it once the theatre creation is stable, but it’ll be something to do in the future (though I have no clue if the campaign / 3D engine can handle realistic OOBs – the ROKA has 37 divisions and 30 brigades unattached to a division).
-
Looks very good, well done! Will there be any issues for multiplayer use ?
-
Looks very good, well done! Will there be any issues for multiplayer use ?
textures have no effects on mp
-
Looks very good, well done! Will there be any issues for multiplayer use ?
I guess there won’t be crashing issues from what dema has said, but visual cues might look a little different – e.g. you might be able to recognise roads a bit more than the rest of your package, so if you give directions to other people for a visual target relative to that road, it might be just slightly harder to spot the target.
There is also a small chance that the network does something weird (e.g. it sees what’s supposed to be a blue slate roof and turns it into a swimming pool). I don’t think it should be an issue overall though!
-
Job well done……Much appreciated!
-
Okay, I have uploaded Rev. 1 (the screenshots have also been updated)– I don’t think I’ll make significant changes to the architecture now, but more training should help. Try it and let me know what you think! (Note: There’s some artifacts around tile borders of the tiles – they’ll be fixed with the next revision.)
-
The link suffers from its success =).