Cloned Falcon 4 Voices - Add Voice Frags In the ORIGINAL Voices (Long)
-
@mav-jp said in Cloned Falcon 4 Voices - Add Voice Frags In the ORIGINAL Voices (Long):
@tomcattwo said in Cloned Falcon 4 Voices - Add Voice Frags In the ORIGINAL Voices (Long):
@mav-jp
Mav, let me know if you need any help. I’d really like to learn more about how the eval portion of the code works. I can see some immense opportunities. Right now, all I can do is “replace” things. But the whole world opens up if we can change how the code evaluates in-game events.
Regards,
TC2I change the code everyday to create new comms but I have always been limited by existing frags
I had to tweak them , assemble then and sometimes I couldn’t create what I wanted
If the tool works well the first job is to replace the existing bad phraseology by lore realistic ones
Then we could create better comms as
Some parts of the game
Really need it@Mav-jp , yes I already cloned a couple of frags for Mideast128 that sounded like the splices were not real good (examples: “platform” and “furball”) and fixed all voices for frags “2 6” through “6 9” for all voices (using splices). If there are other frags you’d like me to clone in all voices, send me a list and I’ll do them and send you the. wav files.
R/
TC2 -
-
Really interested to see how this progresses…I kinda want to install this locally and mess around with it
-
@jstnj ,
Jump on in! First, I recommend you download Mideast128 Theater and fly around some at different airbases/airstrips to hear what it sounds like in the game - I think you’ll be pleasantly surprised. From my perspective, the clones are certainly good enough that there is no loss of immersion.I’ve provided pretty much everything you need in terms of instructions, download links, and tools needed, so give it a go!
I’d be happy to help out @Mav-jp whenever he is ready to work on improving the sound for the next iterations of Falcon BMS. Having more folks who know how to use the tool can only help. PM me if you have questions.I am contemplating whether or not I want to try to train the synthesizers using a different corpus of readings (LibriTTS) to see if I can get the synthesizer to produce better recognition with some degree of the ability to recognize punctuation. That is about a month-long effort (because I’d have to train the new “baseline” synthesizer to ~300,000 steps from scratch - that alone would take a week, then single-voice train all 14 Falcon voices again), with no guarantee it will be any better at all. But I can’t tackle that until I finish at least the first half of my TE project - it will take me another 2-4 weeks for that.
Regards,
TC2 -
-
@Tomcattwo
Hi Tomcat.
Need some help.
I can extract .wav files from “Falcon.tlk” using “TlkTool”.
But I need to know where can I find the texts that corresponds to every .wav, for example “100.wav” says “May day may day…”, and I need to find the texts that corresponds to a concrete .wav, to can modify and test only certain .wav’s, not all .wav’s.
Example: I want to change a .wav that says “Turning right…”, and I don’t know what .wav number is.And it’s crazy to hear .wav’s one by one until reach the one I need, there are thousands of them. I need to know where are the files related with “Falcon.tlk”, that link the ID’s from for example “100.wav” to “100: Turn right…” on another file. And what program will be needed to do this if aren’t plain texts file.
Thank’s in advance for any information. Nobody can give me this information along the years, to can mod some things on Falcon 4 BMS, mainly to change voices for others that I like much and can understand better.
My objective is to replace some .wav’s on “Falcon.tlk” with others that can be better understood for me, it’s for own purposes.
Note: When recompressing any .wav file to “falcon.tlk” with TlkTool, succeeds, but the .wav file looks bad into “.tlk”, and I can’t keep recompressing all the thousands of files in “falcon.tlk” for every test I run. Does anyone know the author of the TlkTool program or know what to do to get recompression of a single .wav file to work?. Thank’s in advance.
Another Note: And I found the files related to the sound, the list of linked ID’s yet located. The only question finally is how to recompress a single .wav file to “falcon.tlk” without this errors on TlkTool. Thank’s. -
Hello @cchaparro , congratulations (I think) on starting down the road to understanding Falcon BMS’s voice communications! First let’s start with a lesson on what goes into making BMS’s voice reply when it is supposed to do so.
BMS voice responses consist of one or more voice fragments (called “frags” for short) that BMS has to chain together to provide a single response. There are over 3000 frags used in BMS. Frags are listed by their FragID number in a file called fragFile.xml, which is located in the …Falcon BMS 4.37\data\Sounds folder (the same folder that contains Falcon.tlk, and all the other sound files we will consider in this lesson).
For example, when the pilot provides an input that prompts BMS to provide a voice response (for example, if our pilot, callsign “Mustang Four One” inputs “ttttt1” which requests BMS to provide the current atmospheric pressure, QNH), BMS has to first determine if the request was sent on the correct frequency (in this case the approach frequency), then it has to parse the input to identify that “the pilot has just requested to hear QNH information be sent on this frequency”). It then has to evaluate how to put the necessary frags together, using the correct voiceIDs (those are the numerical filenames of the correct .wav files in Falcon.tlk - each .wav has it’s own unique VoiceID which is the numerical part of its filename, and there are 40000 or more unique VoiceIDs in the Falcon.tlk file).
BMS uses the CommFile.xml and evalFile.xml files and internal hardcoding to determine which frags it needs, and their order,
to formulate the response. In our example, it needs to reply:“Mustang Four, Kunsan Approach, QNH is 2984.”
So, it needs:
- a frag for the callsign (“Mustang”)
- one for the flight number (“Four”)
- a frag for the airbase (“Kunsan”)
- a frag for the Agency (“Approach”)
- a frag for the response (“QNH is”)
- and then four frags for the numbers (“two”, “nine”, “eight”, “four”).
Nine frags just for this simple reply!
BMS also has to identify which voice is to be used for the reply. There are fourteen voices in Falcon BMS: voices 0 through 11 are pilot and AWACS/FAC voices. Voice 12 is the female ATC, and Voice 13 is the male ATC.
So every frag must have at least 2, but no more than 14, VoiceIDs associated with it - those are the .wav files. In our example, BMS will need to provide either a female ATC (Voice 12) or a male ATC (Voice 13) VoiceID. It picks one of the two at random to use for the reply. Let’s say it picks the female ATC (Voice 12). So for each of the 9 frags needed for our reply, it needs to select the Voice 12 VoiceID for each frag. How does it know which VoiceID to pick for each frag? The answer is in the magic of the fragFile.xml file, which identifies which VoiceIDs belong to each and every frag.
So now it knows what frags it needs for the response, and which VoiceIDs (.wav files) it needs to play. How does it know what to print on the screen for the reply? The answer is that it can find what to print in the F4Talk.csv file. The F4Talk.csv file is “comma separated variables” file, able to be opened and viewed as a text file using any text file viewer such as Notepad, Notepad+, or TextPad (my personal favorite), or in a spreadsheet file (such as Excel). F4Talk.csv lists every frag by its FragID and what is to be printed for each of the 14 voices, separated by commas. This file can be edited using any text editor, so you can change what you want to see printed. Note: F4Talk.csv only determines what gets printed on the screen. It does not directly affect the audio response information.
Since Falcon BMS know the nine FragIDs and the associated VoiceID (and voice number - Voice 12 in our example), Falcon BMS knows what to print because F4Talk.csv can identify exactly what to print. BMS then generates the response by playing each of the 9 identified VoiceIDs, in order, and prints the line as identified by the FragIDs/Voice positions from F4Talk.csv on the screen!
This is part one of my response. I will try to answer more of your questions in my next response below.
Regards,
Tomcattwo
(VoiceClone) -
@cchaparro
You asked: "The only question finally is how to recompress a single .wav file to “falcon.tlk” without this errors on TlkTool. "TLDR (too long, didn’t read): No, there is not a way I have found yet to recompress a single (or multiple) .wav file(s) to Falcon.tlk without errors. The only way I’ve found that works is compressing every .wav file into a new Falcon.tlk file
The long version:
I have not yet found a way to correctly add or delete a single (or multiple) .wav files to Falcon.tlk. The TlkTool only seems to work correctly if you create an entirely new Falcon.tlk file using every VoiceID .wav file needed. This is unfortunate. What I have had to do is to create every new or updated/corrected .wav file first, and then when I am ready to make a new Falcon.tlk file, I start with the entire set of decompressed 40,000 .wav files, and replace the ones I want to fix/add with the files I made, then compress a new Falcon.tlk using TlkTool.Important note: .wav files used in Falcon.tlk MUST be 16bit, Mono PCM files with 8KHz sampling rate Any other type will not work correctly in Falcon BMS!!
Another really useful tool for the type of work you are doing is TLKTOOLGUI by Khronik. TlkToolGui uses four files to create a TlkToolGui “project”:
- falcon.tlk
- fragFile.bin
- F4Talk.csv - NOTE: You must rename “F4Talk.csv” to “F4Talk_1_0_0.csv” or TlkToolGui will not process it correctly
- EvalFile.bin
You can make EvalFile.bin and fragFile.bin from the .xml files using TlkTool.
Once the project is created, you can view and hear every single VoiceID (Falcon.tlk. wav file) , arranged by Voice number (Voice 0 through Voice 13) and listed by FragID in this easy to read graphical user interface. This makes it very easy to find and identify individual .wav files and fragIDs. So, if you want to find the .wav files for “Mustang” for example, open the project in TlkToolGui, go to any voice, search down the list for “Mustang” and you’ll find the FragID. Look at that FragID in fragFile.xml, and it will list all the VoiceIDs for that frag.
I have not tried to use TlkToolGui to update/correct/fix/compress or decompress Falcon.tlk files. It uses TlkTool to do that work, so it also cannot properly insert a single (or multiple) .wav files into a correctly functioning Falcon.tlk file. I don’t think @Khronik or @lightning have been on the forum for a long time, so I’ve no idea if TlkTool can be fixed to allow for proper insertion of a single (or multiple) .wav files into Falcon.tlk properly.
My process:
-
I start with the latest BMS Version’s KTO sound files. I use TlkTool to decompress the base KTO Falcon.tlk file’s .wav files.
-
I determine exactly which VoiceIDs I need to change (I use huge spreadsheets to keep track of which FragIDs and VoiceIDs I need to change), and create the new .wav files. I use Correntin’s SV2TTC Voice Cloning Toolbox to create new cloned .wav files as I described in detail above so that the voices match the original Falcon voices. The .wav’s are put into 16bit, Mono PCM files with 8KHz sampling rate format (I use Goldwave). They are named using their corresponding VoiceID.wav
-
I update the fragFile.xml and F4Talk.csv files as needed (you only need to correct fragFile.xml if you are adding new frags, for example, adding brand new airbase names - airbase name FragIDs are called directly in the stations+ils.dat file in the …Falcon BMS 4.37\data\campaign folder but the frags and VoiceID/print info info needs to be put into the fragFile.xml and F4Talk.csv)
-
I paste my new .wav files into the decompressed “base” .wav files, replacing when prompted.
-
I compress the entire .wav set into a new Falcon.tlk file.
-
I create a new project in TlkToolGui using the new Falcon.tlk, F4Talk.csv, fragFile.bin and evalFile.bin files, and pray that it works correctly. If it does (HUZZAH!!) then I go test my new .wav files to see how they sound.
-
Then I make sure I have archived the old base soundfiles safely, copy the new soundfiles (Falcon.tlk, F4Talk.csv, fragFile.xml) into …Falcon BMS 4.37\data\Sounds, crank up Falcon, and test the new sounds in game.
It’s a long, complicated process, but it does work if you do it the right way. It takes a LOT of work. Adding new airbase names or changing callsigns requires additional changes to TTSEval.xml and to stations+ils.dat files and (for new callsigns) changes to the .strngs files using Mission Commander. Each new callsign requires 30 new cloned .wav files. Each new airbase name requires 6 new .wav files.
Please feel free to PM me or ask additional questions here.
Regards,
Tomcattwo
(VoiceClone) -
@Tomcattwo
WOW. Don’t know what more to say than… WOWThis was a unique and so impressive tutorial, guide, and a master class of how to use the right tools with the right knowledge to do the right job.
What I’m trying to do, compared with the amazing job that you are doing, is … a grain of sand in the desert.
For now I’ll stop these attempts to recompress independent .wav files until investigate or hope that the TlkTool or something similar will allow this recompression in the near future.
Replace some .wav files and recreate the entire “falcon.tlk” file from the thousands of previously uncompressed .wav files, to verify that they work, it involves hundreds and hundreds of disk read and write operations, and a lot of waiting time between tests.At least, there is a way to do it, which is very positive.
And users to thank enormously for sharing this information, which, as I say, I have been trying to obtain for years without positive results.
Thank’s for all all all this … WOW … really impressive. No words to say. Thank’s thank’s and … Thank’s.
-
@cchaparro
Glad to help. This area is sort of a “speciality” for me. I must pass thanks along to @MeuMestre , whose original work in this area was the key that opened the door for me. I am just paying it forward.I am sure you know this already, but for those that may not, each Add-on theater may have its own …\Sounds folder, which can contain its own unique sound files. This is how we can add new airbase names, for instance, that are unique to the theater. In that case there are additional files that need to be changed besides just Falcon.tlk and F4Talk.csv. That is a lesson for another thread.
ATIS works differently from all of the above, as it uses built-in Microdsoft Voice Recognition software. That is an entirely new lesson for yet another thread.
DM me if you have other questions or suggestions.
Regards,
Tomcattwo
(VoiceClone) -
@Tomcattwo and he can translate thousands of languages.
-
@Tomcattwo said in Cloned Falcon 4 Voices - Add Voice Frags In the ORIGINAL Voices (Long):
Since Falcon BMS know the nine FragIDs and the associated VoiceID (and voice number - Voice 12 in our example), Falcon BMS knows what to print because F4Talk.csv can identify exactly what to print. BMS then generates the response by playing each of the 9 identified VoiceIDs, in order, and prints the line as identified by the FragIDs/Voice positions from F4Talk.csv on the screen!
This thread contains such interesting technical background information! I was wondering based on the above: are the selected frag ids accessible anywhere in shared memory? I would be interested in trying to map them to their texts and generate a synthesized voice externally, as a proof of concept.
Getting the subtitles might work as well, if those are easier to access.
-
@Ricky
The individual VoiceIDs (i.e., each numbered .wav file that has been compressed into the falcon.tlk file) are already “mapped to their texts”. This is because the F4Talk.csv file (which is a text file which maps all the “subtitles” to each FragID), aligns each phrase to its corresponding FragID, and FragFile.xml maps each FragID to the corresponding VoiceIDs it uses.For example, if you open F4Talk.csv using a text editor such as Notepad, Notepad++, Textpad etc., at the very top you will see a “legend” which describes what each line in F4Talk.csv contains. Each line contains 18 values, each separated by a comma.
Fragment,# Voices,Voice 0,Voice 1,Voice 2,Voice 3,Voice 4 (F),Voice 5,Voice 6 (F),Voice 7,Voice 8,Voice 9,Voice 10,Voice 11,Voice 12 (F),Voice 13,Summary,Eval
Voices 4, 6 and 12 are female voices. Voices 0-11 are Pilots/Tanker Boomers, AWACS/JSTARs etc. Voices 12 and 13 are the female and male Air Traffic Controllers.
Which voice does what is mapped in a file called “VoiceRange.dat”, which is in the Campaign folder.
For example, at FragID 2, you will see the line:
2,12,abort mission,mission is aborted,mission abort,abort mission,abort mission,abort mission,mission is aborted,mission abort,abort mission,abort mission,mission abort,mission abort,,,abort mission,
In this FragID (#2) line there are the text of 12 VoiceIDs (12), and the text for each Voice (Voice 00 through Voice 11) are listed. Note that the 12 voices don’t necessarily each say the same phrase. Voices that don’t have a VoiceID.wav file for this phrase (because they don’t need to say it in the game) are simply skipped by not putting anything in between their corresponding commas. In our example, The “abort mission” phrase isn’t needed by the ATCs, so that is why there are 3 “,” toward the end of this line, corresponding to Voice12 (F) and Voice 13, the ATCs. The last is the “Summary” Phrase, which just lets you know what this line’s set of phrases is for - Abort Mission There is a comma after the last Abort Mission, phrase - and that is for the last element, “Eval”, which just wasn’t included in this line (leaving Eval off does not seem to cause any ill effects that I have found).
Then if you open the FragFile .xml, and find FragID 2, it will list:
<Frag id="2"> <Speaker voice="0" tlkId="26" /> <Speaker voice="1" tlkId="27" /> <Speaker voice="2" tlkId="28" /> <Speaker voice="3" tlkId="29" /> <Speaker voice="4" tlkId="30" /> <Speaker voice="5" tlkId="31" /> <Speaker voice="6" tlkId="32" /> <Speaker voice="7" tlkId="33" /> <Speaker voice="8" tlkId="34" /> <Speaker voice="9" tlkId="35" /> <Speaker voice="10" tlkId="36" /> <Speaker voice="11" tlkId="37" /> </Frag>
So you know exactly which VoiceID is recorded with each voice for this set of phrases.
If you were to decompress falcon.tlk (using TlkTool) into it’s 40,000+ .wav files, and listen to 26.wav, you’d hear “Abort Mission” (in Voice 00) and if you were to play 27.wav, you’d hear “Mission is aborted” in a different voice (Voice 01) etc.
But you really don’t have to go to all this trouble to map out 40,000+ .wav files…A 133+ coder named Khronik built a small tool called TalkToolGUI, which does all that work for you! How to use it is described above. Here’s a screenshot of what TlkToolGui looks like on one of the Theater projects I created:
Using TlkToolGUI, you can listen to every phrase in every voice if you wish. TlkToolGUI will even translate the F4Talk.csv to other languages (limited), though I haven’t tried that function myself. Note, however, that I don’t use TlkToolGUI for adding to, deleting from, compressing or decompressing falcon.tlk files. I use TlkTool itself using command line parameters as it’s more controllable and reliable (this is not Khronik’s fault - some stuff in TlkTool works and some stuff doesn’t seem to always work as advertised).
Nevertheless, TlkToolGUI is pretty slick, huh? So, Ricky, you don’t need to do the work, it’s already done for you
If you want a list of every phrase mapped with every VoiceID in a spreadsheet form, please PM me. I might be able to help.
Regards,
Tomcattwo
(VoiceClone) -
Thanks so much for the detailed explanation! I really appreciate it.
I’m sorry I probably didn’t phrase my question clearly. What I’m really looking for is whether the fragIDs can be pulled from shared memory somewhere while they’re being used to generate the audio. What I would like to try is to then use an external program to map them to the values in the csv file. The program could then use the phrase as input for different external text to speech engines.
It’s all just an experiment really. It would be so cool to be able to hear the ATC phrases we know so well spoken by a modern speech engine in real time.
-
@Ricky
Ahh I see. That is sort of the “next evolution” in Falcon Voice. That could mean also “on the fly” translation to different languages too.I am not privy to the code that drives BMS in memory, or even the evaluation code from command to response. So I can be of no help there.
Good luck on your project. Start a new thread in Sound subtopic once you get started
Regards,
Tomcattwo
(VoiceClone) -
Is real time AI voice cloning even worth it? I mean as many times as I’ve used it they often falter in keeping a steady pitch throughout the course of conversation. Not to mention the terrible lags and eventually my actual voice coming out at times (happened 1/10 times, pretty nig ratio I’d say)