Hi, I am currently working on a personal project to automate captioning of Jap AV files from Japanese to english. I am a native english speaker hence ability to decipher Japanese as a language will be an issue. Is there anyone here preferably with some knowledge of the Japanese language who wishes to collaborate? Essentially the process will be 1. To identify a japanese AV title 2. Me to caption it and provide 2 files: Jap.srt and Eng.srt 3. You to proof read it.

Sample English SRT 1 00:00:01,66 --> 00:00:41,75 Hi hi yeah yeah yeah yeah it's hot yeah yeah on the way home yeah 2 00:00:42,13 --> 00:00:51,95 Also, I'm still in Japan after eating something like that, so I went home during the summer vacation. 3 00:00:51,95 --> 00:00:57,24 I think I've just been around at home, but I'm glad 4 00:00:57,52 --> 00:01:04,62 You're at home Yeah Well, we're having a festival meeting from now on 5 00:01:04,62 --> 00:01:11,77 I'm gonna go to make rice for my grandfather on the back 6 00:01:11,77 --> 00:01:18,15 It's not free anyway, that old man is also old, and sometimes he has a wait-and-see

Sample Jap SRT 1 00:00:01,66 --> 00:00:41,75 ひ ひ ふ ん ええ うん ござい ます うん は あ 暑い わ ね 帰り に うん ああ 2 00:00:42,13 --> 00:00:51,95 また そんな なんか 食べ て まだ 日本 は ある 私 は だ から ね 全く 夏休み で 帰省 3 00:00:51,95 --> 00:00:57,24 し た と 思え ば 家 で ごろごろ し て ばかり な ん だ から けど 良かっ た 4 00:00:57,52 --> 00:01:04,62 あんた が 家 に い て くれ て うん まあ これから 祭 の 打ち合わせ が あっ て あんた 5 00:01:04,62 --> 00:01:11,77 裏 の お 爺 ちゃん の ご飯 作り に 行っ て き な よ なんで 私 が いい 6 00:01:11,77 --> 00:01:18,15 じゃ ない どうせ ヒマ でしょ あの お 爺 ちゃん も 年 だ し え き たまに 様子見 が

The idea may be cool, but I doubt anyone here knows Japanese

Yea. Jus giving it a shot. Any popular translation besides Jap to Eng? I can give it a shot to translate it to English.

Hello, I've learnt Japanese for 2 years at school (as well as German and Spanish), but I forgot most of them. I'm totally unable to get a dialog, however, I can still catch some words. For JAV, it's often enough to notice the google speech recognition is still unable to get Japanese. Your project is very interesting, you should test it by auto-captioning a movie which already has subtitles and compare the results. I think I did something similar: ffmpeg + google api. I tried German, Italian and Japanese. The latter was by far the worst result. German & Italian results wasn't so bad, enough to understand the main topic, but needed a lot of corrections to make the dialogs fully understandable. Actually, your project may be the same some of us (non native english speaker) did to caption English movies. As I said before, you should try on an already subtitled movie, or on a short movie sample first (if a very short sample, I can help sometimes, depending on my free time). Else, in adult movies, beside English and Japanese, German and Italian are very popular languages.

Yea will test tat direction out. That said biggest issue with Jap to Eng is really language construct sov Vs svo structure. For Spanish, Italian etc, accuracy is definitely better.

> language construct sov Vs svo structure. Yes, but not only. Speech recognition doesn't mark silence, punctuation, etc... and Japanese in its writed form doesn't use spaces between words. Thus, you have to check when the dialog ends. Look at index 5 and 6 from your example: 裏 の お 爺 ちゃん の ご飯 作り に 行っ て き な よ なんで 私 が いい じゃ ない どうせ ヒマ でしょ あの お 爺 ちゃん も 年 だ し え き たまに 様子見 が I'm gonna go to make rice for my grandfather on the back It's not free anyway, that old man is also old, and sometimes he has a wait-and-see Now, from what I understand, if I "split" the sentences like this: ...裏 の - お 爺 ちゃん の ご飯 作り に 行っ て き なよ - なんで? - 私 が いいじゃ ない - どうせ ヒマ でしょ あの お 爺 ちゃん も 年 だ し え き たまに 様子見 が... Then the translation becomes: ...on the back. - Don't go to make your grandfather's rice - Why? - I don't like him - Anyway, that old man is also old, and sometimes he waits... So as you can see, even if the speech recognition is perfect (but it's never perfect...) the proof reading is very important and takes a lot of time... From which movie did you take that sample?

that's a interesting perspective. So far I am able to do a speaker identification. However once it's pushed to SRT, that separation of speaker dissappears I am using this JAV as reference to get a sense of the accuracy of my transcription and translation. https://www.avsubtitles.com//movie.php?movid=2214

Using Nima-007, there are 2 ways to process the transcribed file. 1. Via timeline This approach loses the speaker identification 2. Via speaker identification This approach loses the timeline identification but results in a more understanding translation