Shows
Daily Paper CastTaming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis
🤗 Upvotes: 12 | cs.CV, cs.LG, cs.SD, eess.AS
Authors:
Ho Kei Cheng, Masato Ishii, Akio Hayakawa, Takashi Shibuya, Alexander Schwing, Yuki Mitsufuji
Title:
Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis
Arxiv:
http://arxiv.org/abs/2412.15322v1
Abstract:
We propose to synthesize high-quality and synchronized audio, given video and optional text conditions, using a novel multimodal joint training framework MMAudio. In contrast to single-modality training conditioned on (limited) video data only, MMAudio is jointly trained with larger-scale, readily available text-audio data to learn to generate semantically ali...2024-12-2423 min
Bilingual Harukist ChannelSetting Free the BearsThis is English Japanese bilingual reading of John Irving’s novel translated by Haruki Murakami.2022-01-0805 minBilingual Harukist ChannelAuggie Wren’s Christmas StoryThis is English Japanese bilingual reading of Auggie Wren’s Christmas story. Written by Paul Auster. Japanese translation is Haruki Murakami.2021-12-2505 minBilingual Harukist ChannelCathedralThis is English Japanese bilingual reading of “Cathedral”. Japanese translation is Murakami’s version.2021-12-1805 minBilingual Harukist ChannelWinter DreamsThis is English Japanese bilingual reading of Winter Dreams. Japanese translation is Murakami’s version.2021-12-1005 minBilingual Harukist ChannelThe Long GoodbyeThis is English Japanese bilingual reading of The Ling Goodbye. Japanese translation is Murakami’s version.2021-11-2603 minBilingual Harukist ChannelThe Catcher in the RyeThis is English Japanese bilingual reading of The Catcher in the Rye. Japanese translation is Murakami’s version.2021-11-2004 minBilingual Harukist ChannelThe Headless HawkThis is English Japanese bilingual reading of The Headless Hawk. Japanese part is translated by Murakami.2021-11-1205 minBilingual Harukist ChannelThe Great GatsbyThis is English Japanese bilingual reading of The Great Gatsby. Japanese translation is Murakami’s version.2021-11-0503 minBilingual Harukist ChannelWhat I Talk About When I Talk About RunningThis is English Japanese bilingual reading of Haruki Murakami.2021-10-2904 minBilingual Harukist ChannelAbsolutely on MusicThis is English Japanese bilingual reading of Haruki Murakami.2021-10-2203 minBilingual Harukist ChannelFirst Person SingularThis is English Japanese bilingual reading of Haruki Murakami.2021-10-1704 minBilingual Harukist ChannelThe Yakult Swallows Poetry CollectionThis is English Japanese bilingual reading of Haruki Murakami.2021-10-0804 minBilingual Harukist ChannelCarnavalThis is English Japanese bilingual reading of Haruki Murakami.2021-10-0204 minBilingual Harukist ChannelWith The BeatlesThis is English Japanese bilingual reading of Haruki Murakami.2021-09-2405 minBilingual Harukist ChannelOn a Stone PillowThis is English Japanese bilingual reading of Haruki Murakami2021-09-1903 minBilingual Harukist ChannelCreamThis is English Japanese bilingual reading of Haruki Murakami.2021-09-1104 minBilingual Harukist ChannelMen without womenThis is English Japanese bilingual reading of Haruki Murakami.2021-09-0304 minBilingual Harukist ChannelSamsa in LoveThis is English Japanese bilingual reading of Haruki Murakami.2021-08-2805 minBilingual Harukist ChannelScheherazadeThis is English Japanese bilingual reading of Haruki Murakami.2021-08-1404 minBilingual Harukist ChannelAn Independent OrganThis is English Japanese bilingual reading of Haruki Murakami.2021-08-0604 minBilingual Harukist ChannelYesterdayThis is English Japanese bilingual reading of Haruki Murakami.2021-07-3003 minBilingual Harukist ChannelDrive My CarThis is English Japanese bilingual reading of Haruki Murakami.2021-07-2304 minBilingual Harukist ChannelHoney PieThis is English Japanese bilingual reading of Haruki Murakami.2021-07-1603 minBilingual Harukist ChannelSuper-Frog Saves TokyoThis is English Japanese bilingual reading of Haruki Murakami.2021-07-0904 minBilingual Harukist ChannelThailandThis is English Japanese bilingual reading of Haruki Murakami.2021-07-0204 minBilingual Harukist ChannelAll God’s Children Can DanceThis is English Japanese bilingual reading of Haruki Murakami.2021-06-2505 minBilingual Harukist ChannelLandscape with FlatironThis is English Japanese bilingual reading of Haruki Murakami.2021-06-1803 minBilingual Harukist ChannelUFO in KushiroThis is English Japanese bilingual reading of Haruki Murakami.2021-06-1104 minBilingual Harukist ChannelA Shinagawa MonkeyThis is English Japanese bilingual reading of Haruki Murakami.2021-06-0404 minBilingual Harukist ChannelThe Kidney-Shaped Stone That Moves Every DayThis is English Japanese bilingual reading of Haruki Murakami.2021-05-2803 minBilingual Harukist ChannelWhere I’m Likely To Find ItThis is English Japanese bilingual reading of Haruki Murakami.2021-05-2103 minBilingual Harukist ChannelHanalei BayThis is English Japanese bilingual reading of Haruki Murakami.2021-05-1404 minBilingual Harukist ChannelChance TravellerThis is English Japanese bilingual reading of Haruki Murakami.2021-05-0704 minBilingual Harukist ChannelFireflyThis is English Japanese bilingual reading of Haruki Murakami.2021-04-3003 minBilingual Harukist ChannelCrabsThis is English Japanese bilingual reading of Haruki Murakami.2021-04-2303 minBilingual Harukist ChannelThe Ice ManThis is English Japanese bilingual reading of Haruki Murakami.2021-04-1604 minBilingual Harukist ChannelThe Rise and Fall of Sharpie CakesThis is English Japanese bilingual reading of Haruki Murakami.2021-04-0903 minBilingual Harukist ChannelTony TakitaniThis is English Japanese bilingual reading of Haruki Murakami.2021-04-0203 minBilingual Harukist ChannelThe Year of SpaghettiThis is English Japanese bilingual reading of Haruki Murakami.2021-03-2604 minBilingual Harukist ChannelThe Seventh ManThis is English Japanese bilingual reading of Haruki Murakami.2021-03-2003 minBilingual Harukist ChannelNausea 1979This is English Japanese bilingual reading of Haruki Murakami.2021-03-1203 minBilingual Harukist ChannelA “Poor Aunt” StoryThis is English Japanese bilingual reading of Haruki Murakami.2021-03-0504 minBilingual Harukist ChannelMan-Eating CatsThis is English Japanese bilingual reading of Haruki Murakami.2021-02-2604 minBilingual Harukist ChannelDabchickThis is English Japanese bilingual reading of Haruki Murakami.2021-02-1902 minBilingual Harukist ChannelA Perfect Day For KangaroosThis is English Japanese bilingual reading of Haruki Murakami.2021-02-1203 minBilingual Harukist ChannelHunting KnifeThis is English Japanese bilingual reading of Haruki Murakami.2021-02-0504 minBilingual Harukist ChannelA Folklore For My GenerationThis is English Japanese bilingual reading of Haruki Murakami.2021-01-2906 minBilingual Harukist ChannelThe MirrorThis is English Japanese bilingual reading of Haruki Murakami.2021-01-2204 minBilingual Harukist ChannelAIRPLANEThis is English Japanese bilingual reading of Haruki Murakami.2021-01-1504 minBilingual Harukist ChannelThe New York Mining DisasterThis is English Japanese bilingual reading of Haruki Murakami.2021-01-0804 minBilingual Harukist ChannelBirthday GirlThis is English Japanese bilingual reading of Haruki Murakami.2021-01-0204 minBilingual Harukist ChannelBlind Willow, Sleeping WomanThis is English Japanese bilingual reading of Haruki Murakami.2020-12-2502 minBilingual Harukist ChannelThe Elephant VanishesThis is English Japanese bilingual reading of Haruki Murakami.2020-12-1805 minBilingual Harukist ChannelThe SilenceThis is English Japanese bilingual reading of Haruki Murakami.2020-12-1104 minBilingual Harukist ChannelKinoThis is English Japanese bilingual reading of Haruki Murakami2020-12-0408 minBilingual Harukist ChannelThe Last Lawn of the AfternonThis is English Japanese bilingual reading of Haruki Murakami.2020-11-2704 minBilingual Harukist ChannelThe Dancing DwarfThis is English Japanese bilingual reading of Haruki Murakami.2020-11-2004 minBilingual Harukist ChannelA Slow Boat To ChinaThis is English Japanese bilingual reading of Haruki Murakami.2020-11-1305 minBilingual Harukist ChannelTV peopleThis is English Japanese bilingual reading of Haruki Murakami.2020-11-0604 minBilingual Harukist ChannelA WindowThis is English Japanese bilingual reading of Haruki Murakami.2020-10-3004 minBilingual Harukist ChannelFamily AffairThis is English Japanese bilingual reading of Haruki Murakami.2020-10-2303 minBilingual Harukist ChannelThe Little Green MonsterThis is English and Japanese bilingual reading of Haruki Murakami.2020-10-1705 minBilingual Harukist ChannelBARN BURNINGThis is English Japanese bilingual reading of Haruki Murakami.2020-10-1004 minBilingual Harukist ChannelLederhosenThis is English Japanese bilingual reading of Haruki Murakami.2020-10-0205 minBilingual Harukist ChannelThe Fall of the Roman Empire, The 1881 Indian uprising, Hitler’s invasion of Poland, and the realmThis is English Japanese reading of “The Fall of the Roman Empire, The 1881 Indian uprising, Hitler’s invasion of Poland, and the realm of raging winds” written by Haruki Murakami2020-09-2604 minBilingual Harukist ChannelSleepThis is English Japanese bilingual reading of Haruki Murakami.2020-09-1903 minBilingual Harukist ChannelOn Seeing the 100% Perfect Girl One Beautiful April MorningThis is English Japanese bilingual reading of Haruki Murakami.2020-09-1205 minBilingual Harukist ChannelThe Kangaroo communiqueThis is English Japanese bilingual reading of Haruki Murakami.2020-09-0504 minBilingual Harukist ChannelThe Second Bakery AttackThis is English Japanese bilingual reading of Haruki Murakami.2020-08-2905 minBilingual Harukist ChannelThe wind-up bird and Tuesday’s womenThis is English Japanese bilingual reading of Haruki Murakami.2020-08-2205 minBilingual Harukist ChannelKilling CommendatoreThis is English Japanese bilingual reading of Haruki Murakami.2020-08-1504 minBilingual Harukist ChannelCOLORLESS TSUKURU TAZAKI AND HIS YEARS OF PILGRIMAGETHIS IS ENGLISH AND JAPANESE BILINGUAL READING OF HARUKI MURAKAMI.2020-08-0804 minBilingual Harukist Channel1Q84This is English Japanese bilingual reading of Haruki Murakami2020-08-011h 22Bilingual Harukist ChannelAfrerDarkThis is English Japanese bilingual reading of Haruki Murakami.2020-07-2504 minBilingual Harukist ChannelKafka on the ShoreThis is English-Japanese bilingual reading of Haruki Murakami.2020-07-1804 minBilingual Harukist ChannelSputonik SweetheartThis is English-Japanese bilingual reading of Haruki Murakami2020-07-1104 minBilingual Harukist ChannelThe Wind-Up Bird ChronicleThis is bilingual reading of Haruki Murakami2020-07-0404 minBilingual Harukist ChannelSouth of boarder, West of sunThis is bilingual reading of Haruki Murakami.2020-06-2705 minBilingual Harukist ChannelNorwegian WoodThis is bilingual reading of Haruki Murakami.2020-06-2004 minBilingual Harukist ChannelDance Dance DanceThis is bilingual reading of Haruki Murakami.2020-06-1305 minBilingual Harukist ChannelHard-Boiled Wonderland and the End of the worldThis is bilingual reading of Haruki Murakami.2020-05-3012 minBilingual Harukist ChannelA Wild Sheep ChaseThis is bilingual reading of Haruki Murakami’s novel.2020-05-2305 minBilingual Harukist ChannelPinball,1973This is bilingual reading of Haruki Murakami’s novel.2020-05-1603 minBilingual Harukist ChannelHear the Wind SingThis is bilingual reading of Haruki Murakami’s novel.2020-05-0605 min