Jen-Tzung Chien received his Ph.D. degree in electrical engineering from National Tsing Hua University, Hsinchu, Taiwan, in 1997, and is currently the Lifetime Chair Professor in National Yang Ming Chiao Tung University, Hsinchu, Taiwan. He has authored more than 250 peer-reviewed articles in machine learning, deep learning, and Bayesian learning with applications on speech and natural language processing, and three books including Bayesian Speech and Language Processing, Cambridge University Press, 2015, Source Separation and Machine Learning, Academic Press, 2018, and Machine Learning for Speaker Recognition, Cambridge University Press, 2020. He was a Tutorial Speaker of AAAI, IJCAI, ACL, KDD, ICASSP, COLING and Interspeech. He received the Best Paper Award in IEEE Workshop on Automatic Speech Recognition and Understanding in 2011, and IEEE International Workshop on Machine Learning for Signal Processing in 2023.

Spoken dialogue systems have become crucial to build a wide range of virtual assistants for customer service, entertainment and health. These systems are composed of various components including automatic speech recognition, text-to-speech, and natural language generation, which involve delicate multimodal machine learning towards multilingual generative models. This talk will focus on state-of-the-art generative models in individual components and address how the pre-trained foundation models are utilized to re-shape the architecture via adapters, re-function the foundation via prompting and re-program the dialogue via flow control. We will also explore the challenges and opportunities through learning and integrating these components into a comprehensive conversation system.

Chin-Hui Lee is a professor at School of Electrical and Computer Engineering, Georgia Institute of Technology. Before joining academia in 2001, he had accumulated 20 years of industrial experience ending in Bell Laboratories, Murray Hill, as the Director of the Dialogue Systems Research Department. Dr. Lee is a Fellow of the IEEE and a Fellow of ISCA. He has published 30 patents and about 600 papers, with more than 55,000 citations and an h-index of 80 on Google Scholar. He received numerous awards, including the Bell Labs President's Gold Award for speech recognition products in 1998. He won the SPS's 2006 Technical Achievement Award for “Exceptional Contributions to the Field of Automatic Speech Recognition''. In 2012 he gave an ICASSP plenary talk on the future of automatic speech recognition. In the same year he was awarded the ISCA Medal in Scientific Achievement for “pioneering and seminal contributions to the principles and practice of automatic speech and speaker recognition''. His two pioneering papers on deep regression accumulated over 2200 citations and won a Best Paper Award from IEEE Signal Processing Society in 2019.

With recent advances in deep neural networks and large pre-trained models, the baseline performances for automatic speech recognition (ASR) of resource-rich languages have improved a great deal. However, only a few applications have been deployed in our daily life. Part of the reason was due to past black-box approaches to ASR without leveraging upon speech knowledge sources, resulting in unsatisfactory recognition results in many situations. On the other hand, knowledge-based approaches, such as automatic speech attribute transcription (ASAT), were not practiced in the machine-learning community due to difficulties to integrate speech knowledge into building ASR system. Since speech attributes are usually language-universal, they serve as an ideal set of fundamental units to build speech models. They also share common distinct features among different languages such that good models can also be established for speech processing to detect speech cues needed to correct unexpected ASR results. In this talk, we will discuss ways the O-COCOSDA community can contribute to developing robust multilingual ASR systems for many resource-limited languages in this region.

Dr. Satoshi Nakamura is a full professor at The Chinese University of Hong Kong, Shenzhen. He is also a professor emeritus at Nara Institute of Science and Technology (NAIST) and Honorarprofessor of Karlsruhe Institute of Technology, Germany. He received his B.S. from Kyoto Institute of Technology in 1981 and Ph.D. from Kyoto University in 1992. He was an Associate Professor in the Graduate School of Information Science at NAIST from 1994-2000. He was Department head and Director of ATR Spoken Language Communication Research Laboratories in 2000-2004, and 2005-2008, respectively, and Vice president of ATR in 2007-2008. He was Director General of Keihanna Research Laboratories and the Executive Director of Knowledge Creating Communication Research Center, National Institute of Information and Communications Technology, Japan 2009-2010. He moved to Nara Institute of Science and Technology as a full professor in 2011. He established the Data Science Center at NAIST and was a director from 2017 to 2021. He also served as a team leader of the Tourism Information Analytics Team at the AIP center of RIKEN Institute, Japan, from 2017-2021. He is currently a full professor at The Chinese University of Hong Kong, Shenzhen, China. His research interests include modeling and systems of spoken language processing, speech processing, spoken language translation, spoken dialog systems, natural language processing, and data science. He is one of the world leaders in speech-to-speech translation research. He has been serving various speech-to-speech translation research projects, including C-Star, A-Star, and the International Workshop on Spoken Language Translation IWSLT. He is currently the chairperson of ISCA SIG SLT (Spoken Language Translation). He also contributed to the standardization of the network-based speech translation at ITU-T. He was a committee member of IEEE SLTC 2016-2018. He was an Elected Board Member of the International Speech Communication Association, ISCA, from 2012 to 2019. He received the Antonio Zampolli Prize in 2012 and retained the title of IEEE Fellow, ISCA Fellow, IPSJ Fellow, and ATR Fellow.

After long research, speech translation technology has reached the level of providing a service using a smartphone. However, there are still various problems in realizing automatic simultaneous interpretation that produces the interpretation output before the end of the utterance. In this talk, I will introduce the recent research activities on automatic simultaneous speech translation and the simultaneous speech translation system developed for IWSLT shared tasks. The talk also includes research activities on speech translation, preserving para-linguistic information, and utilizing the pre-trained Large Language Models.