Speech Research @ UoA
Nau mai, Haere mai

We are a group of researchers who are fascinated by the power of speech and languages. We spend hours discussing about vowels, accents, the way people talk, the way people pause before they talk, how we can use speech to understand peoples’ emotions, language-background, age, if they are healthy and what they had for breakfast (now.. that’s not true!). If you want to meet the team and get to know the specific areas everyone works on – this is the place …

We are a very friendly bunch. So, feel free to contact us via email or say a quick “Hi” if you see us in Auckland.

Dr. Catherine Inez Watson

Professor  in the Electrical, Computer, and Software Engineering department, The University of Auckland.

About me: I joined the Department of Electrical, Computing, and Software Engineering at the University of Auckland in February 2004. Before that I had been at Macquarie University, Sydney, Australia for 8 years. I spent one year as a research fellow in the Department of Information Science at the University of Otago in 1995. I did my undergraduate degree and PhD at the University of Canterbury, both in the Department of Electrical and Electronic Engineering. Between my undergraduate and postgraduate degrees I worked in Medical Physics at the Christchurch Public Hospital.

My research specialty is speech analysis; using both signal processing and acoustic phonetic techniques. My research focus is speech production in machines and people. The two main languages I work in are New Zealand English and te reo Māori.

Fun fact about me: I think spectrograms of speech would be fun to turn into a needle work project.

Webpage: https://unidirectory.auckland.ac.nz/profile/c-watson

Dr. Peter J. Keegan (Waikato-Maniapoto, Ngāti Porou)

Senior Lecturer, Te Puna Wānanga, Faculty of Education and Social Work, The University of Auckland.

About me: I grew up on a dairy farm in central Taranaki. I worked for 8 years as a Telecommunications Technician at NZ Post/Telecom. I joined the University of Auckland in December 2000 for work on an assessment project (https://e-asttle.tki.org.nz/). I completed an undergraduate degree (Linguistics/Māori) at The University of Waikato and postgraduate degrees at The University of Waikato and Victoria University of Wellington.

Research interests: Māori language and education, especially measurement/assessment, the description of Māori (especially sound/lexical change).

Fun fact about me: I edit a website. www.maorilanguage.info

Webpage: https://unidirectory.auckland.ac.nz/profile/p-keegan,http://peterjkeegan.github.io/

 

 

 

Dr. Elaine Ballard

Senior Lecturer, Speech Science Division of the School of Psychology, Faculty of Science, University of Auckland.

About me: I am a NZBC (New Zealand Born Chinese) who grew up here in Auckland back in the time when there were not that many Asians around. I have always been interested in languages so it’s no surprise that I ended up majoring in languages and literature for my BA, MA (Hons) at Auckland and in Linguistics for my PhD at Cornell.

Research interests: Bilingualism, Phonetics and Phonology, Sound Change, Language Variation, Language Acquisition

Fun fact about me: I have a third degree black belt in Aikido.

Webpage: https://unidirectory.auckland.ac.nz/profile/e-ballard

Dr. Justine Hui

Lecturer, Department of Mechanical and Mechatronics Engineering, University of Auckland

 About me: I have a conjoint degree in BE(Hons) in Computer Systems Engineering and BA(Hons) (BA in Linguistics and Japanese and BA(Hons) in Asian Studies) at the University of Auckland. This was followed by several years in industry, while I completed a ME part-time with Dr. Catherine Watson on speech synthesis. I then went on to do my PhD at Sophia University, Japan, at the Speech Communication Lab with Prof. Takayuki Arai on the relationship between hearing deficits and speech/melody perception in elderly listeners. 

After my PhD, I came back to Auckland as a postdoctoral research fellow (PI: A/P Yusuke Hioka) examining how language familiarity affects our understanding of speech in different acoustic environments while being involved in the MPAi (Māori pronunciation aid) project with Catherine, Peter and Dr Danielle Lottridge.

I am currently a lecturer at the Department of Mechanical and Mechatronics Engineering.

My research interests include phonetics and linguistics, psychoacoustics, speech perception and production of hearing impaired listeners and second language speakers, hearing assistive technologies, Audio VR (VR acoustics), learning equity of second language speakers, second language acquisition

Fun fact about me:  I sew to destress. 

Webpage: https://unidirectory.auckland.ac.nz/people/profile/justine-hui; https://ctjustinehui.github.io/

Dr. Jesin James

Senior Lecturer in the Electrical, Computer, and Software Engineering department, The University of Auckland.

About me: I strongly believe that if you can combine your passion with your career, you will never feel that you are working. Engineering offers the possibility to combine your passion with technology and make a difference. I enjoy languages, music and interacting with people – and being a lecturer in Electrical, Computer, and Software Engineering working on speech technology for languages is definitely tying everything up nicely. I am originally from Kerala, India. Being a multilingual and moving to a new country has made me appreciate languages and how language knowledge can help me contribute to technology development for those languages

My main research areas are speech signal processing, under-resourced languages, machine learning and engineering education.

Fun fact about me: I love thrifting and travelling.

Webpage: https://unidirectory.auckland.ac.nz/profile/jjam194

Nicholas Eng

PhD student at the Department of Mechanical Engineering.

About me: I am investigating the creation of synthetic voices using noisy speech samples.

My main research areas are speech synthesis, speech perception, signal processing, machine learning

Fun fact about me: I dab into music production from time to time

Webpage: https://unidirectory.auckland.ac.nz/people/profile/neng668

Brooke Ross

PhD Student, Department of Speech Science in the School of Psychology, Faculty of Science

About me: I moved to Auckland from Nelson-Tasman in 2013 to complete a BA in Linguistics and Media studies. I went on to do my MA in Linguistics at Victoria University of Wellington in 2018. I returned to UoA in 2019 to start my PhD research supervised by Dr Elaine Ballard and Associate Professor Catherine Watson. My PhD research research looks at sound change in the New Zealand English vowel space, with a particular focus on sound changes happening in Auckland.

Research interests: New Zealand English, Acoustic Phonetics, Sound Change and Variation.

Fun fact about me: Since so much of my work is indoors, I spend most of my free time outdoors, Swimming, Running, Gardening, etc.

Isabella Shields (Ngāti Porou)

PhD student at the Department of Electrical, Computer, and Software Engineering

About me: Originally from the deep south (Dunedin), I moved to Auckland in 2015 to study. I completed my conjoint BA in French and BE(Hons) in Electrical Engineering in 2019.

I am investigating the acoustics of te reo Māori /r/. I also work as a Research Assistant.  My areas of interest are Māori phonetics, acoustic phonetics, sound change.

Fun fact about me:  I collect hobbies; sewing, travelling, painting, drawing, embroidery, playing guitar, lego, rewatching the same TV shows, and most recently, needle felting.

Webpage: https://unidirectory.auckland.ac.nz/profile/ishi836

Clara Zhang

PhD student, Department of Mechanical and Mechatronics Engineering

About me: I received my B.E. in Mechatronics Engineering in 2021 at the University of Auckland, New Zealand. From 2021, I am pursuing my PhD degree in Mechatronics Engineering, specializing in speech and acoustic signal processing. My PhD research focuses on speech enhancement for non-native listeners under different acoustic environments, under the supervision of Associate Professor Yusuke Hioka, Dr Justine Hui, and Associate Professor Catherine Watson.

My research interests include signal processing, speech perception of second-language listeners, phonetics, and speech enhancement.

Fun fact about me: I learned Japanese by listening to Japanese songs and watching variety shows.

Himashi Rathnayake

PhD student, Department of Electrical, Computer, and Software Engineering

About me: In 2022, I graduated from the University of Moratuwa, Sri Lanka, with a BSc (Hons) in Computer Science and Engineering. I am currently pursuing my PhD under the supervision of Dr. Jesin James and Dr. Catherine Watson. My research is focused on creating a culturally congruent speech emotion recognition system for te reo Māori.

My research interests include speech emotion recognition, and natural language processing.

Fun fact about me: I enjoy reading novels and listening to music from time to time.

Ken Liu

Masters student, Department of Electrical, Computer, and Software Engineering

About me: I am currently doing a Master of Engineering in the field of multi-modal emotion recognition. This project was in progress under Jesin James.

My research interests include computer vision, speech signal processing, emotion recognition, and machine learning.

Fun fact about me: I love painting and traveling.

Henry An

Masters student, Department of Electrical, Computer, and Software Engineering

About me: I studied a B.E. in software engineering at UoA from 2019 to 2022. I am currently doing a Masters of Engineering on visualising TTS training under Jesin James.

Research interests: Speech technology, machine learning

Fun fact about me: I have never broken a bone.

Itay Ben-Dom

PhD student, Department of Electrical, Computer, and Software Engineering

About me: My PhD research focuses on speech emotion recognition in Parkinsonian speech using machine learning, with special focus on the glottal signal.

Research interests: Speech emotion recognition, signal processing, machine learning

Epri Wahyu Pratiwi

PhD student, Department of Mechanical and Mechatronics Engineering

About me:
I’m currently working on my Ph.D. in Mechatronics Engineering at the University of Auckland, under the supervision of Dr. Justine Hui and Associate Professor Yusuke Hioka. My academic journey started with a Master’s in Electrical Engineering in National Central University (NCU) Taiwan and a Bachelor’s in Engineering Physics in Institut Teknologi Sepuluh Nopember Surabaya (ITS), Indonesia.
My main focus is on understanding how people with bimodal and bilateral cochlear implants experience listening. This research, part of engineering and hearing science, could make a real difference in improving technology for those with hearing loss. I’ve also worked as a Research Assistant in Academia Sinica Taipei, where I implement speech enhancement on cochlear implants simulation and used deep learning for assessing music quality in hearing aids.

Fun fact about me:
Beyond academia, I am the founder of Climate Rangers Jakarta, a young group community I initiated in May 2017. Through this community, I am dedicated to providing young people in Jakarta with a better understanding of climate change.

Qing Guan

ME student, Department of Electrical, Computer and Software Engineering

About me:

I graduated from the University of Auckland in 2023 and am currently pursuing my Master’s degree under the supervision of Dr. Catherine Watson and co-supervision of Dr. Justine Hui. My research focuses on how body positions affect speech features.

My research interests include signal processing and speech analysis.

Fun fact about me: I enjoy playing piano and drums and consider myself a music enthusiast.

Anthony White

ME student, Department of Electrical, Computer and Software Engineering

Alumni

Daniel Cho

MSc student, Department of Electrical, Computer, and Software Engineering

About me: My Masters research focused on the various emotions within speech and methods to detect them.

My research interests would be within emotion and speech acoustic analysis.

Fun fact about me: In my spare time I enjoy tinkering with cars and racing them

Binu Abeysinghe

Masters student, Department of Electrical, Computer, and Softwaree Engineering

About me: I am a recent post-graduate who has begun a master’s thesis in the area of natural language processing and machine learning. My part 4 project involved a detection and classification end-end system for the secondary human emotions in speech. This was done under the supervision of Jesin James and Catherine Watson, as well as with the aid of my good friend, Gavin Cao as my project partner.

My research interests are synthetic accent detection and how to qualitatively access synthetic voices for under-resourced languages.

Fun fact about me: I helped make the website you are reading this on.

Webpage: https://binu.nz/

Mohammad Hassaan Mirza

 Masters student, Department of Electrical, Computer, and Software Engineering

About me: I completed my Bachelors Honours in Computer Systems Engineering in 2020, with my Part IV Honours Project being to develop a system for automatically transcribing classical piano music to sheet music, and measuring the accuracy of the transcription. The project was conducted under Dr.Catherine Watson and Itay Ben-Dom, in collaboration with fellow student Kimberley Evans-Parker.

I am currently doing my Masters of Engineering under Catherine Watson and Jesin James. My project focuses on implementing a system to identify emotions in speech through the use of machine learning, building upon previously done work in identifying valence and arousal levels in speech. I hope to also be able to represent the changing of valence and arousal levels through data visualisation.

My research intrests include Signal processing, machine learning, and emotion recognition

 Fun fact about me: I love to sketch and write, and often use them as ways to wind down in times of stress.