![Page 1: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/1.jpg)
Real-Time Speech Recognition Subtitling in Education
Respeaking 2009
Dr Mike Wald University of Southampton
![Page 2: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/2.jpg)
Traditional supports create
a reliance on intermediaries
![Page 3: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/3.jpg)
Finding experienced or qualified interpreters, note-takers or re-speakers at university level is difficult
![Page 4: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/4.jpg)
Speech Recognition: real time access to spoken language
![Page 5: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/5.jpg)
Speech Recognition: also supports note taking
![Page 6: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/6.jpg)
SR in classrooms is VERY DIFFICULT!
• Star Trek expectations
• Special vocabulary
• Spontaneous speech not writing
• Dialogue and interaction
• Andtherearenospacesbetweenwordswhenpeo
pletalksoitisunclearwherewordsbeginandend
![Page 7: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/7.jpg)
How to Wreck a Nice Beach You Sing Calm Incense
![Page 8: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/8.jpg)
This is a demonstration of the problem of the readability of text created by commercial speech recognition software used in lectures they were designed for the speaker to dictate grammatically complete sentences using punctuation by saying comma period new paragraph to provide phrase sentence and paragraph markers when people speak spontaneously they do not speak in what would be regarded as grammatically correct sentences as you can see you just see a continuous stream of text with no obvious beginnings and ends of sentences normal written text would break up this text by the use of punctuation such as commas and periods or new lines by getting the software to insert breaks in the text automatically by measuring the length of the silence between words we can improve the readability greatly
![Page 9: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/9.jpg)
This is a demonstration of the problem of the readability of text created by commercial speech recognition software used in lectures
they were designed for the speaker to dictate grammatically complete sentences using punctuation by saying comma period new paragraph to provide phrase sentence and paragraph markers
when people speak spontaneously they do not speak in what would be regarded as grammatically correct sentences
as you can see you just see a continuous stream of text with no obvious beginnings and ends of sentences
normal written text would break up this text by the use of punctuation such as commas and period or new lines
by getting the software to insert breaks in the text automatically by measuring the length of the silence between words we can improve the readability greatly
![Page 10: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/10.jpg)
1998 LIBERATED LEARNING Pilot Project
Today we will be discussing applied interactions in psychotherapy and how it related to Canadian law I will be covering chapters 7 8 and nine in preparation for next week;s midterm exam
But first are they are questions about what we discussed yesterday lets move forward by asking the following question how does
![Page 11: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/11.jpg)
It gives you something to compare your notes to and
if you miss a class the notes are still accessible
![Page 12: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/12.jpg)
It’s very helpful when the lecturer moves on while
you’re still writing down a point as you can look at
the screen
![Page 13: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/13.jpg)
It’s forced me to be more reflective with my own
teaching style and approach
![Page 14: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/14.jpg)
It makes me ask myself what is teaching, why am I teaching this way, is there
a better way?
![Page 15: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/15.jpg)
Initial Research Summary
• Helps students access lectures
• Students thought it had great potential
• Improved teaching but gave teachers extra work
• Challenges: accuracy, readability, and ease of use
![Page 16: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/16.jpg)
Accuracy & UnderstandingStop / Proceed with Caution / GO
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
![Page 17: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/17.jpg)
28 KB/s speech signal
320 b/s (240 words/minute) Real Time Editor
Corrects 15 errors/minute
320 b/s (240 words/minute)
![Page 18: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/18.jpg)
Speaker(s)
uncorrected words
Human intermediar(y/ies) can interact with a post processing system through the selection and correction of errors.
SR Systemusing voice & language models
possible feedback?
Post-processing system for automatic error identification and correction
Corrected words
SR Post-processing Enhancement
![Page 19: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/19.jpg)
Text with Errors
Text without Errors
Post-processing Enhancement
using statistical (including alternates lists and confidence levels), linguistic, context, phonemic, visual, signal and noise information
![Page 20: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/20.jpg)
Text with Errors
Text without Errors
Post-processing Enhancement
Topic Detection
&
Machine Translation
![Page 21: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/21.jpg)
Text can reduce the memory demands of spoken language
Speech can better express subtle emotions
Images can communicate moods, relationships and complex information holistically.
Multimedia
![Page 22: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/22.jpg)
Easy to find WHOLE of recording but NOT a PART
Analogy
Text book with front cover but no contents page, index or page numbers
![Page 23: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/23.jpg)
![Page 24: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/24.jpg)
![Page 25: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/25.jpg)
synchronised images (e.g. Slides)user created synchronised bookmarks, tags, notes with associated links to other resources
audio / video
synchronised text captions
multimedia start time multimedia end time
![Page 26: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/26.jpg)
Synchronised
Multimedia
Collaborate/Reflect
Search
Reason/Summarise
Organise/Index
Notes
Tags
Bookmarks
Text Captions
Images/Slides
Video
Audio
&
Links
![Page 27: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/27.jpg)
![Page 28: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/28.jpg)
![Page 29: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/29.jpg)
![Page 30: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/30.jpg)
![Page 31: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/31.jpg)
Synote Supports learners to …
• search text and replay sections
• read transcript rather than listen to speech
• read text of slide images • insert bookmark to continue later
• tag/highlight sections (e.g. not understood)
• add synchronised notes
• link to other web pages/resources
![Page 32: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/32.jpg)
Synote Support teachers to…
•index and tag their recordings
•provide synchronised slides and captions
•respond to learner tags
•link to web pages/resources •link to sections of existing multimedia
![Page 33: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/33.jpg)
Synote Demo
![Page 34: Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton](https://reader036.vdocuments.mx/reader036/viewer/2022062516/56649e555503460f94b4ca7c/html5/thumbnails/34.jpg)
Questions ?
www.synote.org