navigable, customizable tts for algebra

Slide 1

Navigable, Customizable TTS for Algebra

Lois FrankelEducational Testing Service (ETS)

Neil SoifferDesign Science, Inc.

CSUNMarch 1, 2013Copyright 2012 Educational Testing Service. All rights reserved.1Presentation OverviewProject Motivation, Goals, and BackgroundDevelopment to date: discussion and demonstrationResearch StudiesNext Steps2Copyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Lois2Project MotivationText-to-speech (TTS) vs. Math-to-speechLimited choice of syntax/semanticsLimited screen-reader supportLimited or no within-expression navigationDifficult to authorWorks in limited environmentsETS Experience3Handouts are available at: www.atia.org/orlanohandoutsCopyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.LoisGreen font, Voiced GRE3Project GoalsFocus on instruction, not assessmentHigh school algebraClassroom-like synthesized speech style: ClearSpeakCustomization featuresInteractive navigationAuthor in WordPlayback in Word and IE

4Handouts are available at: www.atia.org/orlandohandoutsCopyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Lois4Not Part of ProjectBrailleSupport for other formatsAuthoring by people with visual impairmentSolving math problemsTutoring/CurriculumIntegration with standardized testing

5Handouts are available at: www.atia.org/orlandohandoutsCopyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Lois (other formats = large print, pdf, ebooks)solving math problems = students showing their work, working through solutions, etc. tutoring/curriculum: not supporting online tutoring or curriculum enhancements, etc. 5Behind the Scene: MathMLThe language of math on the webHTML5, eBooksBrowser support & MathJaxDesigned for AccessibilityDaisy playersIE + MathPlayer + AT 6Handouts are available at: www.atia.org/orlandohandoutsCopyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Neil 6What is MathML? x 3

7Handouts are available at: www.atia.org/orlandohandouts

Copyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Neil7Project Software for MathMathML Tools:Authoring: MathType+WordAudio rendering: MathPlayerScreen readerWindow-Eyes (Word & IE)8Copyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Neil MathType is common math editor in Word and elsewhere; MathPlayer is free plug-in for IE and, coming in the future with other browsers and WordFor the project, GW Micro is part of the grant but other AT vendors will support8Many ways to Speak MathHow do you say9Handouts are available at: www.atia.org/orlandohandouts

Copyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Neil: talk about the options for speaking these to get audience to think about how they might say themRe x^(i+1) how to differentiate from x^i + 19Many ways to Speak MathHow do you say10Handouts are available at: www.atia.org/orlandohandouts





Copyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Neil: talk about the options for speaking these to get audience to think about how they might say themRe x^(i+1) how to differentiate from x^i + 114How Math Speech WorksRulesPreferencesExact Speech15Handouts are available at: www.atia.org/orlandohandoutsCopyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Rules try to get it right,preferences allow authors to choose: eg, x to the i vs x to the i powerExact speech for very special cases15ClearSpeakMany ways to speak mathClearSpeak design philosophyOther design philosophiesMathSpeakSimpleSpeech

16Handouts are available at: www.atia.org/orlandohandoutsCopyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.NeilClearSpeak: use common ways of speaking math with pauses or words to disambiguateEx: the fraction with numerator 1 and denominator x+1Mention MathSpeak (follows braille), SimpleSpeech (make simple things simple)

16DemonstrationAuthoringUsing MathType in Word to enter math and use preferencesEntering exact speech in WordPlaybackWord with Window-Eyes17Handouts are available at: www.atia.org/orlandohandoutsCopyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.continue the n+1/n+2 exampleShorten slide to make more basic/generic17Project Goals RecapClearSpeakAuthor in WordAllow setting preference and exact speechPlayback in Word and IEInteractive navigation18Handouts are available at: www.atia.org/orlandohandoutsCopyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Last Neil slide before back to Lois18Creating Expressions and Questions for StudiesLevelDegree of complexityRelevance of speech variationsProvide useful dataTest comprehension, not computationGuide decisions about speech rules, preferences

18Copyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Lois resume hereTypical of expressions found in high school algebra curricula, targeting second half of courseOf sufficient complexity to offer areas of potential ambiguity that a successful speech style should resolve for the listener; that is, not be so simple that students would instantly understand them, no matter how they were spokenOf sufficient simplicity that excessive computation is not required in order to answer the math questions posed about the statement.Speaks significantly differently in at least two of the three speech styles being tested (ClearSpeak (the style we are developing) vs. the existing MathSpeak and Simple Speech styles).Lends itself to math questions that will be diagnostic of aspects in which the speech style succeeds or fails to make a math expression clear, as opposed to those that assess whether the student can do the mathematics required to work with the expression. For example, if a speech style makes the start or end of a fraction clear, a student is more likely to correctly identify the denominator of the fraction than if the style fails to make those boundaries clear.

19Feedback Study #1Compared ClearSpeak, MathSpeak, SimpleSpeechFractions, exponents, parenthesesFocus: was expression understood?Research/Statistical consultation16 HS students: blind or low vision

20Handous are available at: www.atia.org/orlandohandoutsCopyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Beth: completed16 high school students with blindness or low vision, currently enrolled in Algebra 1 or aboveSome students had additional disabilities, including TBI, auditory processing, autism spectrum, other physical disabilitiesLois

15 completed the study; 1 partially

Thanks to Nan Kong and Eric Hansen for experiment design and data analysis

20Results Summary ClearSpeak more understandable than Simple Speech or MathSpeakStudents overall preferred ClearSpeak Students got more answers correct with ClearSpeak than with Simple Speech or MathSpeak21Handouts are available at: www.atia.org/orlandohandoutsCopyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.ClearSpeak student comments:

[Liked]: Fraction, numerator, denominator, and raised to the power of.The phrase raised to the power could just be to the power, raised to the power 2 could just be squared, and the phrase the fraction with numerator and denominator could be made with less words. [power 2 was on purpose because of the nature of the expression, for pedagogical purposes; i.e. idea is to divide and subtract exponents, so using 2 rather than squared desirable]

I would take the times out.Simple Speech: I liked that it used squared and to the to refer to exponents; The way it reads negative exponents should be change (for example, to the negative 3rd instead of super negative 3 end super)I liked that it was organized so I could hear all the parts clearly.I couldnt understand the statement.; It said super for all the exponents and it sounded like it skipped part of it.I wouldnt say superscript. I would say exponent.I would take out the superscript for the exponents. I would put a pause between the end of the exponent and the beginning of the next variable. Use exponent.MathSpeak:I liked that it used cubed to refer to the third power; the superscript and baseline for exponents should be changedI liked that it was clear about what fraction was supposed to be which, and that it was organized in a clear order.I think taking out some of the things like the level deep and if not need the baseline, these made it a really long and confusing read.I would insert multiply or times after baseline and before next variable or I would pause between them; otherwise, it was too fast.Dont like dot. Prefer multiply by because it gives me time.I still dont understand what nested one deep and superscript mean. I think I may understand that superscript means to the power of something and subscript is to the base of something. Like x^7, the 7 is the superscript and x7, the 7 is the subscript. Teachers note: The student then erased that, and rewrote: 5x7

21One Expressions Results22Handouts are available at: www.atia.org/orlandohandoutsP+Very FamiliarSomewhat FamiliarSomewhat UnfamiliarVery UnfamiliarSimple Speech30817MathSpeak31546ClearSpeak*66810

Copyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Essential Differences:Simple Speech: fraction ... x to the ninth...z super negative 7 end super... over...y squared...end fractionMathSpeak: start fraction nested one deep x superscript 9 baseline...z superscript minus 7 over ... end fraction nested one deepClearSpeak: the fraction with numerator x raised to the power 9 times... and denominator y raised to the power 2...

Results (from Nan Kong & Eric Hansen)Students got the ClearSpeak versions right significantly more often (95% confidence level) than the other speech styles.

ClearSpeak student comments:

[Liked]: Fraction, numerator, denominator, and raised to the power of.The phrase raised to the power could just be to the power, raised to the power 2 could just be squared, and the phrase the fraction with numerator and denominator could be made with less words. [power 2 was on purpose because of the nature of the expression, for pedagogical purposes; i.e. idea is to divide and subtract exponents, so using 2 rather than squared desirable]But note nested bug in MathSpeak, and that turned most students off, so MathSpeak would likely have fared better without the bug

I would take the times out.Simple Speech: I liked that it used squared and to the to refer to exponents; The way it reads negative exponents should be change (for example, to the negative 3rd instead of super negative 3 end super)I liked that it was organized so I could hear all the parts clearly.I couldnt understand the statement.; It said super for all the exponents and it sounded like it skipped part of it.I wouldnt say superscript. I would say exponent.I would take out the superscript for the exponents. I would put a pause between the end of the exponent and the beginning of the next variable. Use exponent.MathSpeak:I liked that it used cubed to refer to the third power; the superscript and baseline for exponents should be changedI liked that it was clear about what fraction was supposed to be which, and that it was organized in a clear order.I think taking out some of the things like the level deep and if not need the baseline, these made it a really long and confusing read.I would insert multiply or times after baseline and before next variable or I would pause between them; otherwise, it was too fast.Dont like dot. Prefer multiply by because it gives me time.I still dont understand what nested one deep and superscript mean. I think I may understand that superscript means to the power of something and subscript is to the base of something. Like x^7, the 7 is the superscript and x7, the 7 is the subscript. Teachers note: The student then erased that, and rewrote: 5x7

2223ClearSpeakSimpleSpeakMathSpeakVery familiar48%24%24%Somewhat familiar50%50%31%Not very familiar2%9%23%Very unfamiliar0%16%22%Very easy to understand52%17%16%Somewhat easy to understand33%37%38%Somewhat hard to understand13%24%23%Very hard to understand3%22%22%Very sure understood71%42%39%Somewhat sure understood24%37%33%Not sure understood5%7%11%Definitely did not understand0%14%17%Familiarity and UnderstandingCopyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.23Feedback Study #2Can Prosody help understanding?PausesPitchRate ChangesVolume ChangesRejected pitch and volume changesnon-speech sounds not an optionFocus on pauses, rate-change vs. start/end

24Handouts are available at: www.atia.org/orlandohandoutsCopyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.24Feedback #2 Example vs.

With Pauses / rate changeExpression 1Expression 2With end rootExpression 1Expression 2


Copyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Add audio for both sqrt examples: Lois to add: 4 for sq rt, just 2 for parens: uniform & nth/gradations. break into 2 slides. Possibly make clones first.

Talk about problems with parens and a couple of solutions25Feedback #2 Another ExampleNested Parentheses:

Uniform PausesNon-uniform pausesNon-uniform pauses and First Paren 26Handouts are available at: www.atia.org/orlandohandouts

Copyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Add audio for both sqrt examples: Lois to add: 4 for sq rt, just 2 for parens: uniform & nth/graduations. break into 2 slides. Possibly make clones first.

Talk about problems with parens and a couple of solutions26NavigationSimple char by char mode for beginnersPowerful navigation features for expertsReadDescribeMultiple ways of movingTreeCharacter, Placemarker, SemanticAfter tree move: read or describe10 Placemarkers, 2 Cursors, Where am I?


Copyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Neil:Slide Too long break up or reduce27Participation OpportunitiesBlind or Visually Impaired StudentsTeachers28Handouts are available at: www.atia.org/orlandohandoutsCopyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.Lois or Neil28Contact InformationLois Frankel: [email protected] Soiffer: [email protected] Brownstein: [email protected]: http://www.dessci.com/en/products/mathplayer/

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R324A110355 to the Educational Testing Service. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

29Handouts are available at: www.atia.org/orlandohandoutsCopyright 2012 Educational Testing Service. All rights reserved.Copyright 2012 Educational Testing Service. All rights reserved.

Thank you for attending this session30Handouts are available at: www.atia.org/orlandohandoutsCopyright 2012 Educational Testing Service. All rights reserved.LF: Will we use handouts? Can adapt ones used for AER and/or project meeting if so; also we need to get a fairness review before uploading the ppt. 30

navigable, customizable tts for algebra

Documents