Gary Strong

Speech Prostheses as a Metaphor for Verbal Short-term Memory

  • Gary W. Strong is Program Director for Interactive and Intermedia Technologies at the National Science Foundation, and Coordinator for the NSF on the National Science and Technology Council Committee on Computing, Information, and Communication.

  • Lap-top computers with touch-screens and speech synthesizers can be effective prostheses for people who are unable to speak. Observations of children who have never been able to speak due to physical disability indicate that their synthetically-produced speech is slower than real time speech due to the complexity of language selection from a touch-screen and that their sentences are initially "telegraphic" in grammatical style. Nevertheless, synthetic speech is better than no speech in nearly all situations. But, even more important, remarkable serendipitous benefits have been observed, benefits never anticipated, let alone predicted, by speech therapists.

  • Speech prostheses through dedicated programs on laptops are given some people who are born without the capacity to speak in order to enable them to converse and express their needs. However, observation shows that sometimes in the beginning an inordinate amount of time is spent by users playing with the device by themselves rather than employing it to communicate with others. However, long-term observation suggests that this activity is more than merely play. The way the device is employed resembles an outward expression of verbal short-term or working memory. The user selects words and constructions for himself or herself rather than for others in a way that suggests a rehearsal or recall instead of an intent to communicate with others. If this is true, the computer speech prosthesis may be a standing-in for a verbal short-term memory that was never developed because the users are congenitally without speech.

  • If it is true that such users do not have a verbal short-term memory, they will be unable to internally speak to themselves in a way that normal speakers take for granted. Without this ability, it is possible that the sense of self is impoverished in comparison to that of normal speakers. Therefore, initial behavior with a speech prosthesis may be that associated with discovering for the first time that one has something to say and that it can be sculpted before communication takes place. In this way, the device serves as not only a metaphor for the internal speech process, but as an actual external replacement for an important short-term working memory process that plays a role in the developing sense of self.

    References:

    Strong, G.W. (1994) Augmentation of self-reflective activity for non-speakers. Proceedings of IWHIT94, International Workshop on Human Interface Technology 94, Aizu-Wakamatsu, Japan: The University of Aizu, Pp. 96-99. Can be downloaded from URL: Postscript Doc

    Edwards, A. D. N., (Ed.) (1995). Extra-Ordinary Human-Computer Interaction: Interfaces for Users with Disabilities. Cambridge Series on Human-Computer Interaction. New York, Cambridge University Press. Described at URL: Description

    General description of Short-term (Working) Memory and reference at URL: Working memory

    General reference on assistive technology Ried, S.; Strong, G.W.; Wright, L.; Wood, A.; Goldman, A. & Bogen, D. (1995) Computers, assistive devices, and augmentative communication aids: Technology for social inclusion. Journal of Head Trauma Rehabilitation 10(5): 80-90.

    Pictures of the Dynavox and downloadable demonstration software are available at URL: Dynavox

    Picture of a new competitor product at URL: picture

    Description of a competitor for traditional computer platforms at URL: Alt platform

    Report calling for interface research for people with disabilities. More Than Screen Deep: Toward an Every-Citizen Interface to the Nation's Information Infrastructure. NATIONAL ACADEMY PRESS, Washington, D.C. 1997. More than Screen Deep