ASSISTIVE COMMUNICATION DEVICE, METHOD, AND APPARATUS
20210390881 · 2021-12-16
Inventors
Cpc classification
G09B5/06
PHYSICS
G09B21/007
PHYSICS
International classification
Abstract
Communication devices, apparatuses, and methods to assist an individual to develop communicative ability are disclosed. A communication device includes a communication module to generate a user interface to display word tiles for selection, compile a sentence upon selection of word tiles, and output the sentence. A first word tile is associated with a word and at least two stimuli of the word, the at least two stimuli including at least two of a text transcription of the word, a visual depiction of the word, and a vocal expression of the word. The communication device includes a training module to collect interaction data, the interaction data including indications of interactions of a user account with the communication module over a plurality of trials, and to progressively diminish at least one of the at least two stimuli of the word of the first word tile based on the interaction data.
Claims
1. A communication device comprising: a communication module to generate a user interface, the user interface to: display word tiles for selection; compile a sentence upon selection of word tiles; and output the sentence; wherein a first word the is associated with a word and at least two stimuli of the word, the at least two stimuli including at least two of: a text transcription of the word, a visual depiction of the word, and a vocal expression of the word; and a training module to collect interaction data, the interaction data including indications of interactions of a user account with the communication module over a plurality of trials, and to progressively diminish at least one of the at least two stimuli of the word of the first word the based on the interaction data.
2. The communication device of claim 1, wherein the at least two stimuli of the word comprise the text transcription of the word and the visual depiction of the word, and wherein the training module is to progressively diminish visibility of the visual depiction of the word.
3. The communication device of claim 2, wherein the training module is further to progressively emphasize visibility of the text transcription of the word.
4. The communication device of claim 1, wherein the at least two stimuli of the word comprises the vocal expression of the word, and wherein the training module is to progressively diminish audibility of the vocal expression of the word.
5. The communication device of claim 1, wherein the at least two stimuli of the word comprises the vocal expression of the word, and wherein the training module is to delay output of the vocal expression of the word.
6. The communication device of claim 1, wherein the training module is to progressively diminish the at least one of the at least two stimuli of the word according to a pre-determined training schedule.
7. The communication device of claim 1, wherein the training module is to progressively diminish the at least one of the at least two stimuli of the word when the interaction data indicates that a user of the user account has improved at building an association between the at least two stimuli of the word.
8. The communication device of claim 1, wherein the communication device comprises a processor and a memory, the memory to store programming instructions executable by the processor to execute the communication module and the training module.
9. A method for assisting an individual to develop communicative ability, the method comprising: providing a communication interface to the individual, the communication interface displaying word tiles for selection by the individual to form a sentence, wherein a first word tile is associated with a word and at least two stimuli of the word, the at least two stimuli including at least two of: a text transcription of the word, a visual depiction of the word, and a vocal expression of the word; facilitating an interaction of the individual with the communication interface to build a plurality of sentences; and progressively diminishing at least one of the at least two stimuli of the word based on the interaction of the individual with the communication interface.
10. The method of claim 9, wherein the at least two stimuli of the word comprise the text transcription of the word and the visual depiction of the word, and wherein the method comprises progressively diminishing visibility of the visual depiction of the word.
11. The method of claim 10, wherein the method comprises progressively emphasize visibility of the text transcription of the word.
12. The method of claim 9, wherein the at least two stimuli of the word comprises the vocal expression of the word, and wherein the method comprises progressively diminishing audibility of the vocal expression of the word.
13. The method of claim 9, wherein the at least two stimuli of the word comprises the vocal expression of the word, and wherein the method comprises delaying output of the vocal expression of the word.
14. The method of claim 9, wherein the method comprises progressively diminish the at least one of the at least two stimuli of the word according to a pre-determined training schedule.
15. The method of claim 9, wherein the method comprises: collecting interaction data, the interaction data including indications of interactions of a user account with the communication module over a plurality of trials; and progressively diminishing the at least one of the at least two stimuli of the word when the interaction data indicates that a user of the user account has improved at building an association between the at least two stimuli of the word.
16. A communication apparatus comprising: a communication display surface displaying word tiles, wherein a first word the is associated with a word and displays a text transcription of the word and a visual depiction of the word; and an implement manipulable to diminish visibility of the visual depiction of the word.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0007]
[0008]
[0009]
[0010]
[0011]
[0012]
[0013]
[0014]
[0015]
[0016]
[0017]
[0018]
[0019]
DETAILED DESCRIPTION
[0020] According to relational frame theory, an individual may develop stimulus relations as the individual engages in communication in which a caregiver interacts with the individual using a variety of stimuli and reinforces successful stimuli associations. For example, a caregiver may show a physical object to an individual and reinforce the individual speaking the name of the object. With further teaching to develop relations, a caregiver may gesture toward the object, and ask “what is it?” and reinforce the individual selecting the written word of the object from a communication board. With relational frame theory, additional relations may emerge, such as when the caregiver says the word of the object, and the individual selects the written word of the object. As such, stimulus relations between auditory, visual, and textual stimuli may be developed.
[0021] An example process 100 for the development of such stimulus relations is depicted in
[0022] Stimulus relationships may be developed through stimulus equivalence or mutual entailment of stimuli by the individual engaging with multiple stimuli during the course of communication. An individual may thereby develop equivalencies or mutual entailments between several different stimuli with respect to a particular concept.
[0023] According to the methods described herein, the individual may be progressively weaned off of reliance on one or more of the stimuli, such as the visual stimuli, to strengthen the individual's reliance on other stimuli (e.g., written stimuli). For example, where an individual has developed a strong reliance on visual stimuli to communicate, the individual may be encouraged to rely more heavily on textual or auditory stimuli, i.e., by engaging in written or oral communication. The individual may thereby strengthen their communicative ability.
[0024] Thus, a communication device, method, and communication apparatus are provided to assist an individual to develop communicative ability. Use of the communication device, method, and communication apparatus involve progressively diminishing certain stimuli to reduce the individual's reliance on the diminished stimuli and to reinforce the individual's reliance on other stimuli.
[0025]
[0026] The memory stores a communication application (e.g., a software application) to assist an individual with communication and to assist the individual to develop communication capabilities. The communication application includes a communication module to generate a user interface. The user interface displays word tiles for selection by an individual, compiles a sentence upon the individual's selection of word tiles, and outputs the sentence to communicate with another person. An example of such a user interface displaying word tiles is provided in
[0027] With continued reference to
[0028] The processor may include any quantity and combination of a processor, a central processing unit (CPU), a microprocessor, a microcontroller, a field-programmable gate array (FPGA), and similar. The network interface may include programming logic enabling the communication device 250 to communicate over one or more computing networks, is configured for bidirectional data communications through any network used, and accordingly can include a network adaptor and driver suitable for the type of network used. The memory includes a non-transitory computer-readable medium for storing programming instructions executable by the processor. The memory may include volatile storage and non-volatile storage. Volatile storage may include random-access memory (RAM) or similar. Non-volatile storage may include a hard drive, flash memory, and similar.
[0029] The communication device 250 may communicate over one or more computing networks with a data collection server. The data collection server may store interaction data from a plurality of communication devices 250. The interaction data stored on the data collection server may be accessed as a cloud computing service to allow an individual to share their data across multiple communication devices 250. The interaction data may further be used to gain insights into the progress of the individual, or a plurality of individuals, using the communication devices 250. Analysis of the interaction data may be incorporated into aspects of the communication module or the training module. For example, training schedules may be informed by analysis of the interaction data.
[0030]
[0031] A word tile may also be associated with a vocal expression of the word. For example, selection of a word tile may cause the communication device 250 to produce an audible sound of the vocalization of the word, either upon selection of the word tile or upon output of a sentence constructed using the user interface 300.
[0032] Thus, word tiles are associated with words, and are associated with at least two stimuli of the word, wherein the stimuli may be a text transcription of the word, a visual depiction of the word, or a vocal expression of the word.
[0033] Selection of a word tile may provide for the selection of additional selection of word tiles so that the user may select several words to be compiled into a sentence for output. For example, selection of a first word tile 302 may generate a group 304 of additional word tiles, and selection of a word tile from the group 304 may generate an additional group 306 of still additional word tiles, the selection of which may generate an additional group 308 of still additional word tiles, and so on.
[0034] Word tiles may be organized into groups, categories, and/or hierarchies to provide for ease of selection. These groups, categories, and/or hierarchies may be connected in that the selection of one word tile may generate an additional group of word tiles for selection by the user. Thus, an individual may select word tiles by navigating through a mapping of connected word tiles. For example, the word tile “I want” may be connected to the word tiles for “Eat” and “Drink” and “Item” as these word tiles represent logical ideas that could follow “I want”. The word tiles may be stored in a word tile library stored in a database with the appropriate linkages.
[0035] The mapping of connected word tiles may be hard-coded by a user or may be generated dynamically. Thus, in some examples, the selection of a particular word tile may cause the communication module to pull a following additional group of word tiles from a word tile library and present the group of word tiles to the individual in a particular order according to pre-determined rules. These pre-determined rules may be configured to follow grammar rules. Further, these pre-determined rules may be programmed by a person, such as a caregiver of the individual, to develop a library of connected word tiles that would suit the needs of the individual in that commonly used word tiles are presented to the individual most conveniently. For example, a caregiver may develop a library of connected word tiles in which the word tile “I want” is connected to “to play”, and “to play” is connected to the individual's favourite activities, such as “baseball” or “cards”.
[0036] In other examples, the selection of a particular word tile may cause the dynamic generation of a following group of word tiles, wherein the following group is generated according to a predictive algorithm. The predictive algorithm may involve presenting the individual with word tiles which have been most commonly selected by the individual in the past, which may change over time. As another example, word tiles may be presented to the individual based on commonly selected word tiles according to interaction data stored on the data collection server. A predictive algorithm may include a machine learning algorithm.
[0037] As word tiles are selected, the selected word tiles may be stored in a sentence container 310. As word tiles are included in the sentence container 310, additional sentence structure elements, such as articles, prepositions, or other words and/or punctuation, may be generated and inserted as appropriate into the sentence container 310. The communication module may include a natural language processor and/or generator to generate such additional sentence structure elements.
[0038] Once a sentence is formed, the individual may press a button (e.g. “speak”) to output the sentence and empty the sentence container 310 for another sentence to be compiled. Outputting the sentence may involve playing an audio transcription of the compiled sentence for a caregiver to hear. The user interface 300 may also include a button to undo an action such as an addition of a word tile to the sentence container 310 (e.g. an “undo” button).
[0039] The user interface 300 may also include buttons to output simple responses, such as “yes” and “no” for ease of communication. The user interface 300 may also include buttons to navigate larger menus of word tiles (e.g. a “forward” button to flip to another page of word tiles).
[0040] The user interface 300 may also provide a mechanism for a caregiver to provide input to the communication application regarding the performance of the individual. For example, as shown in
[0041] The user interface 300 may include additional user interface elements to assist a user to construct a sentence. As shown in
[0042] As another example, as shown in
[0043] As an individual practices communication with the user interface 300, the individual may become more proficient with the use of some of certain word tiles. Early in the training process, the individual may be relying on one of the stimuli associated with the word tiles more than other stimuli. For example, the individual may be relying on the visual depiction of the word shown on a particular word tile to conclude that that word tile is the word tile they wish to select. As the individual becomes more proficient with the use of certain word tiles, the training module of the communication application may note this development in the interaction data. For example, an individual requiring less prompting (guidance) to select a word tile from a menu of word tiles may indicate that the individual is more proficient at selecting that word tile. This may further indicate that the individual has improved at building an association between the stimuli associated with the word tile and is ready to be weaned off of reliance on the visual stimuli. As an individual becomes more proficient at selecting a particular word tile, the training module may cause the communication module to generate that word tile with one of the stimuli diminished in other trials using the communication device 250, thereby reducing the individual's reliance on that diminished stimuli, and encouraging the individual to rely on the other stimuli. For example, the communication module may diminish the visibility of the visual depiction of the word on a given word tile. Thus, as the individual continues to practice with the user interface 300, the individual is progressively weaned off of reliance on the visual depiction of words, and encouraged to draw on associations with text transcriptions of words to communicate. In other examples, the textual stimuli may be diminished. In still other examples, the auditory stimuli may be diminished. Thus, an individual can thereby be assisted to develop communicative ability.
[0044] Thus, in use, a method for as assisting an individual to develop communicative ability is provided. The method involves providing a communication interface (such as, for example, user interface 300) to the individual. The communication interface displays word tiles for selection by the individual to form a sentence, wherein word tiles are associated with a word and at least two stimuli of the word. The stimuli may include a text transcription of the word, a visual depiction of the word, and a vocal expression of the word.
[0045] The method further involves facilitating interaction of the individual with the communication interface to build a plurality of sentences. Thus, a caregiver may interact with the individual to induce the individual to communicate using the communication interface.
[0046] The method further involves progressively diminishing at least one of the at least two stimuli of the word based on the individual's interaction with the communication interface. Thus, as the individual continued to use the communication interface, the individual may progressively develop communicative ability language and communication skills.
[0047] In some examples, it is contemplated that any of the three stimuli associated with word tiles, visual, textual, or auditory, may be progressively diminished in this manner. For example, the auditory and/or visual stimuli may be progressively diminished in favour of the text stimuli. The mode of progressively or systematically diminishing stimuli may be selected to suit the needs of any given individual.
[0048] Further, the progression of the diminishment of stimuli may follow any pre-determined training schedule to suit the needs of any given individual. For example, some stimuli may be diminished linearly, according to a schedule of thresholds, or any other algorithm. Moreover, different modes for diminishing stimuli are also contemplated.
[0049]
[0050]
[0051] In examples in which the auditory stimuli is to be diminished, the audibility of the vocalization of the word may be decreased by, for example, lowering the volume of the output sound, or by delaying output of the sound to provide the user with an opportunity to vocalize the sound themselves. Such diminishment, including the degree by which audibility is decreased, and the duration of delay of the sound, may be configured by a caregiver through a user interface. Further, such diminishment may be updated as the user progresses as discussed herein.
[0052]
[0053]
[0054] The category selector menu 602 may be used to select a category for a new word tile. The word selector menu 604 may be used to search for a word in a database of pre-configured word tiles to be added to the individual's pool of word tiles, or to add a new word to the database. The new word creator menu 606 may be used to input the text transcription of a word, provide grammatical information relating to the word (such as whether the word is a noun, verb, adjective, is plural, etc.), to add a visual depiction of the word (e.g. by taking a picture using the communication device 250 or by selecting an image from a gallery), and to add a vocal expression of the word (e.g. by recording the spoken word). A user, such as a caregiver on an individual, may thereby add word tiles to a library of word tiles to be used by the communication application. In other examples, vocal expressions of words may be associated with computer-generated vocalizations of the words.
[0055]
[0056]
[0057] The communication apparatus 700 further includes, an implement which is manipulable to diminish visibility of the visual depiction of the word of a word tile. For example, the implement may include a slider which is configured to slide into a slot on the communication display surface to cover the visual depiction of a word of a word tile.
[0058] In use, a method for as assisting an individual to develop linguistic capabilities is provided. The method includes providing a communication interface (such as, for example, the communication apparatus 700) to the individual. The communication interface displaying word tiles for selection by the individual to form a sentence. A word tile is associated with a word and displays a text transcription of the word and a visual depiction of the word.
[0059] The method further involves facilitating interaction of the individual with the communication interface to build a plurality of sentences. Thus, a caregiver may interact with the individual to induce the individual to communicate using the communication interface.
[0060] The method further involves progressively diminishing visibility of the visual depiction of a word of a word tile word based on the individual's interaction with the communication interface. Thus, as the individual continued to use the communication interface, the caregiver may note the individual's progress with developing proficiency selecting word tiles, and progressively cover the visual depictions of words accordingly. The individual may thereby progressively develop communicative ability.
[0061] In other examples, the visual depiction of the word may be diminished by the caregiver cutting away portions of the visual depiction of the word, drawing over the visual depiction of the word, or otherwise obscuring view of the visual depiction of the word.
[0062] In still other examples, the communication device 250 or the communication apparatus 700 may be used to develop communicative ability for people suffering from dementia. Auditory voices may interfere with the communicative ability of some individuals suffering from dementia. Thus, in such examples, where an auditory stimulus is included, the auditory stimuli may be progressively diminished throughout trials to reduce interference caused by auditory stimuli.
[0063] Thus, an individual may be assisted to develop communicative ability using the communication device, communication apparatus, and/or methods described herein. The individual may be assisted on an ongoing basis or over a predetermined number of teaching trials. By encouraging an individual to rely on different stimuli, the individual may become more proficient at communicating in different modes. For example, by encouraging the individual to rely on text to communicate, the individual may be assisted in the development of the proficiency of using other devices such as computers. As another example, by encouraging the individual to rely on auditory prompts or stimuli to communicate, the individual may be assisted in the development of proficiency in oral communication.
[0064] It should be recognized that features and aspects of the various examples provided above can be combined into further examples that also fall within the scope of the present disclosure. The scope of the claims should not be limited by the above examples but should be given the broadest interpretation consistent with the description as a whole.