MUSEUM ACCESS FOR ALL SYSTEM AND METHOD

Abstract

A method facilitates communication of program information to visitors to museums and other facilities that include displays for educational interest. The method operates with a system including an APP downloadable onto a smartphone or other module, a facility system located within the museum or other facility and ability to communicate between the facility system and the smartphone or other module. The method facilitates communication with visitors needing any one or more of visual assistance, hearing assistance, and/or sensory assistance. The system includes a protocol translator which facilitates operation of the inventive method.

Claims

1. A method of facilitating informed access by visitors to a facility with a plurality of displays, including the steps of: a) providing each visitor with a handheld module having a computer with memory, a display, ability to communicate audio and visual programming, and a wireless transmitter and receiver; b) downloading an APP into said memory, said APP facilitating said module being able to communicate programming designed for people who are deaf or hard of hearing and/or people who are visually impaired; c) each facility having a facility system including video and audio playback equipment, a computer with a memory on which multimedia programming designed for people who are deaf or hard of hearing and/or people who are visually impaired is stored for playback to visitors and a wireless transmitter and receiver; d) said APP storing on each module a list of facilities in which a facility system is installed; e) when a visitor is inside or outside a chosen facility location, said visitor selecting said chosen facility location and when said visitor is near, adjacent or within said chosen facility location, said handheld module communicating wirelessly with said facility system, whereupon informational content regarding displays in said facility can be downloaded onto said module; f) said visitor choosing on said APP whether multimedia programming for people who are deaf or hard of hearing and/or people who are visually impaired is to be played for said visitor; g) sensing by a proximity sensor adjacent each particular display for proximity of a visitor's module and, responsive to sensing such proximity, causing informational content concerning said particular display to be communicated on said module for said visitor; and h) listening by a protocol translator for time code requests from modules, and when a time code request is received by said protocol translator, said protocol translator sends a request to said facility system to sync multimedia programming from an adjacent exhibit to cause said multimedia programming to play on said module in a chosen format, whether for people who are deaf or hard of hearing and/or people who are visually impaired.

2. The method of claim 1, wherein said facility is a museum.

3. The method of claim 1, wherein said module comprises a smartphone.

4. The method of claim 2, wherein said module comprises a smartphone.

5. The method of claim 1, wherein the APP also facilitates communication of programming designed to be received by visitors with sensory processing difficulties.

6. The method of claim 1, wherein each said proximity sensor includes a beacon.

7. The method of claim 5, wherein said sensory processing difficulties comprise autism.

8. The method of claim 3, wherein said smartphone is owned by a visitor.

9. The method of claim 1, wherein said module is owned by said facility and rented to a visitor.

10. The method of claim 1, wherein said protocol translator receives playback status of said playback equipment.

11. The method of claim 1, wherein said wireless transmitter and receiver of said module are connected to a worldwide wireless communication system known as the Internet.

12. The method of claim 1, wherein said module comprises a smartphone.

13. The method of claim 12, wherein said facility is a museum.

14. The method of claim 1, wherein said multimedia programming can be translated in any one of a plurality of languages.

15. The method of claim 3, wherein said multimedia programming can be translated in any one of a plurality of languages.

16. A method of facilitating informed access by visitors to a museum with a plurality of displays, including the steps of: a) providing each visitor with a handheld module having a computer with memory, a display, ability to communicate audio and visual programming, and a wireless transmitter and receiver; b) downloading an APP into said memory, said APP facilitating said module being able to communicate programming designed for people who are deaf or hard of hearing and/or people who are visually impaired; c) each museum having a facility system including video and audio playback equipment, a computer with a memory on which multimedia programming designed for people who are deaf or hard of hearing and/or people who are visually impaired is stored for playback to visitors and a wireless transmitter and receiver; d) said APP storing on each module a list of museums in which a facility system is installed; e) when a visitor is inside or outside a chosen museum location, said visitor selecting said chosen museum location and when said visitor is near, adjacent or within said chosen museum location, said handheld module communicating wirelessly with said facility system, whereupon informational content regarding displays in said museum can be downloaded onto said module; f) said visitor choosing on said APP whether multimedia programming for people who are deaf or hard of hearing and/or people who are visually impaired is to be played for said visitor; g) sensing by a proximity sensor adjacent each particular display for proximity of a visitor's module and, responsive to sensing such proximity, causing informational content concerning said particular display to be communicated on said module for said visitor; and h) listening by a protocol translator for time code requests from modules, and when a time code request is received by said protocol translator, said protocol translator sends a request to said facility system to sync multimedia programming from an adjacent exhibit to cause said multimedia programming to play on said module in a chosen format, whether for people who are deaf or hard of hearing and/or people who are visually impaired.

17. The method of claim 16, wherein said module comprises a smartphone.

18. The method of claim 16, wherein each said proximity sensor includes a beacon.

19. The method of claim 16, wherein said sensory processing difficulties comprise autism.

20. The method of claim 16, wherein said protocol translator receives playback status of said playback equipment.

21. The method of claim 16, wherein said wireless transmitter and receiver of said module are connected to a worldwide wireless communication system known as the Internet.

22. The method of claim 16, wherein said multimedia programming can be broadcast in any one of a plurality of languages.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0039] FIG. 1 shows a schematic representation of the inventive system.

[0040] FIG. 2 shows a first portion of a flowchart showing the operation of the inventive system.

[0041] FIG. 3 shows a continuation of the flowchart shown in FIG. 2.

[0042] FIG. 4 shows a flowchart of the visual assistance operation of the present invention.

[0043] FIG. 5 shows a flowchart of the sensory assistance portion of the inventive system.

[0044] FIG. 6 shows a flowchart for the inventive protocol translator.

[0045] FIG. 7 shows a flowchart for the museum audio visual network media tracking process.

[0046] FIG. 8 shows a representation of a prior art system.

SPECIFIC DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0047] FIG. 1 shows a schematic representation of an example of the present invention.

[0048] In museums today, inclusivity is always being discussed. Unfortunately, there is nothing on the market that addresses all disabilities. Most often, products are used that cover hearing assistance, and even fewer that cover visual assistance. Applicant is unaware of any such system devised to address sensory processing difficulties such as Autism. For most museums, cost is a big factor, and many are not able to invest in multiple high-cost systems. Instead, they decide to go without the assistive supports and risk violating the Americans with Disabilities Act.

[0049] The present invention is an all-encompassing solution that provides museums with the tools and content necessary to make sure those visitors who need support have access to it. The unique media tracking element of the present invention allows for it to be added to a pre-existing or new multimedia system. While current audio tour systems typically need an input from the visitor to select which exhibit they are currently in, the location-based system of the present invention allows for visitors to roam a museum and gain access to the assistive content needed based on the closest exhibit.

[0050] Innovations that the present invention provides include: [0051] a) Hearing, visual, sensory, and translation capability all supported under a single application and system; [0052] b) Sensory support for each exhibit to help those individuals with a sensory sensitivity or diagnosis like Autism; [0053] c) Visitors can easily use their own device, as renting out devices is typically a concern in the post-COVID world; [0054] d) Location-based contentcontent is only accessible when in the museum; and [0055] e) Media tracking capabilities for pre-existing museum audio/video systems or new installations. This capability makes syncing between handheld devices and live video content possible.

[0056] With reference to FIG. 1, the system includes the following components:

[0057] A visitor-owned smartphone/device or museum-owned device 101 that can be rented to the visitor. This handheld device 101 contains the A4A (Access For All) APP that the visitor or museum can download via an Internet connection. When outside of the museum, the visitor can select which museum they are planning to visit and download the content from the web-hosted content server 305. Museum-specific content cannot be accessed outside of the museum. This device utilizes both Bluetooth and Wireless technologies. As is well known, a smartphone has a built-in computer with a memory, a display, ability to communicate audio and visual programming and a wireless transmitter and receiver. The same is true of modules rentable from museums.

[0058] Examples of devices usable with the system include Apple iPhone, iPod Touch, iPad, or iPad Mini; Google Android Device, a Bluetooth beacon 201 (equipment installed within the museum for use with A4A APP). The Bluetooth beacon device 201 only transmits a programmed identification number. It continuously broadcasts its device number via Bluetooth for the handheld device 101 to detect. The APP on the handheld device 101 determines what content is available to the visitor by selecting the closest beacon device 201 via Bluetooth signal strength and distance calculations determined by the APP by incorporating the Bluetooth beacon's manufacturers SDK into the APP. The Bluetooth Beacon devices 201 are located in the exhibit spaces of the museum. One example of a proximity beacon device usable with the present invention is the Estimote Proximity Beacon device.

[0059] The Protocol Translator device 301 (equipment installed within the museum for use with A4A APP) is the communications manager between all aspects of the A4A APP and system. It receives location information and requests from the handheld device 101, initializes and sets up the timelines for the Timeline-Base Video Player 302, receives playback communication from each of the exhibit multimedia players in the Museum AV (audio-visual) Network 401, and translates the playback status of the individual multimedia players to the Timeline-Based Video Player 302 in order to consistently track the exhibit media state, request current playback state and timecode from the Timeline-Based Video Player 302, and send the current timecode location and status to the handheld device 101. The AMX NX Series Controller is an example of a protocol translator usable in the present invention. The museum AV network 401 comprises a facility system including video and audio playback equipment.

[0060] A Timeline-Based Video Player 302 (equipment installed within the museum for use with A4A APP) is used to track the playback of the various multimedia elements of the Museum AV Network 401. This feature allows for the A4A APP and installed system to be backwards compatible to any Museum AV System 401. The setup of the Timeline-Based Video Player 302 is completed by uploading an xml setup document to the Protocol Translator 301 that contains media timeline information of the various multimedia exhibits within the existing Museum AV Network 401. The Protocol Translator 301 then translates the xml data into the network protocol that the Timeline-Based Video Player 302 understands. This creates an individual timeline for each of the multimedia exhibits within the Museum AV Network 401. The display of this device shows the playback status of each timeline that tracks a multimedia element of the Museum AV Network 401. The 7.sup.th Sense Delta Pico Server (general computer with Delta Server software) is an example of a timeline-based video player usable in the present invention.

[0061] The Live Audio Network 303 (equipment installed within the museum for use with A4A APP) allows for the handheld device 101 to receive lip-synced audio over the A4A Network 304. There is one live audio device for each exhibit media player in the Museum AV Network 401. This live audio device connects to the audio output of the exhibit media player, then streams that audio over the A4A Network 304 via an RTP protocol. The handheld device 101 then connects to the associated RTP stream of the Live Audio Network 303, as indicated by the downloaded museum-specific setup file, and delivers lip-synced audio to the visitor through headphones or other connected listening devices. A Barix AudioPoint 3.0 can be employed as the Live Audio Network 303.

[0062] The A4A Network 304 (equipment installed within the museum for use with A4A APP) is an interconnectivity of the Protocol Translator 301, Timeline-Based Video Server 302, and the Live Audio Network 303 through a CAT6 or better cable network that includes a router, switches, and wireless access points as understood by those skilled in the art. The handheld device 101 connects to the A4A network 304 through a wireless network connection. The A4A Network 304 is connected to the Internet which is where the web-based content server is hosted 305. This allows the visitor to download the A4A APP and museum-specific content to their handheld device 101 if they did not do so prior to entering the museum.

[0063] The Museum AV Network 401 is the current audio/video system that is installed within museums. This network can be installed at the same time as the A4A system or can be pre-existing. A typical Museum AV System consists of video and audio playback equipment, projectors, monitors, touchscreens, computers, audio DSP system, amplifiers, and speakers, a wireless transmitter and receiver along with proximity sensors adjacent each display, to name a few. The purpose of the A4A APP and installed system is to provide local content support on the visitor's handheld device 101 that syncs to the multimedia content within the Museum AV Network 401.

[0064] With reference now to the flowchart of FIG. 2, the visitor interacts with the A4A system through a downloaded APP on their personal or rented handheld device 101. The APP has stored therein information concerning each museum or other facility that has the inventive system installed therein. If the APP detects that the visitor is not at a museum location with the A4A Network and System, no content will be available on the APP. Outside of a museum location, a visitor can select the museum they will be visiting to pre-download the content but will not be able to access the content in any of the Hearing Assistance or Translations, Visual Assistance or Sensory Assistance Home Screens. When the APP detects the visitor at a museum location, the downloaded content is available as the handheld device 101 detects the closest Bluetooth Beacon 201.

[0065] As shown in the flowchart of FIG. 3, in the Hearing Assistance or Translations section of the APP on the handheld device 101, the APP is continuously detecting the closest Bluetooth Beacon 201. At any point in the flowchart, if a closer Bluetooth Beacon 201 is found, the APP will revert to the Detection State (bolded at the top of the flowchart). When a new Bluetooth Beacon 201 is detected, the APP uses the downloaded museum-specific setup file to determine if this exhibit area has any content for Hearing Assistance. Hearing Assistance support comes in two types: live audio streaming from the Live Audio Network 303, and captions synchronized to the exhibit video player within the Museum AV Network 401. Volume level of the content on the handheld device 101 is controlled by the visitor.

[0066] If the museum-specific setup file indicates that there is live audio for this exhibit area, the APP connects to the Live Audio Streaming Channel 303 for that Bluetooth Beacon 201 number. If the museum setup file indicates that the Museum AV Network 401 has a media component, the APP opens a communication port with the Protocol Translator 301 and sends a Timeline Request. The Protocol Translator starts the Timecode Process (see FIG. 6) and sends the received current timing information back to the APP on the handheld device 101. The APP then processes the information and either stops all video content or plays the video file starting at the location specified by the timing information.

[0067] With reference to the flowchart of FIG. 4, in the Visual Assistance section of the APP on the handheld device 101, the APP is continuously detecting the closest Bluetooth Beacon 201. At any point in the flowchart, if a closer Bluetooth Beacon 201 is found, the APP will revert to the Detection State (bolded at the top of the flowchart). When a new Bluetooth Beacon 201 is detected, the APP uses the downloaded museum-specific setup file to determine if this exhibit area has any content for Visual Assistance. Visual Assistance support comes in two types: descriptive audio of the surrounding exhibit area, and descriptive audio synchronized to the exhibit video player within the Museum AV Network 401. Volume level of the content on the handheld device 101 is controlled by the visitor.

[0068] If the museum setup file indicates that the Museum AV Network 401 has a media component, the APP opens a communication port with the Protocol Translator 301 and sends a Timeline Request. The Protocol Translator starts the Timecode Process (see FIG. 6) and sends the received current timing information back to the APP on the handheld device 101. The APP then processes the information and either stops all audio content or plays the audio file starting at the location specified by the timing information. If the museum setup file indicates that the Museum AV Network 401 does not have a media component, the APP plays a descriptive audio file for the exhibit and gives the visitor playback control of the audio.

[0069] With reference to the flowchart of FIG. 5, in the Sensory Assistance section of the APP on the handheld device 101, the APP is continuously detecting the closest Bluetooth Beacon 201. At any point in the flowchart, if a closer Bluetooth Beacon 201 is found, the APP will revert to the Detection State (bolded at the top of the flowchart). When a new Bluetooth Beacon 201 is detected, the APP uses the downloaded museum-specific setup file to determine if this exhibit area has any content for Sensory Assistance. Sensory Assistance consists of video behavioral supports that play on the handheld device 101 for each exhibit. The visitor has volume level and playback control of the content on the handheld device 101.

[0070] The Timecode Process used by the Protocol Translator 301 is unique in this system (see the FIG. 6 flowchart). The Protocol Translator 301 is continually listening on all communication ports for Timecode Requests from handheld devices 101 that are connected to the A4A Network 304. Each unique communication port number correlates to a Media Timeline running on the Timeline-Based Video Player 302. When a Timecode Request is received, the Protocol Translator 301 sends a Media Location Request to the Timeline-Based Video Player 302 for the Media Timeline that corresponds to the current exhibit location as determined by the closest Bluetooth Beacon 201. The Protocol Translator 301 receives a string back from the Timeline-Based Video Player 302 and determines whether the current exhibit media is stopped or playing. The Protocol Translator 301 then completes the comparisons and/or calculations between the received timing data and the initial exhibit media timing information taken from the xml setup documented uploaded to the Protocol Translator 301, and sends the correct exhibit media status information to the handheld device 101. This status allows the handheld device 101 to sync to the exhibit media player that resides on the Museum AV Network 401, allowing for Hearing Assistance captioning, Translations, or Visual Assistance descriptive audio to sync with the onscreen media content.

[0071] The Media Tracking Process of the media content within the Museum AV Network 401 (see FIG. 7 flowchart) creates a unique system that allows for the A4A APP to be retrofitted to existing Museum AV Systems or used in new installations. While the Protocol Translator 301 is listening for Timeline Requests from any handheld device 101 on the A4A Network 304, it is also listening for Media Tracking information from video playback equipment on the Museum AV Network 401. Each exhibit is assigned a number that corresponds to the Media Timeline used for tracking playback content on the Timeline-Based Video Player 302 and is stored in the downloaded museum-specific setup file on the handheld devices 101 which is used to determine the communication port with the Protocol Translator 301.

[0072] When a video player on the Museum AV Network 401 begins the playback of a file, the player sends a string to the Protocol Translator 301 that indicates the exhibit number and which video has started playing. Once this is received by the Protocol Translator 301, it calculates the cue and video frame positions based on the frame rate and timeline settings as indicated in the xml setup document. The Protocol Translator 301 then sends two commands to the Timeline-Based Video Player 302: a move to frame command and play command for the timeline that corresponds to the exhibit number.

[0073] When a video player on the Museum AV Network 401 stops the playback of a file, the player sends a string to the Protocol Translator 301 that indicates the exhibit number, and that the player has stopped all playing content. Once this is received by the Protocol Translator 301, it sends a stop command to the Timeline-Based Video Player 302 for the timeline that corresponds to the exhibit number.

[0074] This creates an external system that allows for tracking of live media content within a museum and can be standardized across video media player manufacturers. The benefit to this system makes the museum more accessible for others in a way that enhances the experience. If a visitor enters an exhibit with live media content, the Media Tracking process allows for better media visual descriptions, captioning, and translation options that are automatically synchronized on their personal handheld device 101.

[0075] In the operation of the present invention, the museum visitor downloads the A4A APP onto their device from the Apple APP Store or the Google Play Store or other APP source. As the visitor enters the museum and opens the APP, it will ask the user to choose a museum from a list of museum subscribers. Choosing a museum subscriber downloads custom content to their device. These files include xmls files that contain local system information for the Subscriber Synchronization System (i.e., network communication ports and IP information, beacon information, GUI layout information, and general APP configuration information), and audio and video files that are custom created content that include captions, video descriptor files, and social stories.

[0076] The visitor then walks around the museum with one of the three available APP assistance areas openHearing Assistance, Visual Assistance or Sensory Assistance. As the visitor encounters a beacon, the APP knows where the visitor is in the museum space. [0077] 1. If the beacon configuration has no media or audio/video contentno interaction with Subscriber Synchronization System. [0078] a. Hearing Assistanceplays a video for that area indicating there are no captions or audio in this area. [0079] b. Visual Assistanceplays the visual descriptor audio file for the area. [0080] c. Sensory Assistanceplays the social story video for the area. [0081] 2. If the beacon configuration has media or audio/video content (i.e., a video playing on a monitor with audio)there is communication with Subscriber Synchronization System. [0082] a. Hearing Assistancethe xml setup document data indicates that this beacon area needs more information about the local media and/or needs to connect to a live audio feed. As required, the APP connects to the live audio RTP stream and sends a LOCATION message to the Protocol Translator to determine the current timecode of the area's video through communication with the Timeline-Based Video Player. The Protocol Translator sends back a message indicating the current timecode and plays the caption video file for the area starting from the specified timecode, so that the caption file on the device is playing in sync with the video on-screen. [0083] b. Visual Assistancethe xml setup document data indicates that this beacon area needs more information about the local media. The APP then sends a LOCATION message to the Protocol Translator to determine the current timecode of the area's video through communication with the Timeline-Based Video Player. The Protocol Translator sends back a message indicating the current timecode and plays the visual descriptor audio file for the area from the specified timecode, so that the visual descriptor file on the device is playing in sync with the video on-screen. [0084] c. Sensory Assistanceplays the social story video for the area.

[0085] Equipment utilized in the A4A system includes the following as examples: [0086] 1. BeaconEstimote proximity beacon to determine location of device through iBeacon and Eddystone protocols. [0087] 2. Museum Audio/Video Systemthis is the current audio/video system of the museum (equipment will be different at every museum and can include projectors, monitors, audio players, video players, computers, etc.). The A4A APP and the Subscriber Synchronization System can be easily integrated into new and currently installed audio/video systems. [0088] 3. Live Audio NetworkBarix AudioPoint 3.0 system, which streams audio over the network via an RTP protocol and has very low latency (so that the audio lip-syncs to videos). The APP uses their SDK to connect and disconnect from streams of audio. [0089] 4. Subscriber Synchronization Systemcurrently using an AMX NX-3200 as the Protocol Translator in combination with a 7.sup.th Sense Design Delta Pico Server as the Timeline-Based Video Player. A Medialon Control Server may also be suitable as a Protocol Translator. [0090] a. AMX NX-3200/Protocol Translatorcustom coding that handles all communication protocols [0091] including communication and socket control of connected handheld devices and video players located on the Museum Audio/Video Network, and [0092] monitoring timelines on Delta Pico (Timeline-Based Video Player). [0093] b. 7.sup.th Sense Delta Pico/Timeline-Based Video Playerincludes custom timeline programming that tracks the Museum Audio/Video System media. [0094] i. c. Communication between Protocol Translator and the Timeline-Based Video PlayerSetupa custom created xml setup document contains information about all of the media content of the Museum Audio/Video System (video file names, number of files, length of files) is ingested by the Protocol Translator during configuration and sent via network communication to the Timeline-Based Video Player. This setup communication creates a custom, dedicated timeline for each media exhibit within the Museum Audio/Video System and mimics the playback of that exhibit. [0095] ii. Museum Audio/Video System Trackingas the Museum Audio/Video System is running, the playback devices send communication to the Protocol Translator when the video starts and stops. This communication controls the dedicated timeline for that video on the Timeline-Based Video Player, so that it can be used to track the video player. [0096] iii. Visitor Usagewhen the Protocol Translator receives the LOCATION command from a handheld device, the Protocol Translator then polls the Timeline-Based Video Player to determine what the current timecode of the video for that area is.

[0097] As such, an invention has been disclosed in terms of preferred embodiments thereof which fulfill each and every one of the objects of the invention as set forth hereinabove, and provides a new and useful museum access for all system and method of great novelty and utility.

[0098] Of course, various changes, modifications and alterations in the teachings of the present invention may be contemplated by those skilled in the art without departing from the intended spirit and scope thereof.

[0099] As such, it is intended that the present invention only be limited by the terms of the appended claims.