A system and method for remote activation of a storage operation of pictorial information

Abstract

The invention relates to a system and method for remote activation of a storage operation of pictorial information. According to the invention, it involves a wearable sensor device, comprising at least one sensor providing sensor signals, a processing unit for running computer programs, a first memory unit for storing instructions for said computer programs and a wireless signaling unit for communicating with external devices over a wireless link. The system further comprises at least one second digital device comprising a recording apparatus adapted to be set in a mode of continuous capture of pictorial information, a second memory unit, and a wireless signaling system for communicating with external devices over a wireless link. The processing unit in the wearable sensor device is adapted to send commands over said wireless link by means of one-way transmission to said second digital device to trigger, in response to said commands, storage of at least part of said continuously captured pictorial information in the memory of said at least one second device.

Claims

1. A system for remote activation of a storage operation of pictorial information, comprising: a wearable sensor device, comprising at least one sensor providing sensor signals, a processing unit for running computer programs, a first memory unit for storing instructions for said computer programs and a wireless signaling unit for communicating with external devices over a wireless link; and at least one second digital device comprising a recording apparatus adapted to be set in a mode of continuous capture of pictorial information, a second memory unit, and a wireless signaling system for communicating with external devices over a wireless link; wherein said processing unit in said wearable sensor device is adapted, based on said sensor signals, to send commands over said wireless link by means of one-way transmission to said second digital device to trigger, in response to said commands, storage of at least part of said continuously captured pictorial information in the memory of said at least one second device.

2. The system according to claim 1, wherein said commands are generated by sensor signals tracking the movements, proximity, or the location of a user carrying said wearable sensor device.

3. The system according to claim 2, wherein said sensor device is a sensor module that is detachably attached to an apparel or equipment of a user.

4. The system according to claim 2, wherein said sensor device is a sensor module that is integrated in a garment worn by the user.

5. The system according to claim 2, wherein said sensor device is a sensor module that is integrated in a shoe of the user.

6. The system according to claim 2, wherein said sensor device is a sensor module that is integrated in a sport equipment used by the user.

7. The system according to claim 1, wherein said second device is a digital camera.

8. The system according to claim 1, wherein said second device is a smartphone equipped with at least one digital camera.

9. The system according to claim 1, wherein said second device includes more than one digital camera.

10. The system according to claim 1, wherein said at least one second device is adapted to store captured pictorial information as video clips in its memory a predetermined period of time before and/or after having received a storage command.

11. The system according to claim 1, wherein said sensor device has a programming interface enabling a connection to an external computerized device having a user interface for adding and/or modification of computer program instructions and/or parameters for said wearable sensor device, and for storing said instructions and/or parameters in said first memory unit.

12. The system according to claim 1, wherein said external computerized device comprising said programming interface is a mobile phone.

13. The system according to claim 1, wherein said external computerized device comprising said programming interface is a wristop computer.

14. The system according to claim 1, wherein said external computerized device comprising said programming interface is a tabletop computer.

15. The system according to claim 11, wherein said external computerized device comprising said programming interface and said second digital device comprising a recording apparatus are the same device.

16. The system according to claim 15, wherein said external computerized device comprising said programming interface is a wearable digital camera or a mobile phone.

17. The system according to claim 1, wherein said wearable sensor device comprises at least a 3D accelerometer sensor.

18. The system according to claim 1, wherein said wearable sensor device comprises electromagnetic sensors for 3D-detection of the direction of earth's magnetic field.

19. A method of activating a storage operation of pictorial information, comprising the steps of: providing a wearable sensor device comprising at least one sensor providing sensor signals, a processing unit for running computer programs, a first memory unit for storing instructions for said computer programs and a wireless signaling unit for communicating with external devices over a wireless link; providing at least one second digital device comprising a recording apparatus adapted to be set to continuous capture of pictorial information, a second memory unit, and a wireless signaling system for communicating with external devices over a wireless link; producing in said wearable device, based on said sensor signals, commands to instruct storage of pictorial information in said second device; transmitting by means of one-way transmission said commands over said wireless link to said second device; and receiving said commands in said at least one second device and triggering in response to said commands storage of at least part of said continuously captured pictorial information in a memory of said at least one second device.

20. The method according to claim 19, further comprising the step of storing in said at least one second device captured pictorial information as video clips of a predetermined length, wherein each clip is captured a predetermined period of time before and/or after a storage command was issued.

21. The method according to claim 19, further comprising the step of adding and/or modifying computer program instructions and/or parameters through a programming interface in said sensor device with a an external computerized device having a user interface for said wearable sensor device, and storing said instructions and/or parameters in said first memory unit.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG. 1 shows the main components of the inventive system;

[0033] FIG. 2 shows a wristop computer and a camera unit according to the invention in more detail;

[0034] FIG. 3 shows an example of the inventive system when it is carried by the user;

[0035] FIG. 4 shows an example of the inventive system where one component of the system is stationary;

[0036] FIG. 5 shows a schematic representation of video clips recorded according to the present invention;

[0037] FIG. 6 shows a flowchart diagram over a video clip storage procedure according to the invention.

[0038] FIG. 7 shows a further example of the inventive system when it is carried by the user;

[0039] FIG. 8 shows the main components of the inventive system; and

[0040] FIG. 9 shows a sensor unit suitable for use in the inventive system in more detail.

DETAILED DESCRIPTION OF THE INVENTION

[0041] In FIG. 1 is shown the main components of one embodiment of the inventive system: a wearable sensor device 1, a unit 2 containing a video camera and external computerized device 3. All the connections between the devices need not to be two-way communications. The wearable sensor device 1 is only sending commands to the camera unit 2, not receiving anything from it.

[0042] The general-purpose computer, or a service accessed with it from the internet, may provide a programming interface for the wearable sensor device 1. Apart from obvious configuration data input needs, an important task for the programming interface in the computerized device 3 or the camera unit 2, in some embodiments and if so equipped, is to allow for input, setting or editing of computer program instructions and/or parameters for the various applications running in the wearable sensor device. These applications depend on the sensors available and the needs of the user, and the parameters may typically be sensor value trigger levels for activation of the camera unit 2, video clip recording times, and so on.

[0043] It is clear that a two-way communication between the computerized device 3 and the wearable sensor device 1 facilitates the use of a programming interface on the computerized system 3 that may be used for editing of predefined applications running in the wearable sensor device 1, or for adding new ones. It is also within the scope of the present invention to store the data and customized views in a format that is transferable as data files to other similar devices or social media, or to remote services e.g. on the internet for bookkeeping and further processing.

[0044] In FIG. 2 is shown a wearable sensor device 1 and a camera unit 2 in more detail. The wearable sensor device computer 1 has a processing unit 21 for running computer programs and applications. A user interface unit 22 may receive manual commands (on/off, start, reset, etc.) entered by the user e.g. by push-buttons 26. A first memory unit 23 is provided for storing operating systems, application program code and instructions and parameters. A sensor interface 24 provides a wired connection to one or more built-in sensors (not shown). Typically, the sensors may be accelerometers, gyroscopes or other orientation-detecting switches such as a compass, timers, GPS-devices, shock sensors, inclinometers and so forth.

[0045] A wireless signaling unit 25 is used for a one-way communication of commands to the camera unit 2 over a wireless link, as shown by the arrow L. The commands trigger storage of at least part of continuously captured pictorial information in a second memory unit of the camera unit 2.

[0046] The wireless communication protocol may be a Bluetooth LE or the proprietary ANT+protocol. These are using direct-sequence spread spectrum (DSSS) modulation techniques and an adaptive isochronous network configuration, respectively. Enabling descriptions of necessary hardware for various implementations for wireless links are available e.g. from the Texas Instrument's handbook Wireless Connectivity which includes IC circuits and related hardware configurations for protocols working in sub-1- and 2.4-GHz frequency bands, such as ANTI, Bluetooth, Bluetooth low energy, RFID/NFC, PurePath Wireless audio, ZigBee, IEEE 802.15.4, ZigBee RF4CE, 6LoWPAN, Wi-Fi, GPS.

[0047] The wireless signaling unit 25 may also provide a programming interface enabling a two-way connection to the external computerized device 3, which has a user interface for adding and/or modification of computer program instructions and/or parameters for said wearable sensor device, as described above. The instructions and/or parameters are stored in the first memory unit 23. The camera device 2 may have the required programming interface in-built, which may be used by a wired or a wireless connection with the wearable sensor device 1.

[0048] In FIG. 3 is shown a man on a snowboard with a wearable sensor device 30 attached to his wrist and a camera 31 attached to his forehead by some suitable fastening means. Obviously, the camera may be attached to any part of the clothing, or to the snowboard. The wireless link L provides for wireless communication from device 30 to 31. The wearable sensor device 30 will send instructions to the camera 31 to start the storage of video clips when the ground speed, the vertical speed or acceleration (on a steep slope) exceeds a certain limit, or the gravity (lack of vertical acceleration) is zero or below a certain limit, indicating that the snowboarder makes a jump in the air. The trigger function may be based on any physical variable the sensors may detect, and the trigger rules may be built by an editing function to consider any combination of such variables, as will be explained later in the examples. Design and editing software for the triggers are as such known in the art and are, for example, stored in the external computerized device 3 as shown in FIG. 1, or in a camera unit 2, such as a mobile phone provided with a camera and a programming interface application for the wearable sensor device 30.

[0049] During the performance, the camera 31 interprets received command signals and the storage of pictorial information is initiated.

[0050] In FIG. 4 is shown the use of the present inventive system in a downhill ski-related embodiment, where the camera 41 is stationary, and the wearable sensor device 40 attached to the wrist (or elsewhere) of a user is operated to activate the camera 41 via the wireless link L to store a clip when the user approaches or passes across the line-of-sight of the camera. In this way, repetitive footage can be taken e.g. for training and comparison purposes. It is clear that within the scope of the invention, any number of cameras can be placed at a distance from each other or at different angles with respect to a performance spot or along a route. The triggering of the cameras may be manual or automatic, using e.g. proximity sensing. The conditions at some spots of performance, like at a take-off ramp, easily create conditions that can make the sensors in the wearable sensor device 41 to trigger the camera to store the event. Other performance spots to be monitored, e.g. on board of a sailing ship, are more likely to be manually or proximity triggered.

[0051] FIG. 5 is a schematic and exemplary representation of video clips VC1 . . . VC6 stored in a semiconductor or magnetic memory. Thumbnail icons T1-T6 showing the content of each of the video clips may be looked at for immediate decision on how to deal with each of the clips. Of course, a variety of editing technologies exist and it is not in the scope of the present invention to deal with the art of managing video clips in detail.

[0052] FIG. 6 shows a flowchart diagram over the basic video clip storage procedure according to the invention. In step 60, the wearable sensor device 1 of FIG. 2 is monitoring the sensor signals. If the input signals at test 61 exceed the pre-set trigger threshold for start storing video footage, a command to this effect is issued in 62 and communicated in step 63 to the camera unit 2. The camera unit in this example, records the next 15 seconds of video footage, and stores it with the previous 15 seconds of captured video in one video clip, as those shown in FIG. 5. Obviously and equally well, the whole length of the clip could be stored from video material captured before the command, and stop at the command. Vice versa, the whole length of the clip could be stored from video material captured after the command and stopped after a predetermined time.

EXAMPLES

[0053] 1. Ski Stunts

[0054] A user wants to record his greatest jumps/stunts. After each successful jump, when a button 26 is pressed, 30 seconds of recorded video footage preceding the pressing of the button is stored. User can pre-define the length of stored footage.

TABLE-US-00001 //if user is on moving and in free fall for longer that 2 seconds: record a movie clip 30 sec before and 30 after. if (SUUNTO_SPEED > 10 && SUUNTO_G[2] < 1) { ACTIVATE.CAMERA_AFTERTHOUGHT(30); clipsrecorded = clipsrecorded+1; //show how many clips have been recorded RESULT = clipsrecorded; }

[0055] 2. Using a Stationary and a Moving Device

[0056] Here, a user wants to get short clips without the need/possibility to manipulate the wearable sensor device on the go. Proximity determination can be based on signal strength sensing, or it can be location-based using GPS. For example, the Bluetooth Low Energy (BLE) standard features proximity sensing. Other signal strength based options include ultrasound sensing e.g. in underwater applications or short-range inductive wireless (telemetric) devices. In location based proximity sensing, both the wearable sensor device and the camera may have GPS capabilities, or only the device on the move, provided that the position of the stationary device is fed into the GPS navigator.

[0057] Snowboarding with a helmet mounted camera, divers passing a stationary camera, sailing boats passing a turning point buoy are examples of useful applications in this regard. Filming a skateboarding trick with a camera mounted in the rail is also feasible, e.g. by using the following code:

TABLE-US-00002 //If user is approaching the camera start filming. if(SUUNTO_PROXIMITY < 15 && recording == 0){ ACTIVATE.CAMERA_RECORD( ); recording = 1; //when user is gone, stop recording }else if (SUUNTO_PROXIMITY >15 && recording == 1){ ACTIVATE.CAMERA_STOP( ); recording = 0; }

[0058] 3. Mountain Biking

[0059] User wants to film the downhill trails (maintaining a high speed) of a mountain bike expedition. He wants to automatically control the camera when hands can't be removed from the handlebar.

TABLE-US-00003 if (SUUNTO_SPEED > 30 && recording == 0) { ACTIVATE.CAMERA_RECORD( ); recording = 1; }else if (SUUNTO_SPEED < 20 && recording == 1){ ACTIVATE.CAMERA_STOP( ); recording = 0; }

[0060] During such a trail, the user likely wants to record the best views, spots, jumps etc. Here the buttons of the wearable sensor device can be used to bookmark the spots for future editing. The bookmarks can be saved to camera memory as short clips, to allow faster editing.

[0061] With reference to FIGS. 7-9, an further embodiments of the inventive system and method for remote activation of a storage operation of pictorial information is shown and described herein as follows:

[0062] The inventive system as used by a person 70 in FIG. 7 comprise wearable sensor devices 71, 72 and/or 73, which can be integrated in the equipment, like in a shoe (sensor 71), a garment (sensor 72), or worn detachably on the body (sensor 73), or simply in a pocket (not shown). If a sensor 71-73 includes acceleration sensors, its location on the person or the sports equipment may preferably be selected according to the sports discipline or other movements during the performance that will initiate the desired commands to be sent over wireless link 76 to a digital recording device, here depicted as a smart phone 74 rigged onto a stick 75. In response to such commands, the camera in the device 74 will be triggered to store in its memory (a second memory unit) a predefined part of the continuously captured pictorial information. Obviously, a camera 74 can also be carried on the person 70, as shown in FIG. 3.

[0063] The sensor device 71-73 comprises at least one sensor. The sensors included in the device may track movements by means of measuring acceleration, directions by means of a magnetometer, location by means of a GPS sensor, the height or changes in height by means of an air pressure sensor, etc. The sensor device may also have a timer function that may send out one or intermittently several commands at predetermined time(s), or it may delay the sending of a command with a predetermined time delay after the triggering event has occurred, for example.

[0064] The receiving device 74 may perform further calculations based on the received commands over the wireless link. If commands are sent out in a sequence, e.g. based on a timed function or asynchronously as sequential triggering events occur, the receiving device may measure the received signal strength and start the storage of the continuously captured pictorial information when the signal strength reaches a predefined threshold level, for example. Similar location-based calculations can be made based on a GPS sensor that is set to send a command when reaching a particular location, for example.

[0065] As explained in connection with FIG. 1, a programming interface on a separate computerized system, or in-built in the camera 74, may be used for editing of applications running in the wearable sensor device 74. Such a programming interface is for example a software having capability of controlling a wired or wireless interface 25, 93 of the wearable sensor device, to access the first memory unit (items 23 and 92 in FIGS. 2 and 9, respectively) in the wearable sensor device and to store the additions/changes made in that first memory unit.

[0066] With reference to FIG. 8, the inventive system shown comprises a wearable sensor device 80, here attached to a chest belt 81, a one-way wireless link 82 and a recording device 83.

[0067] The sensor device is now described in more detail in connection with FIG. 9. The sensor device 90 comprises at least one sensor 91, a processing unit 94 for running computer programs, a first memory unit 92 for storing instructions for the computer programs, and a wireless signaling unit 93 for communicating with external devices over a one-way wireless link 95. The processing unit 94 is adapted, based on sensor signals received from the sensor(s) 91, to send commands over the wireless link 95 to a second digital device 74, 83 to trigger, in response to said commands, storage of at least part of the continuously captured pictorial information in a memory of the second digital device.

[0068] According to some embodiments, the sensor device is a sensor module that is detachably attached to an apparel or equipment of a user. According to some embodiments, the sensor device is a sensor module that is integrated in a garment worn by the user. According to some further embodiments, the sensor device is a sensor module that is integrated in a shoe of the user. According to some embodiments, the sensor device may also be a sensor module that is integrated in sports equipment used by the user.

[0069] The second digital device is according to some embodiments a wearable digital camera 83. According to some embodiments, the second digital device is a smartphone 74 equipped with at least one digital camera. According to some embodiments, the second device may more than one digital camera.

[0070] According to the invention, the second digital device may be adapted to store captured pictorial information as video clips in its memory a predetermined period of time before and/or after having received a storage command, as explained in connection with FIG. 6.

[0071] According to the inventive method, the activation of a storage operation of pictorial information includes the steps of: [0072] providing a wearable sensor device 71-73; 80; 90 comprising at least one sensor 91 providing sensor signals, a processing unit 94 for running computer programs, a first memory unit 92 for storing instructions for said computer programs and a wireless signaling unit 93 for communicating with external devices 74, 83 over a wireless link 76, 82, 95; [0073] providing at least one second digital device 74, 83 comprising a recording apparatus adapted to be set to continuous capture of pictorial information, a second memory unit, and a wireless signaling system for communicating with external devices over a wireless link; [0074] producing in said wearable sensor device 71-73; 80; 90, based on said sensor signals, commands to instruct storage of pictorial information in said second device 74, 83; [0075] transmitting by means of one-way transmission of information said commands over said wireless link 76, 82, 95 to said second device; [0076] receiving said commands in said at least one second device 74, 83 and triggering in response to said commands storage of at least part of said continuously captured pictorial information in a memory of said at least one second device.

[0077] The second digital recording device 74, 83 is here by default presumed to have a second memory unit to capture, and when so instructed, to store selected pictorial information in a predetermined way, for example as clips as discussed in connection with FIGS. 5 and 6. The captured pictorial information may thus be stored as video clips of a predetermined length. Each clip may thus be captured a predetermined period of time before and/or after a storage command was issued.

[0078] It is to be understood that the embodiments of the invention disclosed are not limited to the particular systems and process steps disclosed herein, but are extended to equivalents thereof as would be recognized by those ordinarily skilled in the relevant arts. It should also be understood that terminology employed herein is used for the purpose of describing particular embodiments only and is not intended to be limiting.

[0079] Reference throughout this specification to one embodiment or an embodiment means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrases in one embodiment or in an embodiment in various places throughout this specification are not necessarily all referring to the same embodiment.

[0080] In addition, various embodiments and example of the present invention may be referred to herein along with alternatives for the various components thereof. It is understood that such embodiments, examples, and alternatives are not to be construed as de facto equivalents of one another, but are to be considered as separate and autonomous representations of the present invention.

[0081] Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Numerous specific details are provided, such as examples of lengths, widths, shapes, etc., to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention can be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the invention.

[0082] While the forgoing examples are illustrative of the principles of the present invention in one or more particular applications, it will be apparent to those of ordinary skill in the art that numerous modifications in form, usage and details of implementation can be made without the exercise of inventive faculty, and without departing from the principles and concepts of the invention. Accordingly, it is not intended that the invention be limited, except as by the claims set forth below.