APPARATUS FOR CONTROLLING RADIOFREQUENCY SENSING
20240282182 ยท 2024-08-22
Inventors
Cpc classification
G08B13/2494
PHYSICS
G08B13/1672
PHYSICS
G08B29/188
PHYSICS
G08B13/1645
PHYSICS
G08B21/0492
PHYSICS
International classification
Abstract
The invention relates to an apparatus for controlling radiofrequency sensing and audio sensing of a network. The network (100) comprises a plurality of network devices (111, 112, 113, 114), e.g. smart lights, and is adapted to perform radiofrequency sensing and audio sensing utilizing one or more of the network devices. A context parameter providing unit (121) provides context parameters, wherein the context parameters are indicative of a context in which the radiofrequency sensing and the audio sensing is performed. A controlling unit (122) controls the radiofrequency sensing and the audio sensing of the network in dependency of each other based on the context parameters. This allows for an improved monitoring of an area.
Claims
1. An apparatus for controlling radiofrequency sensing and audio sensing of a network, wherein the network comprises a plurality of network devices, and wherein the network is adapted to perform radiofrequency sensing and audio sensing utilizing one or more of the network devices, wherein the apparatus comprises: a context parameter providing unit for providing context parameters, wherein the context parameters are indicative of a context in which the radiofrequency sensing and the audio sensing is performed, and a controlling unit for controlling the radiofrequency sensing and the audio sensing of the network in dependency of each other based on the context parameters, wherein the controlling of the radiofrequency and the audio sensing performed in dependency of each other refers to a controlling based on functional relations between changes in sensing parameters of the radiofrequency sensing and the audio sensing; wherein the controlling unit is adapted to apply predetermined instructions that define a specific orchestration of the radiofrequency sensing and the audio sensing of the network based on the context parameters, wherein the specific orchestration refers to a specific controlling of sensing parameters utilized for the radiofrequency sensing and the audio sensing of the network in dependency of each other; wherein an orchestration refers to the application of interrelated instructions with respect to the radiofrequency sensing and the audio sensing, and said interrelated instructions indicate a controlling relationship between certain sensing parameters of the radiofrequency sensing and the audio sensing such that when one of these sensing parameters is changed or adapted also the respective related sensing parameter is also changed or adapted according to the controlling relationship.
2. The apparatus according to claim 1, wherein the context parameters are indicative of a current sensing situation referring to at least one of a frequency dependent transmission range of audio and/or radiofrequency signals of one or more network devices, a spatial confinement of the audio and/or radiofrequency signals, at least one physical dimension of a to be sensed subject, a possible and/or allowable radiofrequency and/or audio frequency range provided by one or more of the network devices, an allowable nonaudible sound pressure dose and current or expected presence, absence or constellation of one or more subjects in a sensing area of the network.
3. (canceled)
4. The apparatus according to claim 1, wherein the instructions include as specific orchestration adapting the wavelength of audio signals utilized for the audio sensing to be similar to the wavelength of the radiofrequency signals utilized for radiofrequency sensing, when the context parameters indicate a predetermined current sensing situation that requires an increase of the sensing accuracy.
5. The apparatus according to claim 4, wherein the instructions include a specific orchestration adapting the wavelength of audio signals utilized for the audio sensing to be different from the wavelength of the radiofrequency signals utilized for radiofrequency sensing, when the context parameters indicate a predetermined current sensing situation that requires an increase in the sensing diversity.
6. The apparatus according claim 5, wherein the instructions include as specific orchestration coordinating a wavelength hopping of audio signals utilized for the audio sensing with a wavelength hopping of the radiofrequency signals utilized for radiofrequency sensing, when the context parameters indicate a predetermined current sensing situation with an environmental audio and/or radiofrequency noise above a predetermined threshold and/or when the context parameters indicate a predetermined current sensing situation that requires an increase of the sensing accuracy.
7. The apparatus according to claim 6, wherein the coordination of the wavelength hopping refers to a synchronization of the wavelength hopping of audio signals utilized for the audio sensing with the wavelength hopping of the radiofrequency signals utilized for radiofrequency sensing in the time domain.
8. The apparatus according to claim 1, wherein the specific orchestration can further include coordinating a beam steering of the audio signals utilized for the audio sensing with a beam steering of the radiofrequency signals utilized for radiofrequency sensing such that both the radiofrequency sensing and the audio sensing is performed with respect to the same sensing area at the same time.
9. The apparatus according to claim 1, wherein the instructions include as specific orchestration performing audio sensing with a higher spatial resolution than the radiofrequency sensing, when the context parameters indicate an occurrence of a fall in the context of a fall detection or when the context parameters indicate an occurrence of a gesture in the context of a gesture detection.
10. The apparatus according to claim 1, wherein the controlling of the audio sensing comprises filtering out parts of the audio sensing signal based on the context parameters, wherein the audio sensing is performed based on the filtered audio sensing signals.
11. The apparatus according to claim 1, wherein for applications that more than one sensing task is desired to perform by the sensing network, the controlling comprises assigning the radiofrequency sensing and the audio sensing to different sensing tasks based on the context parameter and adapting the radiofrequency sensing and the audio sensing to the respective assigned tasks.
12. The apparatus according to claim 1, wherein the controlling comprises assigning the radiofrequency sensing and the audio sensing to different parts of a sensing area based on the context parameter and adapting the radiofrequency sensing and the audio sensing to perform the respective sensing in the assigned part.
13. A network comprising: a plurality of network devices adapted to enable the network to perform radiofrequency sensing and audio sensing in a sensing area concurrently, an apparatus according to claim 1.
14. A method for controlling radiofrequency sensing and audio sensing of a network, wherein the network comprises a plurality of network devices, and wherein the network is adapted to perform radiofrequency sensing and audio sensing utilizing one or more of the network devices, wherein the method comprises: providing context parameters, wherein the context parameters are indicative of a context in which the radiofrequency sensing and the audio sensing is performed, and controlling the radiofrequency sensing and the audio sensing of the network in dependency of each other based on the context parameters. wherein the controlling of the radiofrequency and the audio sensing performed in dependency of each other refers to a controlling based on functional relations between changes in sensing parameters of the radiofrequency sensing and the audio sensing; applying predetermined instructions that define a specific orchestration of the radiofrequency sensing and the audio sensing of the network based on the context parameters, wherein the specific orchestration refers to a specific controlling of sensing parameters utilized for the radiofrequency sensing and the audio sensing of the network in dependency of each other; wherein an orchestration refers to the application of interrelated instructions with respect to the radiofrequency sensing and the audio sensing, and said interrelated instructions indicate a controlling relationship between certain sensing parameters of the radiofrequency sensing and the audio sensing such that when one of these sensing parameters is changed or adapted also the respective related sensing parameter is also changed or adapted according to the controlling relationship.
15. A computer program product for controlling radiofrequency sensing and audio sensing of a network, wherein the computer program product is adapted to cause an apparatus according to claim 1 when run on the apparatus.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In the following drawings:
[0031]
[0032]
[0033]
DETAILED DESCRIPTION OF EMBODIMENTS
[0034]
[0035] The network devices 110, 111, 112, 113 are in this example all adapted to transmit and receive radiofrequency signals 114 and audio signals 115 for performing radiofrequency and audio sensing. Examples, for more details on how to perform generally radiofrequency sensing can be found, for instance, in the book Wireless AI: Wireless Sensing, Positioning, IoT and Communications by K. J. R. Liu et al., Cambridge University Press, 2019. Examples, for more details on how to perform general audio sensing can be found, for instance, in the article Contactless Infant Monitoring using White Noise by A-Wang et al., The 25th Annual International Conference on Mobile Computing and Networking (MobiCom '19), article 52, pages 1 to 16 (2019) or in the article AIM: Acoustic Imaging on a Mobile by W. Mao et al., MobiSys '18: Proceedings of the 16.sup.th Annual International Conference on Mobile Systems, Applications, and Services, pages 468 to 481 (2018). However, in other embodiments also only a part of the network devices can be adapted to transmit and/or receive radiofrequency signals 114 or audio signals 115. Preferably, at least some of the network devices 110, 111, 112, 113 further provide a lighting functionality and can thus be regarded as smart lights in a smart home environment. In this example, the network 100 comprises further the apparatus 120, wherein the apparatus 120 comprises a context parameter providing unit 121 and a controlling unit 122. The apparatus 120 is adapted to control the radiofrequency sensing and the audio sensing of the network 100. For example, the apparatus 120 can be in a wired or wireless communication 123 with one or more of the network devices 110, 111, 112, 113. However, the apparatus 120 can also be realized as an integral part of one or more of the network devices 110, 111, 112, 113, wherein in this case the apparatus 120 can be realized, for instance, as general or dedicated hardand/or software on one or more of the network devices 110, 111, 112, 113. Moreover, the apparatus 120 can also refer to software running on a dedicated or general computing structure, for instance, a personal computer, a smartphone, a cloud, etc. For example, if the apparatus 120 is realized as being part of a cloud the apparatus 120 can be adapted to communicate indirectly with one or more of the network devices 110, 111, 112, 113, for instance, via a respective application provided on a device being in communication with the network.
[0036] The context parameter providing unit 121 is adapted for providing context parameters. The context parameters are generally indicative of a context in which the radiofrequency sensing and the audio sensing of the network is performed. For example, the context parameters can refer to an environment in which the radiofrequency sensing and the audio sensing is performed, wherein in this case the context parameters refer to external network parameters. In this case the context parameters can refer, for instance, to a physical dimension of a to be sensed or monitored subject, a position and/or characteristic of a confinement of the radiofrequency and/or audio sensing of the network, a position and/or characteristic of different subjects, like furniture in a sensing environment of the network 100, etc. Such external network parameters can be provided, for instance, by an input of a user during a configuration of the network 100. However, such external network parameters can also be a result of a measurement in the environment of the network 100. Such a measurement can be performed, for instance, by a camera, a LIDAR or any other measurement device that allows to provide indications on the environment of the network 100. In a preferred embodiment at least some of the external network parameters are derived from results of the radiofrequency and/or audio sensing itself. The context parameters can, however, also refer to internal network parameters indicative of an internal situation of the network and/or the network devices of the network 100. Such internal network parameters can refer, for instance, to general possible and/or allowable radiofrequency and/or audio frequency ranges provided by one or more of the network devices, to generally provided functionalities of the network devices, to a position of one or more network devices with respect to each other, etc. Moreover, the internal network parameters can also refer to a current sensing situation, and then be indicative of a current setting of the network devices, for instance, to currently utilized radiofrequency or audio sensing parameters, to a current status of one or more of the network devices, for instance, whether they are in a sleep state, in a awake state, in a sensing state, etc. Generally, it is preferred that the context parameters independent on whether they refer to external or internal network parameters, are indicative of a current sensing situation of the network 100, i.e. refer to a current state of the environment and/or internal working of the network. In particular, in this case it is preferred that the current sensing situation is derived from results of the radiofrequency and/or audio sensing of the network itself.
[0037] Based on their nature, the context parameters can be, for instance, stored on a storage device to which the context parameter providing unit 121 has access for providing the respective context parameters. However, if the context parameters are directly measured and refer to a current situation, i.e. to a current measurement, a context parameter providing unit 121 can also be adapted to receive these context parameters in real time, for instance, during a direct communication with the respective sensing device or during an indirect communication with the respective sensing device.
[0038] The controlling unit 122 is adapted to control the radiofrequency sensing and the audio sensing of the network in dependency of each other based on the context parameters. For example, the controlling unit is adapted to apply predetermined instructions that define a specific orchestration of the radiofrequency sensing and the audio sensing. In this case, predetermined instructions can be stored on a storage unit together with the respective context parameters which indicate a respective current sensing situation in which the predetermined instructions should be applied. Thus, the controlling unit 122 can be adapted to determine based on the context parameters directly or based on the current sensing situation derived from the context parameters which predetermined instructions should be applied. The predetermined instructions can be predetermined by a user, for instance, based on experience, knowledge on the respective situation that could be applied, based in respective sensing goals, etc. However, the predetermined instructions can also be provided by a manufacturer or by a professional based on general knowledge of optimal instructions in commonly occurring sensing situations. Generally, the controlling unit 112 is adapted to control the radiofrequency sensing and the audio sensing of the network in dependency of each other, in particular, to orchestrate the radiofrequency sensing and the audio sensing. Such an orchestration refers to taking into account during the controlling of one of the sensing modalities a current controlling of the respective other sensing modality. For example, if the controlling refers to an amending of sensing parameters of the radiofrequency sensing current or also amended sensing parameters of the audio sensing are taken into account. Thus, the radiofrequency sensing and the audio sensing are not performed independent of each other.
[0039]
[0040] In the following more detailed embodiments will be described, for instance, with respect to preferred combinations of sensing situations and respectively preferred predetermined instructions for the orchestration of the radiofrequency sensing and the audio sensing of the network.
[0041] Generally, occupancy detection and activity-tracking can be performed with active sound sensing, for instance, sound sensing utilizing Amazon Alexa device hardware performing both the transmission and reception functions required by the active audio sensing. Also WiFi-based or ZigBee-based radiofrequency sensing achieves a good presence detection performance but lacks the capability to locate people accurately. On the other hand, audio sensing is capable to locate people very well, mainly due to its ability to widely vary the audio wavelength. For example even with commonly available commodity speaker/microphone hardware the available audio wavelength ranges from 1.7 cm at 20 KHz audio to 1700 cm at 20 Hz. However, audio sensing is relatively easily adversely affected by noise in the environment, e.g. HVAC noise. Additionally, as most of the audio wavelengths available from the commodity speakers/microphone hardware are within the human-audible regime, this imposes application constraints on the wavelength choice. For instance, the current context of the room will determine which specific subset of the available audio sensing frequencies are regarded currently acceptable to the end user.
[0042] In this context, the inventors have found that the fusion of radiofrequency sensing and passive audio sensing can provide an efficient and affordable way, for instance, to detect the presence of persons in a room, to track persons in a building, to monitor breathing and/or daily activities, to detect fall accidents, etc.
[0043] Generally, sound waves and radiofrequency signals are fundamentally different. Sound waves are longitudinal compression waves of the air, while electromagnetic waves are transverse waves made up of electric and magnetic fields. Nevertheless, there are also some similarities between wireless and acoustical wavelengths, for instance, which subset of wavelengths, as compared to a certain physical object size, will interact strongly with the object itself or diffract around an edge of an object. How audio and radiofrequency signals propagate across a room is not just affected by a room's layout but also, for instance, by building materials and the chosen radio/audio frequency. For example, detailed measurements of wireless radiofrequency attenuation for various building materials and showcases how in practice the building materials used within the specific building space strongly determine how uniform the radio waves fill up the room are readily available. Moreover, besides the construction form of the room itself, also the clutter inside the room, such as furniture, and people can interact with the wireless waves, for instance, it is generally recommended to model this clutter as scatter sources even at 2.4 GHz radio frequencies. Further, also for audio waves the sound-wave propagation is affected by the construction form and building materials in the room. For instance, the Sound Transmission Class, or STC of a wall describes the difference in magnitude between the incident audio sound wave and the transmitted sound wave. For instance, a sustaining wall in a house or office has an STC of 33, meaning that it can attenuate 33 dB from one side of the wall to the other. While in general the signal attenuation of a 2.4 GHz wireless radiofrequency signals by building materials is often considerably less than audio sensing, there are in praxis many exceptions, such as for metal-coated glass surfaces often used in commercial buildings. In addition, unlike for sound, the wireless radiofrequency signal attenuation of a dividing wall will disproportionally increase with higher WiFi frequencies. For audio applications, often a Noise Reduction Coefficient metric, or NRC is utilized which describes for a certain building material the fractional ratio between the reflected sound wave and the incident sound wave. The NRC varies widely from room to room, e.g. smooth concrete surface reflects most sound energy, while a sound absorbing felt-covered surface has almost perfect absorption and hence will hinder a multipath audio sensing signal propagation in the room. For example, it is well known that an acoustical designer may deliberately add diffusion elements, e.g. a rough brick wall, to a building space, e.g. when creating a home theatre for an audiophile customer. The added sound diffusion elements prevent for specific audio frequencies the formation of unwanted audio standing waves in the home theatre. However, unlike in the millionaire's home theatre, normal rooms, e.g. an office conference room, typically may have many smooth surfaces such as glass, smooth walls, stone floors, etc., which are known to create more echoes and reflections and thereby create standing audio waves in the room which are diametral to audio sensing. As a room with all-absorptive surfaces is not a going to be a good-sounding room, even a high-end home theatre will use a balance of diffusion and absorption to achieve a good audio environment. Reflection of audio sensing sound waves off surfaces in the room can also lead to reverberation, i.e. unwanted prolonging of a sound in human perception. Prior art teaches that reverberation occurs in a small room with height, width, and length dimensions of approximately 17 meters or less. This is due to the human brain keeping a sound in memory for up to 0.1 seconds, hence if a reflected, sufficient sound wave reaches the ear within 0.1 seconds of the initial sound, it causes reverberation, consequently, the sound sensing pings become more pronounced and annoying to the human ear. Audio waves also interact with physical objects via diffraction, i.e. change in propagation direction of waves as they pass through an opening or around a barrier in their path. Diffraction enables the sound wave to travel around corners, around obstacles and through openings. The amount of diffraction, i.e. the sharpness of the bending around an object/opening, is dependent on the chosen audio wavelength. Diffraction effects are known to increase with increasing audio wavelength. If the wavelength becomes smaller than the obstacle or opening, no noticeable diffraction of the audio waves occurs. Audio waves also experience refraction effects, i.e. the bending of the path of the waves. Audio waves, being longitudinal waves physically interacting with matter, are known to get much stronger refracted than wireless waves, for instance when the waves move through different layers of air with gradually varying temperatures. The most well-known example for refraction is sound waves traveling over water, since water has a moderating effect upon the temperature of air, the air directly above the water tends to be cooler than the air far above the water. Sound waves travel slower in cooler air than they do in warmer air. For this reason, the portion of the wavefront directly above the water is slowed down, while the portion of the wavefronts far above the water speeds ahead. Subsequently, the direction of the sound travelling over the water changes, refracting downwards towards the water. In audio sensing, certain environments such as industrial facilities, e.g. food processing, will have temperature gradients between a first zone and a second zone serving different tasks, e.g. cooking vs preparing.
[0044] Based on the above described background the invention as already described above can utilize, for instance, already existing microphone sensors as well as the ZigBee radios to provide an orchestrated hybrid audio- and sound-sensing solution, for instance, for a richer activity tracking or fall detection. A simple audio speaker can be utilized for the audio sensing, for instance, as standalone network device or embedded into a network device with another functionality, like a light controller, a wall switch or a fixture such as a downlight with Amazon Alexa.
[0045] Moreover, based on the above background, it is clear that for wireless waves generally the shape of a room, its building materials, its environmental parameters and the size of objects present in the room can determine an optimal audio wavelength choice for audio sensing and an optimal wireless radiofrequency signal wavelength choice for radiofrequency sensing. Further, the maximum obtainable sensing performance of the hybrid sensing system for a chosen combination of a first audio-sensing wavelength and a second radiofrequency-sensing-wavelength can be determined, for instance, based on the above parameter. Hence, in practice a first room, e.g. living room, may be a suitable acoustic environment for a first combined set of audio and radiofrequency-sensing sister wavelengths, while a second room, e.g. kitchen, may require a different combined set of wavelengths. Additionally, of course also the expected to-be-detected activities can be taken into account when selecting the respective sensing wavelengths.
[0046] Compared to radiofrequency sensing, audio sensing is vastly more versatile. Unlike radiofrequency sensing, when utilizing audio sensing already the most basic audio-sensing system using only a standard low cost microphone and speaker hardware allows to vary the audio sensing wavelength over a wide range from 1.7 cm up to 17 m, i.e. from 20 KHz to 20 Hz, hence audio sensing is well-equipped for dynamic-frequency sensing, while radiofrequency sensing is generally ill-equipped for wide-range dynamic-frequency sensing as very expensive hardware will be needed to vary the wireless carrier frequency over a wide range of frequencies. Audio sensing allows for both short wavelengths comparable to WiFi, e.g. 2.5 cm, as well as providing extremely long wavelengths well beyond the ability of a commodity wireless radio, for instance, it is easily possible for audio sensing to generate even a wavelength similar to the torso size of the to be tracked person, hence maximizing the interaction of the audio waves with the torso. On the other hand, if, for instance, such a 70 cm wavelength should be generated with an wireless radio, the associated 400 MHz wireless frequency would require a very different radiofrequency front-end than current commonly utilized 2.4 GHz-only commodity WiFi/Zigbee radios. In an example, the similar wavelength may comprise wavelengths having the same order of magnitude, and wherein the different wavelength may comprise wavelengths having a different order of magnitude. Additionally, or alternatively, similar wavelength may refer to be similar if the difference between the wavelengths, e.g., wavelengths of the audio and radiofrequency signals does not exceeds a threshold, and wherein different wavelength may refer to be different if the difference between the wavelengths, e.g., wavelengths of the audio and radiofrequency signals exceeds a threshold.
[0047] In this context, the invention, as for instance described with respect to
[0048] For example, ZigBee-based or WiFi-based radiofrequency sensing achieves a good presence detection performance but can at a given moment be prone to wireless interference in a certain wireless wavelength regime, e.g. disturbances due to 5 GHz video streaming or actuation of a 2.4 GHz microwave oven. For instance, if the radiofrequency sensing may be forced by the disturbances to switch to 2.4 GHz instead of 6 GHz WiFi, the new longer wireless wavelength will result in a loss of the capability to precision-locate the people. On the other hand, audio sensing has been shown to be capable to locate people well, mainly due to its ability to widely vary the audio wavelength. Even with commodity audio speaker/microphone hardware, the available audio wavelengths range from 1.7 cm for a 20 KHz audio tone to 1700 cm for a 20 Hz tone. However, audio sensing can be adversely affected by ambient audio noise in the environment. Hence if at a given moment a certain ambient-noise disturbance starts to occur, it immediately renders certain audio wavelengths useless for audio sensing purposes. As audio noise in real life environment is mostly below 10 kHz, especially lower audio sensing frequencies are affected by ambient audio noise. The presence of environmental audio noise may hence limit the available bandwidth of the audio sensing system and consequently its sensing performance. Additionally, many of the audio frequencies utilized by prior-art audio-sensing systems are within the human-audible regime. This imposes real-time application constraints on which subset of audio wavelengths are at a given moment acceptable to be used for audio sensing depending on the context in the room.
[0049] The inventors have in this context recognize that radiofrequency signals and audio signals have different propagation mechanisms and are subject to different sources of interference. In an embodiment, the control unit as for instance described above can thus utilize instructions that lead to a controlling such that the audio sensing uses an audio wavelength similar or vastly different than the wavelength used by the radiofrequency sensing. Such instructions can thus refer to a deliberate orchestration of the audio sensing and radiofrequency sensing, in particular, with respect to the used wavelengths, to be applied in a given situation. Such an orchestration can, for instance, both minimize interference from external noise sources and enrich the sensing-signal diversity and thereby enhance the accuracy of the hybrid context awareness sensing system. Exemplarily such instructions can be utilized in situations in which a double-checking of the sensing results between the two sensing modalities is desired, e.g. for estimating the size of an object, like is it has to be differentiated between a child and an adult.
[0050] As described above, for controlling the sensing modalities in dependency from each other several parameters, i.e. context parameters, can be taken into account, for instance, when choosing an instructions referring to an orchestration of the audio-sensing wavelength and the radiofrequency sensing wavelength. The context parameters can refer, for example, to a) a frequency-dependent transmission range of the radiofrequency sensing and audio sensing signals, respectively, b) a respective spatial confinement of the radiofrequency and audio sensing signal, e.g. using shorter radiofrequency wavelength results in the sensing to be more directional/confined, c) a currently available/allowable radio- and audio wavelengths, for instance, due to hardware constrictions, audio/wireless noise interference and other real-time application constraints, d) a currently available/allowable radio- and audio messaging rate, whereas the messaging rate is determined by wireless congestion, missed-message rate, power consumption and real-time application constraints such as unobtrusive embedding of audio sensing signals in the background music, e) an accuracy of the audio- and radiofrequency signal arrival angle detection for a pair of network devices monitoring a specific area within the room for the currently chosen sensing wavelengths, respectively, for instance, for a first detection zone, the angel of arrival detection with the chosen radiofrequency-sensing wavelength may enable less precise localization of a subject, while audio beamforming may allow for a more accurate and faster positioning or tracking, or f) a physical dimension of a to-be-traced subject, e.g. forklift vs person.
[0051] In an embodiment, the control unit can be adapted to utilize instructions for the controlling of the orchestration between radiofrequency and audio sensing that evolve over time depending on the context parameters. For instance, the room's context parameters, e.g. referring to the room as a hospital room, can at first indicate to utilized instructions that only allow for a very low dB audio level signal materially inaudible to an human occupant and hence restricting the applied audio sensing tone. This restriction will result in audio-sensing only being able to deliver rather inaccurate breathing detection.
[0052] Ultrasound at sufficient sound pressure levels can cause hearing damage in humans even if it cannot be heard. Even a device designed to be safe for humans may cause nuisance or harm to pets and other animals due to their extended range of hearing. For ultrasound devices emitting airborne ultrasound the auditory and subjective risks are the most relevant. Safety standards and guidelines have been developed with the goal of protecting against hearing damage in humans. Audio sensing will cause humans and animals to be exposed to ultrasound for longer durations than previously considered.
[0053] Hence, it is desired that the inaudible audio sensing solution considers the hearable sound and ultrasound perception and safety in humans and animals, when the audio sensing selects the sound pressure levels and length of duration of administering the audio sensing sound levels. Specifically, the instructions used for inaudible audio sensing can limit the audio sensing sound pressure depending on the current position of the human with respect to the audio sensing transmitter as well as the presence and location of animals in the room.
[0054] In this case, the instruction can indicate a controlling of the sensing modalities such that the radiofrequency sensing nodes, i.e. network devices, utilize mm-wave WiFi to perform breathing and fall detection, albeit covering with the radiofrequency sensing only a small portion of the room due to the confined nature of mm-wave WiFi sensing signals, wherein at the same moment the audio sensing is controlled to choose an audio wavelength well suited to perform coarse motion detection at the full room level to eliminate the motion detection blind spots left by the mm-wave WiFi sensing. If after a while a change in the room's context parameter suddenly allows for human-audible audio sensing, for instance, since the context parameter indicate that a person in the room is listening to music, the control unit can adapt the utilized instructions such that the breathing detection task is assigned to the audio sensing network devices and the radiofrequency sensing network devices switch to 5 GHz WiFi that is well suited to perform coarse motion detection at the full room level. The audio sensing network devices can then be controlled to select audio sensing wavelengths which are optimal for tracking the breathing motion of an individual. Moreover, in an embodiment the control unit can be adapted to utilize instructions that lead to an initiation of an audio sensing change after radiofrequency sensing results indicate a fall, i.e. imply that an elderly person may be now lying on the floor. In particular, based on this result, the audio sensing system can be controlled, as indicated by the instructions, to switch to a highly audible, intrusive audio sensing signal, e.g. a linear increase in the emitted audio frequency over 100 ms from 20 Hz to 16 KHz, that enables the audio sensing to perform high-accuracy breathing detection and precision-location of the height of the chest and thereby deduce whether the elderly is really lying on the floor. Further, in such an embodiment, the apparatus can also be adapted to ask a user, for instance, via audio speaker of the network devices to provide feedback via a gesture which is sensed by the audio- and/or radiofrequency sensing modality. In an embodiment, the instructions can also indicate, for example, to tune the sensing modalities, to occasionally perform an elaborate, partially audible audio-sensing calibration scan even if the context parameters indicate the presence of people, e.g. the audible calibration can last less than 2 s. For instance, the instructions can refer to using such an audio scan at times when the context parameters of a room indicate that the person will not be annoyed by the audible noise, e.g. when the audio-sensing calibration just comprise a couple of additional audio sensing beeps at the end of every public-service-announcement message via the intercom.
[0055] In an embodiment the context parameter can also be indicative of occurring mechanical vibrations of a network device, for instance, being a luminaire that can, for example, swing in the wind, or influences by mechanical work in the environment. Such mechanical vibrations can be especially diametral for fine grained radiofrequency sensing, e.g. breathing detection. To minimize the negative impact of vibrations on the sensing performance, the instructions corresponding to such situations indicated by respective context parameters, can refer to a switching or adjust of at least one of the sensing wavelengths of at least one of the sensing modalities such that the updated sensing wavelength is substantially longer than the current physical oscillation of the network device.
[0056] Certain wireless protocols such BLE or WiFi sequentially hop through different radiofrequency channels, for instance, of a predefined set of frequency channels, to avoid wireless interference. Generally, presently available WiFi/BLE radio stacks can be utilized to allow for a purposefully orchestration of the frequency hopping to optimize the radiofrequency sensing performance. In an embodiment, the orchestration of the sensing modalities can thus refer to a coordination of the radiofrequency sensing and audio sensing such that the frequency hopping of the audio sensing and the frequency hopping of the radiofrequency sensing system are purposefully coordinated and synchronized. Compared to just independently running two independent frequency-hopping schemes for radiofrequency sensing and audio sensing in parallel, this orchestration ensures that the sensing modalities will deliver reproducible performance compared to an uncoordinated frequency hopping which will continuously yield different permutations of BLE sensing wavelength and audio sensing wavelengths. Hence, synchronizing the time-series of the applied respective audio- and radiofrequency-wavelengths will, for instance, enable machine learning sensing AI algorithms that can be utilized for one or both sensing modalities to be better trained and hence improve the performance of the sensing system. In the simplest implementation, in this embodiment the audio sensing can be controlled to utilize on purpose the identical sequence of audio wavelengths as the frequency hopping radiofrequency sensing modality. For example, in a simple case the radiofrequency sensing modality performs a single hop from 2.4 GHz referring to a wavelength of 12.5 cm to 5 GHz referring to a wavelength of 6 cm. In this case the audio sensing modality will be controlled accordingly to hop from a first 2.8 kHz audio tone referring to a wavelength of 12.5 cm to a second 5.7 kHz audio tone referring to a wavelength of 6 cm. Using at any given moment the same wavelength for both the audio sensing and radiofrequency sensing ensures that the respective audio and radiofrequency waves interact with objects of similar sizes. Thus, both the audio and wireless sensing signals are for instance predominantly interacting with a to-be-tracked robot in a warehouse but not with other smaller objects also present in the space.
[0057] In an embodiment, the instructions can also refer to an orchestration for dynamically coordinating the sensing wavelengths of the two sensing modalities. This instructions can, in particular, be applied when the context parameters indicate that a lot of ambient noise is present in a room. For instance in a conference room during an cocktail reception, audio sensing may temporarily only be able to utilize a first audio wavelength which is reasonably free of audio interference, e.g. 2.8 kHz, while a second audio wavelength may suffer from major audio interference, e.g. 5.7 kHz. The controlling unit can then control the radiofrequency sensing based on the instructions to switch to the same new wavelength selected for the audio sensing.
[0058] In an embodiment, the orchestration of the respectively chosen audio- and radiofrequency sensing can also take a bleeding of audio sensing signals and/or wireless sensing signals into adjacent rooms into account. For instance, if utilizing audible audio-sensing frequencies is acceptable given the current context parameter of a first room, the audio sensing signal may still leak through a door to a second room, especially if a low frequency audio signal is chosen. In this case the control unit can be adapted to control the audio sensing to utilize shorter wavelengths in order to avoid disturbing the second room. While diffraction may cause the sound sensing signals to leak into a second room, diffraction actually may be beneficial to a sensing performance within the first room. For instance, diffraction can bend the audio sensing signals around a corner to reach a hidden outmost area of the room far away from the sensing network devices. Thus, in an embodiment, the control unit can utilize instructions that refer to an orchestration that determines when to employ an audio sensing signal with a high degree of the diffractive ability, for instance, when to increase a wavelength of the utilized audio sensing signals. In particular, wavelengths longer than a typical dimension of an object in the room enable the sound sensing to both travel across longer distances and/or to transverse also a highly cluttered space as the long-wavelength sensing signals are able to diffract nicely around any physical obstructions in its path.
[0059] Generally, audio signals reflected at an object with a rough surface tend to be diffuse by the rough surface, wherein rough is in this context defined as a surface comprising structures with a similar scale as the incident wavelength. Such a diffusion on rough surfaces leads to a less sharply confined audio sensing zone since the incoming sound is reflected by the rough surface in a wide variety of directions. In an embodiment, the controlling unit can thus be adapted to utilize instructions that determine a selection of an audio sensing wavelength based on context parameter indicating a rough surface in a room such that an interaction with the rough surface, for instance, at a heating radiator or an industrial machine in a plant, is minimized.
[0060] Similarly, a topography of a surface determines how much of a sensing signal is absorbed by the object. For a nonporous surface, such as smooth concrete, acoustical energy will reflect off the surface with little loss of acoustical energy and hence provide a well energized audio multipath signal. However, if the surface is porous, such as a fiber batting, acoustical energy will penetrate the surface and scatter among the pores and interact and reflect off the fibers. Scattering among the fibers and pores results in frictional losses resulting in conversion of the acoustical energy into heat and an attenuation of the audio sensing signal. Hence, whenever a substantial portion of audio sensing signals are absorbed by a porous surface, a specific set of multipaths in the room is deenergized. Thus, it is preferred that if context parameters indicate such a situation, the control unit is adapted to utilize instructions that allow to mitigate this effect by either controlling the audio sensing such that certain audio sensing wavelengths are utilized that are not influenced by the absorption or that determines to control the radiofrequency sensing such that for respective portions of the room, i.e. sensing area, only the radiofrequency sensing results are determined as being trustworthy, since the wireless radiofrequency signals are absorbed less.
[0061] Generally, for more advanced sensing applications such as localization or breathing detection, it is advantageous to employ active beam steering, which preferentially sends the sensing signal in certain spatial directions. For instance, audio sensing can employ both audio transmit beamforming, for example, via a directional speaker, and/or audio receive beamforming, for example, via a directional microphone array. Similarly, modern WiFi radios are capable to perform transmit-beamforming as well as receive-beam forming utilizing an antenna array. Such exemplary hardware utilized in the sensing modalities allows even for both radiofrequency sensing and audio sensing to use sensing beams swiveling across a sensing area, for instance, to count the people present within a space. Thus, in an embodiment, it is preferred, that the control unit is adapted to utilize instructions that determine a coordination between the audio sensing and radiofrequency sensing modalities with respect to their respective beam-steering directions as well as the chosen respective spatial confinement of their respective sensing signals, e.g. with respect to a beam cross section, divergence, etc. This coordination can determine a modification of the chosen wavelengths as well as a conscious manipulation of the transmit/receive characteristics of the antennas/speaker/microphone array of the respective network devices. In this way, both the radiofrequency sensing and the audio sensing are focused at any given time towards the same spot in the sensing area, which will improve the performance of the sensing as the two sensing modalities can directly, i.e. 1:1, compare their respectively acquired time-series sensor data, as they concurrently observed the same spot in the room at the same moment in time. Preferably, in this case the control unit is adapted to control the sensing modalities to utilize audio- and radiofrequency-sensing frequencies which result in a similar spatial confinement of the respective sensing signals.
[0062] In an embodiment, since the audio-sensing hardware is generally super flexible regarding the choice of its audio wavelength, the control unit can be adapted to utilize instructions that determine an audio wavelength which is vastly different to the WiFi/ZigBee wavelength utilized by the radiofrequency sensing system.
[0063] Moreover, in an embodiment, the radiofrequency sensing can be adapted employ a sensing algorithm analyzing a combination of Doppler shifts, e.g. caused by a walking person, as well as micro-Doppler shifts generated by a complex body, e.g. micro-Doppler from complexly moving legs and arms attached to the human torso. In this case it is preferred that the controlling of the sensing modalities is based on instructions that determine a audio sensing wavelength depending on a Doppler signature reported by the radiofrequency Doppler sensing, wherein in this case the Doppler signature can be regarded as part of the context parameters. For instance, if the radiofrequency Doppler sensing has detected a micro Doppler effect indicative of hand movement but is not able on its own to resolve finger movements, the audio sensing can be controlled to employ a set of audio wavelengths comprising both a first wavelength which is the same wavelength used by radiofrequency sensing as well as a shorter second audio wavelength capable to also resolve minute finger movements. The finger movements may for instance provide additional context information, for instance, whether the person is typing and at which speed the person is typing, e.g. browsing the internet vs writing an article.
[0064] In particular, the above described embodiments show that fusing of purposefully highly-orchestrated radiofrequency sensing and audio sensing will provide a more efficient way to detect the presence of persons, track persons in a building, monitor breathing and daily activities as well as detecting fall accidents. In the following a detailed preferred embodiment for a specific application of the apparatus described already above, will be described.
[0065] Elderly care, both at home or in retirement communities, is an urgent matter for our society. Technologies that assist the elderly in independent living are essential for enhancing care in a cost-effective and reliable manner. The recent Covid-19 pandemic caused many lives in nursing homes. In this situation assistance to provide contactless care and monitoring is essential to avoid contacts and block the spread. Cost-efficient, though sufficiently accurate context awareness sensing is also desired for non-residential venues. Many applications for elderly care or COVID-risk-mitigation systems for commercial offices demand real-time observation of the environment and the resident's activities using an event-driven system. On the other hand, end-user acceptance of current-state-of-the-art sensors such as video cameras is an uphill battle, as cameras can easily be perceived as intrusive by the elderly. Similarly, in retail and hospitality applications, especially in the US, owners of a venue regularly are faced with high costs due to fake lawsuits of people claiming an injury due to a fall to the ground. Hence, big-box retailers such as The Home Depot recently installed in all stores video surveillance to help them to distinguish between real fall events and suspicious fall claims which warrant a deeper investigation. While such cameras may be acceptable for big-box retail, they are certainly undesired in hospitality venues such as restaurants. In addition, the high cost of streaming cameras makes them prohibitive for most small entity retailers.
[0066] Thus, it is proposed to utilize the above described principles of the invention to allow for a richer activity tracking as well as for fall detection. Generally, for instance, commonly used ZigBee radiofrequency sensing is good at presence detection but has difficulties when locating the people. Audio sensing on the other hand can locate the people by varying the audio sensing wavelength, but is subject to interference from noise originating, for instance, from HVACs, fans, etc. Thus, in an embodiment the apparatus is utilized with a network provided in the context of a Retail/Hospitality/Elderly Care Facility. In particular, it is preferred that multiple luminaire comprising embedded sensors in the ceiling are utilized as network devices. For example, modern retail or office or elderly care spaces often feature already a multitude of such luminaires in form of a luminaire network, wherein the sensors of the luminaires can comprise both a wireless radio as well as a microphone. This already present lighting infrastructure can then also be utilized to concurrently perform audio sensing as well as radiofrequency sensing and in this way ensure seamless sensing coverage for the whole building. Thus, the above described apparatus allows for a controlling of the two sensing modalities that allows for an iterative fusion between the audio sensing and the radiofrequency sensing. Preferably, the control unit is adapted for this application to utilize the following instructions for controlling the sensing modalities. The instructions refer to adapting the radiofrequency sensing such that the presence of people can be detected, for instance, with 2.4 GHz radio sensing, due to its larger spatial coverage compared to the inaudible audio signals. The results of the radiofrequency sensing can then be provided by the context parameter providing unit as context parameters. In particular, it is preferred that the sensing result indicates already a rough location of people detected in a sensing area. Generally, the radiofrequency sensing, due to an adjustable radiofrequency transmission power without disturbing people, has a larger sensing range compared to audio sensing. So, once the radiofrequency sensing system has identified the whereabouts of people and provided the result indicating a part of the sensing area in which the people are located the control unit is preferably adapted to select a certain sub-set of audio transmitter and receivers, for instance, of respective network devices, that are than adapted to perform a precision-tracking of the persons pre-identified by the radiofrequency sensing system. The audio sensing system can for instance, be adapted to extract the respiration signal from a time series audio sensing signal quality parameter. Preferably, the control unit is adapted to control the audio sensing to use FWCW signals to locally monitor activities and detect accidents by widely varying audio sensing frequencies. In particular, it is further preferred that the control unit is adapted to control the audio-sensing and the radiofrequency sensing such that they are actively tuned in based on each other's results. In particular, such that both sensing modalities work concurrently and enforce each other. For example, depending on the chosen audio frequency, the audio sensing signals may or may not be perceived obtrusive by humans occupying the space. If human-perceivable audio signals are used, it is preferred to control the audio sensing such to embed audible audio sensing signals into a soothing white noise or in a music soundscape, for instance, commonly used in a hospitality or retail. Near-ultrasonic audio sensing frequencies can, for example, be intermixed in pop-music, so that the near-ultrasonic sound sensing signals are aligned with spikes in the amplitude of the pop song. Moreover, the audio sensing signals, for instance, ping signals, can be deliberately embedded in active sound masking systems, for instance, used for suppressing intelligible speech in office applications. Optionally, the detected audio-sensing information can be fed to a utilized radio sensing algorithm for online training of the algorithm such that the two sensing modality results will eventually converge with enough data for presence detection. In this case, it is preferred to use a Bayesian fusion of the audio sensing and radiofrequency sensing results to improve an accuracy of the sensing, specifically for fall detection.
[0067] In an embodiment, especially suitable in this context, it is preferred that the control unit is adapted to control the audio sensing such that the audio sensing wavelength is varied for intrusive or non-intrusive audio sensing. Such an audio spectrum swipe enables the audio sensing to search the space for humans and measure distance and detect accidents, like a fall, etc. Optionally, the apparatus can be adapted to share sensing knowledge, for example, in form of respective instructions, with other apparatuses across rooms with similar layouts, e.g. for senior living or nursing homes. For example, a room layout similarity can be determined based on audio sensing results combined with meta data.
[0068] Generally, low wavelength audio signals attenuate slower than higher wavelength audio signals. Hence, audio sensing using sensing signals with a lower wavelength has a wider sensing range. Moreover, while audio sensing using higher wavelength can only sense in a direct line of sight and can thus be blocked easily, audio sensing signals with a lower wavelength are diffracted around corners and hence can be utilized to sense beyond blocking objects. Hence, in an embodiment it is preferred, that if the context parameter indicate, for instance, based on a radiofrequency sensing of the environment, that a blocking object is present in the sensing are, e.g. an order picking robot in an warehouse, the controlling unit can be adapted to lower a frequency of the audio sensing signals, for instance, based on the indicated size of the object, to sense also behind the object.
[0069] In a preferred embodiment the controlling unit is adapted to control the network to perform audio sensing only, radio sensing only, or a combination of both depending on the context parameters referring, for instance, to the network environment, a room type, interference characteristics, etc., e.g. if a room is too audio-noisy, or a microwave oven is on.
[0070]
[0071] Generally, human body mass is known to absorb sound. Hence, due to the presence of the person in the space, the channel state information (CSI) in the direction of the person will significantly change. Assuming we have M speakers and N network devices, i.e. receivers, the signal can be modeled as shown below with CSI {H.sub.i,j},
[0072] Thus, in step 306 the audio sensing can be controlled, for instance, by amending a respective sensing algorithm to utilize the CSI of the received signals for fall detection.
[0073] If it is determined in step 303 that the audio noise is too high in step 308 it can be determined if radio interferences are also present or if radiofrequency sensing is possible. If radiofrequency sensing is possible the radiofrequency sensing can be controlled in step 309 to perform fall detection utilizing the CSI of the radiofrequency signals for fall detection. Generally, the radiofrequency sensing and audio sensing as described above can be performed also concurrently, if the noise in the respective modality is not too high for a sensible sensing accuracy. In this case, in step 307 the sensing results of both modalities can be fused, for instance, based on predetermined rules, or a Bayesian fusion model using respective probabilities for each sensing result.
[0074] After a fall has been detected in step 310, both sensing modalities can be controlled to perform a close monitoring. Specifically, the microphone array can be controlled to detect via audio sensing the respiration rate to confirm the detected accident in step 311. If generally no or only minimum activities are detected for more than a few minutes, an emergency call can in step 312 be used to handle this situation. Moreover, since the network includes both an audio speaker and a microphone, the network can talk to the caregiver to confirm, for instance, when a fall is detected.
[0075] Generally, in the above examples, the audio sensing and the radiofrequency sensing are controlled such that they are actively tuned in based on each other's sensing results. Thus, in the above embodiments, both sensing modalities work concurrently and enforce each other.
[0076] Further preferred details of the two sensing modalities are described in the following. With respect to the radiofrequency sensing it is preferred, for instance, when using WiFi that a network device can act as both a transmitter and receiver. In this case, it is preferred that multiple network devices are utilized in order to meet detection accuracy requirements. Moreover, it is preferred that the audio sensing is controlled to utilize non-obtrusive audio signals to human, i.e. signals with a frequency above 16 KHz, for tracking purposes, but to utilize obstructive audio sensing signals to confirm if an accident is detected. The obtrusive audio signals allow the audio sensing to use higher audio bandwidth, hence enabling higher-quality detection.
[0077] In one specific embodiment, it is preferred to control the audio sensing to embed the audio sensing signals into a soothing white noise signal. Similarly, the audio sensing's ping messages can be hidden in an acceptable music, e.g. music soundscape used in hospitality or big-box retail. The audio sensing ping signals can also be deliberately embedded in active sound masking systems. Moreover, a series of orthogonal sequences of audio sensing signals can also be used in a time order to provide timing information, which could also help to determine the spatial positioning of a human in a room. In particular, it is noted that the microphones and the speakers transmitting the audio signals are preferably not synchronized as they are preferably not co-located. Preferably, the audio sensing is controlled to transmit a beaconing signal, for instance, a radiofrequency signals over the network using BLE, ZigBee, or PoE, from an audio-sensing master network device to synchronize the start and stop for the audio sensing. However, since a wireless or wired network can already provide a synchronous network clock, also global time stamps can be utilized to define a start and stop time for all of the audio sensing network devices within a detection area. Optionally, the control unit can be adapted to control the sensing modalities such that the sensing area is divided up into a first set of sensing zones which can be optimally served by the radiofrequency sensing and second sensing zones better served by the audio sensing. In particular, the sensing zones can be divided and assigned based in context parameters, since depending on its building materials, shape and room size, each sub-space in a building can provide different challenges for radiofrequency sensing signals and audio sensing signals, respectively. Hence the building space can also be divided up by the control unit, for instance, by using predetermined decision criteria, into a first set of sensing zones which can be optimally served by the radiofrequency sensing and a second set of sensing zones which is currently better served by the audio sensing. In the zones well covered by the audio sensing, the radiofrequency sensing can be controlled to refer to a basic radiofrequency sensing. However, also in this case, if the context parameters for instance, including sensing result of any of the sensing modalities, indicate the presence of a person, the control unit can be adapted to control again both sensing modalities to be used in concert in the vicinity of the person. For example, if an area generates a lot of audio ambient noise but the area is relatively small, such as a kitchen, radiofrequency sensing can be the first choice as the ambient audio noise may impact the audio sensing performance. However, in an area with strong wireless interference or much wireless traffic, audio sensing can be the first choice. For zones which both are well-covered by the audio sensing and are even tolerant to the audible nuisance of the audio sensing signal, the radiofrequency sensing can be controlled to be scaled back to basic radiofrequency sensing that is generally less wireless bandwidth consuming and the audio sensing can be controlled to perform a highly-granular sensing. However, if in a small room a lot of ambient noise is present, such as a conference room during an cocktail reception, radiofrequency sensing can be controlled to act as the primary sensing modality, since the reception's ambient audio noise may impact the audio sensing performance. In addition, the scaled-back audio sensing can be controlled to use only those audio frequencies which are reasonably free of audio interference. However, in another room of the building currently experiencing strong wireless interference or wireless traffic congestion, audio sensing may be the first choice. Optionally, the radiofrequency sensing and audio sensing zones can be semi-overlapping/interleafed in the sensing area to create distance between neighboring zones utilizing the same sensing modality.
[0078] Generally, audio sensing can be configured to filter out certain distances based on the audio time-of-flight delay, e.g. for disregarding any reflected audio sensing signals arriving between 1.5 m to 3 m range from a speaker. This makes it, for instance, possible to exclude a certain corridor within an open plan office from triggering an occupied status of a nearby group of desks. However, this possibility is preferably used with a synchronized network clock such that it is most easily achieved by placing both the audio transmitter and receiver in the same physical unit, i.e. network device. The delay in the audio echo can then be easily measured by using a self-orthogonal waveform with good auto correlation. Thus, controlling the audio sensing to use this possibility can complement the capabilities of radiofrequency sensing, for instance, using Zigbee, which can be based on RSSI and thus lacks ranging capabilities.
[0079] Generally, if in the above described embodiments, situations, like specific activities, room usages, room characteristics, etc. are described, these sensing situations can be inferred from respective context parameters, for instance, from sensing results of any one of the modalities, information on the room, a time of day, object location and sizes, etc. The concurrently described controlling strategies can then be provided in form of controlling instructions corresponding to the respective sensing situation and/or context parameters. These instructions can then be stored on a respective storage and utilized by the control unit based on the respective context parameters when controlling the sensing modalities based on the respective context parameters. Thus, even if not mentioned in all embodiments explicitly, the controlling is based on the respective context parameters that indicate the sensing situation that corresponds to the respective instructions used for controlling. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.
[0080] In the claims, the word comprising does not exclude other elements or steps, and the indefinite article a or an does not exclude a plurality.
[0081] A single unit or device may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
[0082] Procedures like the providing of the context parameters or the controlling of the radiofrequency and audio sensing, etc., performed by one or several units or devices can be performed by any other number of units or devices. These procedures can be implemented as program code means of a computer program and/or as dedicated hardware.
[0083] A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.
[0084] Any reference signs in the claims should not be construed as limiting the scope.
[0085] The invention relates to an apparatus for controlling radiofrequency sensing and audio sensing of a network. The network comprises a plurality of network devices, e.g. smart lights, and is adapted to perform radiofrequency sensing and audio sensing utilizing one or more of the network devices. A context parameter providing unit provides context parameters, wherein the context parameters are indicative of a context in which the radiofrequency sensing and the audio sensing is performed. A controlling unit controls the radiofrequency sensing and the audio sensing of the network in dependency of each other based on the context parameters. This allows for an improved monitoring of an area.