Method for categorizing a scene comprising a sub-scene with machine learning

Abstract

A method for identifying a scene, comprising a computing device receiving a plurality of data points corresponding to a scene; the computing device determining one or more subsets of data points from the plurality of data points that are indicative of at least one sub-scene in said scene, said at least one sub-scene displayed on a display device that is part of said scene, wherein said at least one sub-scene does not represent said scene; the computing device categorizing said scene, disregarding said at least one sub-scene, wherein the categorizing includes interpreting said scene by a computer vision system such that said at least one sub-scene is not taken into account in the categorizing of said scene.

Claims

1. A method for categorizing a scene, comprising: a computing device receiving a plurality of data points corresponding to said scene; the computing device determining one or more subsets of data points from the plurality of data points, wherein said one or more subsets of data points are indicative of at least one sub-scene in said scene, said at least one sub-scene displayed on a display device that is part of said scene, wherein said at least one sub-scene does not represent said scene; the computing device categorizing said scene, disregarding said at least one sub-scene, wherein the categorizing includes interpreting said scene by a computer vision system such that said at least one sub-scene is not taken into account in the categorizing of said scene.

2. The method of claim 1, wherein said scene is an indoor scene.

3. The method of claim 1, wherein said scene is an outdoor scene.

4. The method of claim 1, wherein said scene comprises a series of subsequent scenes defining said scene.

5. The method of claim 1, wherein said scene comprises a traffic scene from a viewpoint inside a vehicle looking out of said vehicle.

6. A device comprising an AI system for categorizing a scene, said AI system comprising a computing device running a computer program performing: receiving a plurality of data points corresponding to said scene; determining one or more subsets of data points from the plurality of data points, wherein said one or more subsets of data points are indicative of at least one sub-scene in said scene, said at least one sub-scene displayed on a display device that is part of said scene, wherein said at least one sub-scene does not represent said scene; categorizing said scene, said computer program disregarding said at least one sub-scene, wherein the categorizing includes interpreting said scene by a computer vision system such that said at least one sub-scene is not taken into account in the categorizing of said scene.

7. A non-transitory computer readable medium having stored thereon computer program instructions that, when executed by a processor in a computing device, configure the computing device to perform: receiving a plurality of data points corresponding to a scene; determining one or more subsets of data points from the plurality of data points, wherein said one or more subsets of data points are indicative of at least one sub-scene in said scene, said at least one sub-scene displayed on a display device that is part of said scene, wherein said at least one sub-scene does not represent said scene; categorizing said scene, said computer program instructions disregarding said at least one sub-scene, wherein the categorizing includes interpreting said scene by a computer vision system such that said at least one sub-scene is not taken into account in the categorizing of said scene.

8. An AI system comprising a computing device executing the computer program instructions of claim 7.

9. An apparatus comprising the AI system of claim 8, wherein said scene comprises a representation of a surrounding of said apparatus comprising said scene, said AI system providing instructions to adjust at least one physical parameter of said apparatus based upon said categorizing of said scene.

10. The apparatus of claim 9, selected from a vehicle and a robot system.

11. A monitoring system comprising the AI system of claim 8, wherein said scene comprises a representation of a surrounding of said monitoring system, said AI system providing a signal based upon said categorizing of said scene.

12. A surveillance system comprising the monitoring system of claim 11.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Embodiments of the invention will now be described, by way of example only, with reference to the accompanying schematic drawings in which corresponding reference symbols indicate corresponding parts, and in which:

(2) FIGS. 1A-D schematically depict flow charts of example methods to categorize various aspects from a scene comprising a display device.

(3) FIG. 2 schematically depicts an embodiment for monitoring the indoors of an elderly home with a television display.

(4) FIG. 3 schematically depicts an embodiment for monitoring a square with display devices.

(5) FIG. 4 schematically depicts an embodiment built into a self-driving car analyzing a scene comprising a billboard.

(6) The drawings are not necessarily to scale.

DESCRIPTION OF PREFERRED EMBODIMENTS

(7) The following detailed description describes various features and functions of the disclosed systems and methods with reference to the accompanying figures. In the figures, similar symbols identify similar components, unless context dictates otherwise.

(8) FIGS. 1A-D schematically depict flow charts of example methods (1, 1′, 1″ and 1′″) for categorizing wherein a computing device 3 receives data points (100, 100′) from scene 10, comprising a display device 2 and a sub-scene 10′, resulting is a categorized action 21, a categorized scene 20 and one or more categorized subjects 22.

(9) In FIG. 1A, method 1 categorizes an action in scene 10 resulting in a categorized action 21. Computing device 3 receives a plurality of data points 100 corresponding to scene 10. Computing device 3 determines a subset of data points 100′ indicative of sub-scene 10′ displayed on display device 2, and categorizes an action in scene 10 and disregards sub-scene 10′.

(10) In FIG. 1B, method 1′ categorizes a scene 10 resulting in a categorized scene 20. Computing device 3 receives a plurality of data points 100 corresponding to scene 10. Computing device 3 determines a subset of data points 100′ indicative of sub-scene 10′ displayed on display device 2, and categorizes scene 10 and disregards sub-scene 10′.

(11) In FIG. 1C, method 1″ categorizes one or more subjects in scene 10 resulting in one or more categorized subjects 22. Computing device 3 receives a plurality of data points 100 corresponding to scene 10. Computing device 3 determines a subset of data points 100′ indicative of sub-scene 10′ displayed on display device 2, and categorizes one or more subjects in scene 10 and disregards sub-scene 10′.

(12) In FIG. 1D, method 1′″ categorizes a scene 10 resulting in a categorized scene 20 and a categorized action 21. Computing device 3 receives a plurality of data points 100 corresponding to scene 10. Computing device 3 determines a subset of data points 100′ indicative of sub-scene 10′ displayed on display device 2, and categorizes scene 10 while disregarding sub-scene 10′, wherein a categorized action 21 is deducted from a categorized scene 20.

(13) In another method a categorized scene 20 is deducted from one or more categorized actions (21). For example, a box match scene with various billboards can be categorized directly or can be categorized by the activity or series of actions by boxers fighting in a ring.

(14) In yet another method a categorized scene 20 is deducted from one or more categorized subjects (22). For example, a box match scene with various billboards can be categorized directly or can be categorized by a one or more subjects such as a boxing ring, boxers, trainers, crowd and various attributes in scene 10.

(15) The methods (1, 1′, 1″ and 1′″) may include one or more operations, functions, or actions as depicted in FIGS. 1A-D and may result in one or more categorized objects as depicted by the blocks 20, 21 and 22. Although the blocks 20, 21 and 22 are depicted in a sequential order, these blocks may in some instances be performed in parallel, and/or in a different order than those described herein. Also, the various blocks may be combined into fewer blocks, divided into additional blocks, and/or removed based upon the desired implementation.

(16) In addition, for the methods (1, 1′, 1″ and 1′″) and other processes and methods disclosed herein, the flow charts show functionality and operation of possible implementations of embodiments. In this regard, each method may represent a module, a segment, or a portion of program code, which includes one or more instructions executable by a processor for implementing specific logical functions or steps in the process. The program code may be stored on any type of computer readable medium or memory, for example, such as a storage device including a disk or hard drive. The computer readable medium may include a non-transitory computer readable medium, for example, such as computer-readable media that stores data for short periods of time like register memory, processor cache and random-access memory (RAM). The computer readable medium may also include non-transitory media or memory, such as secondary or persistent long-term storage, like read only memory (ROM), optical or magnetic disks, compact-disc read only memory (CD-ROM), for example. The computer readable media may also be any other volatile or non-volatile storage systems. The computer readable medium may be considered a computer readable storage medium, a tangible storage device, or other article of manufacture, for example.

(17) In addition, for the methods (1, 1′, 1″ and 1′″) and other processes and methods disclosed herein, computing device 3 may represent circuitry that is wired to perform the specific logical functions in the process. For the sake of example, the methods (1, 1′, 1″ and 1′″) shown in FIGS. 1A-D will be described as implemented by an example computing device, such as the computing device 3 depicted in FIG. 2. The methods (1, 1′, 1″ and 1′″) can also be described as implemented by an autonomous vehicle, as depicted in FIG. 4, as the computing device may be onboard the vehicle or may be off-board but in wireless communication with the vehicle. It should be understood that other entities or combinations of entities can implement one or more steps of the example methods (1, 1′, 1″ and 1″).

(18) FIG. 2 schematically depicts an application of an embodiment for monitoring an indoors scene of an elderly home 50. In the indoors scene of elderly home 50 there is a television, as display device 2. A video camera, as image capturing device 4, captures scene 10 and transmits its data points 100 including sub-scene 10′ with data points 100′ to computing device 3, operationally coupled to video camera 4. Although scene 10 comprises a man with a gun 21, computing device 3 does not categorizes scene 10 in FIG. 2 as threatening or as a crime scene since computing device 3, when categorizing the scene, is trained to disregard the data points 100′ of sub-scene 10′ displayed on the television 2.

(19) In another application computing device is categorizing, within scene 10, an action, a pose, a subject or a combination thereof.

(20) FIG. 3 schematically depicts an application of an embodiment for monitoring an outdoors scene of a square 51. In the outdoors scene of square 51 there is a wide screen, as display device 2 and a merchandise wagon 7 with a display device 2′. A video camera, as image capturing device 4, monitors the square and is operationally coupled to computing device 3. Similar to the indoors example of FIG. 2, the monitoring of an outdoors scene of square 51 is complicated by the display devices (2 and 2′). Wide screen 2 is showing a fighting scene 22 while the display device 2′ on the merchandise wagon 7 shows similar architecture as the surroundings of the square.

(21) In this application, computing device 3, when categorizing the people 8 on the square, will, by disregarding the sub-scene on wide screen 2, deduct that the number of people on the square in view of camera 4 is nine. For instance, such information can be used for monitoring and controlling a crowd in an open space.

(22) Additional in this application, computing device 3, when categorizing the houses 9 on the square, will, by disregarding the sub-scene on display device 2′, deduct that the number of houses in view of camera 4 is three and by doing so, it will also increase the correct categorization for merchandise wagon 7 since computing device 3 is not misled by the display device 2′.

(23) Would this application serve as a surveillance system then the system of FIG. 3 would not trigger an alarm for an alleged fighting incident 22 as displayed on the wide screen (display device 2). Adjacent, the surveillance system could trigger an alarm for merchandise wagon 7 for trespassing the square by an unauthorized vehicle.

(24) FIG. 4 schematically depicts an embodiment built into a self-driving car 5 analyzing a scene 10 comprising a billboard, as display device 2. Computing device 3 is operationally coupled with image capturing device 4 and receives a plurality of data points 100 corresponding to scene 10. Computing device 3 determines a subset of data points 100′ indicative of sub-scene 10′ displayed on billboard 2, categorizes scene 10 and disregards sub-scene 10′. As a result the self-driving car 5 will not be misled by the image of car 23 on billboard 2.

(25) The billboard 2 can be a traditional poster, a digital billboard or a screen configured to display a static image, a (time) series of images, or a video movie.

(26) Further, an example system may take the form of a non-transitory computer-readable medium, which has program instructions stored thereon that are executable by at least one processor to provide the functionality described herein.

(27) An example system may take the form of any vehicle or a subsystem of any vehicle that includes such a non-transitory computer-readable medium having such program instructions stored thereon. Therefore, the terms “computing device” and “autonomous vehicle” can be interchangeable herein. However, in some examples, the computing device may be configured to control the vehicle in an autonomous or semi-autonomous operation mode.

(28) In yet another application, an embodiment is built into a robot so the robot will correctly interpreter its surrounding and the scene wherein the robot is operating.

(29) It may be readily understood that certain aspects of the disclosed systems and methods can be arranged and combined in a wide variety of different configurations, all of which are contemplated herein.

(30) It will also be clear that the above description and drawings are included to illustrate some embodiments of the invention, and not to limit the scope of protection. Starting from this disclosure, many more embodiments will be evident to a skilled person. These embodiments are within the scope of protection and the essence of this invention and are obvious combinations of prior art techniques and the disclosure of this patent.