Indicators Of Compromise By Analyzing Data Based On Rolling Baseline

Abstract

Techniques are disclosed for identifying indicators of compromise in a variety of objects. The objects may be finished products or components thereof. The indicators of compromise in the objects are determined/detected by analyzing their data which may reside in a cloud. The analysis is performed by an instant baseline engine that first establishes a rolling baseline with a centroid of a conceptual hypercube. The centroid represents the normal population of data packets. Data packets far enough away from the centroid indicate an anomaly that may be an indicator of a compromise of/in the respective object. An early detection of such indicators of compromise in the objects can prevent catastrophic downstream consequences for the concerned party/parties.

Claims

1. A system comprising computer-readable instructions stored in non-transitory storage medium and at least one microprocessor coupled to said non-transitory storage medium for executing said computer-readable instructions, said at least one microprocessor configured to: (a) analyze data from one or more objects; (b) establish a rolling baseline of said data by assigning each packet of said data to a cluster of packets amongst a plurality of clusters of packets of said data; (c) score, based on its distance from a centroid of said rolling baseline, each packet of said data; and (d) identify based on said distance an indicator of compromise in said one or more objects; wherein said one or more objects comprise one or both of a component and a product.

2. The system of claim 1 wherein said indicator of compromise is manifested as said data being one or both of unintelligible and obfuscated.

3. The system of claim 1 wherein said indicator of compromise is manifested as said data being unintentionally encrypted.

4. The system of claim 1 wherein said indicator of compromise is manifested as said data being misreported.

5. The system of claim 4 wherein said data being misreported is due to one or both of: (e) at least one unauthorized remote command executed on said one or more objects; and (f) at least one unauthorized message sent by said one or more objects.

6. The system of claim 1 wherein said data resides in a cloud.

7. The system of claim 6 wherein said cloud is one or more of a generic automation testing cloud, a device specific cloud, a vendor specific cloud and a component cloud.

8. The system of claim 6 wherein when said one or more objects comprise a component, said cloud is a component cloud used for electronic design automation (EDA) of said component.

9. The system of claim 8 wherein said indicator of compromise signifies a pattern of failure of said component.

10. The system of claim 6 wherein a training dataset is created from said data, said training dataset used to generate an optimal operational model of said one or more objects to facilitate establishing said rolling baseline line in element (b) above.

11. The system of claim 1 wherein said indictor of compromise is manifested as one or more of an overload of a CPU, an overuse of a memory, an overuse of a disk storage, an overuse of a network bandwidth and an overage of thermal output of said one or more objects.

12. A computer-implemented method executing computer-readable instructions by at least one processor, said computer-readable instructions stored in a non-transitory storage medium coupled to said at least one processor, and said computer-implemented method comprising the steps of: (a) analyzing data from one or more objects, said one or more objects comprising one or both of a component and a product; (b) establishing a rolling baseline of said data by assigning each packet of said data to a cluster of packets amongst a plurality of clusters of packets of said data; (c) scoring, based on its distance from a centroid of said rolling baseline, each packet of said data; and (d) identifying based on said distance an indicator of compromise in said one or more objects.

13. The computer-implemented method of claim 12 manifesting said indicator of compromise as said data being one or both of unintelligible and obfuscated.

14. The computer-implemented method of claim 12 manifesting said indicator of compromise as said data being encrypted.

15. The computer-implemented method of claim 12 manifesting said indicator of compromise as said data being misreported.

16. The computer-implemented method of claim 12 providing said data to be residing in a cloud.

17. The computer-implemented method of claim 16 providing said cloud to be one of a generic automation testing cloud, a device specific cloud, a vendor specific cloud and a component cloud.

18. The computer-implemented method of claim 16 when said one or more objects are comprising a component, providing said cloud to be a component cloud that is used for electronic design automation (EDA) of said component.

19. The computer-implemented method of claim 16 utilizing said data as a training dataset for generating an optimal operational model of said one or more objects for facilitating of said establishing of said rolling baseline line in step (b) above.

20. The computer-implemented method of claim 12 manifesting said indicator of compromise as one of an overage and an underage of one or more of a CPU usage, a memory usage, a disk storage usage, a network bandwidth usage and a thermal output associated with said one or more objects.

Description

BRIEF DESCRIPTION OF THE DRAWING FIGURES

[0044] FIG. 1 is a conceptual diagram illustrating the anomalous subject and device identification system of the present design.

[0045] FIG. 2 is a detailed diagram illustrating various embodiments with various types of sensors used in the anomalous subject and device identification system of the present technology.

[0046] FIG. 3 is a diagram emphasizing the embodiments utilizing one or more cameras according to the instant principles.

[0047] FIG. 4 is a diagram emphasizing the embodiments utilizing one or more assets sensors and xmitters at a manufacturing or fabrication site according to the instant principles.

[0048] FIG. 5 is a variation of FIG. 4 also incorporating cameras and other subjects.

[0049] FIG. 6 is a diagram emphasizing the embodiments utilizing personal-device sensors at a site.

[0050] FIG. 7 is a variation of FIG. 6 also incorporating cameras and unattended devices at the site.

[0051] FIG. 8 is a variation of FIG. 7 also incorporating wireless antennas installed at the site.

[0052] FIG. 9 is a diagram emphasizing embodiments where sensors of the present design are embodied in various computing devices.

[0053] FIG. 10 illustrates some of the many types of objects by analyzing whose data the instant baseline engine detects indicators of compromise in them in a highly preferred set of embodiments.

[0054] FIG. 11 shows a preferred variation of the embodiments of FIG. 10 where the instant baseline engine analyzes smart TV data residing in a cloud in order to detect indicators of compromise in the TVs.

[0055] FIG. 12 shows a preferred variation of the embodiments of FIG. 10 where the instant baseline engine analyzes component data residing in a component cloud in order to detect indicators of compromise in the components.

DETAILED DESCRIPTION

[0056] The figures and the following description relate to preferred embodiments of the present invention by way of illustration only. It should be noted that from the following discussion, alternative embodiments of the structures and methods disclosed herein will be readily recognized as viable alternatives that may be employed without departing from the principles of the claimed invention.

[0057] Reference will now be made in detail to several embodiments of the present invention(s), examples of which are illustrated in the accompanying figures. It is noted that wherever practicable, similar or like reference numbers may be used in the figures and may indicate similar or like functionality. The figures depict embodiments of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.

[0058] The techniques described herein may employ computer code that may be implemented purely in software, hardware, firmware or a combination thereof as required for a given implementation. The system and methods of the present technology will be best understood by first reviewing an anomalous subject and device identification system 100 as illustrated in FIG. 1. System 100 is a surveillance system comprising any number of sensors 104A, 104B, . . . 104N connected via a communication network (not shown) to a rolling baseline computation engine 110 at a site or an organization or an establishment or facility or property or environment 102.

[0059] Reference numerals 104A . . . 104N may represent anywhere from a single sensor up to hundreds or thousands or even more sensors as depicted by the dotted line, that may generate data for rolling baseline engine or for short baseline engine 110. Furthermore, non-limiting examples of these sensors are shown in FIG. 1. These include as one or more sensors 104A termed as asset sensors installed on or near or in vicinity or in proximity of valuable or sensitive assets, such as a manufacturing equipment or tools at a manufacturing or chip fabrication facility.

[0060] The sensors in FIG. 1 also include one or more sensors 104B that are picture or video cameras and one or more sensors 104C that are audio sensors such as microphones. These further include one or more sensors 104D that are wireless sensors such as wifi or bluetooth or Zigbee sensors and the like—these are termed as personal-device sensors because they are responsible for sensing/communicating with devices carried by subjects 206. FIG. 1 also shows one or more such sensors operating at a kiosk 105.

[0061] Any number and type of sensors 104A-N may be installed on one or more computing devices, such as mobile devices including cellular phones including smartphones. Sensors 104A-N may also be on tablets, and wearable devices such as smartwatches, even desktops, etc. It should further be noted that sensor(s) 104A may be one or more asset sensors, sensor(s) 104B may be one or more cameras, sensor(s) 104C may be one or more microphones that may or may not be integrated with camera(s) 104B, sensor(s) 104D may be one or more wireless personal-device sensors, examples of which were noted above, etc.

[0062] In this disclosure, unless otherwise explicitly noted, we may use reference numerals, for example reference numeral 104B to refer to a single sensor or multiple sensors of a given type, in this case camera or cameras. Any of sensors 104 may be operating in one or more kiosks, such as kiosk 105 at site 102. These sensors may be installed on one or more computing devices, fixed or mobile, enterprise or personal.

[0063] According to the present technology, sensors 104A . . . 104N gather data that is related to various subjects or targets 106. Subjects may be sentient beings, such as any sentient life forms or beings including animals or human beings shown in FIG. 1. Subjects also include non-living or non-sentient beings such as robots, automatons, cyborgs, as well as objects or assets of interest or value at site 101. In FIG. 1, sensors 104 are monitoring/surveilling human subjects 106 at site 102 and providing that data to baseline engine 110 for analysis, in order to accrue the benefits of the instant anomalous subject and device identification system 100 of the present design. Baseline engine 110 used by the present technology is the rolling baseline data surveillance system taught in detail in above-incorporated references including U.S. Pat. No. 10,542,026 issued on 21 Jan. 2020 to Christian.

[0064] Explained further, baseline engine 110 analyzes each packet of data gathered by sensors 104. As a part of this analysis, it assigns each packet of data to a cluster of packets amongst clusters of packets of data. The clustering is done preferably by utilizing k-means clustering, specifically by utilizing Eq. (1) of the above-incorporated references and teachings. As a result, baseline engine 110 establishes a rolling or evolving baseline 120 for the data that signifies the mean or normal behavior of the packets.

[0065] Baseline 120 is based on a conceptual hypercube 180 with a centroid 182 as shown in FIG. 1 representing the normal population of packets. For brevity, we may just refer to centroid 182 to be the centroid of baseline 120, rather than spelling out fully that centroid 182 is the centroid of hypercube 180 of baseline 120. Thus, as data packets from sensors 104A-N arrive via a communication network (not shown) at baseline engine 110, it scores these backets based on their distance from centroid 182 of baseline 120.

[0066] Since baseline 120 with centroid 182 signifies the “normal” behavior of packets, packets that are very far away from centroid 182 represent an anomaly. In this way, anomalous subject and device identification system 100 identifies anomalous subjects among subjects 106 that are associated with anomalous packets of data. Once again, for even a more detailed explanation of the workings of baseline engine 110 of anomalous subject and device identification system 100, that is responsible for establishing a rolling baseline 120 and then identifying anomalous data packets, the reader is referred to the above-incorporated references including U.S. Pat. No. 10,542,026 issued on 21 Jan. 2020 to Christian.

[0067] Now let us take a more detailed look at the present technology by reviewing its various embodiments and by taking advantage of FIG. 2. FIG. 2 shows an anomalous subject and device identification system 200 of the present design operating at a site 202. Site 202 has a number of subjects 206 per above explanation. In the example shown in FIG. 2, these subjects or targets are humans or people marked by reference numerals 206A, 206B, . . . . Also shown are a number of sensors of various types 204A, 204B, 204C, 204D, . . . per above discussion. Any number and types of such sensors 204A-N or simply sensors 204 may be present at site 202. All these sensors are connected to a network backbone 208 that is in turn connected to baseline engine 110 of the above teachings. Network backbone 208 is an electronic communications network based on techniques known in the art.

[0068] Furthermore, sensors 204 are collecting data about people 206A, 206B, . . . or simply people 206 at site 202 and supplying it to baseline engine 110 for analysis such that any malicious or anomalous subjects/actors/people/beings amongst people/beings 206 or any anomalous devices at site 202 can be identified. This process depends upon the type of sensor(s) involved. The results of analysis performed by baseline engine 110 and any other related data is stored in an appropriate data storage mechanism for archival and analytics. Such a storage mechanism may be a database on premises at site 202 or in cloud 230 shown in FIG. 2 or a combination thereof.

[0069] Let us now study the various embodiments utilizing the different types of sensors at a given site based on the present principles while referring to FIG. 2.

[0070] Camera(s): Camera(s) or simply camera 204A visually monitors people 206. In various embodiments, camera 204A may be a standard video camera such as a closed-circuit television (CCTV) camera, or a more specialized camera such as a stereoscopic video camera or a thermal camera. Regardless, camera 204A supplies its data as video packets via network backbone 208 to baseline engine 110 of the above discussion.

[0071] Baseline engine 110 then establishes a rolling baseline 120A with conceptual hypercube 180A and centroid 182A for these video packets. It then identifies anomalous video packets as compared to baseline 120A per above-incorporated references and teachings. Anomalous video packets are associated with a specific subject/person, exemplarily person 206C amongst subjects/person 206 at site 202. Based on the analysis performed by baseline engine 110 and identification of anomalous video packet(s) by engine 110, anomalous subject and device identification system 200 of FIG. 2 identifies person 206C as an anomalous subject or a malicious actor. Its findings can then be accessed directly via an appropriate user interface (not shown) and/or stored in cloud 230 for archival and analytics.

[0072] Note that in the present and other embodiments discussed in this disclosure, the correspondence of the reference numeral of the baseline to the type of sensor 204 must not be taken too strictly. For example, any number of baselines may be established by baseline engine 110 based on the video stream from a single camera depending on the analysis performed by the baseline engine for a given implementation. There may be one baseline geared towards security aspects, another baseline geared towards training aspects, another towards behavioral aspects, etc. Conversely, data streams from multiple sensors may be combined into a single baseline also, as per the requirements of a given implementation.

[0073] As already mentioned, camera 204A may be a standard video camera such as the one typically integrated with today's cellular phones or smartphones or a more specialized camera or a CCTV camera. The analysis performed by baseline engine 110 for its rolling baseline 120A calculation may then be based on facial recognition and motion tracking of subjects/people/beings 206. Facial recognition and object tracking or simply tracking of people 206 in the video data from camera 204A are performed based on techniques known in the art by data processing module 220 shown in FIG. 2. Preferably, for this purpose data processing module 220 performs form or skeletal motion analysis on the video stream(s).

[0074] Data processing module 220 is also responsible for performing any other data preprocessing tasks before supplying its output as data packets to baseline engine 110 for analysis. In various embodiments, data processing module 220 may be implemented as a single module or it may be comprised of various submodules per the needs of an implementation and based on techniques known in the art. In a preferred embodiment, it is implemented as a shim compatibility layer to baseline engine 110.

[0075] Each subject or person 206A, 206B, . . . at site 202 is identified by a hash signature or an alternate identifying signature/marker/information or simply an identifier for object tracking performed by data processing module 220. The movement data of each signature is then fed to baseline engine 110. Preferably, the movement data comprises (x, y, z) coordinates or other equivalent location information of the respective individual/subject/being at site 202 at various points in time. Alternately or in addition, the movement data comprises his/her speed and direction of movement at the given location and the given point in time.

[0076] As that person moves in a building or site, object tracking function of module 220 tracks the movements of the person in the building having the assigned identifier. If there are more than one cameras 204A, object/facial recognition and tracking is performed on video data streams of all such cameras by module 220. The movement data of tracked people 206 with their respective identifiers is then fed to baseline engine 110 for analysis per above. There are a number of useful scenarios or situations that can be captured by the embodiments. A non-exhaustive list of these includes: [0077] 1. Erratic/distressed movement pattern: In one embodiment, rolling baseline 120A signifies the average or mean behavior of crowd 206 by a given set of movements or movement pattern/patterns of people 206 that is considered “normal”. An individual or person, such as person 206C with an exemplary hash signature or simply hash or identifier C1369F4789DA, exhibiting an erratic or stressful or distressed movement pattern or patterns may signify an anomaly. In this case, baseline engine 110 will determine the distance of video packets associated with person 206C to be far enough away from centroid 182A of baseline 120A to signal an anomaly. This anomaly is then reported by engine 110 per prior teachings. Anomalous subject and device identification system 200 can then take appropriate actions based on the anomalies reported by baseline engine 110.

[0078] 2. Audio signatures: In a related variation, camera 204A may be integrated with microphone 204B in a single product/device. In such a variation, audio packets of data or audio data stream from microphone 204B are combined with video packets or video data stream from camera 204A to advantageously enhance facial recognition and object tracking of people 206 at site 202. For example, if site 202 is a theatre or studio or the like where the audio signature of each tracked individual may be distinguishable enough, such an audio signature may further help data processing module 220 to recognize and locate each individual with his/her identifier at site 202. Additional embodiments benefiting from audio or microphone sensors 204B are discussed further below.

[0079] As already mentioned, camera 204A may be a stereoscopic camera. Such a stereo camera has the advantage of providing depth information or size information of the object, thus better aiding facial recognition and object tracking of subjects 206 discussed above. In still other variations, camera 204A may be a thermal-video camera, that may or may not also be a stereo camera. Let us study this variation now in greater detail.

[0080] Thermal camera(s): In such a variation, a given site 202, such as a building or an arena or a school or any other site shown in FIG. 2, is fitted with one or more thermal cameras 204A. As per above, for brevity, we may refer to thermal camera(s) 204A in the singular with the knowledge that data streams from multiple cameras 204A will be combined by anomalous subject and device identification system 200 for analysis. Camera 204A may just be a pure thermal camera and capture the infrared spectrum of the electromagnetic radiation only. In such an implementation, data processing module 220 recognizes and tracks objects or people 206 based on just their temperature readings or thermal signature alone.

[0081] However, in other variations, camera 204A is a bi-spectrum camera because it captures both visible and infrared spectrums of the electromagnetic radiation. Preferably, thermal camera 204A is also a stereoscopic or stereo camera because then it can capture depth/size information. Regardless, thermal camera 204A working in conjunction with data processing module 220, identifies and tracks each individual person amongst persons/people 206 at site 202 and further, reads their body temperatures. Thus, each individual/person along with his/her identifier per above, is also associated with a body temperature reading that is taken in real-time or near real-time. The temperature readings of each tracked/identified person are then provided to baseline engine 110 for analysis.

[0082] Such an embodiment is shown in greater detail in FIG. 3. FIG. 3 is a variation of FIG. 2 showing our site 202 now configured with an entrance 212 denoted by a dotted and dashed line. People 206A, 206B, . . . or simply people 206 are shown entering site/building 202 through entrance 212. People 206 may be a few, or in dozens, or in thousands or even more in number at crowded site 202. There are one or more thermal cameras 204A, which we will simply refer to as camera 204A per above, targeted or aimed at entrance 212. As people 206 enter the building, camera 204A captures their visible and infrared video streams. More specifically, person 206A has a temperature reading of 210A, person 206B has a temperature reading of 210B, and so on as shown.

[0083] These visible and infrared video data streams or simply data streams are communicated to data processing module 220 via network backbone 208. Data processing module 220 identifies and tracks each subject 206A, 206B, . . . amongst subjects 206 per above, and associates a temperature reading with them. It then communicates this information to baseline engine 110 for analysis.

[0084] Preferably, module 220 communicates data packets containing the following information to engine 110: [0085] 1. A timestamp at which the observation is made by camera 204A. [0086] 2. An object identifier assigned to each subject/person 206A, 206B, . . . per above. [0087] 3. (x, y, z) coordinates or location information of each identified subject/person at site 202. [0088] 4. A temperature reading of the identified subject/person at timestamp in (1) above.

[0089] These data packets are then parsed by baseline engine 110 which then establishes a baseline 120A for the normal temperature readings for the individuals and identifies anomalous individuals per prior teachings. Preferably, an anomalous threshold value is provided as an input to baseline engine 110. For example, a normal threshold value of 38° C. or 100.4° F. is provided to baseline engine 110 that incorporates this value into baseline 120A with centroid 182A. It then identifies as anomalous any subjects with body temperatures above the normal threshold value.

[0090] A number of very useful scenarios are discovered/caught by the present embodiments of the anomalous subject and device identification system of the present design. The present technology allows an early detection of potential health and security threats in a reliable and flexible manner. A non-exhaustive list of useful scenarios identified/caught by the present design includes: [0091] 1. Elevated body temperature: Continuing with the above discussion, any individuals, such as person 206C, showing a temperature reading equal to or greater than this normal threshold value are then identified as anomalous by baseline engine 110. If there are multiple thermal cameras 204A, then video data streams from these cameras is processed by combining them at or by data processing module 220 that then tracks objects/people across the various data streams of different cameras and identifies anomalous subjects with elevated body temperatures per above teachings. Preferably, the temperature reading performed by thermal camera(s) 204A is accurate within an error tolerance of less than or equal to 0.3° F. [0092] 2. Mask detection and/or enforcing mask wearing: The facial recognition capabilities of module 220 also allow detection of facial masks worn by individuals/personals. Preferably, the facial recognition capabilities are not degraded as a result of subjects wearing masks. Therefore, anomalous subject and device identification system 200 of FIG. 3 is able to detect which subjects amongst subjects 206A-E are wearing facial masks. Baseline engine can then establish baseline 120B based on wearing of masks by the subjects as the normal behavior, and any subjects not wearing a mask can be signaled as an anomaly. Hence, mask wearing can be appropriately enforced upon those individuals/subjects who are not wearing masks.

[0093] Furthermore, while an anomalous subject with elevated body temperature per above, signifies a problem/anomaly, but if that individual is also not even wearing a mask, then that is even a greater anomaly or problem or threat, and baseline engine 110 can identify him/her as such. [0094] 3. Enforcing social distancing: Based on the capabilities of the present design and specifically the present embodiments, system 200 is able to enforce social distancing amongst subjects, such as that needed during the Covid-19 pandemic. Because the subjects are assigned an identifier and their location, speed and movements are known/tracked, the system can determine which subjects are not following social distancing guidelines. In the present case, proximity to other subjects may be a dimension on the hypercube of the respective baseline established by engine 110. A proximal distance, for example 6 feet, can be provided as an input to baseline engine 110 representing the minimum threshold value. If a given subject is in repeated violation of the minimum threshold value/distance, then this situation and the subject can be conveniently identified and flagged by baseline engine 110. [0095] 4. Weapons detection: Depending on the image/object recognition capabilities of data processing module 220, data streams captured by cameras 204A can be used to determine if a subject is carrying a weapon at site 202. Of course, the present technology can support additional specialized sensors for weapons detection, such as metal or ballistic detectors at the site, instead of or in addition to sensors 204 shown in FIG. 2. Such sensors allow system 200 to specifically detect guns, knives and other prohibited articles. As a result, system 200 in conjunction with baseline engine 110 can identify any anomalous individuals that may be carrying a prohibited weapon at site 202 per prior teachings. [0096] 5. Thermal signatures: The above capabilities utilizing thermal cameras of the present technology also allow system 200 to harvest thermal signatures of subjects at site 202. For instance, each subject may have a slightly different normal body temperature that can be captured and cataloged by the system in an appropriate database. Similarly, an overall infrared signature of the bodies or forms of each subject may also be captured and cataloged in the database.

[0097] Microphone(s): While referring back to FIG. 2, a given site 202, such as a building or an arena or any other location, is fitted with one or more microphones 204B. As per above, for brevity, we may refer to microphones 204B in the singular with the knowledge that data streams from multiple microphones will be processed by anomalous subject and device identification system 200 for analysis per above.

[0098] While typically microphones will come integrated with cameras 204A, this is not necessarily the case. It is conceivable to have a site where audio signatures of subjects alone are used for identification and tracking and for determination of anomalous subjects. Examples of such audio sensitive sites include theaters, studios, etc. Moreover, the audio signatures may be combined with video signatures for better tracking of objects.

[0099] Data processing module 220 of FIG. 2 may correlate an audio signature or identifier of a subject amongst subjects 206 based on audio stream from microphone 204B, with a video signature or identifier of the subject based on video stream from camera 204A to pinpoint the location of the subject with greater fidelity.

[0100] It can then better provide the movement patterns or temperature readings of these subjects to baseline engine 110 for analysis per above teachings.

[0101] Asset sensor(s): While still referring to FIG. 2, a given site 202, such as a manufacturing or a chip fabrication facility or any other location containing important or valuable assets, is fitted with one or more asset sensors 204C. For the purposes of present discussion an asset is a subject that is not a sentient being but still a valuable and/or sensitive item or thing whose monitoring is required. Examples include manufacturing equipment, apparatus, vaults/safes, valuable paraphernalia, or any other item of value at site 202 whose monitoring is justified. As per above, for brevity, we may refer to asset sensors 204C in the singular with the knowledge that data streams from multiple asset sensors will be processed by anomalous subject and device identification system 200 for analysis per above.

[0102] Asset sensor 204C captures data from one or more xmitters installed in or near or around assets present at the site. In the embodiments where site 202 is a manufacturing or chip fabrication facility, an xmitter can be any sensor installed in or near a manufacturing equipment or asset that senses/monitors the asset and transmits the sensed/monitored data to asset sensor 204C. An xmitter at a manufacturing or any other site can be based on any suitable wired or wireless technology including blue-tooth, cellular network, radio frequency identification (RFID), Zigbee, etc.

[0103] Exemplarily, such an xmitter monitors the asset to ensure that it stays at a given location. Alternatively or in addition, such an xmitter may perform measurements of one or more manufacturing parameters for and/or in conjunction with the asset/equipment/tool, such as, reading the value of a voltage, a current, a pH, etc. It then transmits this reading or sensed data, either by a wired connection or wirelessly to an asset sensor of the present design, such as asset sensor 204C.

[0104] FIG. 4 shows such an embodiment in greater detail. More specifically, FIG. 4 is a variation of FIG. 2 where site 202 is a manufacturing facility, for example, a chip fabrication facility or fab. Facility 202 has a manufacturing line 214 that has various manufacturing assets or tools 216A, 216B, 216C and 216D as shown. These assets are being monitored by various xmitters of the present principles. Specifically, xmitter 218A is in charge of monitoring asset/equipment 216A and 216B, xmitter 218B is monitoring asset 216C and xmitter 218C is monitoring asset/equipment 216D.

[0105] Data surveilled or monitored by xmitters 218A-C is then transmitted, by wire or wirelessly, on-demand or at regular intervals or on realtime or near-realtime basis, to asset sensor(s) 204C. Asset sensor 204C may be any wireless sensor receiving data packets from xmitters 218A-C based on techniques known in the art. For instance, asset sensor(s) 204C may communicate with xmitters 218A-C using one or more of blue-tooth, cellular network, radio frequency identification (RFID), a Zigbee or any other suitable wireless technologies required for a given implementation.

[0106] Asset sensor 204C then communicates this data to data processing module 220 as shown. In the present embodiment, data processing module 220 performs any necessary processing of data received from xmitters 218A-C before providing it to baseline engine 110 for analysis. In an exemplary embodiment, data processing module 220 normalizes data between one or more assets. In the same or another variation, module 220 correlates data between assets of the same type or of different types. In any event, the processed data is provided to baseline engine 110 for analysis. Baseline engine now establishes a rolling baseline for assets 216A-D based on data received from xmitters 218A-C and identifies any assets or subjects that may be anomalous.

[0107] In the preferred embodiment, baseline engine 110 establishes a rolling baseline for each different type of asset or manufacturing tool/equipment. For example, if site 202 is a fab then baseline engine 110 may establish a rolling baseline 120B with centroid 182B for chemical vapor deposition tools, and another baseline for metrology tools, etc. as shown. Note that in FIG. 4, to avoid clutter, only one such baseline with its centroid are shown marked by reference numerals 120B and 182B respectively.

[0108] FIG. 5 shows a variation of FIG. 4 containing a camera(s) 204A from the embodiments of FIG. 3 explained earlier. Also shown are human subjects 206F, 206G and 206H. Camera 204A is in charge of monitoring/surveilling people 206F-H present at site 202 per earlier explanation. There is also a data processing module 220 in FIG. 5 of above discussion. In the present embodiments, in addition to its functions already described above, data processing module 220 also correlates data between human subjects 206F-H and manufacturing subjects or assets 216A-D. If camera or cameras 204A are also thermal cameras, then temperature readings 210F-H of subjects 206F-H respectively are also available as shown.

[0109] There are a number of useful scenarios that are identifiable by the variations shown in FIG. 4 and FIG. 5. A non-exhaustive list of these scenarios includes: [0110] 1. Dwell times: Examples of useful correlations between data from asset sensor(s) 204C and camera(s) 204A include which human subjects 206F-H have been in the vicinity or proximity of manufacturing subjects/assets 216A-D during a given time window, the dwell times of subjects 206F-H around subjects 216A-D in a given time window, the temperature readings of these subjects, where else in facility 202 have subjects 206F-H been, their past security profile/performance, etc. This correlated data is then provided to baseline engine 110 which analyzes it and identifies anomalous subjects or assets at site 202 per prior explanation. [0111] 2. Training or other security issues/threats: If a subject/human 206B has shown a greater than normal dwell time around a malfunctioning asset 216D, then this may signify a training problem. If asset 216D has been involved in security incidents in the past, then this may signal a security issue or threat associated with subject/human 206B. [0112] 3. Espionage: If an unauthorized universal serial bus (USB) device with malware is inserted into subject/asset 216C that is exfiltrating data, then baseline engine 110 will catch this incident. More specifically, data transmission/download patterns of subject/asset 216C as compared to its baseline will signify a greater than “normal” activity. Such an anomalous activity will be identified based on the distance of the number of transmitted/downloaded data packets from the centroid of the hypercube of the baseline per prior teachings. This is one form of espionage that is identifiable by the present technology.

[0113] As another example, if an unauthorized subject/human has excessive dwell time around a sensitive asset, then this might signify another form of espionage.

[0114] Similarly, a variety of other useful scenarios that are based on correlating data related to subjects 206F-H and captured by camera(s) 204A with the data related to subjects 216A-D captured by asset sensor(s) 204C, are conceivably caught and are identifiable by the embodiments explained in relation to FIG. 4-5.

[0115] Personal-device sensor(s): In a highly preferred set of embodiments, a given site 202 of FIG. 2, such as a building or an arena or any other site, is fitted with one or more personal-device sensor(s) 204D. Personal-device sensors 204D are wireless sensors based on one or more of a blue-tooth sensor, a cellular signal sensor, a radio frequency identification (RFID) sensor/reader, a Zigbee sensor or any other suitable wireless technology sensor required for a given implementation. Personal-device sensors 204D are in charge of communicating with the devices carried by various subjects at site 202.

[0116] If a personal-device sensor is a blue-tooth sensor, it is responsible for communicating with blue-tooth personal-devices, if it is a cellular signal sensor, it is responsible for communicating with cellular personal-devices such as cellular phones, if it is an RFID reader, it is responsible for communicating with RFID personal devices such as RFID tags, which may be active, passive or semi-active tags. If the personal device sensor is a Zigbee sensor, it is responsible for communicating with Zigbee personal-devices such a Zigbee end-devices.

Depending on the requirements of an implementation and the capabilities of a particular wireless technology, any of the communication above may be bi-directional or uni-directional i.e. only from the personal-devices to the personal-device sensor. Moreover, more than one sensors of the same or different type may be integrated into a single composite sensor/device in the present or any other embodiments of this disclosure.

[0117] A personal-device carried by a subject may or may not actually be owned by him/her or be his/her “personal” device in a manner of ownership. However, for the purposes of this disclosure any device carried by the subject is termed as a personal-device. Such subjects are typically human beings and the devices carried by them may be cellular phones including smartphones, tablets, wearable devices such as smartwatches, laptop computers, etc. Note however that there are situations that a personal-device is unattended or not carried by any subject. Such a situation is discussed in detail in the embodiments explained below.

[0118] FIG. 6 shows the present embodiments in greater detail. Specifically, FIG. 6 is a variation of FIG. 2 emphasizing the wireless sensor capabilities based on the instant principles. Site 202 of FIG. 6 shows a number of human subjects carrying devices. Specifically, FIG. 6 shows subject 206I carrying a device 222I, 206J carrying a device 222J, 206L carrying devices 222L and 224L, 206M carrying device 222M and subject 206N carrying devices 222N, 224N and 226N at site 202. Note that subject 206K is not carrying any device while subject 206L is carrying two devices 222L and 224L, and subject 206N is carrying three devices 222N, 224N and 226N. While the present design supports only having one personal-device sensor, the embodiment of FIG. 6 explicitly shows two personal-device or wireless sensors 204D1 and 204D2 which may be based on any number of available wireless technologies, some examples of which were listed above.

[0119] Now, based on triangulation and trilateration techniques known in the art and the availability of sufficient number of sensors 204D, the present design is able to determine where each device carrying subject is on the premises of site 202. For this purpose, our data processing module 220 may again be utilized with the necessary algorithms for locating devices 222, 224 and 226 with their respective subjects 206 at site 202. As noted, two such exemplary algorithmic techniques include triangulation and trilateration.

[0120] As a consequence, module 220 may determine that individual/subject 206I is in region R1 of site 202, individuals/subjects 206J and 206L are in region R2 and subjects 206M and 206N are in region R3. Furthermore, data processing module 220 of the present design also assigns an identifier to each device that it detects at site 202. Note that subject 206K who is not carrying any device will not be detected by sensors 204D1 and 204D2 alone. For this purpose, we will defer to embodiments discussed further below.

[0121] Now, given the above setup, the wireless embodiments of FIG. 6 are able to provide a number of important capabilities for identifying anomalous situations. A non-exhaustive list of such capabilities and situations/scenarios is provided below: [0122] 1. Location detection: As noted above, with two or more personal-device sensors 204D1-2, the system is able to determine the location of each device carrying subject at site 202 using techniques including triangulation and trilateration. [0123] 2. Anomalous movement patterns: Based on location detection, the system is further able to determine movement patterns or speed and direction at a given point in time of subjects 206I, 206J, 206L, 206M, 206N. If any subjects exhibit erratic or distressed movements, they can be identified by baseline engine 110 per above teachings. [0124] 3. Anomalous dwell patterns: In a similar manner, system 200 is able to identify different from normal dwell times of any subjects 206I, 206J, 206L, 206M, 206N at sensitive locations at site 202 based on their authorization level. Similarly, the proximity of one subject to other subjects that is not regarded normal for a given implementation, etc. can also be identified and an anomalous subject identified per above. [0125] 4. Excessive beaconing in unused media access control (MAC) address space: Let us consider the scenario where site 202 of FIG. 6 is an office building with a local area network (LAN) powered by one or more personal-device or wireless sensors or antennas 204D1-2 and subjects 206I-N are expected to be employees. Prior to joining the LAN, a device beacons in an unused MAC address space, that is, by not using its real or correct MAC address. However, this beaconing is still detected by sensors 204D1-2 and data processing module 220 assigns it an identifier.

[0126] Only after a device joins the LAN, it beacons with its correct MAC address and at which point system 200 can use its real MAC address as the device identifier. If it is expected that employees 206I-N will be connected to the LAN, then a device that continues to beacon in the unused MAC address space for a greater than normal period of time, will be identified as a suspect device by baseline engine 110.

[0127] More specifically, baseline engine 110 will establish rolling baseline 120D with a normal behavior of data streams from sensors 204D1-2 indicating that devices at site 202 start communicating with their real MAC address within a “normal” time window. This time will be a dimension in the conceptual hypercube with centroid 182D of baseline 120D. If a device such as device 222J carried by employee 206J beacons in the unused MAC address space for greater than normal time, then it will be far away enough from centroid 182D along this dimension to signify an anomaly. Such an anomaly may indicate a breach or security incident or a threat, or a technical issue. As a result, employee 206J with device 222J will be flagged/signaled as an anomaly by engine 110. These and other useful scenarios are easily identifiable and caught by anomalous subject and device identification system 200 of the personal-device sensor embodiment shown in FIG. 6.

[0128] Personal-device sensor(s) together with camera(s): In a highly useful set of embodiments personal-device sensors 204D of FIG. 2 and FIG. 6 work together with cameras 204A of FIG. 2 to provide additional fidelity to our anomalous subject and device identification system. Such an embodiment is shown in FIG. 7. Just like the embodiments of FIG. 5, cameras for the present embodiments are a desirable but not necessary type of sensor to accrue the benefits of the present technology.

[0129] FIG. 7 is a variation of FIG. 6 but also with cameras 204A of the prior teachings. FIG. 7 also shows a device 222M in region R2 of site 202 that is not carried by any subject. In the example of FIG. 6, two cameras 204A1 and 204A2 are explicitly shown as well as data processing module 220 that amongst other things, performs object tracking and facial/image/object recognition. In a manner analogous to the embodiments of FIG. 5, cameras in FIG. 7 add fidelity to the embodiments of FIG. 6 while also providing additional capabilities as discussed further below. For example, cameras 204A1 and 204A2 of FIG. 7 are able to detect and track subject 206K who is not carrying any wireless device detectable by personal-device sensors 204D. Of course, camera(s) 204A1-2 are able to afford all the capabilities to the embodiment of FIG. 7 as already explained in reference to the embodiments of FIG. 3.

[0130] Moreover and very importantly, system 200 with cameras 204A1-2 working in conjunction with data processing module 220 as well as personal-device sensors 204D1-2 is now able to associate a specific subject with each device. Anomalous subject and device identification system 200 of FIG. 7 assigns an identifier to each subject as well his/her associated device(s) per above. As the subject moves around the building/site 202, the system is able to ascertain the physical proximity or correlation between the subject and his/her devices.

[0131] Data streams from sensors 204A1-2 and 204D1-2 processed by module 220 are then provided to baseline engine 110. Based on data streams from cameras 204A1-2, baseline engine establishes one or more baselines 120A1, 120A2, 120A3, . . . 120AN for the dimensions of conceptual hypercube of interest with correspondent centroids 182A1, 182A2, 182A3, . . . 182AN. Similarly, based on data streams from wireless sensors 204D1-2, baseline engine establishes one or more baselines 120D1, 120D2, 120D3, . . . 120DN for the dimensions of conceptual hypercube of interest with correspondent centroids 182D1, 182D2, 182D3, . . . 182DN. It then scores each incoming packet from these data streams against the above baselines by computing the distance of the packet from the respective centroids on a certain dimension of interest. If the distance is far enough or greater than what is normal for the respective baseline, it identifies that packet as an anomalous packet and signals an anomaly identifying the associated subject and/or device per prior teachings.

[0132] Such a capability allows a number of important scenarios to be discovered/caught by anomalous subject and device identification system 200 of FIG. 7 based on the present technology. A non-exhaustive list of these include: [0133] 1. Lack of a device carried by a subject: A subject, such as subject 206K detected and tracked by cameras 204A who is not carrying any device may indicate a suspect situation for site 202. In this case, one dimension of the conceptual hypercube will exemplarily be the number of devices carried by a subject. If the number is 0 or too high above the normal, then this indicates an anomaly for site 202. Such a scenario along with the anomalous subject is identified by the present technology per above teachings. [0134] 2. Unattended device: Device 222M that is not carried by any subject may also be a suspect situation. Such a device 222M can be detected by one or more of various types of appropriate sensors supported by the present design, including cameras 204A1-2 and personal-device sensors 204D1-2 of FIG. 7. Device 222M will have an assigned identifier by data processing module 220 per above. If there has been no subject associated with this device identifier, then the device itself and alone is identified by the system as an anomalous device. On the other hand, if the device identifier has been associated with subject 206M with his/her own identifier, then system 200 is able to ascertain that subject/human 206M was previously associated with or carrying device 222M. Subject 206M may or may not be on site 202 at that point in time.

[0135] Any of the above scenarios may simply signify an innocuous situation, such as a lost device. On the other hand, these may also indicate a more serious security incident/threat associated with device 222M and subject 206M. Regardless, the above scenarios along with the subject and/or device in question are signaled by baseline engine 110 as anomalies and identified. [0136] More specifically, in these scenarios, one dimension of the conceptual hypercube will exemplarily be the number of subjects associated with a device. If the number is 0 or greater than 1, then this indicates an anomaly for site 202. Per above, if there is a prior association of an anomalous device with a subject then that subject is also identified, otherwise just the device itself is identified as anomalous by the anomalous subject and device identification system 200 of the present design. [0137] 3. Transfer of a device: In an analogous manner, if a device that was once associated with one subject is now associated with another subject, such a situation also rises to a level of concern or anomaly. Again, such an anomaly caught by the present design may be innocuous or a more serious security exposure or threat. In this case also, one dimension of the conceptual hypercube will be the number of subjects associated with a device. If the number is 0 or greater than 1, then this indicates an anomaly for site 202.

[0138] Wireless sensors with site instrumentation: In addition to or alternatively of cameras, in some embodiments the wireless sensors of the present design are augmented by wireless antennas instrumented/installed at the site. Like cameras, these local antennas and instrumentation provide additional fidelity to the anomalous subject and device identification system of the present design.

[0139] FIG. 8 assists us in explaining these embodiments in greater detail. FIG. 8 is a variation of FIG. 7 but with an RF infrastructure containing two additional radio antennas 232A and 232B installed in regions R2 and R3 of site 202. Preferably, these antennas are wifi antennas or access points operating in a radio frequency range of 2.5 to 5 GHz. Preferably still, the antennas are cellular antennas. Such antennas may be used in conjunction with a spectrum analyzer (not shown specifically) in order to read and analyze the cellular signals from the devices. Based on the signal strength and/or other network techniques in the art, these antennas assist in knowing whether a device is close to an antenna or not. This knowledge further supplements the object tracking by data processing module 220 enabled by cameras 204A.

[0140] Any number of antennas 232A, 232B or more, installed in the local infrastructure at site 202 can operate in one or more of at least two configurations: (i) the antennas act as a booster for wireless sensors 204D1-2 by collecting data on the ground close to the devices at site 202 and then communicating it to sensors 204D1-2 either by wire or wirelessly, (ii) the antennas themselves operate as sensors 204D installed at optimal locations at site 202 for maximum signal coverage/strength. In other words, they may supplement existing wireless sensors 204D, but instead of or in addition to, may also act themselves as wireless sensors 204D.

[0141] In the absence of cameras 204A, antennas 232A and 232B assist in the determination of the location of a device with respect to the antennas in conjunction with wireless sensors 204D. As explained earlier in reference to the embodiments of FIG. 6-7, this is accomplished by using network algorithmic techniques including triangulation and trilateration, etc. preferably performed by data processing module 220. Any number and type of such antennas based on various available wireless technologies may be installed at site 202 depending on the requirements of an implementation. The various antennas at the site may all be based on the same or different wireless technologies depending on the types of wireless devices they need to communicate with.

[0142] Using sensors on computing devices: In a highly useful set of embodiments, sensors available on computing devices are used to accrue the benefits of the anomalous subject and device identification system of the present design. The benefit of these embodiments is that instead of requiring separate sensors, sensors that are already ubiquitously present in today's computing devices are utilized. Exemplary computing devices include laptops, tablets, cellular phones including smartphones, wearable devices (including smartwatches and medical devices), security devices, etc.

[0143] Let us take advantage of FIG. 9 to discuss these embodiments in greater detail. FIG. 9 is a variation of FIG. 2 showing an anomalous subject and device identification system 300 of the present design operating at a site 302. Camera and microphone sensors 204A and 204B respectively of the prior discussion are now embodied in a tablet 234, wireless asset sensor 204C is now embodied in cellular phone or smartphone 236 and wireless personal-device sensor 204D is now embodied in cellular phone or smartphone 238. By being embodied here we mean that the sensor in question may be integrated with the device or operably connected to it, such as via a USB port.

[0144] Kiosk 205 discussed further below has a computing device 240 installed in it. Device 240 may be a tablet or a cellular phone/smartphone or even a laptop or the like. Not all of sensors 204A-D above need to be embodied in computing devices. In other words, any subset of the sensors may be separately installed as in the embodiments of FIG. 2-8. Also shown in FIG. 9 is data processing module 220 that works in conjunction with the sensors per prior discussion.

[0145] All the relevant teachings of the prior embodiments apply to the present embodiments also, except that the sensors are now on economically and ubiquitously available on (personal) computing devices. One of the advantages of the present embodiments is that a given site, such as site 302 can be quickly provisioned with the instant anomalous subject and device identification system 300. This is because the computing devices housing the sensors of interest, such as devices 234, 236, 238 and 240 are cheaply and readily available. Moreover, they have a small form factor, such that they can be easily and flexibly deployed at site 302 for optimal results. In an interesting application of the present embodiments, mobile devices with police officers containing cameras, microphones and/or other sensors are used to surveil a location on a short notice per above teachings.

[0146] Kiosks: Referring to FIG. 2, the present technology lends itself well to showcasing its capabilities at a kiosk 205 at site 202. Kiosk 205 may be instrumented with one or more sensors 204. These sensors may further be embodied in a computing device installed or operating in the kiosk.

[0147] Referring now to the embodiment of FIG. 9, kiosk 205 shows a computing device 240 that may be a tablet operating in it. Exemplarily, tablet 240 may be instrumented with a camera, such as camera 204A and a microphone, such as microphone 204C along with a data processing module 220 of the present design. Then, guests/subjects 206 at site 306 may use the kiosk to take their temperature reading or to ensure that their mask is detectable or to get familiarized with the capabilities of anomalous subject detection system 300 at site 302.

[0148] Data layering: In the preferred embodiment, the present technology is implemented by storing the data streams from various sensors, such as sensors 204 at site 202/302 as separate data-tracks or layers in a file. Each data layer or track in the data file corresponds to a data stream from a sensor. For example, there may be a radio frequency (RF) data layer, a cellular layer, a blue-tooth layer, a video layer, an audio layer, etc. This layering may be performed by data processing module 220.

[0149] Additionally, as object recognition is performed, an underlying subject/device data layer containing characteristics of the objects being recognized and to whom an identifier is assigned per above, is also created. For instance, if the object recognition function recognizes two persons amongst persons 206 with identifiers 78X67 and Y6790 with heights 6 foot, 3 inches and 5 feet, 6 inches respectively, then this data is stored in the underlying subject/device data layer in the data file.

[0150] Where there are multiple sensors of the same type, such as cameras 204A1 and 204A2 in FIG. 7-8, the data streams from these sensors can be stored as separate data layers. Alternatively, the data streams may first be combined into a composite data layer of video type by data processing module 220 and then stored in the data file. The present design thus affords the above multilayer approach to data streams obtained from various sensors.

[0151] Forensic analysis: As already mentioned, the embodiments of FIG. 2-9 utilize cloud 230 for archiving the findings of baseline engine 110 and for performing analytics on the archived data. Such analytics or forensic analysis, that preferably utilizes machine learning (ML) and artificial intelligence (AI) techniques, can be extremely useful. This is because it can allow answering hard questions for establishments and allow them to limit liability and/or manage risk.

[0152] For example, let us consider that a site, such as site 202/302 of the of the prior discussion is a restaurant/school. Then a claim by a patron/student 206 that he/she got infected with Covid-19 virus while at the restaurant/school on a given date may be challenged by uncovering evidence in the archive that the patron/student was not wearing a mask on that date at the restaurant/school. In another interesting application of the above embodiments for performing mask wearing enforcement/detection, a local government may audit a chain of hotels or restaurant based on the above-discussed instant archived data in cloud 230 to determine if they have been allowing patrons without masks.

[0153] Furthermore, as the data streams from sensors 204 about subjects at site 202/302 is stored in a database, whether the database is on-premise at site 202/302 or in cloud 230, this allows the creation of profiles for individual subjects. This capability is also very useful because any analytics performed on the output of baseline engine 110 can then be matched against the profile of the subject in question to determine whether a specific behavior matches his/her profile. If it does not, then system 200/300 updates the subject or target profile accordingly. The profiling capability further allows system 200/300 to blacklist or whitelist subjects as needed.

[0154] In yet another variation, the anomalous subject and device identification system of the present design further analyzes data from subjects based on their police record. For example, one dimension of the hypercube of the baseline established by baseline engine 110 may be the number of arrests or warrant or charges, etc. for the subjects. This information may then be utilized to determine if a given subject scored on that dimension is likely to be associated with an anomalous situation based on above teachings.

[0155] Overall: Any of the embodiments taught above may utilize a wired or a wireless connection as appropriate to facilitate communication between sensors, devices and ground infrastructure. Furthermore, backbone 208 discussed in various embodiments above may also be wired or wireless. Furthermore, various capabilities of the above embodiments may be combined (mixed and matched) depending on the number and types of various sensors and/or devices involved in an implementation.

[0156] Furthermore, exemplary sites/locations that may benefit from the anomalous subject and device identification system with its above-taught embodiments include airports, train stations, subways, central bus stations, embassies and consulates, government buildings, stadiums, arenas, venues, convention centers, Fortune 500 companies' headquarters or key offices, hospitals, universities/colleges, schools, restaurants and hospitality centers, office buildings, etc.

Indicators of Compromise by Analyzing Data Based on ROLLING Baseline

[0157] In a highly preferred set of embodiments, the data from a variety of objects is analyzed by the rolling baseline engine of the prior teachings to determine whether any of those objects may be compromised. The term object in these embodiments is used to refer either to a finished product or simply put, a product, or a component assembly or simply put, a component, that may be assembled into or integrated into or to produce a product.

[0158] Exemplarily, a product may be consumer product or device or an end-user device, such as a smart television, a smartphone, a tablet, a thermostat, a smart fridge, a security camera, a microphone, a measuring device, any other internet of things (IoT) devices, or any other consumer product/device. The product may also be an industrial product/device, or any other conceivable finished product. A component may be any conceivable component or assembly that is used in the assembly or manufacturing of a finished product, including the ones listed above. A component may also be a semi-conductor component, a chip or a circuit board or circuitry that may eventually be used in a product including the ones listed above.

[0159] A few examples of objects covered by the present embodiments are shown in FIG. 10. More specifically, FIG. 10 shows one or more smart meters 400A sending their operational data, such as their measurements or meter readings at defined intervals or on-demand to our rolling baseline engine 110 via network backbone 208. FIG. 10 also shows one or more circuit boards 400B that send their prediction, simulation, test, verification, performance, etc. data during their manufacturing to baseline engine 110 via network backbone 208. Network backbone 208 may be connected to other systems and Information Technology infrastructure not explicitly shown in FIG. 10, and which is responsible for consuming and processing data generated by objects 400. Depending on the embodiment network backbone 208 may be a secure or private network or the internet or any combination thereof.

[0160] FIG. 10 also shows one or more digital cameras 400C that may include security cameras, sending camera footage, such as a surveillance footage, to baseline engine 110. FIG. 10 further shows one or more microphones 400D that send their operational data such as audio recordings to baseline engine 110. Also shown in FIG. 10 are one or more semi-conductor components or chips 400E that send their manufacturing data, such as prediction, simulation, test, verification, performance, etc. data to baseline engine 110 via network backbone 208. There are also one or more smart computing devices 400F, which may include a smart phone, a tablet, or any other mobile computing device that send data to baseline engine 110.

[0161] Also shown in FIG. 10 are one or more smart home devices 400G, including Google Nest™ and Amazon Echo™ devices, such as temperature or atmospheric sensors, that send their data, such as temperature readings, carbon monoxide readings, etc. to baseline engine 110 via network backbone 208. FIG. 10 also shows one or more smart TVs 400H that sends their data, such as performance data or playtime, or any other statistics to baseline engine 110. Further, the dotted line 400I in FIG. 10 indicates that any number and types of such objects/devices may benefit from the present technology not explicitly shown in FIG. 10, that send their manufacturing and/or operational or any other kind of data to baseline engine 110 for analysis.

[0162] As noted above, data in the present embodiments refers to any data that any of the above objects generates during its course of manufacturing/construction/assembly or operation thereafter. Exemplarily, if the object is a semiconductor chip or a circuit, the manufacturing data may be its prediction, simulation, test, verification, performance, etc. data. Such data may also take the form of “pings” or “heartbeats” or measurements or surveillance footage or an operational report, performance data, or any other type of conceivable data generated by an object or expected to be generated by the object during its lifecycle.

[0163] Baseline engine 110 then analyzes this data and determines if the corresponding object or objects that generated the data may be compromised. This determination is referred to as finding/detecting/generating an indicator of compromise in the object(s) by baseline engine 110. Per prior teachings, this is accomplished by identifying anomalous packets far enough away from centroid 182 representing the normal population of packets of data received from the objects.

[0164] In one example, object 400A is a smart tachometer, such as the one used in a modern car, that may normally transmit its reading every 1 minute to rolling baseline engine 110. Now, if the frequency of such a reading inexplicably changes to 5 minutes, then this may indicate a compromise of/in object/device 400A as detected by baseline engine 110. Explained further, a hacker or a malware may have intruded object 400A and may be executing remote and unauthorized commands on it and/or be sending/receiving unauthorized messages to/from it. Remote command execution on an IoT device, such as object 400A of FIG. 10, may result in consuming its limited computational and network resources. As a consequence, it may now only be able to report its readings at every 5 minutes, instead of every 1 minute, as originally intended.

[0165] Our baseline engine 110 is able to detect such a misreporting or deviation of reporting frequency of object 400A as a far enough distance from centroid 182 of hypercube 180 of baseline 120, per prior teachings. This deviation is identified as an anomaly or an indicator of compromise of/in device/object 400A in the present embodiments. As a consequence, remedial or corrective actions may be immediately taken, which otherwise could have resulted in a catastrophic outcome, such as a car accident.

[0166] In a desirable set of variations of the present embodiments, the rolling baseline engine detects indicators of compromise in objects by analyzing their data sent to or residing in a cloud. There are many different types of clouds in which such internet-enabled objects may send their manufacturing, operational or any other data. These include at least: [0167] 1. Generic automation testing clouds or farms. These clouds, also sometimes referred to as device farms or simply farms, are clouds that are used for the functional automation and testing of various devices, including IoT devices. Current providers of such farms include Saucelabs™, Xamarin™, Perfecto™, etc. These clouds offer generic testing environments for many types of devices running a variety of operating systems, including iOS™ and Android™. The devices testable in these cloud environments include not only mobile phones, but also a variety of IoT devices, such as smart refrigerators, smart TVs, smart thermostats, etc. that run on the supported operating systems.

[0168] As a consequence, a developer team or a concerned party can test the application(s) developed for the “target” device or devices with a target operating system(s) without incurring the capital expense of owning and operating those devices and operating systems. This is especially helpful when the developer needs to support an application on not only a single target device, e.g. iPhone 13, but a range of devices e.g. iPhone 6S through iPhone 15 running a variety of iOS versions, including iOS 10 through iOS 15. [0169] 2. Device specific clouds These clouds are specific to a given type of object or device. For example, Amazon web services (AWS™) device farms offer mobile application testing services for iOS and Android devices. Although the devices currently testable are mobile phones, the types of such devices are expected to grow and include other types of iOS and Android devices, including smart home and industrial devices, IoT devices, etc. [0170] 3. Vendor specific clouds There are also clouds that receive data from specific vendor devices. For example, LG™ TV, Sony™ TV, Samsung™ TV, and other vendors have dedicated clouds that are intended to receive a variety of operational data from various types of smart appliances include smart TVs that are operating in production. There are also vendor specific clouds for smart TVs, smart fridges, toasters, and any conceivable internet-enabled device. Such clouds could also be specific to individual model numbers of the specific types of objects/devices. [0171] 4. Component clouds

[0172] These clouds are used for receiving manufacturing data from individual components of a product. Such components include semi-conductor components or microchips or simply a chips or circuit boards or assemblies. The components may also be contained in a consumer/finished product in which case the finished product may be sending its operational data to a device cloud discussed above while its individual component(s) would send their data to a component cloud. [0173] Alternatively, the component cloud may be used for electronic design automation (EDA), also sometimes referred to as electronic computer-aided design (ECAD), of the component(s) during their design and manufacturing and before they have gone into production. Some of the current industry participants operating such component clouds include Invidia™, Lattice™, Siemens™, Synopsis™, etc.

[0174] Those skilled in semiconductor design will appreciate the tremendous benefits of using the cloud for EDA/ECAD. In a cloud-based EDA model, the various design, prediction, simulation, verification, testing and any other EDA artifacts are kept in the component cloud. The cloud is operated by a specialized contract manufacturer or a fab or a foundry or in some cases a large vertically integrated company. Regardless, the cloud-based EDA approach greatly optimizes the time to market, cost, quality and other metrics of EDA for a given chip, instead of the traditional approach of chip design where a company vertically owns and operates the various aspects of EDA for a given chip/product.

[0175] The object/device clouds presented above may be multi-functional where a single cloud serves as more than one of the above presented clouds. They clouds may also be tiered or hierarchical such that data in component clouds is combined or integrated with the data in the next higher tier and so on.

[0176] Regardless of the type and structure of clouds, including the ones described above, a variety of objects, which includes finished or end-products or simply products and/or their components, send their design, manufacturing, testing, performance and/or operational data to a cloud. The purposes of doing so may be various including their testing/verification, monitoring, improvement and optimization, analysis, etc. As noted, the objects may send their data to the cloud either before their operation in production i.e. during their design and development, as well as during post-deployment operation, or both.

[0177] In the present variations, the rolling baseline engine analyzes this data sent by the objects to the cloud or the data residing in the cloud, and detects any anomalies with the associated contexts per prior teachings. A deviation or anomaly thus detected serves as an indicator of a compromise of/in the associated objects. In other words, if the data in the cloud observed by our baseline engine is far enough away from the centroid of the “normal” population of the data packets, then this acts as an indicator of compromise of the objects. The rolling baseline engine may also itself operate inside the same cloud, a different cloud, or on-premise while analyzing the object data in the cloud.

[0178] The anomaly indicating a compromise may manifest itself in a variety of ways including an overload/overuse/overage or underload/underuse/shortage of any number of metrics or resources of the objects. These include CPU usage, memory usage, disk storage usage, network bandwidth usage, thermal output, to name a few. The manifestation may also be in the form of misreporting of operational data, such as underreporting or overreporting of data. It may also be in the interpretability of data, e.g. if the data report is obfuscated, unintentionally encrypted, or is incomprehensible or gibberish.

[0179] Early knowledge of an indicator of a compromise allows the developer or a concerned party to patch or address the vulnerability that caused the compromise/infiltration. If the object in question is a component, then this prevents a costly production of products with inherent flaws or vulnerabilities in them, only to cause bigger and more consequential damage in the future. If the object is a product, then this facilitates their firmware/software updates or patches or recalls to ensure their secure operation and satisfied customers. The anomaly/anomalies detected by instant baseline engine may also denote technical flaw(s) in the design/manufacturing of the objects and not just necessarily exploited vulnerabilities.

[0180] FIG. 11 shows an embodiment 450 with an exemplary device cloud 452 feeding data to our rolling baseline engine 110, with baseline 120, hypercube 180 and centroid 182 of normal population of operational data per prior teachings. In the embodiment shown in FIG. 12, rolling baseline engine 110 is obtaining data from cloud 452 for analysis, however per above, baseline engine 110 may itself be in cloud 452.

[0181] Regardless, baseline engine 110 analyzes data packets sent to or residing in cloud 452 that originated from various smart TVs 400H. Based on this analysis, it detects indicators of compromise in one or more smart TVs 400H. Per above, cloud 452 may be specific to a given TV manufacturer, and even a given TV model. By way of example, operational data of a given batch of smart TVs 400H of a given model may indicate that 10% of such TVs are getting their current data/time settings reset or corrupted. This may indicate a compromise through an exploit of a vulnerability of that model of TVs. Baseline engine 110 can detect such an indicator of compromise so adequate patches/updates could be made.

[0182] FIG. 12 shows an embodiment 500 with an exemplary component cloud 454 containing design and manufacturing data received from component chips 400E. By way of example, if component 400E is a memory chip or a microcontroller or a central processing unit or processor (CPU) or a graphics processing unit (GPU) or the like, with specific model number, then data that may be EAD data is sent by chips 400E to component cloud 454. Cloud 454 may be dedicated for the EDA of chip 400E or it may be shared for the EDA of a number of different types/models of semiconductor chips.

[0183] Our baseline engine 110 analyzes this data and detects if a malware or a hacker has compromised components 400E that may be causing their test data to deviate from the normal. Similarly, if verification data generated by an EDA job, such as functional verification data, is unintelligible or incomprehensible or gibberish or misreported, then this may be an indicator of compromise of chips 400E. Similarly, if thermal data from chips 400E indicates an overage of thermal output, this may be because a malware is overloading the chip, or is otherwise causing a design defect in the heat flow of chips 400E.

[0184] Thusly, an immediate response can be launched that could avoid expensive chip/circuit failures down the line. Similarly, EDA simulation data, such as logic simulation data may be encrypted by an attack, resulting in large enough distance from centroid 182 of normal population of data packets per prior teachings. This may be an indicator of compromise, such as a ransomware attack. In a similar manner, the data may be obfuscated in some other way indicating a compromise.

[0185] The proactive knowledge of the compromise allows a developer or a concerned party to patch the flaw or vulnerability in the objects that allowed the exploit to occur. This limits the damage and avoids further and potentially more catastrophic compromises to occur in the future.

[0186] Data in component clouds is used for improvements and optimization of the chip design. The present technology is thus effectively used in component clouds with its rolling baseline engine for detecting patterns of failures and/or compromise of/in the components/chip. In another exemplary scenario, if rolling baseline engine 110 detects a security flaw in 20% of chips 400E that are sending their data to cloud 454, then this may indicate a pattern of failure or exploitation of the chips. A timely knowledge of such a failure or flaw is useful for efficiently patching the flaw before additional batches of chips 400E are produced and before a “mass compromise” occurs. This is especially the case once these components are fitted into potentially millions or more of consumer products that are then sold.

[0187] In yet another interesting application, if a batch of cell phones from a deployment, such as a military deployment is detected to be sending unauthorized pings to a network by engine 110, then this is an indicator of compromise of those cell phones. In this application, the rolling baseline engine analyzes the data on a network, which may be a secure cloud, and determine such pings as an overuse of network bandwidth and an anomaly or an indicator of compromise. Early detection of such a compromise by baseline engine 110 of the present design can prevent serious downstream consequences.

[0188] In still another variation, the operational data received from various objects in the various clouds taught above, is used as training data for modeling the operational behavior of the respective objects using artificial intelligence (AI) techniques. Such modeling is then used to determine the normal patterns of behavior i.e. for generating an optimal operational model of the objects. This optimal operational model signifies or is correlated to centroid 182 of hypercube of 180 of baseline engine 110 shown in FIG. 10-12, and thus facilitates establishing rolling baseline 120 for the objects.

[0189] In view of the above teaching, a person skilled in the art will recognize that the apparatus and method of invention can be embodied in many different ways in addition to those described without departing from the principles of the invention. Therefore, the scope of the invention should be judged in view of the appended claims and their legal equivalents.