SYSTEMS AND METHODS FOR DETECTING APPLIED RAILCAR BRAKES USING THERMAL IMAGING

Abstract

Systems and techniques for detecting applied railcar brakes within a railroad classification yard using thermal imaging. A thermal imaging system, including at least one thermal imaging sensor, captures a plurality of thermal images of railcar wheels. The thermal imaging system determines if a temperature within a user-defined bounding box on the thermal sensor's field of view exceeds a predetermined temperature threshold for a specified amount of time. Alternatively, a neural network may be used to detect the excessive temperature. In response, the thermal imaging system sends a system message, and a computer processor receives the system message and, if it meets reporting criteria, identifies the particular railcar and generates and transmits an alert message indicating a potential issue with the wheels of the railcar.

Claims

1. A method for detecting potential issues within a railroad classification yard, the method comprising: capturing, using at least one thermal imaging sensor, a plurality of thermal images of at least a portion of an object within the railroad classification yard; determining, based on analysis of the plurality of thermal images, that a temperature within a bounding box on the plurality of thermal images has exceeded a predetermined temperature threshold assigned to the bounding box for a predetermined amount of time; generating, in response to determining that the temperature within the bounding box has exceeded the predetermined temperature threshold assigned to the bounding box for the predetermined amount of time, an alert message indicating a potential issue with the object within the railroad classification yard; and transmitting the alert message to at least one of a classification yard control system or a user interface prior to a railcar associated with the object being released to free-roll within the railroad classification yard.

2. The method of claim 1, further comprising, sending an automated signal to at least one component of the railroad classification yard to cause one or more physical components of the railroad classification yard to be actuated to release a brake system of the railcar associated with the object.

3. The method of claim 1, wherein generating the alert message includes: sending, in response to determining that the temperature within the bounding box has exceeded the predetermined temperature threshold assigned to the bounding box for the predetermined amount of time, to a logic controller: a system message including one or more of: data indicating a camera location, a timestamp, and the predetermined temperature threshold that was exceeded; and the plurality of thermal images captured by the at least one thermal imaging sensor.

4. The method of claim 3, wherein the logic controller is configured to: determine that the system message meets a reporting criteria; and in response to determining that the system message meets the reporting criteria: determining an identification of the object associated with the system message; and generating and transmitting the alert message indicating the potential issue with the object within the railroad classification yard.

5. The method of claim 1, wherein the bounding box includes a user-defined area of a field of view of the at least one thermal imaging sensor.

6. The method of claim 5, wherein the field of view of the at least one thermal imaging sensor is configured to view the at least a portion of the object.

7. The method of claim 1, wherein the object includes a wheel of a railcar, and wherein the potential issue is potential applied brake condition.

8. The method of claim 1, wherein determining that the temperature of the object has exceeded the predetermined temperature threshold comprises utilizing a neural network.

9. The method of claim 1, wherein the at least one thermal imaging sensor includes one or more of: a mobile sensor attached to a vehicle or an aircraft; and a stationary sensor disposed within the railroad classification yard.

10. The method of claim 1, wherein the object includes one or more of: a rail heater, a classification yard retarder, an air plant, air conditioning equipment, electrical switching equipment, a motor, a gearbox, hydraulic equipment, a rail switch, a rail derail, and signaling equipment.

11. A system for detecting potential issues within a railroad classification yard, the system comprising: at least one thermal imaging sensor configured to: capture a plurality of thermal images of at least a portion of an object within the railroad classification yard; determine, based on analysis of the plurality of thermal images, that a temperature within a bounding box on the plurality of thermal images has exceeded a predetermined temperature threshold assigned to the bounding box for a predetermined amount of time; and send, in response to determining that the temperature within the bounding box has exceeded the predetermined temperature threshold assigned to the bounding box for the predetermined amount of time, a system message and the plurality of thermal images; and a logic controller configured to: receive the system message and the plurality of thermal images from the at least one thermal imaging sensor; determine that the system message meets a reporting criteria; and sending, in response to determining that the system message meets the reporting criteria, an alert message to at least one of a classification yard control system or a user interface prior to a railcar associated with the object being released to free-roll within the railroad classification yard.

12. The system of claim 11, wherein sending the alert message includes: determining an identification of the object associated with the system message; generating the alert message indicating a potential issue with the object within the railroad classification yard; and transmitting the alert message to at least one of a classification yard control system or a user interface prior to a railcar associated with the object being released to free-roll within the railroad classification yard.

13. The system of claim 11, wherein the system is configured to send an automated signal to at least one component of the railroad classification yard to cause one or more physical components of the railroad classification yard to be actuated to release a brake system of the railcar associated with the object.

14. The system of claim 11, wherein the system message includes one or more of: data indicating a camera location, a timestamp, and the predetermined temperature threshold that was exceeded.

15. The system of claim 11, wherein the bounding box includes a user-defined area of a field of view of the at least one thermal imaging sensor.

16. The system of claim 15, wherein the field of view of the at least one thermal imaging sensor is configured to view the at least a portion of the object.

17. The system of claim 11, wherein the object includes a wheel of a railcar, and wherein the potential issue is potential applied brake condition.

18. The system of claim 11, wherein the at least one thermal imaging sensor includes one or more of: a mobile sensor attached to a vehicle or an aircraft; and a stationary sensor disposed within the railroad classification yard.

19. The system of claim 11, wherein the object includes one or more of: a rail heater, a classification yard retarder, an air plant, air conditioning equipment, electrical switching equipment, a motor, a gearbox, hydraulic equipment, a rail switch, a rail derail, and signaling equipment.

20. A computer-based tool for detecting potential issues within a railroad classification yard including non-transitory computer readable media having stored thereon computer code which, when executed by a processor, causes a computing device to perform operations comprising: capturing, using at least one thermal imaging sensor, a plurality of thermal images of at least a portion of an object within the railroad classification yard; determining, based on analysis of the plurality of thermal images, that a temperature within a bounding box on the plurality of thermal images has exceeded a predetermined temperature threshold assigned to the bounding box for a predetermined amount of time; generating, in response to determining that the temperature within the bounding box has exceeded the predetermined temperature threshold assigned to the bounding box for the predetermined amount of time, an alert message indicating a potential issue with the object within the railroad classification yard; and transmitting the alert message to at least one of a classification yard control system or a user interface prior to a railcar associated with the object being released to free-roll within the railroad classification yard.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0024] For a more complete understanding of the present disclosure, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:

[0025] FIG. 1 is a diagram illustrating a railcar brake detection system, according to particular embodiments.

[0026] FIGS. 2A and 3A illustrate thermal images with bounding boxes, according to particular embodiments.

[0027] FIGS. 2B and 3B illustrate non-thermal images, according to particular embodiments.

[0028] FIG. 4 illustrates a thermal imaging system message, according to particular embodiments.

[0029] FIGS. 5A and 5B are images indicating a false positive, according to particular embodiments.

[0030] FIG. 6 illustrates an embodiment of a thermal imaging system that may be used to eliminate false positives, according to particular embodiments.

[0031] FIG. 7 illustrates a chart that may be generated by the railcar brake detection system of FIG. 1, according to particular embodiments.

[0032] FIG. 8 is a chart illustrating a method for detecting applied railcar brakes within a railroad classification yard using thermal imaging, according to particular embodiments.

[0033] FIG. 9 is an example computer system that can be utilized to implement aspects of the various technologies presented herein, according to particular embodiments.

[0034] It should be understood that the drawings are not necessarily to scale and that the disclosed embodiments are sometimes illustrated diagrammatically and in partial views. In certain instances, details which are not necessary for an understanding of the disclosed methods and apparatuses or which render other details difficult to perceive may have been omitted. It should be understood, of course, that this disclosure is not limited to the particular embodiments illustrated herein.

DETAILED DESCRIPTION

[0035] The disclosure presented in the following written description and the various features and advantageous details thereof, are explained more fully with reference to the non-limiting examples included in the accompanying drawings and as detailed in the description. Descriptions of well-known components have been omitted to not unnecessarily obscure the principal features described herein. The examples used in the following description are intended to facilitate an understanding of the ways in which the disclosure can be implemented and practiced. A person of ordinary skill in the art would read this disclosure to mean that any suitable combination of the functionality or exemplary embodiments below could be combined to achieve the subject matter claimed. The disclosure includes either a representative number of species falling within the scope of the genus or structural features common to the members of the genus so that one of ordinary skill in the art can recognize the members of the genus. Accordingly, these examples should not be construed as limiting the scope of the claims.

[0036] A person of ordinary skill in the art would understand that any system claims presented herein encompass all of the elements and limitations disclosed therein, and as such, require that each system claim be viewed as a whole. Any reasonably foreseeable items functionally related to the claims are also relevant. The Examiner, after having obtained a thorough understanding of the disclosure and claims of the present application has searched the prior art as disclosed in patents and other published documents, e.g., nonpatent literature. Therefore, as evidenced by issuance of this patent, the prior art fails to disclose or teach the elements and limitations presented in the claims as enabled by the specification and drawings, such that the presented claims are patentable under the applicable laws and rules of this jurisdiction.

[0037] A typical train is composed of one or more locomotives (sometimes referred to as engines) and one or more railcars being pulled and/or pushed by the one or more engines. Trains are typically assembled in a railroad classification yard. In typical operations of a classification yard, hundreds or thousands of railcars are moved through classification tracks to route each of the railcars to a respectively assigned track, where the railcars are ultimately coupled to their assigned train based upon the train's route and final destination. Once the train is fully assembled, the train then departs the railyard and travels to its destination.

[0038] To assemble an outbound train, railcars are decoupled from incoming trains and sorted or humped to various classification tracks of a railroad classification hump yard. This may involve pulling connecting pins on railcars and allowing the railcars to free-roll on a grade due to gravity. A major efficiency factor within gravity feed classification yards is how well a free-rolling railcar moves on grade. At times railcar brakes may unknowingly be applied and can cause unexpected behavior during movement. These unanticipated behaviors range from a railcar not reaching expected coupling velocities, stopping short of coupling, or stalling within the switching area. Each one of these issues offers its own risk to operations and safety. Stopping short and reduced velocity can cause extra time for train crews to reassemble railcars. Stalls offer a higher risk of railcars colliding causing derailments, damage to transported payloads, and unnecessary hazards to working crews.

[0039] To address these and other problems with free-rolling railcars in a classification yard, the disclosed embodiments provide systems and methods for detecting applied railcar brakes within a railroad classification yard using thermal imaging. In some embodiments, the disclosed systems and methods utilize one or more thermal imaging sensors (e.g., infrared thermal cameras) to detect excessive heat in wheels of a railcar due to brakes being mistakenly or unknowingly applied to the railcar. Once detected, the disclosed systems and methods may send one or more notifications of the excessive heat to alert personnel of the potential problems. In some embodiments, for example, an alert may be displayed on a physical classification yard pin puller board in order to alert personnel of the potential problem with a particular railcar before the railcar is decoupled from adjacent railcars (e.g., its pins are pulled). This may prevent problems with the railcar (e.g., not reaching expected coupling velocities, stopping short of coupling, stalling within the switching area, etc.) during humping operations of the classification yard, thereby increasing the efficiency of operations of the classification yard.

[0040] The disclosed embodiments will now be described in reference to FIGS. 1-9. FIG. 1 is a diagram illustrating a railcar brake detection system, according to particular embodiments. FIGS. 2A and 3A illustrate thermal images with bounding boxes, according to particular embodiments. FIGS. 2B and 3B illustrate non-thermal images, according to particular embodiments. FIG. 4 illustrates a thermal imaging system message, according to particular embodiments. FIGS. 5A and 5B are images indicating a false positive, according to particular embodiments. FIG. 6 illustrates an embodiment of a thermal imaging system that may be used to eliminate false positives, according to particular embodiments. FIG. 7 illustrates a chart that may be generated by the railcar brake detection system of FIG. 1, according to particular embodiments. FIG. 8 is a chart illustrating a method for detecting applied railcar brakes within a railroad classification yard using thermal imaging, according to particular embodiments. FIG. 9 is an example computer system that can be utilized to implement aspects of the various technologies presented herein, according to particular embodiments.

[0041] It should be understood that although the disclosed embodiments have been described in reference to detecting excessive heat in wheels of railcars within a classification yard, the disclosed embodiments may be easily modified to be used in a variety of other applications to detect out-of-range temperatures. As a few examples, the disclosed systems and methods may be used for rail heater inspections, entering/leaving a yard inspections, classification yard retarder inspections, air plant inspections, electrical switching inspections, motor and gearbox inspections, and the like. This disclosure anticipates any appropriate application of the disclosed systems and methods, including applications outside the railroad environment.

[0042] FIG. 1 is a diagram illustrating a railcar brake detection system 100, according to particular embodiments. Railcar brake detection system 100 includes a computing system 110, a client system 130, a network 140, and a thermal imaging system 160. Computing system 110, client system 130, network 140, and thermal imaging system 160 are communicatively coupled together using any appropriate wired or wireless communication system or network (e.g., network 140). In some embodiments, thermal imaging system 160 is located within a railroad classification yard 120. In other embodiments, some components of thermal imaging system 160 (e.g., a thermal imaging camera 162) may be located within railroad classification yard 120 while other components of thermal imaging system 160 (e.g., a neural network 168) are located outside of thermal imaging system 160. While FIG. 1 illustrates a certain number and arrangement of computing system 110, client system 130, and thermal imaging system 160, other embodiments may have any other appropriate arrangement and number of these components.

[0043] In general, railcar brake detection system 100 detects issues with wheels (e.g., mistakenly or unknowingly applied brakes) of railcars 121 within railroad classification yard 120 in order to prevent issues with humping operations within railroad classification yard 120. To do so, thermal imaging system 160 captures, using one or more thermal imaging cameras 162, images 190 of the wheels of railcars 121 as they pass by a specific location within railroad classification yard 120. The images may be still images or videos. Thermal imaging system 160 (or computing system 110 in some embodiments) utilizes various techniques as described herein to detect temperatures within images 190 that exceed a predetermined threshold. For example, a user may draw or otherwise indicate one or more bounding boxes within a field of view of thermal imaging camera 162 (e.g., a portion of the field of view that covers some or all of the wheels of passing railcars 121) in which to detect temperatures. If any portion within a particular bounding box exceeds a predetermined threshold temperature for the particular bounding box, thermal imaging system 160 may send a thermal imaging system message 165 to computing system 110 to indicate that the predetermined threshold temperature was exceeded. Once thermal imaging system message 165 is received by computing system 110, computing system 110 may further process/analyze the thermal imaging system message 165 and then take actions to notify users and systems within computing system 110 of the potential fault with the wheels of railcar 121. As one example, computing system 110 may send a notification 170 to client system 130 to indicate that the railcar 121 associated with the excessive heat has a potential issue with the wheels of the railcar 121 (e.g., the brakes may be applied to the wheels of the railcar 121). As another example, computing system 110 may send alert instructions 180 to a classification yard pin puller board 125 within railroad classification yard 120 to visually display a wheel alert 181 to personnel responsible for pulling pins on railcars 121 during humping operations. As a result, problems with railcars 121 (e.g., not reaching expected coupling velocities, stopping short of coupling, stalling within the switching area, etc.) during humping operations of the classification yard may be avoided or reduced, thereby increasing the efficiency of operations of the classification yard.

[0044] Computing system 110 may be any appropriate computing system in any suitable physical form. As example and not by way of limitation, computing system 110 may be an embedded computer system, a system-on-chip (SOC), a single-board computer system (SBC) (such as, for example, a computer-on-module (COM) or system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, a mainframe, a mesh of computer systems, a mobile telephone, a personal digital assistant (PDA), a server, a tablet computer system, an augmented/virtual reality device, or a combination of two or more of these. Where appropriate, computing system 110 may include one or more computer systems; be unitary or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks. Where appropriate, computing system 110 may perform without substantial spatial or temporal limitation one or more steps of one or more methods described or illustrated herein. As an example, and not by way of limitation, computing system 110 may perform in real time or in batch mode one or more steps of one or more methods described or illustrated herein. Computing system 110 may perform at different times or at different locations one or more steps of one or more methods described or illustrated herein, where appropriate. A particular example of a computing system 110 is described in reference to FIG. 9.

[0045] Computing system 110 includes one or more memory units/devices 115 (collectively herein, memory 115) that may store railcar brake detection module 150 and images 190 from thermal imaging system 160. Railcar brake detection module 150 may be a software module/application utilized by computing system 110 to send notifications 170 and alert instructions 180 regarding potential wheel problems with railcars 121, as described herein. Railcar brake detection module 150 represents any suitable set of instructions, logic, or code embodied in a computer-readable storage medium. For example, railcar brake detection module 150 may be embodied in memory 115, a disk, a CD, or a flash drive. In particular embodiments, railcar brake detection module 150 may include instructions (e.g., a software application) executable by a computer processor to perform some or all of the functions described herein.

[0046] In embodiments, the information related to detection of issues with wheels of railcars 121 within the classification yard 120 may be sent to a terminal process control (TPC) system 198. TPC system 198 may use the information to display the thermal data, the wheel issue detection information, generated alerts, generate automatic signals to railcars and/or other components of the classification yard to release the railcar brakes, and/or may provide an interface where users may take remedial action such as generate an automatic signal to release the railcar brakes, etc.

[0047] TPC system 198 may be configured for managing operations of classification yard 120. In embodiments, TPC system 198 may be configured to facilitate operations to plan, control, track, and report the movement of cuts (e.g., train cars) from a hump rolling stock to a destination train. In embodiments, TPC system 198 may represent an integrated, standardized, configurable, reliable, and efficient control system that may utilize a sustainable and maintainable software/hardware design to support operations of classification yard 120.

[0048] In embodiments, TPC system 198 may provide functionality to create a hump list, and to classify each train car in the hump list in accordance with the configuration of the hump list. The hump list may represent a listing of the train cars that are to be classified by classification yard 120 (e.g., the train cars to be routed to their assigned destination train) and may include an indication of the assigned destination track/train of each train car. In embodiments, TPC system 198 may create the hump list by communicating with external systems (e.g., a transportation support system (TSS), etc.) and obtaining train car data associated with the train cars in the rolling stock train to be classified. The hump list may include an indication or identification of each cut (e.g., each group of one or more cars to be included in each cut) that is to be cut from the rolling stock train at the hump.

[0049] In embodiments, as each cut enters the hump, each cut is separated from the rolling stock (e.g., using gravity) and allowed to coast downhill through the hump yard, and each cut may be routed to its assigned destination track/train to be coupled at the appropriate coupling speed. In embodiments, the release speed, the coupling speed, and/or the speed of each cut through the marshalling tracks of classification yard 120 may be determined by the operations of TPC system 198 to control the push engine speed, to control operations of retarders to remove energy from the cuts as the cuts move through the marshalling tracks of classification yard 120, to predict arrival times and/or speeds of the cuts at various devices and/or segments of classification yard 120, etc.

[0050] In a particular embodiment, a machine learning model (e.g., a neural network deployed on a thermal camera or an edge node) may analyzes thermal imagery and, when an issue with wheels (e.g., a misapplied brake condition) is detected, may publishes an event on a message bus. In some embodiments, the event may include a message queuing telemetry transport (MQTT) topic (with a payload (e.g., a JavaScript Object Notation (JSON) payload) distributed by a broker (e.g., an ActiveMQ (AMQ)-compatible broker), although any suitable publish/subscribe mechanism may be used. In embodiments, the payload may include one or more of a camera identifier and location indicator, a timestamp, an event type (e.g., hot-wheel), one or more temperature statistics for wheel and rail regions (e.g., max/average, time-above-threshold), a confidence score, a reference to the associated frame or clip, etc.

[0051] In some embodiments, a programmable logic controller (PLC) may subscribe to the topic and may perform event gating and correlation prior to notifying systems associated with classification yard 120. For example, the PLC may require that a hot-wheel event be detected on opposing track-side cameras within a configurable coincidence window before the event is considered valid. Upon validation, the PLC may send a hot wheel event notification to a system (e.g., TPC system 198) for remediation. Concurrently, a supervisory control and data acquisition system or HMI server may subscribe to the topic to store the payloads and associated imagery for long-term analytics and trending. In embodiments, this may enable historical tuning of thresholds, operator training, generation of metrics such as time-of-day distributions of thermal events, etc.

[0052] Classification yard 120 is a collection of connected railroad tracks 123 for storing and sorting railcars 121. In some embodiments, classification yard 120 is a hump yard that is designed to classify railcars 121 into common train blocks. Classification yard 120 may be composed of various sub-yards that work together to facilitate the classification of railcars 121 into common train blocks on classification tracks 123. For example, classification yard 120 may include a receiving yard, a hump, a bowl, multiple pull leads, and a departure yard. The receiving yard is a storage location for inbound trains and serves as a buffer for downstream processes. Inbound trains that need classification are broken up and prepared for sorting in the receiving yard. The hump works in concert with a series of automated switches and retarders (e.g., hump yard switching equipment 126) to allow gravity to direct railcars 121 to their desired locations in the bowl. The bowl includes multiple classification railroad track 123. Each classification railroad track 123 typically holds railcars 121 assigned to a single specific train block. The bowl helps sort railcars 121 into different classification railroad tracks 123 based on their destination and acts as a holding location to allow time for the aggregation of block volume. Pull leads are the track connections between the bowl and the departure yard. Yard crews will typically pull multiple classification railroad tracks 123 from the bowl to build an outbound train and then move the outbound train to the departure yard. The pull leads are where these railcars 121 are first combined to construct the outbound train. The departure yard acts as a staging location for an outbound train prior to departure from the terminal.

[0053] Classification yard pin puller board 125 is a physical display within railroad classification yard 120 that provides visual instructions to personnel regarding which railcars 121 are to be decoupled (e.g., their pins pulled) from adjoining railcars 121. In some embodiments, classification yard pin puller board 125 displays an identification (e.g., 121A, 121B, etc.) for each railcar 121 that personnel are to decouple from adjoining railcars 121. In some embodiments, classification yard pin puller board 125 is any appropriate electronic display (e.g., an LED display). In operation, computing system 110 sends alert instructions 180 to classification yard pin puller board 125 when it is determined (as described herein) that a particular railcar 121 has a potential issue with at least one of its wheels. In response, classification yard pin puller board 125 displays a wheel alert 181 adjacent to the particular railcar 121 with a potential wheel issue. For example, as illustrated in FIG. 1, wheel alerts 181 are displayed adjacent to railcars 121A and 121G in response to computing system 110 determining that railcars 121A and 121G have potential wheel issues, as described herein.

[0054] Hump yard switching equipment 126 includes equipment or devices within classification yard 120 that direct railcars 121 to specific classification railroad tracks 123 within railroad classification yard 120. In some embodiments, hump yard switching equipment 126 includes automatic track switches and retarders that operate to switch railcars 121 onto specific classification railroad tracks 123. In some embodiments, computing system 110 is electronically coupled to hump yard switching equipment 126 using any wired or wireless technology via network 140. In some embodiments, computing system 110 sends alert instructions 180 to hump yard switching equipment 126 in order to automatically control movements of railcar 121 on railroad track 123. For example, if computing system 110 detects an issue with the wheels of a particular railcar 121, computing system 110 may send alert instructions 180 to hump yard switching equipment 126 in order to direct the particular railcar 121 to a particular railroad track 123 for maintenance.

[0055] Client system 130 is any appropriate user device for communicating with components of railcar brake detection system 100 over network 140 (e.g., the internet). In particular embodiments, client system 130 may be an electronic device including hardware, software, or embedded logic components or a combination of two or more such components and capable of carrying out the appropriate functionalities implemented or supported by client system 130. As an example, and not by way of limitation, a client system 130 may include a computer system (e.g., computer system 1200) such as a desktop computer, notebook or laptop computer, netbook, a tablet computer, e-book reader, GPS device, camera, personal digital assistant (PDA), handheld electronic device, cellular telephone, smartphone, smartwatch, augmented/virtual reality device such as wearable computer glasses, other suitable electronic device, or any suitable combination thereof. This disclosure contemplates any suitable client system 130. A client system 130 may enable a network user at client system 130 to access network 140. A client system 130 may enable a user to communicate with other users at other client systems 130. Client system 130 may include an electronic display that displays graphical user interface 132, a processor such processor 902, and memory such as memory 904.

[0056] Network 140 allows communication between and amongst the various components of railcar brake detection system 100. This disclosure contemplates network 140 being any suitable network operable to facilitate communication between the components of railcar brake detection system 100. Network 140 may include any interconnecting system capable of transmitting audio, video, signals, data, messages, or any combination of the preceding. Network 140 may include all or a portion of a local area network (LAN), a wide area network (WAN), an overlay network, a software-defined network (SDN), a virtual private network (VPN), a packet data network (e.g., the Internet), a mobile telephone network (e.g., cellular networks, such as 4G or 5G), a Plain Old Telephone (POT) network, a wireless data network (e.g., WiFi, WiGig, WiMax, etc.), a Long Term Evolution (LTE) network, a Universal Mobile Telecommunications System (UMTS) network, a peer-to-peer (P2P) network, a Bluetooth network, a Near Field Communication network, a Zigbee network, and/or any other suitable network.

[0057] Railcar brake detection module 150 may be a software module/application utilized by computing system 110 to detect wheel problems with railcars 121 and then send notifications 170 and alert instructions 180 regarding the detected wheel problems with railcars 121. In some embodiments, railcar brake detection module 150 receives or otherwise accesses thermal imaging system messages 165 generated by thermal imaging system 160. Railcar brake detection module 150 analyzes thermal imaging system messages 165 and determines (e.g., using logical/conditional reporting criteria) whether or not the particular thermal imaging system message 165 includes a railcar wheel issue that needs to be reported to users and other systems of railcar brake detection system 100. As a specific example, a particular thermal imaging system message 165 may indicate that a wheel of railcar 121A exceeded a threshold temperature of 135 F. of a particular bounding box for greater than its configured time threshold (e.g., one second). Railcar brake detection module 150 may compare the 135 F. temperature of the particular thermal imaging system message 165 to a configured reporting threshold temperature (e.g., 130 F.) and determine that the detected wheel temperature exceeded the reporting threshold temperature. As a result, computing system 110 may send notification 170 and alert instructions 180 to alert users and other systems of the excessive wheel temperature of railcar 121A.

[0058] In some embodiments, the reporting threshold temperature (e.g., reporting criteria) used by railcar brake detection module 150 to determine if a thermal imaging system message 165 should be reported to users and system of railcar brake detection system 100 may be different for each railroad classification yard 120. For example, the reporting threshold temperature may vary based on the geographical location of railroad classification yard 120 (e.g., a particular railroad classification yard 120 may have a lower reporting threshold temperature due to a higher ambient temperature.) In some embodiments, the reporting threshold temperature may vary based on the time of the year or the date (e.g., lower threshold in Winter as opposed to a higher threshold in Summer). In some embodiments, the reporting threshold temperature may be normalized and may vary daily or hourly (or more frequently) based on real-time ambient conditions (e.g., lower threshold in cooler ambient conditions as opposed to a higher threshold in warmer ambient conditions). The current ambient conditions may be obtained via any available temperature reporting service, a temperature sensor located in thermal imaging system 160, or via bounding box 210B (e.g., the ambient temperature of railroad track 123 as determined by thermal imaging camera 162).

[0059] In some embodiments, thermal imaging system message 165 indicates that a particular temperature threshold has been exceeded at a particular location and time but does not indicate an identification of the associated railcar 121. In these situations, railcar brake detection module 150 may utilize methods to determine an identification of the particular railcar 121 corresponding to the excessive temperature reported in the thermal imaging system message 165. For example, railcar brake detection module 150 may access a database or other system within railcar brake detection system 100 that provides data about the locations/times of each railcar 121 within railroad classification yard 120 and then cross-reference this data with the data contained in thermal imaging system message 165 (e.g., timestamps and locations). As another example, railcar brake detection module 150 may analyze images 190 to find one or more corresponding non-thermal images 190 to thermal imaging system message 165. For example, railcar brake detection module 150 may find one or more non-thermal images 190 with corresponding time stamps and locations to thermal imaging system message 165. Railcar brake detection module 150 may then utilize any appropriate image processing technique (e.g., OCR) to locate and determine a visible railcar identification within the one or more non-thermal images 190. As a specific example, railcar brake detection module 150 may utilize OCR on image 190D in FIG. 3B to determine an identification of WCRC 3298 for railcar 121.

[0060] Thermal imaging system 160 is any appropriate system (including any number of sensors) for capturing images 190 and generating/sending thermal imaging system messages 165. In some embodiments, thermal imaging system 160 is a stationary module located alongside railroad track 123. In other embodiments, thermal imaging system 160 may be mobile (e.g., located on a vehicle or aircraft such as a drone). In some embodiments, thermal imaging system 160 includes a thermal imaging camera 162 for capturing thermal images 190 of railcars 121 (e.g., as they pass by a stationary thermal imaging camera 162). In general, thermal imaging camera 162 is a ruggedized thermal sensor that is able to function outdoors in harsh environments. Thermal images 190 may indicate temperatures of each pixel or region of images 190. Specific examples of thermal images 190 are illustrated in FIGS. 2A and 3A. In some embodiments, thermal images 190 are stored in computing system 110 for analysis and long-term storage. In some embodiments, each thermal image 190 includes a timestamp of when the thermal image was captured along with a geographic location (e.g., GPS coordinates or location name).

[0061] In some embodiments, thermal imaging system 160 includes a regular day/night camera 163 for capturing non-thermal still images 190 or movies of railcars 121 as they pass thermal imaging system 160. Specific examples of non-thermal images 190 captured by regular day/night camera 163 are illustrated in FIGS. 2B and 3B. In some embodiments, non-thermal images 190 are stored in computing system 110 for analysis (e.g., to determine railcar numbers or identifications using OCR) and long-term storage. In some embodiments, each non-thermal image 190 includes a timestamp of when the non-thermal image 190 was captured along with a geographic location (e.g., GPS coordinates or location name such as classification track 123A).

[0062] In some embodiments, a user or other system may interact with thermal imaging system 160 in order to draw or otherwise indicate bounding boxes 210 within a field of view of thermal imaging camera 162. For example, a user may use a web browser on client system 130 to draw or otherwise indicate a first bounding box 210A that includes some or all of wheels 220 of railcar 121. As another example, a user may use client system 130 to draw or otherwise indicate a second bounding box 210B that includes some or all of railroad track 123. In some embodiments, each bounding box 210 has a user-configurable temperature setpoint. For example, bounding box 210A may have a temperature setpoint of 135 F. If a temperature is detected by thermal imaging system 160 within bounding box 210A that meets or exceeds the temperature setpoint of 135 F. for a user-configurable amount of time (e.g., one second), thermal imaging system 160 may generate a thermal imaging system message 165 that indicates the excessive temperature.

[0063] FIG. 2A illustrates an example of a thermal image 190A that does not generate a thermal imaging system message 165 (e.g., no excessive temperature is detected within bounding boxes 210). In this example, the maximum temperature detected within bounding box 210A is indicated to be 61.4 F. Assuming that the temperature setpoint for bounding box 210A is set to 135 F., thermal imaging system 160 determines that the temperature setpoint has not been reached and thus does not generate a thermal imaging system message 165.

[0064] FIG. 3A illustrates an example of a thermal image 190C that does generate a thermal imaging system message 165 (e.g., an excessive wheel temperature is detected within bounding box 210A). In this example, the maximum temperature detected within bounding box 210A is indicated to be 255.0 F. Assuming that the temperature setpoint for bounding box 210A is set to 135 F., thermal imaging system 160 determines that the temperature setpoint has been exceeded and thus generates a thermal imaging system message 165.

[0065] In some embodiments, multiple overlapping bounding boxes 210 may be used by thermal imaging system 160, but each may have a different temperature setpoint. For example, multiple bounding boxes 210A with identical shapes and locations but with different temperature setpoints may be stacked on top of each other in order to capture temperatures of wheels 220. For example, a first bounding box 210A may have a temperature setpoint of 135 F., a second bounding box 210A may have a temperature setpoint of 140 F., a third bounding box 210A may have a temperature setpoint of 145 F., and so on. In this way, computing system 110 may determine a temperature range of wheel 220. For example, if a first bounding box 210A with a 135 F. temperature setpoint is triggered while a second bounding box 210A with a 140 F. temperature setpoint is not triggered, computing system 110 may determine that the temperature of wheel 220 is between 135 F. and 140 F.

[0066] Thermal imaging system 160 generates and sends thermal imaging system messages 165 to computing system 110 when certain conditions are met. FIG. 4 illustrates an example thermal imaging system message 165, according to certain embodiments. In this example, thermal imaging system message 165 includes a location 410, a camera type 420, an event type 430, and a temperature setpoint 440. Location 410 indicates the physical location of thermal imaging camera 162 and may include GPS coordinates or a description of the location (e.g., in classification yard 120 on north side of hump lead). Camera type 420 indicates a type of camera used (e.g., thermal imaging camera 162 or day/night camera 163). Event type 430 indicates a type of event that caused the trigger (e.g., LowAxleMax, LowAxleAvg, AxleMax, etc.). Temperature setpoint 440 is the user-configurable temperature threshold of bounding box 210 that triggers the generation of thermal imaging system message 165.

[0067] Notification 170 is any appropriate alert or message that is sent to another device (e.g., client system 130) by computing system 110 when it is determined that an event of a specific thermal imaging system message 165 needs to be reported. In some embodiments, notification 170 includes an identification of a railcar 121 with potential wheel problems (e.g., excessive heat was detected in wheel 220 by thermal imaging system 160 and then reported via thermal imaging system message 165). In some embodiments, notification 170 includes one or more related images 190 of the identified railcar 121. In some embodiments, notification 170 is displayed on client system 130. As a specific example, notification 170 may be an email message or text message that is sent to a maintenance crew that instructs the maintenance crew to inspect and perform maintenance on the identified railcar 121 with potential wheel issues. In some embodiments, notification 170 may be sent to TPC system 198.

[0068] Alert instructions 180 are any appropriate signals or messages sent from computing system 110 to classification yard pin puller board 125 in order to instruct classification yard pin puller board 125 to display wheel alerts 181. In some embodiments, alert instructions 180 include identifications of at least one railcar 121 that needs a wheel alert 181 displayed on classification yard pin puller board 125. For example, if computing system 110 determines that railcar 121A has an excessive wheel temperature as described here, alert instructions 180 may include an identification of railcar 121A.

[0069] In operation, railcar brake detection system 100 detects issues with wheels 220 (e.g., mistakenly or unknowingly applied brakes) of railcars 121 within railroad classification yard 120 in order to prevent issues with humping operations withing railroad classification yard 120. To do so, thermal imaging system 160 captures, using one or more thermal imaging cameras 162, thermal images 190 of wheels 220 of railcars 121 (e.g., as they pass by a specific location within railroad classification yard 120). Images 190 may be still images or videos. Thermal imaging system 160 (or computing system 110 in some embodiments) utilizes various techniques as described herein to detect temperatures within images 190 that exceed a predetermined threshold. For example, a user may draw or otherwise indicate one or more bounding boxes 210 within a field of view of thermal imaging camera 162 (e.g., a portion of the field of view that covers some or all of wheels 220 of passing railcars 121) in which to detect temperatures. If any portion within a particular bounding box 210 exceeds a predetermined threshold temperature for the particular bounding box for a user-defined period of time (e.g., one second), thermal imaging system 160 may send a thermal imaging system message 165 to computing system 110 to indicate that the predetermined threshold temperature was exceeded.

[0070] Once thermal imaging system message 165 is received by computing system 110, computing system 110 may further process/analyze the thermal imaging system message 165 and then take actions to notify users and systems within computing system 110 of the potential fault with wheels 220 of railcar 121. As one example, computing system 110 may send a notification 170 to client system 130 (or TPC system 198 in some embodiments) to indicate that the railcar 121 associated with the excessive heat has a potential issue with wheels 220 of the railcar 121 (e.g., the brakes may be applied to wheels 220 of the railcar 121) if a reporting criteria (e.g., a threshold temperature) is met. As another example, computing system 110 may send alert instructions 180 to classification yard pin puller board 125 within railroad classification yard 120 to visually display a wheel alert 181 to personnel responsible for pulling pins on railcars 121 during humping operations if a reporting criteria (e.g., a threshold temperature) is met. As a result, problems with railcars 121 (e.g., not reaching expected coupling velocities, stopping short of coupling, stalling within the switching area, etc.) during humping operations of the classification yard may be avoided or reduced, thereby increasing the efficiency of operations of the classification yard.

[0071] In some situations, thermal imaging system 160 may detect false-positives (e.g., false temperature readings that falsely trigger the temperature threshold of a bounding box 210). For example, when the sun and weather conditions are in a particular configuration, reflections and refraction can occur along the top of railroad track 123 (as illustrated in FIG. 5A) that trigger a temperature threshold (as illustrated in FIG. 5B). To address these issues, some embodiments employ two thermal imaging cameras 162 (e.g., 162A-B) on opposite sides of railroad track 123 from each other as illustrated in FIG. 6. To filter out false-positives, thermal imaging system 160 may only send thermal imaging system message 165 if both thermal imaging cameras 162, within a common time period, detect a wheel temperature above a temperature threshold of a bounding box 210 for a set amount of time. If all of these conditions are met, thermal imaging system 160 may determine that wheel 220 is excessively hot and therefore send thermal imaging system message 165 to computing system 110 to indicate the possible issue with wheel 220.

[0072] Rather than employing two separate thermal imaging cameras 162 as described above, alternate embodiments of thermal imaging system 160 may address false positives by employing alternate software-related techniques. As a first example, thermal imaging system 160 may provide within thermal imaging system message 165 a maximum temperature detected with a particular bounding box 210. As a second example, thermal imaging system 160 may provide dynamic temperatures within thermal imaging system message 165. As a third example, thermal imaging system 160 may provide within thermal imaging system message 165 an average temperature detected in a bounding box 210.

[0073] In some embodiments, computing system 110 may store thermal imaging system messages 165 and images 190 to use for generating and displaying metrics regarding thermal wheel issues. For example, FIG. 7 illustrates a chart 700 that may be generated by computing system 110 using stored thermal imaging system messages 165 from thermal imaging system 160. In this example, chart 700 visually provides a breakdown of all thermal events (as indicated by thermal imaging system messages 165 from thermal imaging system 160) for a particular day. For each hour of the particular day (x-axis), chart 700 shows the number of thermal events (y-axis) received via thermal imaging system messages 165. For example, chart 700 indicates that the most thermal events occurred at 0800 hours (30 events). This may enable managers to determine where training resources are needed and when to deploy maintenance crews.

[0074] In some embodiments, railcar brake detection system 100 may utilize machine learning and a neural network 168 to analyze thermal images 190 and determine potential wheel/brake issues with railcar 121. Neural network 168 may be included within thermal imaging system 160, computing system 110, or any other computing system in computing system 110. In one embodiment, inferences may be directly run on thermal imaging camera 162 using SDK. In another embodiment, an on-network computing system such as computing system 110 may run a video inference by connecting to an RSTP stream of thermal imaging camera 162. In either case, a data collection of all uses where excessive heat in wheels 220 could be detected (e.g., all weather conditions, all times of the day, different types of railcar 121, etc.). There may also be example of non-excessive heat wheels or unwanted wheels such as locomotive wheels. In addition, other unwanted hot objects may be including in the collection (e.g., locomotive exhaust stacks, maintenance vehicles, reflections, and the like).

[0075] Once the data collection for training is complete, a labeled dataset of each image may be reviewed and labeled based on the detection method used. Once the dataset is ready, training, validation, and measurements are conducted on the neural network to evaluate its performance. Once completed, neural network 168 can be packaged and deployed in production.

[0076] In some embodiments, railcar brake detection system 100 may utilize computer vision (CV) to first programmatically identify the shape of the railcar wheels of the railcar 121 in the image stream. Once the wheel shape is identified, railcar brake detection system 100 may analyze the temperatures specifically within the shape of the wheel and may calculate a differential temperature by comparing the wheel temperature to the ambient temperature of the rail or ground.

[0077] FIG. 8 is a chart illustrating a method 800 for detecting applied railcar brakes within a railroad classification yard using thermal imaging, according to particular embodiments. In some embodiments, method 800 may be performed by one or more components of railcar brake detection system 100 (e.g., railcar brake detection module 150). At step 810, method 800 captures a plurality of thermal images using a thermal imaging sensor. In some embodiments, the thermal images are images 190. In some embodiments, the thermal imaging sensor is thermal imaging camera 162.

[0078] At step 820, method 800 determines that a temperature within a bounding box has exceeded a predetermined temperature threshold assigned to the bounding box for a predetermined amount of time. In some embodiments, the bounding box is a user-defined area of a field of view of the thermal imaging sensor (e.g., bounding box 210). In some embodiments, the field of view is configured to view at least a portion of railcar wheels such as wheels 220.

[0079] At step 830, method 800 sends, using a communications network, a system message in response to determining that the temperature within the bounding box has exceeded the predetermined temperature threshold assigned to the bounding box for the predetermined amount of time. In some embodiments, the system message is thermal imaging system message 165 that is sent by thermal imaging system 160. In some embodiments, the system message includes data indicating a camera location, a timestamp, and the predetermined temperature threshold that was exceeded.

[0080] At step 840, method 800 sends, using the communications network, the plurality of thermal images captured by the thermal imaging sensor. At step 850, method 800 receives the system message sent in step 830 and determines whether the system message meets a reporting criteria. In some embodiments, the reporting criteria is a specific temperature threshold. In some embodiments, the temperature threshold is customizable for a particular location of the classification yard. In some embodiments, the temperature threshold is dynamic and may be based on current weather conditions, seasons, time of day/year, etc. If it is determined in step 850 that the system message meets the reporting criteria, method 800 proceeds to step 860. If it is determined in step 850 that the system message does not meet the reporting criteria, method 800 may end.

[0081] At step 860, method 800 determines an identification of a particular railcar associated with the system message. In some embodiments, step 860 includes accessing a database or other system that provides data about the locations/times of each railcar within the railroad classification yard and then cross-referencing this data with the data contained in system message of step 830 (e.g., timestamps and locations). In some embodiments, step 860 includes analyzing day/night (e.g., non-thermal) images to find one or more corresponding non-thermal images to the system message of step 830. For example, method 800 may find one or more non-thermal images with corresponding time stamps and locations to the system message of step 830. Method 800 may then utilize any appropriate image processing technique (e.g., OCR) to locate and determine a visible railcar identification within the one or more non-thermal images.

[0082] At step 870, method 800 sends a notification indicating the particular railcar may have a wheel issue. In some embodiments, the notification is notification 170 and may include an identification of the particular railcar and an image of the wheels of the railcar that have potential issues due to excessive heat. In some embodiments, the notification is sent to and displayed on a client system such as client system 130.

[0083] At step 880, method 800 sends one or more instructions to a classification yard pin puller board. The one or more instructions sent to the classification yard pin puller board instruct the classification yard pin puller board to display an alert beside the identification of a particular railcar associated with the system message of step 830. In some embodiments, the one or more instructions are alert instructions 180. In some embodiments, the classification yard pin puller board is classification yard pin puller board 125. In some embodiments, the alert is wheel alert 181. After step 880, method 800 may end.

[0084] Particular embodiments may repeat one or more steps of the method of FIG. 8, where appropriate. Although this disclosure describes and illustrates particular steps of the method of FIG. 8 as occurring in a particular order, this disclosure contemplates any suitable steps of the method of FIG. 8 occurring in any suitable order. Moreover, although this disclosure describes and illustrates an example method for detecting applied railcar brakes within a railroad classification yard using thermal imaging including the particular steps of the method of FIG. 8, this disclosure contemplates any suitable method for detecting applied railcar brakes within a railroad classification yard using thermal imaging including any suitable steps, which may include all, some, or none of the steps of the method of FIG. 8, where appropriate. Furthermore, although this disclosure describes and illustrates particular components, devices, or systems carrying out particular steps of the method of FIG. 8, this disclosure contemplates any suitable combination of any suitable components, devices, or systems carrying out any suitable steps of the method of FIG. 8.

[0085] FIG. 9 illustrates an example computer system 900 that can be utilized to implement aspects of the various methods and systems presented herein, according to particular embodiments. In particular embodiments, one or more computer systems 900 perform one or more steps of one or more methods described or illustrated herein. In particular embodiments, one or more computer systems 900 provide functionality described or illustrated herein. In particular embodiments, software running on one or more computer systems 900 performs one or more steps of one or more methods described or illustrated herein or provides functionality described or illustrated herein. Particular embodiments include one or more portions of one or more computer systems 900. Herein, reference to a computer system may encompass a computing device, and vice versa, where appropriate. Moreover, reference to a computer system may encompass one or more computer systems, where appropriate.

[0086] This disclosure contemplates any suitable number of computer systems 900. This disclosure contemplates computer system 900 taking any suitable physical form. As example and not by way of limitation, computer system 900 may be an embedded computer system, a system-on-chip (SOC), a single-board computer system (SBC) (such as, for example, a computer-on-module (COM) or system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile telephone, a personal digital assistant (PDA), a server, a tablet computer system, an augmented/virtual reality device, or a combination of two or more of these. Where appropriate, computer system 900 may include one or more computer systems 900; be unitary or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks. Where appropriate, one or more computer systems 900 may perform without substantial spatial or temporal limitation one or more steps of one or more methods described or illustrated herein. As an example, and not by way of limitation, one or more computer systems 900 may perform in real time or in batch mode one or more steps of one or more methods described or illustrated herein. One or more computer systems 900 may perform at different times or at different locations one or more steps of one or more methods described or illustrated herein, where appropriate.

[0087] In particular embodiments, computer system 900 includes a processor 902, memory 904, storage 906, an input/output (I/O) interface 908, a communication interface 910, and a bus 912. Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement.

[0088] In particular embodiments, processor 902 includes hardware for executing instructions, such as those making up a computer program. As an example, and not by way of limitation, to execute instructions, processor 902 may retrieve (or fetch) the instructions from an internal register, an internal cache, memory 904, or storage 906; decode and execute them; and then write one or more results to an internal register, an internal cache, memory 904, or storage 906. In particular embodiments, processor 902 may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates processor 902 including any suitable number of any suitable internal caches, where appropriate. As an example, and not by way of limitation, processor 902 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in the instruction caches may be copies of instructions in memory 904 or storage 906, and the instruction caches may speed up retrieval of those instructions by processor 902. Data in the data caches may be copies of data in memory 904 or storage 906 for instructions executing at processor 902 to operate on; the results of previous instructions executed at processor 902 for access by subsequent instructions executing at processor 902 or for writing to memory 904 or storage 906; or other suitable data. The data caches may speed up read or write operations by processor 902. The TLBs may speed up virtual-address translation for processor 902. In particular embodiments, processor 902 may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates processor 902 including any suitable number of any suitable internal registers, where appropriate. Where appropriate, processor 902 may include one or more arithmetic logic units (ALUs); be a multi-core processor; or include one or more processors 902. Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.

[0089] In particular embodiments, memory 904 includes main memory for storing instructions for processor 902 to execute or data for processor 902 to operate on. As an example, and not by way of limitation, computer system 900 may load instructions from storage 906 or another source (such as, for example, another computer system 900) to memory 904. Processor 902 may then load the instructions from memory 904 to an internal register or internal cache. To execute the instructions, processor 902 may retrieve the instructions from the internal register or internal cache and decode them. During or after execution of the instructions, processor 902 may write one or more results (which may be intermediate or final results) to the internal register or internal cache. Processor 902 may then write one or more of those results to memory 904. In particular embodiments, processor 902 executes only instructions in one or more internal registers or internal caches or in memory 904 (as opposed to storage 906 or elsewhere) and operates only on data in one or more internal registers or internal caches or in memory 904 (as opposed to storage 906 or elsewhere). One or more memory buses (which may each include an address bus and a data bus) may couple processor 902 to memory 904. Bus 912 may include one or more memory buses, as described below. In particular embodiments, one or more memory management units (MMUs) reside between processor 902 and memory 904 and facilitate accesses to memory 904 requested by processor 902. In particular embodiments, memory 904 includes random access memory (RAM). This RAM may be volatile memory, where appropriate. Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Moreover, where appropriate, this RAM may be single-ported or multi-ported RAM. This disclosure contemplates any suitable RAM. Memory 904 may include one or more memories 904, where appropriate. Although this disclosure describes and illustrates particular memory, this disclosure contemplates any suitable memory.

[0090] In particular embodiments, storage 906 includes mass storage for data or instructions. As an example, and not by way of limitation, storage 906 may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. Storage 906 may include removable or non-removable (or fixed) media, where appropriate. Storage 906 may be internal or external to computer system 900, where appropriate. In particular embodiments, storage 906 is non-volatile, solid-state memory. In particular embodiments, storage 906 includes read-only memory (ROM). Where appropriate, this ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or a combination of two or more of these. This disclosure contemplates mass storage 906 taking any suitable physical form. Storage 906 may include one or more storage control units facilitating communication between processor 902 and storage 906, where appropriate. Where appropriate, storage 906 may include one or more storages 906. Although this disclosure describes and illustrates particular storage, this disclosure contemplates any suitable storage.

[0091] In particular embodiments, I/O interface 908 includes hardware, software, or both, providing one or more interfaces for communication between computer system 900 and one or more I/O devices. Computer system 900 may include one or more of these I/O devices, where appropriate. One or more of these I/O devices may enable communication between a person and computer system 900. As an example, and not by way of limitation, an I/O device may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet, touch screen, trackball, video camera, another suitable I/O device or a combination of two or more of these. An I/O device may include one or more sensors. This disclosure contemplates any suitable I/O devices and any suitable I/O interfaces 908 for them. Where appropriate, I/O interface 908 may include one or more device or software drivers enabling processor 902 to drive one or more of these I/O devices. I/O interface 908 may include one or more I/O interfaces 908, where appropriate. Although this disclosure describes and illustrates a particular I/O interface, this disclosure contemplates any suitable I/O interface.

[0092] In particular embodiments, communication interface 910 includes hardware, software, or both providing one or more interfaces for communication (such as, for example, packet-based communication) between computer system 900 and one or more other computer systems 900 or one or more networks. As an example, and not by way of limitation, communication interface 910 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wire-based network or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a WI-FI network. This disclosure contemplates any suitable network and any suitable communication interface 910 for it. As an example, and not by way of limitation, computer system 900 may communicate with an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or one or more portions of the Internet or a combination of two or more of these. One or more portions of one or more of these networks may be wired or wireless. As an example, computer system 900 may communicate with a wireless PAN (WPAN) (such as, for example, a BLUETOOTH WPAN), a WI-FI network, a WI-MAX network, a cellular telephone network (such as, for example, a Global System for Mobile Communications (GSM) network, a Long-Term Evolution (LTE) network, or a 5G network), or other suitable wireless network or a combination of two or more of these. Computer system 900 may include any suitable communication interface 910 for any of these networks, where appropriate. Communication interface 910 may include one or more communication interfaces 910, where appropriate. Although this disclosure describes and illustrates a particular communication interface, this disclosure contemplates any suitable communication interface.

[0093] In particular embodiments, bus 912 includes hardware, software, or both coupling components of computer system 900 to each other. As an example and not by way of limitation, bus 912 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a front-side bus (FSB), a HYPERTRANSPORT (HT) interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBAND interconnect, a low-pin-count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these. Bus 912 may include one or more buses 912, where appropriate. Although this disclosure describes and illustrates a particular bus, this disclosure contemplates any suitable bus or interconnect.

[0094] Although the present disclosure and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the disclosure as defined by the appended claims. Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the disclosure of the present disclosure, processes, machines, manufacture, compositions of matter, means, methods, or steps, presently existing or later to be developed that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein may be utilized according to the present disclosure. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or steps.

[0095] Moreover, the description in this patent document should not be read as implying that any particular element, step, or function can be an essential or critical element that must be included in the claim scope. Also, none of the claims can be intended to invoke 35 U.S.C. 112(f) with respect to any of the appended claims or claim elements unless the exact words means for or step for are explicitly used in the particular claim, followed by a participle phrase identifying a function. Use of terms such as (but not limited to) member, module, device, unit, component, element, mechanism, apparatus, machine, system, processor, processing device, or controller within a claim can be understood and intended to refer to structures known to those skilled in the relevant art, as further modified or enhanced by the features of the claims themselves, and can be not intended to invoke 35 U.S.C. 112(f). Even under the broadest reasonable interpretation, in light of this paragraph of this specification, the claims are not intended to invoke 35 U.S.C. 112(f) absent the specific language described above.

[0096] The disclosure may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. For example, each of the new structures described herein, may be modified to suit particular local variations or requirements while retaining their basic configurations or structural relationships with each other or while performing the same or similar functions described herein. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive. Accordingly, the scope of the disclosures can be established by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Further, the individual elements of the claims are not well-understood, routine, or conventional. Instead, the claims are directed to the unconventional inventive concept described in the specification.