Devices, systems and methods for tracking and auditing shipment items

Abstract

Devices, systems and methods for tracking and auditing shipment items are provided by capturing video of the shipment items handled in a facility. In certain embodiments, tracking devices are disclosed that can be integrated in a platform for material handling vehicle and manufacturing systems. In certain embodiments, tracking devices can provide tracking and auditing of shipment items by using a method of capturing video of shipment items as they are being handled. In certain embodiments, tracking devices can be coupled to video cameras. The video to be captured may be determined based on the information collected and/or commands received in relation to the items. In some embodiments, the method can create metadata and associate video and item information for shipment items handled in a facility. The video and metadata can be accessed through a communication network at a later time for tracking, auditing and other purposes.

Claims

1. A system for capturing video of shipment items handled in a facility, the system comprising: at least one video camera configured to record video content data for a duration associated with a buffer in coordination with at least one physical operation for loading, unloading, inspecting, or moving shipment items; an auditing server capable of receiving the video content data; and a processing device that is coupled to the at least one video camera and that is configured to: communicate with a shipment instruction server; initiate recording from the at least one video camera for the duration of the buffer and cause continued recording for the duration of the buffer until a triggering event is caused; in response to the triggering event, receive action and item information on a shipment item from the shipment instruction server and determine first and second durations of video content data using an adaptive video capture procedure based on the received action information and item information associated with the at least one physical operation, wherein the adaptive video capture procedure selectively generates a concatenated video file from continuously recorded video content data; retrieve first video content data associated with the first duration of the video content data from the buffer; restart recording from the at least one video camera for the second duration of the video content data by automatically resetting the buffer, thereby dynamically overwriting existing video content data including the first video content data; retrieve second video content data associated with the second duration of the video content data from the buffer; associate the action and item information with the first and second video content data; cause the action and item information and the first and second video content data to be stored on the storage medium; and transmit the action and item information and the first and the second video content data to the auditing server.

2. The system of claim 1, wherein the at least one video camera and the processing device are mounted on a materials handling vehicle.

3. The system of claim 1, wherein the at least one video camera is placed adjacent to a conveyor belt.

4. The system of claim 1, wherein the triggering event is the receipt of item and action information from the shipment instruction server.

5. The system of claim 1, wherein the item information includes an identifier, dimensions and geographical location.

6. The system of claim 1, wherein the processing device further comprises at least one input/output port configured to communicate with a console.

7. The system of claim 1, wherein the processing device further comprises an auxiliary power source that is configured to supply uninterruptible power to the processing device and the at least one video camera.

8. The system of claim 1, wherein the processing device further comprises an integrated power and data connector.

9. The system of claim 8, wherein the integrated power and data connector is a Power over Ethernet connector.

10. A method for capturing video of shipment items handled in a facility, the method comprising: instructing at least one video camera to continuously record for a duration of a buffer until a triggering event is caused in coordination with at least one physical operation for loading, unloading, inspecting, or moving shipment items; receiving action and item information on a shipment item from a shipment instruction server in response to the triggering event; retrieving first video content data associated with a first duration of the video content data from the buffer using an adaptive video capture procedure based on the received action information and item information associated with the at least one physical operation, wherein the adaptive video capture procedure selectively generates a concatenated video file from continuously recorded video content data; restarting recording from the at least one video camera for a second duration of the video content data by automatically resetting the buffer, thereby dynamically overwriting existing video content data including the first video content data; retrieving second video content data associated with the second duration of the video content data from the buffer using the adaptive video capture procedure based on the received action information and item information associated with the at least one physical operation; associating the action and item information with the first and second video content data; causing the action and item information and the first and second video content data to be stored on the storage medium; and transmitting the action and item information and the first and second video content data to an auditing server.

11. The method of claim 10, wherein the action information on the item includes loading, unloading, inspecting, or moving the item.

12. The method of claim 10, wherein the first duration of video content data for loading and unloading is larger than the first duration of video content for moving the shipment item.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Further features of the invention, its nature and various advantages will be more apparent from the following detailed description of the embodiments, taken in conjunction with the accompanying drawings in which:

(2) FIG. 1 is a block diagram illustrating a tracking and auditing system in accordance with some embodiments;

(3) FIG. 2 is a block diagram illustrating a tracking and auditing device for shipment items according to certain embodiments;

(4) FIG. 3 is a block diagram illustrating an additional system suitable for use in tracking and auditing shipment items in accordance with some embodiments;

(5) FIG. 4 is a flow diagram illustrating a process for capturing video of shipment items in a facility according to certain embodiments;

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

(6) Devices, systems and methods are provided for tracking and auditing shipment items by recording targeted video content for these items as they are being handled in a facility. The following description starts with an overview of an exemplary system, which illustrates an operating environment for the exemplary device disclosed herein. From there, further details are provided regarding the exemplary device and overall system components and processes that may be implemented in accordance with various embodiments of the invention. A variety of additional aspects are also discussed below.

(7) In the following description, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific embodiments in which the invention may be practiced. It is to be understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present invention. Furthermore, although exemplary embodiments are discussed in the context of a shipment item being handled in a facility by one or more tracking and auditing devices, systems and methods, one of ordinary skill in the art will appreciate that the principles of the invention are applicable to any tracking, monitoring and auditing item effort in any appropriate field such as law enforcement, manufacturing and compliance.

(8) Referring now to the drawings in which like numerals represent the same or similar elements, FIG. 1 represents an overview of a tracking and auditing system 100 for shipment items handled in a facility. System 100 provides a solution to the problem of tracking and auditing shipment items by recording, storing and indexing video as the items are being handled by material handling vehicle 102. In some embodiments, material handling vehicle 102 can be a fork lift, a tow truck or any other suitable vehicle able to transport and/or handle shipment items.

(9) In some embodiments, one or more video cameras 104 are placed on material handling vehicle 102 in order to record the handling of shipment items in a facility. Video cameras 104 can be placed on various locations of material handling vehicle 102 to obtain multiple viewing angles of a shipment item. For example, one or more video cameras 104 can be placed angled and/or flat on the top, bottom and/or front of material handling vehicle 102 or in any other suitable configuration, in accordance with some embodiments of the invention. Video cameras 104 can be any suitable device able to record video for a duration related to a buffer in both an intermittent and continuous manner. For example, video cameras 104 can be one or more camcorder, digital video camera, closed-circuit camera, smartphone camera or any other suitable video camera and/or combination thereof.

(10) In some embodiments, material handling vehicle 102 can include shipment instruction device 106. Specifically, device 106 can receive, provide, and display information about shipment items in a facility from and to a material handling vehicle's operator. For example, shipment instruction device 106 can be any suitable device such as a mounted computing device or hand held device that can be located on material handling vehicle 102. It is typical to use a shipment instruction device 106 in the field of shipment item transportation and facility logistics. A common and exemplary use of shipment instruction device 106 in the field is the receipt of instructions and information by the material handling vehicle operator. In some embodiments, the operator can receive, via device 106, information on the shipment item such as an identifier, its destination and/or its location in the facility. Additionally, an operator can receive instructions such as loading, unloading, moving and/or inspecting a shipment item in the facility. For example, when the operator receives the instruction to load a shipment item he/she will drive material handling vehicle 102 to the location of the item obtained from shipment instruction device 106. In some embodiments, once there, the operator can scan the shipment item using shipment instruction device 106 and confirm its identifier.

(11) Shipment instruction device 106 can be directly connected to tracking device 108 using a wired and/or wireless connection in accordance with some embodiments of the invention. Tracking device 108 can then receive item and action information from shipment instruction device 106. Tracking device 108 can be powered by power supply 110. For example, power supply 110 can be the material handling vehicle's battery or any other suitable form of direct current power. Tracking device 108 can then power and control one or more video cameras 104 and can record video for a specific duration based on the action information communicated by shipment instruction device 106.

(12) In some embodiments, tracking system 100 can be employed in any scale. For example, a tracking and auditing system can include any number of material handling vehicles 102, comprising any number cameras 104 capable of recording video and any number of tracking devices 108 that can communicate with one another or with any other device. For example, a fleet of fork lifts can have a network of tracking devices that can cross-reference item and action information between different fork lifts, thus the system can be capable of obtaining multiple videos from different fork lifts, for a shipment item.

(13) Turning to FIG. 2, tracking device 108 is disclosed, providing a platform for tracking and auditing shipment items on which system 100 of FIG. 1 may be implemented. As shown therein, tracking device 108 can include computing device 210. In some embodiments, computing device 210 may be equipped with processor 204, transceiver 206 and data storage 208. In some embodiments, any of processor 204, transceiver 206 and data storage 208 can be separate components instead of part of a single component such as computing device 210. Processor 204 can be a microprocessor, a central processing unit or any other suitable circuit capable of executing computer instructions. Transceiver 206 can be any suitable circuit capable of transmitting and receiving data such as a radio frequency transceiver and/or Ethernet transceiver. Data storage 208 can be any suitable computer storage device such as RAM, ROM, PROM, and/or SRAM and is preferably a physical, non-transitory medium.

(14) In some embodiments, tracking device 108 can include one or more video cameras 104 capable of recording video. Video camera 104 can be embedded into device 108 or connected externally to device 108 (e.g., as shown in FIG. 1) using any suitable connection. For example, video camera 104 can include one or more camcorder, digital video camera, closed-circuit camera, smartphone camera or any other suitable video camera and/or combination thereof.

(15) Tracking device 108 can be powered using power supply interface 216 in accordance with some embodiments of the invention. For example, power supply interface 216 can include a Power over Ethernet system capable of passing electrical power along with data that can power video cameras 104. In some embodiments, power supply interface 216 can be an uninterruptible power supply capable of providing emergency power to computing device 210, an inverter that can convert power from a direct current power source such as power supply 110 to an alternating current that can power computing device 210 or any suitable combination thereof. In some embodiments, power supply interface 216 including power supply 110 and computing device 210 can be separate components and/or can be part of a single interface.

(16) In some embodiments, tracking device 108 can include input/output (I/O) interface 214 that is capable to connect with any other suitable device such as a computing device and/or peripheral device or any suitable combination thereof. In some embodiments, I/O interface 214 can connect tracking device 108 and camera(s) 104 to obtain the recorded video of a shipment item. Additionally, I/O interface can be used to directly connect tracking device 108 with an external computer for configuration and maintenance purposes such as the use of tracking device 108 in a different facility requiring different location parameters. In some embodiments, I/O interface 214 can be connected to peripheral components such as a printing device and/or a screen. For example, I/O interface can include one or more HDMI connections for the transfer of video data from camera 104, one or more Firewire connections, one or more USB connections for the transfer of data and/or any suitable combination thereof.

(17) The components of tracking device 108 can communicate using information bus 212. For example, computing device 210 can exchange data with I/O interface 214, power supply interface 216 and one or more video cameras 104 in accordance with some embodiments of the disclosed invention. Information bus 212 can be an internal bus such as a memory bus, system bus and/or any suitable bus capable of connecting for example, processor 204, transceiver 206 and data storage 208. In some embodiments, information bus 112 can be an external bus such as an expansion bus or any other suitable form of bus that can for example, connect computing device 210 with I/O interface 214, video camera 102 and power supply interface 216.

(18) It should be noted that the device in FIG. 2 is merely meant to demonstrate an exemplary embodiment of a tracking device, and should not be construed as limiting in any manner whatsoever. The particular configuration in FIG. 2 can be altered in numerous ways without departing from the principles herein. For example, video camera 104 can be integrated in tracking device 104 or one or more cameras may be integrated and/or externally connected to computing device 210. In some embodiments, computing device 210 and/or any of processor 204, transceiver 206 and data storage 208 can also be integrated in tracking device 104

(19) FIG. 3 shows system 300 for tracking and auditing shipment items in accordance with some embodiments of the invention. System 300 can have a plurality of client devices 308 that can be in communication with auditing server 302 and shipment instruction server 304 over communication network 306. The network may be any type of network such as one that includes the Internet, a local area network, a wide area network, an intranet, etc. Client devices 308 can be used to communicate with auditing server 302 and shipment instruction server 304. Client devices 308, as well as auditing server 302 and shipping instruction server 304, may be configured to communicate via wired or wireless links, or a combination of the two.

(20) Client devices 308 can represent a desktop computer, laptop computer, cell or smart phone, tablet device, or other type of computing device. Each of the client devices 308 can be equipped with one or more computing devices 210, data storage devices 208 (e.g., RAM, ROM, PROM, and/or SRAM) and/or one or more components of computing device 210 such as processors 204 (e.g., a central processing unit) that are capable of executing computer program instructions. Any of client devices 308 may further include a display that is capable of displaying item and action information for a shipment item that can be received by shipment instruction server 304 in accordance with some embodiments of the invention. Client devices 308 can be tracking device 108 and shipment instruction device 106 in separate devices that can communicate via communication network 306. In some embodiments, client devices 308 can be tracking device 108 and shipment instruction device 106 integrated in a single device. For example, the operator of material handling vehicle 102 can receive action and item information for a shipment item on a client device 308 such as a smartphone. The operator can then drive to the shipment item and use the smartphone to scan the item. The smartphone can then activate cameras 104 including its own to record video for tracking and auditing the shipment item.

(21) Auditing server 302 and shipment instruction server 304 can also include one or more processors 204 and one or more data storage devices 208. Auditing server 302 may generally represent any type of computing device that is capable of communicating with a client device 308. In some embodiments, auditing server 302 can cause information such as video content data, action and/or item information to be associated and stored in a database that maintains video and/or information about shipment items in a facility. In some embodiments, shipment instruction server 304 can communicate with shipment instruction device 106 that can be any of client devices 308 to provide item and action information. Additionally, shipment instruction server 304 can also receive information on a shipment item when, for example, an operator scans an item upon locating it in the facility. In some embodiments, auditing server 302 and shipment instruction server 304 comprise one or more mainframe computing devices that execute a web server for communicating with client devices 308 over communication network 306. For example, auditing server 302 and shipment instruction server 304 can be located and configured in the same facility or can be external to the facility.

(22) One useful feature provided by system 300 relates to the fact that a number of client devices 308 such as tracking devices 108 may communicate through communication network 306. This useful feature can permit for example a fleet of material handling vehicles 102 that receive item and action information for different shipment items from shipment instruction server 304 to be able to associate the recorded videos from the different tracking devices 108 based on the locations of the shipment items.

(23) In some embodiments, client devices 308 can be used in manufacturing. For example, client devices 308 including tracking device 108 can be placed next to conveyor belts and can record video of the path that a shipment item has traveled on the conveyor belts by associating the video recordings obtained from different tracking devices 108. The video can then be combined and send to auditing server 302. In some embodiments, client devices 308 can be mounted on locations that can capture the handling of shipment items as they are placed on and/or removed from the conveyor belt by a handler.

(24) It should be noted that the system in FIG. 3 is merely meant to demonstrate an exemplary embodiment of an operating environment, and should not be construed as limiting in any manner whatsoever. The particular configuration in FIG. 3 can be altered in numerous ways without departing from the principles herein. For example, it should be noted that the functionality of auditing server 302 and/or shipment instruction server 304 in FIG. 3 may be carried out by a plurality of servers. Likewise, although the figure depicts a single auditing server 302 and shipment instruction server 304 connected to three client devices 308, any number of auditing servers 302, shipment instruction servers 304 and client devices 308 may be utilized with system 100.

(25) Furthermore, while FIG. 3 illustrates a plurality of client devices 308 in communication with an auditing server 302 and shipment instruction server 304 over a network 306, it should be recognized that the functionality provided by auditing server 302 and shipment instruction server 304 to client devices 308 may be performed locally on each of client devices 308. For example, client devices 308 may utilize an application and/or server that executes locally on client devices 308. Thus, any functionality of auditing server 302 and shipment instruction server 304 which is described herein can alternatively be implemented by a client device 308, and vice versa.

(26) In certain embodiments, client device 308, auditing server 302 and shipment instruction device (e.g., 106 in FIG. 1) can be integrated into a single device (e.g., a standard PC computer, tablet, laptop, smartphone, or other device). For example, such a device may run a standard operating system (e.g., Windows, Linux, OSX, Android, iOS). When a device is, for example, a laptop, tablet, smartphone or similar, a tracking and auditing system is easily transportable. Such a device may or may not be further connected to one or more computers or other devices via a network. When configured to operate without a network connection, tracking and auditing system 300 requires little infrastructure beyond the device itself. Because only a single device may be required to implement tracking and auditing system 300, it is apparent that such an embodiment achieves significant savings in costs, infrastructure and complexity.

(27) Embodiments described herein may be hardware-based, software-based or may comprise a mixture of both hardware and software elements. Thus, while the description herein may describe certain embodiments, features or components as being implemented in software or hardware, it should be recognized that any embodiment, feature or component that is described in the figures or description herein may be implemented in hardware and/or software. In certain embodiments, particular aspects are implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.

(28) FIG. 4 illustrates an exemplary method 400 for tracking and auditing shipment items in accordance with certain embodiments of the invention. Initially, tracking device 108 that can be integrated in client devices 308 may be powered up at step 402. In certain embodiments, tracking device 108 may be powered up by power supply 110 and/or power supply interface 216. For example, tracking device 108 can be directly connected to the battery of material handling vehicle 102. Additionally, tracking device 108 may directly connect to an uninterruptible power supply in the case of the vehicle's power loss.

(29) After tracking device 108 has been powered up, it can communicate with shipment instruction server 304 and/or shipment instruction device 106 at step 404. In some embodiments, the communication can happen through communication network 306 and/or I/O interface 214 using a dedicated connector.

(30) At step 406, tracking device 108 can prepare the one or more video cameras 104 for recording by initiating their buffers. In some embodiments, the buffers of video cameras 104 can be set to be capable to record and store locally video for a specific time duration. In some embodiments, one or more video cameras 104 can have buffers of different durations. Once the buffers of video cameras 104 have been initiated at step 406, then tracking device 108 can record video for the duration of the buffers at 408.

(31) Referring back to FIG. 1 and system 100, an operator of the material handling vehicle 102 can receive action and item information at shipment instruction device 106 regarding a shipment item in accordance with some embodiments of the disclosed invention. For example, the operator may receive a command to inspect a shipment item located in a warehouse facility. As the operator travels through the warehouse, tracking device 108 can instruct video camera(s) 104 to record video for the duration of the one or more cameras' buffers as indicated at step 408 until a triggering event is caused at step 410. Camera(s) 104 can record in a continuous loop and can store the duration of the video content locally in the buffer. Once the buffer has filled up, cameras 104 can continue to record by over-writing the video content stored in the buffer

(32) In some embodiments, a triggering event can be the receipt of the item and action information by tracking device 108. For example, receiving by the tracking device an identification number unique to the shipment item. Specifically, an operator can scan a shipment item in the facility using one or more of client devices 308 including shipment instruction device 106. Subsequently, shipment instruction server 304 can receive item information and the shipment item can be identified by scanning a bar code, a quick response code or any other suitable optical code using an optical receiver device, in accordance with some embodiments of the invention. The shipment item can also be identified by the operator manually providing an identification number to the shipment instruction device that is in contact with the tracking device.

(33) Referring back to system 100, if an operator upon receiving the command to inspect has not reached the shipment item and has not been able to scan or otherwise identify it (e.g., NO at 410) then tracking device 108 can continue to record video for the duration of the buffer in camera(s) 104.

(34) In some embodiments, if an operator upon receiving the command to inspect, reaches the shipment item and scans its barcode, (e.g., YES, at 410) then tracking device 108 can receive item and action information from the shipment instruction device 106 and/or server at step 412. For example, tracking device 108 can access the information that the operator of the material handling vehicle obtained from the shipment instruction device 106.

(35) In some embodiments, the item information can include information on the dimensions of the item, a unique identifier, location in the facility and/or any other suitable information. Action information can include the loading, unloading moving and inspection of the shipment item within the facility or any other suitable handling of the shipment item.

(36) Depending on the action information that the operator of material handling vehicle 102 receives, tracking device 108 can determine at step 414 whether to retrieve recorded video from the buffers of one or more cameras 104 in accordance with some embodiments of the invention. For example, if the action received at step 412 indicates that the operator needs to unload a shipment item (e.g., YES, at 414) then tracking device 108 can retrieve a specific duration of video from the buffer associated with the action of unloading at step 416. For example, as the operator drives to the location of the shipment item, cameras 104 are recording in a loop based on their buffers. The operator would then handle and retrieve the item to scan it (e.g., YES, at 410). Once the shipment item is scanned, step 414 can determine the retrieval duration of the item corresponding to the unload action. In some embodiments, loading, unloading, inspecting and/or moving a shipment item may require different durations of recorded video to be retrieved from the buffers of one or more cameras 104. In some embodiments, receiving the actions of moving and inspecting at step 416 may not require retrieval of video from the buffer (e.g., NO, at 414). For example, if the operator does not need to handle and retrieve an item for inspection, then once the item is scanned (e.g., YES, at 410), tracking device 106 causes cameras 104 to record for a duration associated with inspecting an item (e.g., step 418). In some embodiments, the operator may by pass the system and manually insert a duration for recording an item. Additionally, durations of recorded video can depend on the placement of camera(s) 104 on material handling vehicle 102. For example, a camera located on the rear of the vehicle can have a larger duration of video recording than a camera placed upfront.

(37) If the action information received by tracking device 108 does not require video from the buffer (e.g., NO, at 414) or if tracking device 108 has already retrieved a specific duration of video from the buffer at step 416, then tracking device 108 can restart or continue recording using one or more cameras 104 for a determined amount of time based on the action information at step 418. In some embodiments, tracking device 108 restarts recording by resetting the buffers of the one or more cameras. Specifically, in some embodiments, the duration of the recording for loading and unloading a shipment item may be larger than the duration for moving and/or inspecting a shipment item.

(38) Tracking device 108 can then retrieve the additional recording from the buffers of the one or more cameras at step 420. Tracking device 108 can then store the video in storage 208 in accordance with some embodiments of the invention. For example, tracking device 108 can store the video locally on data storage 208, on any of client devices 308, and/or remotely to a database on auditing server 302 using communication network 306.

(39) At step 422, tracking device 108 can associate the item information that it received from shipment instruction device 106 and/or shipment instruction server 304 with the videos stored in data storage 208. Tracking device can then create and attach metadata to the videos based on the item information received from the shipment instruction device and/or server. For example, metadata can include the identifier of the shipment item, dimensions and/or any other suitable textual information that can assist any keyword searches. In some embodiments, metadata can be provided as a separate file to the video using a text format, it can be directly embedded on the video or it can be in any combination thereof.

(40) In some embodiments, tracking device 108 can then send the videos and metadata associated to the shipment item to auditing server 302 or any other suitable data storage device (e.g., data storage 208) at step 424. Once tracking device 108 has sent the video and metadata to auditing sever 302, it returns to step 408 and can start recording in a loop for the duration of the buffers until a new shipment item has been scanned and/or identified.

(41) One of ordinary skill in the art will appreciate that, aside from providing advantages in tracking and auditing shipment items, the devices, systems and methods discussed throughout the disclosure herein may be applicable to a wide variety of fields that require tracking and auditing. Exemplary areas of potential applicability are logistics, manufacturing, compliance and surveillance, as well as tampering detection.

(42) While there have been shown and described various novel features of the invention as applied to particular embodiments thereof, it will be understood that various omissions and substitutions and changes in the form and details of the devices, systems and methods described and illustrated, may be made by those skilled in the art without departing from the spirit of the invention. Those skilled in the art will recognize, based on the above disclosure and an understanding therefrom of the teachings of the invention, that the particular hardware and devices that are part of the invention, and the general functionality provided by and incorporated therein, may vary in different embodiments of the invention. Accordingly, the particular system components shown in FIG. 1-FIG. 4 are for illustrative purposes to facilitate a full and complete understanding and appreciation of the various aspects and functionality of particular embodiments of the invention as realized in system and method embodiments thereof. Those skilled in the art will appreciate that the invention can be practiced in other than the described embodiments, which are presented for purposes of illustration and not limitation, and the present invention is limited only by the claims which follow.