Multiview display for aiming a weapon in accuracy training

Abstract

The embodied invention features a combination display with four individual displays to facilitate handgun training with a laser firing hand weapon. The first is a target display with a skeletal hand-arm position of a trainee. The second is a side view of the skeletal hand-arm position. The third a target display to provide for viewing the shots as they hit the target. The fourth display is a side view of the trainee as seen by a video camera. All of the displays are recorded during a laser target practice and are immediately available for the trainee to view and learn about correct positioning and stability.

Claims

1. Apparatus that provides a multi view visual display for a weapon accuracy training session comprising: A) a computer, B) a monitor display connected to said computer, wherein said monitor display is separated into four views, C) a hand and arm position sensor connected to said computer, wherein said hand and arm position sensor is used to create a skeletal hands and arms image, D) a trainee interface pad connected to said computer, E) an infrared target camera connected to said computer, F) a side view camera connected to said computer, G) a foot pedal connected to said computer, H) a target, I) wherein said four views further comprises: i. a first view showing said target from said infrared target camera, ii. a hand position reference shown on said first view further comprising: a. a hand calibration bubble, or b. two vertical lines, iii. said skeletal hands and arms image is shown on said first view, iv. a second view showing a side view of skeletal hands and arms image, v. a third view showing a second target view from said side view camera, and vi. a fourth view showing a side view of a trainee, J) a laser training hand gun further comprising: i. an infrared diode laser, ii. a visible light diode laser, iii. an activation circuit that simultaneously activates said infrared diode laser and said visible light diode laser for less than one second, iv. wherein said activation circuit further comprises: a. a capacitor, b. a battery, and c. an electric switch activated by a trigger.

2. The apparatus according to claim 1 wherein said foot pedal is an on off switch for a training session.

3. The apparatus according to claim 2 wherein said two vertical lines establishing an acceptable hand positioning width for said trainee when said foot pedal is pressed.

4. The apparatus according to claim 2 wherein said hand calibration bubble is established for said trainee when said foot pedal is pressed.

5. The apparatus according to claim 2 wherein top and side views are recorded via video during said training session.

6. The apparatus according to claim 2 wherein a score is created during a training session based on the hand position being recorded within a scoring area and a shot position being within a scoring area.

7. The apparatus according to claim 2 wherein a vertical break guide line and a horizontal grip guide line are overlaid on said first view.

8. The method according to claim 7 wherein a score is determined according to: A) a hand position within a. said hand calibration bubble, or b. said two vertical lines, and B) a shot result on said target within a predetermined distance from the target center, when a shot is fired at said target.

9. The method according to claim 8 wherein an average of at least two images from said hand and arm position sensor are used for display and scoring.

10. A method of providing multi view display for a weapon accuracy training session comprising: A) Providing: I) a computer, II) a monitor display connected to said computer, wherein said monitor display is separated into four views, III) a hand and arm position sensor connected to said computer, wherein said hand and arm position sensor is used to create a skeletal hands and arms image, IV) a trainee interface pad connected to said computer, V) an infrared target camera connected to said computer, VI) a side view camera connected to said computer, VII) a foot pedal connected to said computer, VIII) a target, IX) wherein said wherein said four views further comprises: a. a first view showing said target from said infrared target camera, b. a hand position reference shown on said first view further comprising: i. a hand calibration bubble, or ii. two vertical lines, c. said skeletal hands and arms image is shown on said first view, d. a second view showing a side view of said skeletal hands and arms image, e. a third view showing a second target view from said side view camera, and f. a fourth view showing a side view of a trainee, X) a laser training hand gun further comprising: a. an infrared diode laser, b. a visible light diode laser, c. an activation circuit that simultaneously activates said infrared diode laser and said visible light diode laser for less than one second, d. wherein said activation circuit further comprises: i. a capacitor, ii. a battery, and iii. an electric switch activated by a trigger, B) activating said training session when said foot pedal is pressed, C) utilizing said hand and arm position sensor to establish a position of said hand calibration bubble on said first view, D) recording all views, E) stopping said training session when said foot pedal is pressed or a definite time has elapsed, and F) displaying any portion of said recording according to the use of said trainee interface pad.

Description

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

(1) FIG. 1 shows the equipment layout for a laser training session at a shooting range.

(2) FIG. 2 is a close up of the Multiview display.

(3) FIG. 3 shows certain features of the Multiview display.

(4) FIG. 4 illustrates how the hands and arms are converted to a skeletal representation.

(5) FIG. 5 is a side view of a laser training handgun.

(6) FIG. 6 is an exploded view the laser training handgun featuring two diode lasers.

(7) FIG. 7 is a diode laser activation circuit.

DETAILED DESCRIPTION OF THE INVENTION

(8) The goal of the training session is to provide important self-instruction by facilitating views that aid the trainee when shooting at a target. The trainee can compare their hands and arms position when looking at the accuracy of shot placement on the target. The trainee can then learn what position works the best for stability. Additionally, an instructor can view the recorded shot session, and offer insight as to what the trainee needs to address based on the recorded four view video. An important goal is to aid the trainee in quickly advancing in shooting expertise, without long hours of unproductive shooting practice.

(9) The embodied invention provides for initial training on finger movement needed to trigger a shot without disturbing accuracy. Since the weapon does not have live ammunition, only the finger movement and weapon aiming causes a shot to miss the target.

(10) To identify a shot, both an infrared (IR) diode laser and a visible diode laser (red) are activated when the trigger is pressed. Both lasers place a small circle (laser dot) on the target. The visible red dot on the target is used by the trainee to receive feedback on how well their shot was placed. The IR laser dot on the target is used by a scoring camera to identify the position of the shot and issue a score or rating. The scoring camera utilizes a filter that removes visible light, so that only the infrared spectrum is seen. The shot on target is then scored based on the position of the IR dot on the target. The visible diode laser and the IR diode laser are aimed-adjusted so that the red dot and the IR dot appear on the same place on the target.

(11) It is not desirable to for a trainee to have the red laser on at all times when aiming. This will tend to teach the trainee to shoot based on looking at the laser dot position, rather than looking through the weapon sights. To this end, both the red dot and the IR dot are briefly activated when the trigger is pressed, and flash on the target for less than a second. It is a simplification to utilize only the IR dot for scoring.

(12) One of the advantages of the laser dot training is to build confidence that the visible light gun laser is accurately aligned to the gun sights, and that any missed shot is due to how the trigger is pressed or how well the trainee aligns the gun sights to the target. This avoids the trainee developing habits of continually adjusting their aim for the variances of gun shyness or shot anticipation.

(13) Once a trainee has had sufficient training in pulling the trigger properly and aligning the gun sights to the target, the trainee can graduate to live fire.

(14) The weight of the training laser handgun is substantially the same as a similar sized factory issued weapon that is fully loaded with ammunition. Also, the overall balance of the weapon is maintained so as to ensure the same feel when a trainee graduates to live fire. The gun weight is adjusted by adding a weight that is shaped to look like a ammunition magazine. The weight of the magazine, and the distribution of weight, can be adjusted to achieve the desired weight and fee of a real weapon.

(15) The infrared camera preferably utilizes a CMOS sensor, but a CCD is also acceptable.

(16) The pulse electronic circuit that activates both diode lasers consumes very little power and provides for a long life battery.

(17) The laser handgun is highly portable and its two diode lasers, power source, switching, and capacitor are completely housed in the upper portion of the handgun, i.e. above the trigger. This provides for easy adaptability to simulate a broad variety of standard hand weapons.

(18) Another feature is the ability to utilize a low power diode laser that is eye safe (i.e. less than 3 milliwatts). Also the diode lasers only activate briefly, so that the system avoids eye damage.

(19) FIG. 1 shows the equipment set up for a training session. A side view camera 101 is connected to a training computer 103 by wireless or by a suitable connection such as USB or ethernet. The side view camera records a side view of the trainee during a shooting session. An infrared sensor 102 detects the position of the trainee's arms and hands. It does this by an infrared sensor that utilizes two infrared cameras and a few illuminating infrared LEDs which illuminate the arms/hands of the trainees. The infrared cameras are spaced apart so as to provide depth (i.e. distance above the sensor) perspective. The infrared sensor 102 and the side view camera 101 are both connected to the training computer.

(20) A multi-view display 104 is connected to the training computer and is available to the trainee. It will be discussed further in FIG. 2. A target camera 107 is connected to the training computer and located near the target 106 to provide a view of the target as seen by the trainee. The target camera provides enhanced visualization of the laser shots as their position is recorded on the target. The target 106 is in a frame and is moved back and forward between the shooter and desired distance by a rail system 108. A button panel 110 is connected to the training computer 103 and is used to rewind and play the training session recording, and to slow down the speed when desired. A keyboard could equally be used.

(21) The sensor 102 utilizes dual infrared cameras to track the position of the hands and arms. The cameras are separated to provide for depth perception utilizing the differences between the two camera images, similar to what human eyes do. The sensors look only at the infrared wavelength, and the arms-hands are illuminated by three infrared LEDs in line with the cameras. To keep power within the minimum amount that a USB power supply can provide, the LED's are pulsed and matched to the camera's frame rate. The sensor communicates with the training computer which identifies more than 21 points for each hand. Individual fingers are recognized including finger digits. One point represents the Palm of the hand, one point for the wrist, and 4 points for each arm. A rectangular projection of the arms approximates the Ulna and Radius bones with straight lines.

(22) Only hands and arms are shown in a skeletal matter. The arm/hand identifying software will not show a skeletal form if the hands cannot be identified when hand positioning is out of range or is overlapping too much. The hand position and skeletal recognition is preferably done with only a single identifying sensor 102.

(23) To avoid jitter from the sensor which can operate up to 200 frames per second, an average of multiple frames removes the jitter. The image is then a sequential display of frame averaging.

(24) The 2 infrared cameras in the identifying sensor see depth and are able to find or interpret that position of the hands in a 3D area above the sensor. The goal of the software is to provide feedback during a diode laser fire at a target.

(25) FIG. 2 is an illustration of 4 views that are combined on a single monitor display: 1. The upper left screen display 201 shows a hand calibration bubble 206 in crosshairs and a view update target as seen by the trainee with skeletal hands-arms 208. 2. The lower left view screen display 203 shows a side view of the trainee's skeletal hands-arms 208 holding the gun. 3. The upper right screen display 202 is a close up of the target so that shots can readily be seen by the trainee. 4. The lower right screen 204 is a side view video recorder of the trainee holding the gun. The video is recorded during the shooting (i.e. after foot pedal is pressed). It is a laser firing view.

(26) Other orientations of the four view display could equally be used, such as the lower right screen being a side view of the skeletal hands, the upper right display being a side view of the trainee, etc.

(27) All four views are recorded during a shot training session and available for replay so that the trainee can identify their position flaws and variances.

(28) A software development kit associated with the sensor is utilized to create customized programming to: 1. Identify the position of the hands, 2. Determine the position of the hand calibration bubble 206, 3. Determine and the position of the vertical 209 and horizontal 210 dashed lines 4. Overlay the hand calibration bubble 206 and the vertical and horizontal lines (209, 210) on the upper left screen.

(29) Every five seconds, the position of the hands holding the gun is marked 205 on the upper left screen 201 as illustrated in FIG. 2. This provides a record of hand position motion during a trainee multi shot sequence. A number is attached to each circular marking to aid in understanding how the hands moved.

(30) An oval or circular hand calibration bubble 206 is projected on the screen when the trainee presses a foot calibrate pedal 109, which also starts a trainee shot session. The bubble then projects on the screen based on the hands of the trainee position, as measured by the infrared sensor 102.

(31) The hand calibration bubble 206 is used to establish an initial hand position for differing heights of the trainee, and different holding heights. Some trainees are short (or tall) and have short arms (or long); and the hand calibration bubble aids in allowing the training session to establish a circle of acceptable hand positions for their size. The bubble is not to scale on the screen and is an additional visual aid for hand position tracking. The outer size of the bubble is fixed and is not calibrated to the trainee's height, arm length, or body position. The hand calibration bubble will be visible but not locked into position until the trainee presents his hands within the sensor range and is not moving.

(32) As seen in FIG. 2, a shooting width 207 is marked on the screen with two vertical lines when the foot calibrate pedal 109 is pressed. The shooting width provides a recommended window of acceptable hand positions based on the initial position of the hands. The shooting width 207 is fixed and is not calibrated to the trainee's height. Either the shooting width 207, the hand calibration bubble 206, or both could be marked on the screen.

(33) The crosshair dashed lines in the upper left view 201 and the lower right view 204 are based on the camera position. They are not calibrated to a shooter position. The dashed lines are guides for how the gun is held. The vertical dashed line 209 indicates whether the gun is held too high or too low. The horizontal line 210 indicates whether the gun is held too loose or too tight. The guide lines do not have to be centered on the target.

(34) FIG. 3 illustrates how a score for the player is identified when a satisfactory shot is made (within a certain distance from target center 301) and within an acceptable hand position (i.e. hand position within hand calibration bubble 302 or shooting width). For example, the score would be 50% for 6 out of 12 shots when both criteria are met. Flexibility to establish an acceptable shot on target region and the size of the hand calibration bubble is part of the training session design. The target camera 107 is used as part of scoring an acceptable shot.

(35) The embodied invention is primarily conceived for hand gun training when firing a laser dot at a target.

(36) The preferred use of the training apparatus would be at a training facility, such as a shooting range. Use of laser hand guns improves the flexibility for the embodied invention to be used in a variety of locations not specifically designated as a shooting range.

(37) See U.S. Pat. No. 9,891,028 for a method of identifying and scoring shot on a paper target when using a camera.

(38) To identify and score a laser dot on a target, a 1080p wireless camera (10801920 pixels) with over 2 million pixels is used to monitor the paper target, and the paper target is mounted inside a metal frame. The frame is designed to fit the target paper tightly so that the alignment between the camera is maintained and a set up calibration is not needed. The camera is positioned in front of the target, and above it, so that the camera obtains a high resolution image of the target. The alignment between the target and the camera is a set up function at the beginning of a shooting session. Preferably, the camera is kept at a fixed distance and position from the target to simplify the set up and accuracy of the camera's image. Pixel changes due to an infrared laser dot image appearing on the target is used to identify the position of the shot. The image from the camera is continuously monitored to recognize the location of the shot and score its position accurately. Since the laser dot does not disturb or move the target, the ability to recognize a shot is simplified.

(39) FIG. 4 illustrates how the skeletal hands and arms are recognized. The hands and arms 401 are converted to the skeletal representation 402 by identifying and matching the joints such as a wrist, finger digits.

(40) The training computer specifications needed to identify the skeletal hand position, the bubble center, and view projection are: 1. At least 8 and preferably 16 gigabytes of random access memory 2. Intel 17 chip running at 2.8 gigahertz. 3. Dedicated video board with at least 2 gigabytes of memory, and preferably 8 gigabytes. 4. Solid state hard drive. 5. Monitor with at least 1080p resolution for the multi-view display

(41) One main goal of the training session is for the trainee to see how their shooting hand position is related to the and the resultant position of the shot on the target. A display is especially created for the trainee to see this in a multi view display. In particular, the display shows a how a trainee is holding their arms and the gun at the moment of shooting. This can be compared to known faults, such as: 1. pulling trigger quickly causing shots to be low and left of the target. 2. Tight gun gripshots tend to be low and right of the target 3. Incorrect finger trigger positionshots tend to be left of the target 4. Thumb push on gunshots tend to be right of the target 5. Anticipating recoil by moving gunshots tend to be high and left of target 6. Anticipating recoil by aiming lowshots tend to be low of the target 7. Anticipating recoil by pushing gunshots tend to be right and mildly elevated

(42) By replaying the training session video with shooting results on the target, a trainee learns quickly what is causing their aiming problems. To do better, they learn correct positioning faults rather than compensating by changing where they aim. The trainee can operate the speed of the playback, so that the moments just prior to the shot being fired can be scrutinized. Additionally, an instructor can review the display with the trainee, and help the trainee understand why they are inaccurate.

(43) A shot training session begins when the trainee presses the foot pedal. The training computer then establishes the hand calibration bubble (or shooting width zone), starts recording both cameras (side view and target), and watches the target to identify shot position based on any pixel change. The shot training session ends when a timer is done, or the trainee presses the foot pedal again. The video is then stored on the training computer hard drive, the shot scoring is made, and the multi-view video monitor is frozen. The trainee then presses a rewind button on the pad to rewind and look at the four view display. When done, the trainee follows a menu that clears the screen and the training computer becomes ready to start the next training session.

(44) FIG. 6 is an exploded view of the laser training handgun. An upper slide 601, is connected to a lower gun frame 602 and a laser slide 603. A front gun sight 604 and a rear gun sight 616 are attached to the upper slide. Inside the laser slide there is an infrared diode laser 605 and a visible light diode laser 606. Preferably, the visible light diode laser emits a red color laser. The two diode lasers are powered by a battery 608 which rests on a battery frame 607 with a door. A circuit (see FIG. 7) includes a 1000 microfarad capacitor 609 which provide a brief pulse of power so that the diode lasers are on for less than a second, and preferably about 0.2 seconds. To switch the diode lasers on, a micro switch 610 is activated by a trigger frame 611 which in turn is activated by the trigger (with torsion spring) 612, which in turn is pulled by the trainee when firing a shot. To add heft and weight to the gun, a simulated magazine 613 is added, which is inserted/removed by use of a magazine catch 614. The simulated magazine includes a magazine outer cap 615.

(45) FIG. 7 is a simple electrical schematic which simultaneously fires the infrared laser and the red dot laser for approximately 0.2 seconds. A battery 701 provides a constant voltage for the circuit. An optional on-off switch 702 may be added to prevent long term battery drainage when not in use. However, the drainage through the capacitor is very small and the on-off switch is not required. A two position micro switch 703 is used to either charge the capacitor 706, or discharge the capacitor through both the visible diode laser 705 and the infrared diode laser 704. The capacitor 706 is preferably 1000 microfarads. The micro switch 703 is activated by the trigger mechanism when the trainee pulls the trigger back. The capacitor charges very quickly, and the laser can be activated as fast as the trainee pulls the trigger.

(46) This simplified circuit provides for simplicity in building the laser training gun, and a printed circuit board is not required. The circuit is directly soldered together by wiring.

(47) The lasers are only activated briefly, so that the targeting camera and training computer scoring system will be able to identify when a shot is fired. The frame rate of the scoring camera is preferably set to record at least one IR dot on the target; and score it accordingly. To ensure at least one scoring record, the frame rate of the camera must be at least 5 image frames per second to record a 0.2 second long laser dot image. Preferably, the frame rate is faster and at least 30 image frames per second are used to record and identify the position of the IR laser dot image. When recording at a frame rate that records multiple image frames of the IR laser dot, the first frame is preferably chosen as the best representation of where a bullet would go.

(48) The targeting camera sends images to the training computer which looks at the frames to identify the bright IR laser dot, and the background color is relatively easy to filter out. Preferably, the training computer is capable of handling both scoring and display.

(49) As used herein the terms training computer and computer system are intended to refer to a computer-related entity, comprising either hardware, a combination of hardware and software, software, or software in execution capable of performing the embodiments described. The disclosed embodiments which use the central computer refer to being interfaced to and controlled by a computer readable storage medium having stored thereon a computer program. The computer readable storage medium may include a plurality of components such as one or more of electronic components, hardware components, and/or computer software components. These components may include one or more computer readable storage media that generally store instructions such as software, firmware and/or assembly language for performing one or more portions of one or more implementations or embodiments of an algorithm as discussed herein. These computer readable storage media are generally non-transitory and/or tangible. Examples of such a computer readable storage medium include a recordable data storage medium of a computer and/or storage device. The computer readable storage media may employ, for example, one or more of a magnetic, electrical, optical, biological, and/or atomic data storage medium. Further, such media may take the form of, for example, floppy disks, magnetic tapes, CD-ROMs, DVD-ROMs, hard disk drives, and/or solid-state or electronic memory. Other forms of non-transitory and/or tangible computer readable storage media not list may be employed with the disclosed embodiments.

(50) A number of such components can be combined or divided in an implementation of a computer system. Further, such components may include a set and/or series of computer instructions written in or implemented with any of a number of programming languages, as will be appreciated by those skilled in the art. Computer instructions are executed by at least one central processing unit. In addition, other forms of computer readable media such as a carrier wave may be employed to embody a computer data signal representing a sequence of instructions that when executed by one or more computers causes the one or more computers to perform one or more portions of one or more implementations or embodiments of a sequence.

(51) While various embodiments of the present invention have been described, the invention may be modified and adapted to various operational methods to those skilled in the art. Therefore, this invention is not limited to the description and figure shown herein, and includes all such embodiments, changes, and modifications that are encompassed by the scope of the claims.