Magnetic vector sensor positioning and communication system

Abstract

A system is described herein for monitoring the movement of one or more magnet sources located external to a device using the vector data from one or more magnetic vector sensors incorporated in the device to determine a position and/or to communicate information.

Claims

1. A system comprising: a first device comprising a screen and at least one vector magnetic sensor that senses the magnetic field produced by the Earth; a second device comprising a modulating magnetic source, said first device is configured to utilize the at least one vector magnetic sensor which interfaces with the modulating magnetic source in the second device to obtain vector data, said vector data corresponding to information communicated from the second device to the first device; wherein said information comprises an identifier; and, wherein said first device is configured to use said identifier with a location look up table.

2. The system of claim 1, wherein the information is location information corresponding to a location of said second device.

3. The system of claim 2, wherein said location information comprises a latitude and longitude of said second device.

4. The system of claim 2, wherein said location information comprises an altitude of said second device.

5. The system of claim 1, wherein said first device comprises a compass.

6. The system of claim 1, wherein said first device comprises an accelerometer.

7. The system of claim 1, wherein said modulating magnetic source comprises an electromagnet.

8. The system of claim 1, wherein said first device is configured to determine a presence of said second device.

9. The system of claim 1, wherein said first device is configured to use the information for authentication purposes.

10. The system of claim 1, wherein said first device is configured to interpolate a location based on said information.

11. The system of claim 1, wherein said first device is configured to extrapolate a location based on said information.

12. A system comprising: a first device comprising a screen and at least one vector magnetic sensor that senses the magnetic field produced by the Earth; a second device comprising a modulating magnetic source, said first device is configured to utilize the at least one vector magnetic sensor which interfaces with the modulating magnetic source in the second device to obtain vector data, said vector data corresponding to information communicated from the second device to the first device; and a third device comprising a modulating magnetic source.

13. The system of claim 12, wherein said second device and said third device have at least one different magnetic characteristic.

14. The system of claim 13, wherein said at least one different magnetic characteristic comprises a different throw.

15. The system of claim 13, wherein said at least one different magnetic characteristic comprises a different amplitude.

16. The system of claim 13, wherein said at least one different magnetic characteristic comprises a different directionality.

17. The system of claim 13, wherein said at least one different magnetic characteristic comprises a different coding.

18. A system comprising: a first device comprising a screen and at least one vector magnetic sensor that senses the magnetic field produced by the Earth; a second device comprising a modulating magnetic source, said first device is configured to utilize the at least one vector magnetic sensor which interfaces with the modulating magnetic source in the second device to obtain vector data, said vector data corresponding to information communicated from the second device to the first device; and, wherein said first device is configured to verify an environment based on a priori knowledge of the second device.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) A more complete understanding of the present invention may be obtained by reference to the following detailed description when taken in conjunction with the accompanying drawings wherein:

(2) FIG. 1A (PRIOR ART) depicts an electronic device having a device body and a stylus as illustrated in U.S. Patent Application No. 2009/0167727;

(3) FIG. 1B (PRIOR ART) depicts an electronic device having a device body and a stylus as illustrated in U.S. Patent Application No. 2009/0167727;

(4) FIG. 2A (PRIOR ART) depicts a magnetic stylus as illustrated in U.S. Patent Application No. 2009/0167727;

(5) FIG. 2B (PRIOR ART) depicts another magnetic stylus as illustrated in U.S. Patent Application No. 2009/0167727;

(6) FIG. 3A (PRIOR ART) depicts an exemplary hall sensor array used in a smartphone;

(7) FIG. 3B (PRIOR ART) depicts an exemplary cell phone having an exemplary X axis, Y axis, and Z axis;

(8) FIG. 3C (PRIOR ART) depicts an exemplary output display showing vector data corresponding to the X, Y, and Z vectors (i.e., magnitude and direction of the X, Y, and Z magnetic field components) as an electronic device such as the cell phone shown in FIG. 3B is moved about over a period of time;

(9) FIG. 4A (PRIOR ART) depicts an example of a magnet gesturing system;

(10) FIG. 4B (PRIOR ART) depicts another example of a magnet gesturing system;

(11) FIG. 5 (PRIOR ART) depicts the magnetic field B of a magnetic sphere being tracked by a position sensor comprising an array of Hall Effect sensors;

(12) FIG. 6 (PRIOR ART) depicts the communications link from an Arduino to an Android via the use of a coil placed over the magnetometer of the Android;

(13) FIG. 7A depicts a system comprising a first device (e.g., a mobile phone) which has a touchscreen (e.g., a capacitive touchscreen) and a second device, e.g., a stylus, which has a magnet such as the stylus shown in FIG. 2A in accordance with an embodiment of the present invention;

(14) FIG. 7B depicts a system comprising a first device (e.g., a mobile phone) which has a touchscreen (e.g., a capacitive touchscreen) and a second device, e.g., a stylus, which has a magnet such as the stylus shown in FIG. 2B in accordance with an embodiment of the present invention;

(15) FIG. 8A depicts a system comprising a first device (e.g., a mobile phone) which has a touchscreen (e.g., a capacitive touchscreen) and a second device, e.g., a mouse, which has a magnet having a first orientation where the magnetic moment of the magnet is parallel to a surface on which the mouse resides in accordance with an embodiment of the present invention;

(16) FIG. 8B depicts a system comprising a first device (e.g., a mobile phone) which has a touchscreen (e.g., a capacitive touchscreen) and a second device, e.g., a mouse, which has a magnet having a first orientation where the magnetic moment of the magnet is perpendicular to a surface on which the mouse resides in accordance with an embodiment of the present invention;

(17) FIG. 9A depicts a system comprising a first device (e.g., a mobile phone) which has a touchscreen (e.g., a capacitive touchscreen) and a second device, e.g., a joy stick input device, which has a base and a control handle with a magnet therein and a base where the magnet has a first orientation that is perpendicular to an axis of an at rest position of the control handle in accordance with an embodiment of the present invention;

(18) FIG. 9B depicts a system comprising a first device (e.g., a mobile phone) which has a touchscreen (e.g., a capacitive touchscreen) and a second device, e.g., a joy stick input device, which has a base and a control handle with a magnet therein and a base where the magnet has a second orientation that is parallel to an axis of an at rest position of the control handle in accordance with an embodiment of the present invention;

(19) FIG. 9C depicts an exemplary system comprising three first devices which provide multiple detection angles relative to a magnet (which has one polarity orientation) within a second device, e.g., a joy stick input device, in accordance with an embodiment of the present invention;

(20) FIG. 9D depicts an exemplary system comprising three first devices which provide multiple detection angles relative to a magnet (which has another polarity orientation) within a second device, e.g., a joy stick input device, in accordance with an embodiment of the present invention;

(21) FIG. 10A depicts a second device in the form of a mouse having two magnets where each of the two magnets has a first orientation relative to a surface in accordance with an embodiment of the present invention;

(22) FIG. 10B depicts a second device in the form of a mouse having two magnets where each of the two magnets has a second orientation relative to a surface in accordance with an embodiment of the present invention;

(23) FIG. 10C depicts a second device in the form of a mouse having two magnets where one of the magnets has a first orientation relative to a surface and the other one of the magnets has a second orientation relative to a surface in accordance with an embodiment of the present invention;

(24) FIG. 10D depicts a second device in the form of a joy stick input device comprising a base (with one magnet) and control handle (with one magnet) in accordance with an embodiment of the present invention;

(25) FIG. 10E depicts a second device in the form of a joy stick input device comprising a base (with one magnet) and control handle (with two magnets) in accordance with an embodiment of the present invention;

(26) FIG. 10F depicts a second device in the form of a joy stick input device comprising a base (with one magnet) and control handle (with three magnets) in accordance with an embodiment of the present invention; and

(27) FIGS. 11A-11K depict exemplary second devices that comprise one or more magnets that can be detected by one or more vector magnetic sensors of one or more first devices in accordance with an embodiment of the present invention. In particular, the exemplary second devices shown include: (1) a glove (FIG. 11A); (2) a golf club (FIG. 11B); (3) a tool (FIG. 11C); (4) a pet collar (FIG. 11D); (5) a game controller (FIG. 11E); (6) a vending machine (FIG. 11F); (7) a vehicle (FIG. 11G); (8) a gas pump (FIG. 11H); (9) a cash register (FIG. 11I); (10) an automated teller machine (FIG. 11J); and (11) a first device (FIG. 11K).

DETAILED DESCRIPTION OF THE INVENTION

(28) In accordance with one aspect of the invention, vector magnetic sensor-based orientation sensing capabilities of a first device are leveraged to determine the orientation of one or more second devices that may be associated with the first device, where the first device comprises at least one vector magnetics sensor and each of the one or more second devices comprises at least one magnet, where the at least one magnet may be a permanent magnet, an electromagnet, or a electro-permanent magnet. Specifically, a second device may comprise a stylus, a joystick, a game controller, a mouse, a glove, a keyboard, an eyepiece, a laptop, a trackpad, a digital audio player, a computer display, a mobile phone, a mobile device, a tablet, etc. Moreover, the second device could merely be a magnet.

(29) In accordance with a first embodiment of the invention depicted in FIG. 7A, a system 700 may comprise a first device 100 (e.g., a mobile phone 100) comprising a touchscreen 112 (e.g., a capacitive touchscreen 112) and a second device, e.g., a stylus 120a, comprising a magnet 126 such as the stylus 120a shown in FIG. 2A. Unlike the prior art approach described previously in relation to FIG. 1A where the stylus 120 was touched to a capacitive touchscreen 112 such that magnetic field lines of a magnet 126 in the head of the stylus 120 produced a capacitive response, a stylus 120a (or any other second device in accordance with the invention) comprising a magnet 126 doesn't have to touch the touchscreen 112 of the first device 100 because the position of the magnet 126 included in the stylus 120a as determined by one or more magnetic sensors 110 included in the first device 100 is used to provide an interface with the first device 100. In accordance with the invention a second device 122a can be in proximity to a first device 100, where the one or more magnetic sensors 110 of the first device 100 can measure the absolute orientation and location of the second device 120a. Vector data corresponding to the absolute orientation and location of the second device 120a within a coordinate system based on the absolute orientation and location of the first device 100 can be mapped to a location on the touchscreen 112 and otherwise used to communicate with the first device 100. Similarly, a system 710 as depicted in FIG. 7B may comprise a first device 100 and a second device, e.g., a stylus 120b, comprising a magnet 126 such as the stylus 120b shown in FIG. 2B, where generally as long as the orientation of the magnet 126 residing in a second device 120b is known, the absolute location and orientation of the magnet 126 residing in the second device 120b can be determined using the vector data provided by the one or more magnetic sensors of the first device 100. It should be noted that the second device could indeed touch the touchscreen of the first device. Further, the first device need not have a touchscreen in the first place but it could have a regular screen.

(30) As shown in FIG. 8A, a system 800 of the invention may comprise a second device that is a mouse 802a comprising a magnet 126 having a first orientation where the magnetic moment of the magnet 126 is parallel to a surface 804 on which the mouse resides. As shown in FIG. 8B, a system 810 of the invention may comprise a second device that is a mouse 802b comprising a magnet 126 having a second orientation, where the magnetic moment of the magnet 126 is perpendicular to the surface 804 on which the mouse 802b resides.

(31) FIG. 9A depicts an exemplary system 900 of the invention that comprises a second device that is a joy stick input device 902a comprising a base 903 and control handle 905 configured to pivot within the base 903 at a pivot point 904. The control handle 905 includes a magnet 126 having a first orientation that is perpendicular to an axis of an at rest position of the control handle (i.e., where the moveable portion is at rest when not being held by a user), where the distance between the magnet 126 and a pivot point 904 is known, the distance between the bottom of the base 903 and the pivot point 904 is known. Thus an at rest absolute location and orientation of the control handle 905 can be determined and then used to determine the real time absolute location and orientation of the control handle 905 during operation.

(32) FIG. 9B depicts an exemplary system 910 of the invention that comprises a second device that is a joy stick input device 902b that is like the joy stick input device 902a of FIG. 9A except the magnet has a second orientation that is parallel to an axis of an at rest position of the control handle.

(33) FIG. 9C depicts an exemplary system 920 of the invention that comprises a second device that is the joy stick input device 902a where the magnetic sensors of three first devices 100a-100c provide multiple detection angles relative to the magnet 126 (which has one orientation) of the joy stick input device 902c.

(34) FIG. 9D depicts an exemplary system 930 of the invention that comprises a second device that is the joy stick input device 902b where the magnetic sensors of three first devices 100a-100c provide multiple detection angles relative to the magnet 126, (which has one orientation) of the joy stick input device 902d.

(35) Under one aspect of the invention two or more first devices 100 can communicate using one or more communications capabilities available to the first devices 100 such as cellular communications, WI-FI communications, or the like, to share vector data. One skilled in the art will recognize that having more magnetic sensors and having more detection angles enables ambiguities of orientation and location to be resolved more easily to include ambiguities resulting from the second device including multiple magnets.

(36) FIG. 10A depicts a mouse 802c having two magnets 126a 126b, where each of the two magnets 126a 126 has a first orientation relative to the surface 804 (not shown) and FIG. 10B depicts a mouse 802d, where each of the two magnets 126a 126b has a second orientation relative to the surface 804. FIG. 10C depicts a mouse 802e having two magnets 126a 126b, where one of the magnets 126a has the first orientation relative to the surface 804 and the other one of the magnets 126b has the second orientation relative to the surface 804.

(37) FIG. 10D further depicts a joy stick input device 902c comprising a base 903 and control handle 905 configured to pivot within the base 903 at a pivot point 904. The base 903 includes a first magnet 126a having a first orientation where the magnetic moment of the magnet 126a is parallel to a surface 804 (not shown) on which the joy stick input device 902a resides. The first magnet 126a is located beneath the pivot point 904 of the control handle 905. The control handle 905 has a second magnet 126b having a second orientation that is parallel to an axis of an at rest position of the control handle (i.e., where the moveable portion is at rest when not being held by a user), where the distance between the second magnet 126b and a pivot point 904 is known, the distance between the first magnet 126a and the pivot point 904 is known, and the at rest angle of the control handle 905 is known. Thus an at rest absolute location and orientation of the control handle 905 can be determined and then used to determine the real time absolute location and orientation of the control handle 905 during operation.

(38) FIG. 10E depicts a joy stick input device 902d that is similar to the joy stick input device 902c except the control handle 905 includes two magnets 126b 126c having an alternating polarity quadrature pole orientation. FIG. 10F depicts a joy stick input device 902e that is similar to the joy stick input device 902d except the control handle 905 includes three magnets 126b 126c 126d having polarity orientations corresponding to a Barker 3 code.

(39) One skilled in the art will recognize that all sorts of non-alternating coded magnet patterns can be employed including other one-dimensional arrays (e.g., Barker 4, Barker 5, etc.), two-dimensional arrays, and three-dimensional arrays where the magnets can have the same shapes, sizes and field strengths or could have different combinations of shapes, sizes, and field strengths. Moreover, multi-pole printed magnetic structures can be used. Alternatively, the magnets could be electromagnets or electro-permanent magnets enabling them to be switched on and off, their coding varied, or their magnetic fields to be otherwise varied (e.g., field strength) in accordance with a modulation pattern that can be demodulated as a form of communication whereby wave theory and modulation are applied to magnetometers. For example, magnetic properties could be varied in time as a form of modulation.

(40) Generally, coded patterns of conventional magnets or modulating electromagnets or electro-permanent magnets can be used to provide differentiation from individual magnets that are present in an environment in which the first and second devices are present. As such, a first device can identify and authenticate magnets, electromagnets, or electro-permanent magnets associated with a second device to which the first device desires to interface for position tracking or communications purposes. Coded magnetic structures are described in U.S. Pat. No. 8,179,219, the contents of which are hereby incorporated herein by reference. One skilled in the art will understand that an alternating polarity magnetic field is a uniformly alternating polarity magnetic field, whereas a coded polarity magnetic field is not uniformly alternating, and that one can implement a non-alternating polarity code such as a Barker 4 code (+++) with different sized alternating polarity magnets that produce a non-uniformly alternating (or coded) polarity magnetic field.

(41) FIGS. 11A-11K presents exemplary second devices in accordance with the invention that may comprise one or more magnets that can be detected by one or more vector magnetic sensors of one or more first devices 100. A glove 1102 is shown having magnets 126 in the fingers and in the palm of the glove (see FIG. 11A). A golf club 1104 includes two magnets 126 in the head of the club 1104 (see FIG. 11B). A tool 1106 includes two magnets 126 (see FIG. 11C). A pet collar 1108 includes a magnet 126 (see FIG. 11D). A game controller 1110 includes two magnets 126 (see FIG. 11E). A vending machine 1112 includes a magnet 126 (see FIG. 11F). A vehicle 1114 includes a magnet 126 (see FIG. 11G). A gas pump 1116 includes a magnet 126 (see FIG. 11H). A cash register 1118 at a point of sale includes a magnet 126 (see FIG. 11I). An automated teller machine 1110 includes a magnet 126 (see FIG. 11J). Even a first device 100 can include a magnet 126 so it can be treated as a second device by another first device 100 (see FIG. 11K). Generally, one skilled in the art will understand that in accordance with the invention one or more magnets can be associated with most any object and used for providing high resolution positional input relating to the object (or second device) to a first device having one or more magnetic sensors.

(42) The present invention uses vector data corresponding to the absolute orientation and location of a second device relative to the absolute orientation and location of a first device to calculate the motion of the second device (or the first device) over time. In order to accomplish motion calculations, a calibration process is required where the orientation (e.g., 0 degrees from a plane horizontal to the ground and facing in the X direction) and location (e.g., 0, 0, 0) of the first device within a coordinate system must be established and then the location(s) of the one or more magnets 126 in a second object relative to the orientation and location of the first device must be determined. Then, based on a priori knowledge of the arrangement of the one or more magnets 126 associated with the second device, the absolute orientation and location of the second device can be determined. The calibration process will typically involve moving the second device to locations within a predefined pattern (e.g., points on a square, rectangle, circle, figure eight, etc.) where the second device may be some distance away from (i.e., external to) the first device or the second device may be in contact with or near contact with the first device (e.g., using a display of the first device and locations thereon where the second device is used to draw something, trace something, or identify multiple points on the device). Alternatively, the calibration process could involve moving the first device relative to the second device where the location and orientation of the second device is fixed. The calibration process might involve leaving the first device fixed and moving the second device and then leaving the second device fixed and moving the first. The first device may also include an accelerometer where it can determine whether or not it is moving and can calibrate and re-calibrate motion calculations accordingly (e.g., re-calibrate when it recognizes it is stationary). The system may also recognize conditions whereby it requires a re-calibration process to be performed, for example, it may re-calibrate periodically based on a timing schedule or it may re-calibrate because of the occurrence of an event (e.g., a threshold being met, a time limit being surpassed, a measured value being outside an acceptable range, etc.).

(43) Calibration of a system of the invention may involve determining the orientation and location of the first device relative to one or more magnets associated with one or more second devices located at reference locations within an environment. The one or more reference locations may be associated with a stationary object such as the vending machine 1012, gas pump 1016, cash register 1018, or automated teller machine 1020 of FIG. 10. The magnetic field(s) of the one or more magnets located at a given reference location may be modulated to function as a beacon signal that might, for example, identify the reference location by an identifier or provide the coordinates (e.g., latitude, longitude, altitude) of the reference location within an established coordinate system. Generally, an established modulation method and protocol can be employed such that information can be conveyed to the first device by the one or more magnets at one or more reference locations to enable the first device to determine its position within an environment. One skilled in the art of positioning systems will understand that the number of reference locations interfacing with a first device determines the extent to which the first device can resolve ambiguities to determine its two-dimensional or three-dimensional location, which could be at a point, at one of a plurality of possible points, within an area, or within a volume.

(44) Moreover, a first device may move about within an environment whereby the second device(s) with which the first device interfaces varies. Various techniques such as measured magnetic field strength may be used to select among available second devices to be used to determine a location.

(45) Measurements of a vector and local gradient of the magnetic field(s) associated with a magnet(s) of a second device are not required given a priori knowledge of the shape and field strength of the magnetic field(s) of the magnet(s) associated with the second device. Without such a priori knowledge, the vector and local gradient of the magnetic field of a magnet(s) associated with a second device can be measured using the vector data of the one or more sensors of the first device.

(46) The locations of the first device and second device can be determined relative to a location corresponding to location information provided by one or more location information systems such as a Global Positioning System, a Wi-Fi position tracking system, or an Ultra Wideband positioning system.

(47) The movement of a vehicle in which the first device resides, movement of a person holding the first device, or the movement of any other moving object to which the first device is associated with can be determined using the accelerometer capabilities of the first device.

(48) When a second device includes a coded magnetic array such as the Barker 3 array shown in FIG. 10F, multiple arrays of vector magnetic sensors can be used to determine the location and orientation of the second device. Generally, the more complex the coded array, which may be a one-dimensional array, two-dimensional array, or three-dimensional array, the more sensors and computations may need to be applied to resolve ambiguities.

(49) The second device can be a tool (e.g., a scalpel used by a surgeon or even a robot). The second device can be a robotic hand or a finger of a robotic hand.

(50) The vector magnetic sensor array of the first device can track the orientation of a plurality of second devices (e.g., multiple fingers of a robotic hand or the fingers of a glove worn by a person).

(51) The first device can also track orientation of multiple objects such as multiple game pieces near the device (e.g., pieces of a chess game on a game board near a PDA).

(52) Control signals can be conveyed from the first device to the second device to control the movement of the second device (e.g., a feedback control system), where the second device is moved, tracked by the first device, and the first device sends data back to the second device concerning its movement to include new movement instructions.

(53) Alternatively, the second device can be in a fixed location/orientation and the first device can determine its own movement relative to the location/orientation of the second device.

(54) Under one arrangement, a plurality of first devices can be coordinated (e.g., 2androids providing 2 look angles) to determine information pertaining to a second device.

(55) An authentication scenario for a security door access control system could be as follows: A person walks up to a security door. The door has a unique id (like an ip address). The security door has a modulating magnetic source that emits the unique ID of the door. Modulation could be constant (beacon) or it could be strobed based on the door recognizing presence of the phone/person/etc., where it could use any detection method such as radar, IR, Bluetooth, etc. to detect the phone/person/etc. The phone detects the door (emission), takes the door ID and combines it with its own ID and sends a packet to a server via phone communications. The server sends the door a validation code that the door uses to produce a validation emission that the phone then sends back to the server to verify proximity to the correct door. The door knows to open.

(56) With such an authentication approach, most any transaction can be authenticated via ones cellphone.

(57) With a network of modulating magnetic sources (beacons) at known locations within a building, a phone can determine where it's at inside the building as it is moved, for example by a person, about the building.

(58) The beacons would emit their locations (e.g., latitude/longitude/altitude) or provide an identifier that the phone could use with a location look up table.

(59) One of the things that can be made available to the phone is a map of a facility or a home identifying where beacons are in the facility. New beacons can be added and discovered and removed and determined.

(60) If the phone has a compass and an accelerometer, they can be used in combination with the magnetometer to provide information used to interpolate and extrapolate in between beacons.

(61) The phone can verify an environment based on a priori knowledge of the beacon supposedly present and can determine if a beacon is no longer present (for replacement purposes).

(62) Different types of beacons can have different magnetic characteristics (e.g., different throw, different amplitude, different directionality, different coding). Information about the type of beacon (determined based on magnetic characteristics) can provide more information about location, authentication, allow for efficiencies of operation, etc. For example, coils used with electromagnets can be small or very big.

(63) Phones can receive information from RF sources, barcodes, and magnetic stripes.

(64) Two devices each having a magnetometer and a modulating magnetic source can have two-way communications.

(65) Using feedback control, the second device can receive position/motion control information via a wireless link from a first device tracking the position of the second device, which enables the second device to be dumb.

(66) While particular embodiments of the invention have been described, it will be understood, however, that the invention is not limited thereto, since modifications may be made by those skilled in the art, particularly in light of the foregoing teachings.