Virtual reality environment generating apparatus and controller apparatus
11325029 · 2022-05-10
Assignee
- National Institute Of Advanced Industrial Science And Technology (Tokyo, JP)
- Murata Manufacturing Co., Ltd. (Kyoto, JP)
Inventors
Cpc classification
A63F13/218
HUMAN NECESSITIES
A63F13/285
HUMAN NECESSITIES
G06F3/011
PHYSICS
A63F2300/105
HUMAN NECESSITIES
A63F2300/1043
HUMAN NECESSITIES
A63F2300/1037
HUMAN NECESSITIES
A63F13/211
HUMAN NECESSITIES
G06F3/016
PHYSICS
International classification
G08B6/00
PHYSICS
A63F13/285
HUMAN NECESSITIES
A63F13/211
HUMAN NECESSITIES
A63F13/218
HUMAN NECESSITIES
Abstract
The present invention provides a virtual reality environment generating apparatus including a non-base type interface configured to allow a user to haptically touch virtual objects and game characters. The virtual reality environment generating apparatus includes a content creating device 102 configured to create a content based on information from various sensors 108 to 111 and content data, an illusionary tactile force sense evoking device 103 configured to use illusionary tactile force sense data to generate an illusionary tactile force sense evoking function 1713 adapted for the content, an illusionary tactile force sense interface device 101 including a illusionary tactile force sense device 107, and an illusionary tactile force sense device driving control device 112 configured to drivingly control the illusionary tactile force sense device. The virtual reality environment generating apparatus utilizes an illusionary tactile force sense to control reaction based on the illusionary tactile force sense, in accordance with motion of the user's finger and body. The virtual reality environment generating apparatus thus expresses not only three-dimensional videos and sound images but also senses of friction and roughness corresponding to the presence, shape, and texture of a virtual object.
Claims
1. An information processing method comprising: evoking an illusionary tactile three sensation based on at least one of an acceleration pattern, a control pattern, physical quantity pattern, or a control algorithm, each of which being a change in a physical quantity generated by driving data comprising information representing the physical quantity; wherein: the illusionary tactile force sensation is evoked by the change of the physical quantity, wherein sensation of forces that is different from forces generated by the changes presented is generated using illusion with a nonlinear sensory characteristic, the sensation of forces is generated by an illusionary tactile force sense interface device including at least one actuator that temporally varies momentum of the actuator, the driving data comprising information representing the physical quantity is obtained from the illusionary tactile force sense interface device itself, and in synchronization with a mechanism within the illusionary tactile force sense interface device changing a shape of the illusionary tactile force sense interface device, the illusionary tactile force sense interface device presents sensation of force in a first direction that is opposite to a second direction of an intermittent force included in the evoked illusionary tactile force sensation, the presenting the sensation of force in the first direction includes presenting a shape, a sense of friction and a sense of roughness of a virtual object, and the sensation of force in the first direction is continuously presented at higher acceleration than acceleration of the sensation of force in the second direction.
2. The information processing method according to claim 1, wherein the evoked illusionary tactile force sensation has a nonlinear sensory characteristic in a relationship between the physical quantity and the sensory characteristic.
3. The information processing method according to claim 1, wherein the evoked sensation includes at least one of a tactile force sense, an illusionary tactile force sensation, an auditory sense, a sound image, or a video.
4. The information processing method according to claim 1, wherein the physical quantity includes at least one of an amplitude, displacement, velocity, acceleration, force, torque, momentum, stimulus, vibration, rotation, pressure, voltage, sound, sound image, video, angle, manipulation information, or an amount of change thereof.
5. The information processing method according to claim 1, wherein the evoking of the illusionary tactile force sensation is obtained based on at least one of a plurality of different operation points, a plurality of different durations, or a plurality of different frequencies on the nonlinear characteristic.
6. The information processing method according to claim 1, wherein the evoked illusionary tactile force sensation presents at least one of a sense of force, a sense of torque, an illusionary sense of force, or an illusionary sense of torque by using a nonlinear characteristic, even when an integral of physical quantities is zero.
7. The information processing method according to claim 1, wherein the evoked illusionary tactile force sensation has a non-zero sensory quantity by using a nonlinear characteristic, even when an integral of physical quantities is zero.
8. The information processing method according to claim 1, wherein the evoked illusionary tactile force sensation presents at least one of a sensation or an illusion that is different from a physically applied physical quantity, force, or motion, and that does not exist physically.
9. The information processing method according to claim 1, wherein the evoked illusionary tactile force sensation presents at least one of a sensation of force or a sensation of a motion that is different from a physically applied force or motion, and that does not exist physically.
10. The information processing method according to claim 1, wherein the evoked illusionary tactile force sensation controls at least one of resistance, friction, or a sensation based on the illusionary tactile force sensation in accordance with at least one of a motion of a finger or a body.
11. The information processing method according to claim 1, wherein the nonlinear sensory characteristic includes a plurality of different operation points including a first operation point and a second operation point.
12. The information processing method according to claim 1, wherein the evoking is generated or controlled based on an illusionary tactile force sense evoking function.
13. The information processing method according to claim 1, wherein: the evoked illusionary tactile force sensation causes a physical reaction via an object, and the object includes at least one of an illusionary tactile force sense device, a tactile force sense device, a real object, a display member, or a virtual object.
14. The information processing method according to claim 1, wherein the at least one of the acceleration pattern, the control pattern, the physical quantity pattern, or the control algorithm is created or controlled based on an illusionary tactile force sense evoking function that evokes an illusionary tactile force sensation.
15. The information processing method according to claim 1, wherein the at least one of the acceleration pattern, the control pattern, the physical quantity pattern, or the control algorithm changes at least one of vibration, force, voltage, electricity, sound, sound image, video, or momentum.
16. The information processing method according to claim 1, wherein the control algorithm is created or controlled based on at least one of a viscoelastic material property, a hysteresis material property, an oscillation circuit property, a stress property, or property of a material having a nonlinear shear property.
17. The information processing method according to claim 1, wherein the at least one of the acceleration pattern, the control pattern, the physical quantity pattern, or the control algorithm is stored in a memory mounted in an illusionary tactile force sense interface device, and is read from the memory by a drive of a CPU.
18. The information processing method according to claim 1, wherein: the data includes content data, and an illusionary tactile force sense evoking function is created based on the content data related to an illusionary tactile force sensation or a tactile force sensation.
19. The information processing method according to claim 1, wherein the at least one of the acceleration pattern, the control pattern, or the physical quantity pattern are created or controlled based on at least one of an elastic material property, a spring property, a capacitor property, a coil property, a resistor property, a viscoelastic material property, an oscillation circuit property, a stress property, or a shear property.
20. The information processing method according to claim 1, wherein the control algorithm creates or controls the at least one of the acceleration pattern, the control pattern, or the physical quantity pattern based on at least one of an elastic material property, a spring property, a coil property, a resistor property, a capacitor property, a viscoelastic material property, an oscillation circuit, a stress property, or a shear property.
21. The information processing method according to claim 1, wherein the evoked sensation is obtained by a change of at least one of a shape, an arrangement, a position, a direction, a phase, or a place of at least one of an illusionary tactile force sense interface device, a tactile force sense interface device, an illusionary tactile force sense device, a tactile force sense device, a real object, a display member, or a virtual object.
22. The information processing method according to claim 1, wherein the data is obtained by a communication.
23. The information processing method according to claim 1, wherein a sensation of a real object is added to a sensation presented by the illusionary tactile force sensation.
24. The information processing method according to claim 1, wherein: the data is fed back, and a motor, or an actuator, or a device, or an interface that presents illusionary tactile force sense information is controlled or driven based on the fed-back data.
25. The information processing method according to claim 1, wherein the sensation of forces comprises at least one of forces or components of motion that do not exist physically.
26. The information processing method according to claim 1, wherein the sensation of forces is generated without a force component that acts in a fixed direction.
27. A sense presenting method comprising: evoking an illusionary tactile force sensation based on at least one of an acceleration pattern, a control pattern, a physical quantity pattern, or a control algorithm, each of which being a change in a physical quantity generated by driving data comprising information representing the physical quantity, wherein: the illusionary tactile force sensation is evoked by the physical quantity, wherein sensation of forces that is different from forces generated by the changes presented is generated using illusion with a nonlinear sensory characteristic, the sensation of forces is generated by an illusionary tactile force sense interface device including at least one actuator that temporally varies momentum of the actuator, the driving data comprising information representing the physical quantity is obtained from the illusionary tactile force sense interface device itself, and in synchronization with a mechanism within the illusionary tactile force sense interface device changing a shape of the illusionary tactile force sense interface device, the illusionary tactile force sense interface device presents sensation of force in a first direction that is opposite to a second direction of an intermittent force included in the evoked illusionary tactile force sensation, the presenting the sensation of force in the first direction includes presenting a shape, a sense of friction and a sense of roughness of a virtual object, and the sensation of force in the first direction is continuously presented at higher acceleration than acceleration of the sensation of force in the second direction.
28. The sense presenting method according to claim 27, wherein the evoked illusionary tactile force sensation has a nonlinear sensory characteristic in a relationship between the physical quantity and the sensory characteristic.
29. The sense presenting method according to claim 27, wherein the evoked sensation includes at least one of a tactile force sense, an illusionary tactile force sensation, an auditory sense, a sound image, or a video.
30. The sense presenting method according to claim 27, wherein the physical quantity includes at least one of an amplitude, displacement, velocity, acceleration, force, torque, momentum, stimulus, vibration, rotation, pressure, sound, sound image, video, voltage, angle, manipulation information, or an amount of change thereof.
31. The sense presenting method according to claim 27, wherein the evoking of the illusionary tactile force sensation is obtained based on at least one of a plurality of operation points, a plurality of durations, or a plurality of frequencies on the nonlinear characteristic.
32. The sense presenting method according to claim 27, wherein the evoked illusionary tactile force sensation presents at least one of a sense of force or a sense of torque by using a nonlinear characteristic, even when an integral of physical quantities is zero.
33. The sense presenting method according to claim 27, wherein the evoked illusionary tactile force sensation has a non-zero sensory quantity by using a nonlinear characteristic, even when an integral of physical quantities is zero.
34. The sense presenting method according to claim 27, wherein the evoked illusionary tactile force sensation presents at least one of a sensation or an illusion that is different from a physically applied physical quantity, force, or motion, and that does not exist physically.
35. The sense presenting method according to claim 27, wherein the evoked illusionary tactile force sensation presents at least one of a sensation of force or a sensation of a motion that is different from a physically applied force or motion, and that does not exist physically.
36. The sense presenting method according to claim 27, wherein the evoked illusionary tactile force sensation controls at least one of resistance, friction, or a sensation based on the illusionary tactile force sensation in accordance with a motion of a finger or a body.
37. The sense presenting method according to claim 27, wherein the nonlinear sensory characteristic includes a plurality of different operation points including a first operation point and a second operation point.
38. The sense presenting method according to claim 27, wherein the evoking is generated or controlled based on an illusionary tactile force sense evoking function.
39. The sense presenting method according to claim 27, wherein: the evoked illusionary tactile force sensation causes a physical reaction via an object, and the object includes at least one of an illusionary tactile force sense device, a tactile force sense device, a real object, a display member, or a virtual object.
40. The sense presenting method according to claim 27, wherein the at least one of the acceleration pattern, the control pattern, the physical quantity pattern, or the control algorithm are created or controlled based on an illusionary tactile force sense evoking function that evokes an illusionary tactile force sensation.
41. The sense presenting method according to claim 27, wherein the at least one of the acceleration pattern, the control pattern, the physical quantity pattern, or the control algorithm changes at least one of vibration, force, voltage, electricity, sound, sound image, video, or momentum.
42. The sense presenting method according to claim 27, wherein the control algorithm is created or controlled based on at least one of a viscoelastic material property, a hysteresis material property, an oscillation circuit property, a stress property in response to an applied force, or property of a material having a nonlinear shear property.
43. The sense presenting method according to claim 27, wherein the at least one of the acceleration pattern, the control pattern, the physical quantity pattern, or the control algorithm are stored in a memory mounted in the illusionary tactile force sense interface device, and is read from the memory by a drive of a CPU.
44. The sense presenting method according to claim 27, wherein the data includes content data, and an illusionary tactile force sense evoking function is created based on the content data related to an illusionary tactile force sensation or a tactile force sense.
45. The sense presenting method according to claim 27, wherein the at least one of the acceleration pattern, the control pattern, or the physical quantity pattern are created or controlled based on at least one of an elastic material property, a spring property, a capacitor property, a coil property, a resistor property, a viscoelastic material property, an oscillation circuit property, a stress property, or a shear property.
46. The sense presenting method according to claim 27, wherein the control algorithm creates or controls the at least one of the acceleration pattern, the control pattern, or the physical quantity pattern based on at least one of an elastic material property, a spring property, a coil property, a resistor property, a capacitor property, a viscoelastic material property, an oscillation circuit, a stress property, or a shear property.
47. The sense presenting method according to claim 27, wherein the evoked sensation is obtained by a change of at least one of a shape, an arrangement, a position, a direction, a phase, a place of an illusionary tactile force sense interface device, a tactile force sense interface device, an illusionary tactile force sense device, a tactile force sense device, a real object, a display member, or a virtual object.
48. The sense presenting method according to claim 27, wherein the data is obtained by a communication.
49. The sense presenting method according to claim 27, wherein a sensation of a real object is added to a sensation presented by the illusionary tactile force sensation.
50. The sense presenting method according to claim 27, wherein: the data is fed back, and a motor, or an actuator, or a device, or an interface that presents illusionary tactile force sense information is controlled or driven based on the fed-back data.
51. The sense presenting method according to claim 27, wherein the sensation of forces comprises at least one of forces or components of motion that do not exist physically.
52. The sense presenting method according to claim 27, wherein the sensation of forces is generated without a force component that acts in a fixed direction.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
(23)
(24)
(25)
(26)
(27)
(28)
(29)
(30)
(31)
(32)
(33)
(34)
(35)
(36)
(37)
(38)
(39)
(40)
(41)
(42)
(43)
(44)
(45)
(46)
DESCRIPTION OF SYMBOLS
(47) 101 Illusionary tactile force sense interface device 102 Content creating device 103 Illusionary tactile force sense evoking device 104 Content data 105 Audiovisual display 106 Tactile force sense data and illusionary tactile force sense data 107 Illusionary tactile force sense device 107a Illusionary tactile force sense device 107b Illusionary tactile force sense device 108 Acceleration sensor 109 Pressure sensor 110 Myoelectric sensor 111 Position sensor 112 Control device 113 Physical simulator 114 Computer graphics 115 Illusionary tactile force sense evoking function generator 116 Learner 117 Corrector 118 Peripheral device 119 Sound source simulator 205 Communicator 520 Virtual object (physical model) 528 Spring and damper physical model 531 Virtual object 533 Finger 535 Stress exerted by virtual object 814 Eccentric weight 815 Eccentric motor 901 Physical phenomenon 902 Nonlinear sensory characteristics 903 Psychological phenomenon 904 Lateral vibration 905 Illusionary tactile force sense 908 Integral quantity of force sense during duration Ta of operation point A 909 Integral quantity of force sense during duration Tb of operation point B 1002a High rotation number ω1 1002b Low rotation number ω2 1100 Virtual plate 1101 Motion of virtual object 1102 Resistance against movement 1103 Resistance from virtual plate 1104 Frictional force 1105 Viscous resistance 1106 Resistance that acts to push back virtual plate into surface 1107 Thickness error of virtual plate 1108 Senses of resistance and viscosity 1109 Sense of friction 1110 Sense of smoothness and acceleration 1111 Sense of roughness 1112 Vibrational resistance 1113 Acceleration force (negative resistance) 1201 Initial phase θi 1202 Sense of lifting based on illusionary tactile force sense 1203 Sense of lifting based on illusionary tactile force sense 1204 Sense of gravity based on illusionary tactile force sense 1301 Adhesion tape 1302 Housing 1303 Finger insertion section 1403 Shape transformable material 1404 Viscoelastic material 1405 Antiseismic material 1406 Display 1702 Corrector 1703 Motor controller 1704 Motor 1705 Encoder 1706 Viscoelastic characteristics controller 1707 Viscoelastic material 1713 Illusionary tactile force sense evoking function 1714 Correction data 1725 Physical strength 15 dB 1901 Motor FB characteristics controller 1902 Control signal generator 2206 Threshold 2302 Weight 2303 Stretchable material 2400 Constant-velocity rotation 2403 Viscoelastic material A 2404 Viscoelastic material B 2601 Hysteresis material A 2602 Hysteresis material B 3001 Shape of illusionary tactile force sense interface device 3002 Shape transforming motor 3003 Flexible transformable material 3101 Virtual controller 3102 Virtual button 3202 Displacement of center of gravity (phase difference 0°) 3203 Displacement of center of gravity (phase difference 180°)
BEST MODE FOR CARRYING OUT THE INVENTION
(48) Embodiments of the present invention will be described below with reference to the drawings.
(49)
(50) Contents are created based on information from the sensors and content data 104. An illusionary tactile force sense evoking function 1713 corresponding to the content is then generated by an illusionary tactile force sense evoking function generator 115 using illusionary tactile force sense data 106. An illusionary tactile force sense device driving control device 112 controls the illusionary tactile force sense device 107.
(51) The phase, direction, and rotating velocity of eccentric motors 815 of the illusionary tactile force sense device 107 are controlled in accordance with the illusionary tactile force sense evoking function generated by the illusionary tactile force sense evoking device 115. An illusion for a tactile force sense (illusionary tactile force sense) is evoked by a change (acceleration and deceleration pattern) in momentum generated by rotation of eccentric weights 814 caused by the eccentric motors in the illusionary tactile force sense device 107. The use of the illusionary tactile force sense evoking function enables a sense different from a presented force (physical information) generated by a change in momentum to be perceived utilizing the illusion, which is nonlinear sensory characteristics. That is, a force and a motion component which do not exist physically can be perceived. For example, periodically repeated vibration physically has no force information on only a fixed direction owing to a periodic variation in the direction of the force. However, by controlling the acceleration and deceleration pattern of the momentum in accordance with the illusionary tactile force sense evoking function, a continuous force in only one direction can be psychophysically perceived based on an illusionary tactile force sense. The illusionary tactile force sense evoking device 103 includes a learner 116 and a corrector 117 to enable optimization in accordance with users' individual characteristics.
(52) The motion of the fingertip 533 at which the illusionary tactile force sense interface is installed is sensed by a position sensor 111 and an acceleration sensor 108. Position, velocity, and acceleration information obtained by the position and acceleration sensors is used to determine whether or not the fingertip 533 has touched a virtual object generated by a physical simulator 113 in a content creating device 102 and to calculate a force that acts on the virtual object. Furthermore, a contact and a grip force exerted between a real object and a user are sensed by the pressure sensor 109 and the myoelectric sensor 110. The content is formed into videos and sound images by a computer graphics 114 and a sound source simulator 119. The videos and sound images are displayed on an audiovisual display 105. Thus, in connection with virtual reality, conventionally based only on the visual and auditory senses, a practical non-base type illusionary tactile force sense interface is provided. As a result, a virtual reality environment is provided which is based on the visual, auditory, and tactile senses and which can be utilized in everyday life.
(53) Instead of the content creating device, simulation data from a physical simulator in another device (for example, a conventional game machine) may be used.
(54) Alternatively, the user may manually set physical parameters so as to controllably utilize the illusionary tactile force sense evoking device 115.
(55) Furthermore, control is generally performed via the content generating device, which is a central section. However, when CPUs and memories mounted in the illusionary tactile force sense interface device, which is a peripheral section, is used to perform real-time control without using the content generating device and the illusionary tactile force sense evoking device 115, responsiveness in depression of the virtual button and the like, reality, and manipulability are improved. The illusionary tactile force sense interface device can be connected to and utilized with conventional devices.
(56) The use of the illusionary tactile force sense interface device 101 allows conventional information on the tactile force sense to be presented.
(57) The transmission of information and/or the connection between instruments such as devices, peripheral devices, databases, and sensors may be wired or wireless.
(58)
(59) The calibration is performed by the illusionary tactile force sense interface device and the illusionary tactile force sense evoking device. The sensing is performed by the illusionary tactile force sense interface device (acceleration sensor, pressure sensor, and myoelectric sensor) and the position sensor. The content generation is performed by the content generating device. The presentation is performed by the illusionary tactile force sense interface device and the illusionary tactile force sense evoking device.
(60) Information is exchanged with the real space through the peripheral device 118. Thus, a virtual reality environment is generated and controlled which a combination of the virtual space and the real space are handled.
(61)
(62)
(63) In each calibration flow, calibration is performed as follows. A calibration signal is generated. In accordance with the calibration signal, the sensors, the tactile force sense interface, and the illusionary tactile force sense interface device 101 are controlled, with the results of the control sensed.
(64)
(65) A brain wave sensor serving as a biosignal sensor forming a part of the illusionary tactile force sense interface device is installed on the head. At the same time, and the illusionary tactile force sense device, forming a part of the illusionary tactile force sense interface device, is installed on the fingertip.
(66)
(67)
(68) When a grid point p1 is coupled to adjacent grid points p2 to p4, a force vector f12 from the grid point p2 to which grid point p1 is subjected is expressed by:
f12=−k×(∥p2−p1∥−L.sub.0)×(p2−p1)/∥p2−p1μ−c×(v2−v1) (1).
where
(69) pi: position vector of a grid point pi,
(70) vi: velocity vector of a grid point pi,
(71) k: Elastic modulus of a spring,
(72) c: Viscosity coefficient of a damper, and
(73) L.sub.0: Length of the spring in a balanced state.
(74) When the resultant force of the forces exerted on the grid point p1 with a mass m1 by the surrounding grip points p2 to p4 is defined as f1, an equation of motion for the grid point p1 is expressed by:
m1×d.sup.2p1/dt.sup.2=f1=f12+f13+f14 (2).
(75) If the fingertip 533 at which the illusionary tactile force sense interface device 101 is installed touches the grip point p1 in this virtual object physical model 520, the grip point p1 changes to the position p′1 of the fingertip. The reaction force (−f) which acts on the fingertip is expressed by:
−f=(f12+f13+f14)−m1×d.sup.2p′1/dt.sup.2 (3).
The motion of the fingertip 533, based on which whether or not the fingertip 533 has touched a virtual object is determined, is sensed by the position sensor 111 and the acceleration sensor 108.
(76) In the actual numerical simulation, the position p′1, velocity v′1, and force f′1 of the grid point p1 at a point in time t′ are determined from variables p1, v1, and f1 at the preceding point in time t.
(77) That is, the following hold true.
Velocity vector:v′1=v1+(f1/m1)×Δt (4)
Position vector:p′1=p1+v1×Δt (5).
(78) Similarly, the position and velocity of p2 with a mass m2 are calculated.
Velocity vector:v′2=v2+(f2/m2)×Δt (6)
Position vector:p′2=p2+v2×Δt (7).
(79) Finally, the force vector acting between the grip points p1 and p2 is calculated by:
f12=−k×(∥p′2−p′1∥−L.sub.0)×(p′2−p′1)/∥p′2−p′1∥−c×(v′2−v′1) (8).
(80) In every calculation, the position, velocity, and force of each grid point are calculated and saved to the memory. The saved values are used to calculate the position, velocity, and force at the succeeding point in time. As a result, the reaction force exerted on the fingertip 533 is presented. This allows the virtual object expressed as a three-dimensional sound image and a three-dimensional video on the audiovisual display to be touched.
(81) In the VR environment, as in the case of the above-described physical simulation for the virtual object 531, based on information on the motion of the real object in the real space sensed by the peripheral device and the user's motion sensed by the position sensor 111 and the acceleration sensor 108, the real object and the user are modeled in the same VR environment. A contact force and a grip force exerted between the content and the user are then calculated. Thus, a VR space is generated into which the virtual space and the real space are merged.
(82)
(83) Hence, the user can transform and move the virtual object 531 while feeling, for example, such a sense of elasticity as felt when rubber is transformed or such a sense of viscosity as felt when slime is stretched; the senses of elasticity and viscosity indicate the material of the virtual object 531.
(84)
(85) The content data 104 on the illusionary tactile force sense and tactile force sense in the VR space created by the content creating device 102 is loaded. An illusionary tactile force sense function and a tactile force sense function are generated and corrected by the corrector 117 in accordance with the information obtained by sensing and the user's characteristics. In accordance with the functions, the illusionary tactile force sense device 107 is feedback-controlled.
(86) The illusionary tactile force sense interface device utilizes illusions. Thus, the sensitivity for the illusionary tactile force sense and improvement of the sensitivity through learning vary greatly among individual users. Hence, even when the same stimulus is presented, the intensity of feeling varies depending on the user. Consequently, in order to allow the user to perceive the stimulus with the same intensity without depending on the user, the stimulus needs to be learned and corrected.
(87)
(88) Correction data 1714 indicative of individual differences is calculated and saved to the memory or a user characteristics database.
(89) Specifically, in the active learning, after the illusionary tactile force sense interface device 101 is installed, a given force based on the illusionary tactile force sense is sequentially presented in the directions of 0°, 180°, 90°, and 270° according to the instructions. Unlike in the case of the tactile force sense, presenting the illusionary tactile force sense with the presentation direction discreetly varied allows the user to get familiar with and learn the illusionary tactile force sense. As the presentation time elapses, a threshold decreases and sense sensitivity increases, thus making the direction of the force clearer. After one minute of learning, the intensity of the illusionary tactile force sense is gradually reduced. The intensity at which perception has failed is estimated to be a sense threshold for the illusionary tactile force sense. The sense threshold varies depending on the presentation direction and the user and is saved to the memory or the database as correction data required to correct individual static characteristics. As the learning progresses, the threshold converges to a given value corresponding to the illusionary tactile force sense sensory characteristics. The level of learning is determined from a convergence time constant corresponding to the rate of the convergence. Then, an equal-sensory-level contour is determined by a psychophysical paired comparison method.
(90) Similarly, in the unconscious learning method, the user's reaction and action to the illusionary tactile force sense information in each content is sensed to measure the user's illusionary tactile force sense sensory characteristics for a feature quantity (illusionary tactile force sensory intensity and time pattern) relating to the illusionary tactile force sense information in the content. The response characteristics of each user are saved to the memory or the database as individual-difference correcting data.
(91) As described above, the illusionary tactile force sense is a characteristic which varies greatly among individuals individual differences. However, learning and correction allow the illusionary tactile force sense interface device to be used to present the same stimulus intensity as that presented by the conventional tactile force sense interface device.
(92) The characteristics of the illusionary tactile force sense will be described below.
(93) The conventional tactile force sense interface device presents the fingertip or the hand with a force or motion physically representing a physical phenomenon relating to the tactile force sense so that the force or motion can be perceived. However, the present invention relates to a phenomenon in which a force or motion that is different from a physically applied force or motion or does not exist is perceived and recognized. For example, the interface feels lifted through the interface does not actually (physically) be lifted.
(94) The conventional tactile force sense interface device allows the user to feel an external force. Thus, a base which supports a reaction force exerted when a force is presented to the fingertip or the like is essential for the conventional tactile force sense interface device. In contrast, a tactile force sense interface device that utilizes a non-base type vibration motor without a base vibrates around the center of gravity, corresponding to a vibration balance point. The tactile force sense interface device with the non-base type vibration motor fails to allow the user to feel externally pushed. In contrast, the illusionary tactile force sense interface device 101 is of the non-base type but enables the user to feel externally pushed; the illusionary tactile force sense interface device 101 utilizes illusions to present the tactile force sense (Non-Patent Document 3).
(95) The illusionary tactile force sense not only allows illusionary senses to be perceived but also brings about a physical phenomenon in which the arm holding the interface is actually raised. This is because a phantom sensation resulting from an illusion causes the user to unconsciously move the hand or to move the arm muscle as result of reflex. In this regard, the illusionary tactile force sense interface device according to the present invention is greatly different from the conventional tactile force sense interface invented and developed so as to reproduce a physical force that acts between an object and the user's body. The present invention relates to a device configured to evoke tactile force sensory illusions, and to a device configured to effectively evoke an illusionary tactile force sense.
(96) Furthermore, the illusionary tactile force sense interface device 101 according to the present invention also provides the functions and effects of the conventional tactile force sense interface device and thus enables the synergetic effect of both presentation senses to be produced.
(97)
(98)
(99)
(100)
(101) Alternate acceleration and deceleration at the operation points A and B for every 180° of phase shift allow the user to continuously perceive the force sense 905 in a fixed direction. The force physically returns, in one cycle, to the initial condition in which the momentum and the integral value of the force are zero. That is, the force remains around the balance point, thus preventing the acceleration and deceleration mechanism from moving leftward. However, the sensory integral value of the force sense, which is a sensory quantity, is inhibited from being zero. At this time, the perception of the integral 908 of the force in the positive direction lowers, with only the integral 909 of the force in the negative direction perceived.
(102) Here, the time differentiation of the angular momentum corresponds to a torque, and the time differentiation of the momentum corresponds to a force. To allow a torque and a force to be continuously generated in a fixed direction, the rotating velocity of a motor needs to be continuously increased or a linear motor needs to be continuously accelerated. Thus, a method of periodically rotating a rotor or the like is unsuitable for continuously presenting the force sense in a fixed direction. In particular, it is physically impossible for non-base type interfaces utilized in mobile applications or the like to present a force in a fixed direction.
(103) However, since the users have nonlinear sensory characteristics, the technique according to the present invention allows a force and a force pattern inconsistent with physical properties to be illusionarily perceived by utilizing the perception sensitivity for the illusionary tactile force sense characteristics and controlling the acceleration and deceleration pattern of the momentum. For example, the sensitivity is the ratio of the magnitude of a felt stimulus to the intensity of an applied stimulus. In the user's sensory characteristics, the sensitivity varies depending on the intensity of an applied stimulus. The users are sensitive to weak stimuli and insensitive to strong stimuli. Thus, by controlling the phase of acceleration and deceleration of the motor rotation to periodically repeat acceleration and deceleration, the present invention has succeeded in presenting a continuous force sense in a direction in which a weak stimulus is presented. Furthermore, selecting the appropriate operation points A and B with the sensory characteristics also allows a continuous force sensor to be presented in a direction in which a strong stimulus is presented.
(104) An example of a similar device is a driving simulator. However, the driving simulator presents the sense of acceleration of a car by applying an intended force (sense of acceleration) and then slowly returning the simulator to the original position at an unnoticeable, low acceleration. Thus, the force is intermittently presented. Such an eccentric acceleration scheme fails to allow a force sense or a sense of acceleration in a fixed direction to be continuously presented. This also applies to the conventional tactile force sense interface device. However, in spite of the driving method 904 of continuously repeating acceleration and deceleration in the forward and backward direction above the sensory threshold at a short period of, for example, 50 Hz, the present invention utilizes illusions to present a sense of translational force 905 in a fixed direction. In particular, the illusionary tactile force sense interface device 101 that uses illusions is characterized in that a continuous force is perceived in the direction opposite to that of the intermittent force presented by the driving simulator according to the physical technique.
(105) That is, by utilizing the user's nonlinear sensor characteristics that the sensitivity varies depending on the intensity, the present invention prevents the integral of a force generated by periodic acceleration and deceleration or vibration from being haptically offset though the integral is physically zero bit also prevents a force in the positive direction 908 from being perceived. Hence, the force sense for a translation force 905 or a sense of torque can be continuously presented in an intended, negative direction 909 (see
(106) In
(107) Thus, the force sense during the interval of the rotation duration Tb relatively increases, improving the continuity of the force sense 905 in one direction. As a result, the force sense can be freely continuously presented in any direction by appropriately selecting the operation points A and B, appropriately setting the operation point A duration and the operation point B duration, and adjusting the synchronous phases of the two eccentric rotors A and B.
(108) The sensory characteristics vary depending with the user as shown in
(109)
(110) Furthermore, a vibration sensory intensity (VI) indicates the intensity of a vibration component perceived simultaneously with the perception of an illusionary force sense in a fixed direction. The intensity of the vibration component is approximately inversely proportional to the physical quantity (logarithm).
(111) Increasing the frequency (f) relatively reduces the vibration sensory intensity (VI) (
(112)
(113)
(114)
(115)
(116)
(117) The illusionary tactile force sense device 107 can change the initial phase (θi) at the start of rotation in
(118) In this case, if the illusionary tactile force sense interface device 101 itself is heavy, an upward force sense 1202 resulting from the illusionary tactile force and a downward force sense 1204 resulting from the gravity offset each other. In this case, an illusionary tactile force sense 1203 is evoked such that the upward direction thereof resulting from the illusionary tactile force sense is slightly misaligned with the direction opposite to that of gravity. Consequently, a sense of lifted can be inhibited from being reduced or hindered.
(119) To be presented in the direction opposite to the direction of gravitational force, the illusionary tactile force sense may be alternately evoked in directions slightly misaligned with the vertical one and corresponding to angles of 180°+α° and 180°−α°.
(120)
(121) As shown in
(122) The terms “viscoelastic material” and “viscoelastic characteristics” refer to viscosity and/or elasticity.
(123)
(124) In
(125) In
(126)
(127) In
(128) Furthermore, with variable touch buttons with commands on a touch panel varying depending on usage and a context, the following may occur. In particular, for example, when the user attempts to depress any button and if the button is hidden under the finger, as in the case of a cellular phone, the variable command on the button may be hidden and may not be read. Similarly, upon attempting to depress any variable button in a virtual space for VR contents, the user may fail to check the command contents of the button since menu description and commands vary depending on the context. Thus, as shown in
(129) To allow the user to manipulate the virtual controller with information on the depression of the virtual object 531 or the virtual buttons in the virtual controller and with a depression reaction force without feeing wrongness, as in the case of the real object, the time delay between the depression and the presentation of the depression reaction force is to be solved. For example, in an arm type grounding force sense interface, the position of the gripping finger is measured based on the angle of the arm or the like, and the stress to be presented is calculated. Then, the rotation of the motor is controlled, and the motion and stress of the arm are presented. Hence, a response delay may occur. In particular, the buttons are reflexively quickly manipulated during a game, and monitoring and control on the content side may delay manipulations. Thus, the illusionary tactile force sense interface device 101 comprises the CPU and memory configured to monitor the sensors (108, 109, and 110) and to control the illusionary tactile force sense device 107 and the viscoelastic material 1404, enabling real-time control. This improves responsiveness in the depression of the virtual button and the like, thus enhancing reality and manipulability.
(130) Furthermore, the illusionary tactile force sense interface device 101 includes a communicator 205 configured to communicate with another illusionary tactile force sense interface device 101. For example, if the illusionary tactile force sense interface device 101 is installed on each of five fingers, each of the illusionary tactile force sense interface devices is deformed in conjunction with the motion of the corresponding finger owing to the shape deformable material (1403 in
(131) In
(132) In
(133)
(134) The present implementation example is characterized as follows. An illusionary tactile force sense interface device mounted in a controller for a conventional game machine simply changes the intensity or frequency of vibration. However, according to the present implementation scheme, an illusionary tactile force sense presenting technique is used to enable the user to continuously perceive a force in a fixed direction. This characteristic is used to feedback-control the direction and magnitude of the illusionary tactile force in accordance with the motion of the finger 533 and the hand by means of the method shown in
(135)
(136) If the illusionary tactile force sense interface device 101 is used to input choices and intensions through gestures, inputting gestures interactively with the virtual object 531 enables intuitive gesture inputs and manipulations.
(137) The illusionary tactile force sense interface device 101 can be installed not only on the finger 533 or the user's body but also on all articles such as writing utensils such as pencils and ink brushes, convenience goods such as tooth brushes, and toys such as stuffed animals. For example, the illusionary tactile force sense interface device 101 may be installed on or built into the hand of a stuffed animal so that upon holding the hand of the stuffed animal, the user can feel pulled or pushed by the stuffed animal. Furthermore, the illusionary tactile force sense interface device 101 can be used for training on the use and handling of a pencil or an ink brush.
(138) Each illusionary tactile force sense interface device serves as a controller, and an aggregate of illusionary tactile force sense interface devices also forms a large controller. Thus, various forms of controllers can be implemented.
(139)
(140) In accordance with the texture in the content information to be presented, an illusionary tactile force sense evoking function is generated based on the information stored in an illusionary tactile force sense database 1710. The generated function is corrected by a corrector 1702 based on the user's characteristics and the position, acceleration, and pressure information from the illusionary tactile force sense interface device 101. The function is then converted into a control signal by a motor controller 1703 that is a controller for the illusionary tactile force sense device 107. In accordance with the control signal, motors 1704 connected to the eccentric weights are driven. An encoder 1705 monitors the rotation phase, and the motor controller 1703 feedback-controls the motor so that the motor rotates appropriately. This rotation and phase pattern evokes an illusionary tactile force sense.
(141) Furthermore, in order to improve the illusionary tactile force sense evoking effect, the following method is used instead of the illusionary tactile force sense device 107 configured to generate an acceleration and deceleration pattern. That is, viscoelastic characteristics controller 1706 convert the function into a control signal so that the properties of a viscoelastic material are controlled in accordance with the control signal. When the viscoelastic characteristics of the viscoelastic material 1407 are temporally varied, even eccentric rotors configured to rotate at a constant velocity allow the same effect as that of the above-described rotation and phase pattern to be evoked based on motional characteristics of eccentric weights obtained via the viscoelastic material 1407.
(142) The present invention is not limited to the above-described two schemes. Any material and method may be used provided that the material and method allow the vibration and momentum to be varied in accordance with a control pattern for evoking an illusionary tactile force sense.
(143)
(144) The illusionary tactile force sense evoking function F 1713 generated by the illusionary tactile force sense evoking function generator is determined based on a direction vector u(x,y,z) for the direction in which a force is to be presented, an illusionary tactile force sensory intensity II, the vibration sensory intensity VI, and response characteristics R(P,I,D). Thus, a phase pattern θ(t)=F(u, II, VI, R) is calculated which is required to control the rotational acceleration and deceleration of the eccentric rotors. In this case, P, I, and D denote a proportional gain, an integration gain, and a differentiation gain for PID control, respectively (a specific calculation method is shown in the embodiment in
(145) Correction data 1714 is saved to a user characteristics database 1711; the correction data 1714 is obtained from the above-described equal-sensory-level contour of the illusionary tactile force sense and the equal-sensory-level contour of user's own illusionary tactile force sense. The correction data is read by the corrector 1702 and used to correct a possible individual difference. Furthermore, the sensitivity S (S=S(CP, PG, FI) for the illusionary tactile force sense varies depending on the contact pressure CP between the fingertip and the illusionary tactile force sense interface device 101, the adverse effect PG of the gravitational force in connection with the posture, and an inertia force FI resulting from acceleration when the device is moved. The sensitivity S is determined through preliminary subject experiments and saved to the illusionary tactile force sense data 1710. Then, correction is performed by adding the correction data 1714 to the sensitivity S as an increase in threshold for the equal-sensory-level contour of illusionary tactile force sense. As a result, a corrected illusionary tactile force sense evoking function is determined.
(146)
(147) In the illusionary tactile force sense device 107, based on the illusionary tactile force sense evoking function and the illusionary tactile force sense function data 1710, the motor 1704, which allows illusionary tactile force sense information to be presented, is feedback-controlled to present the desired sense.
(148) The illusionary tactile force sense device 107 also has the function of presenting a tactile force sense (tactile force sense device). The illusionary tactile force sense and the tactile force sense are simultaneously presented to exert a synergetic effect for improving texture.
(149)
(150) In the present device, the motor 1704 is separately controlled by both a motor feedback (FB) characteristics controller configured to control the feedback characteristics of the motor 1704 and a control signal generator configured to convert an illusionary tactile force sense evoking pattern into a motor control signal. In the present invention, controlling the synchronization of the phase pattern θ(t)=F(u, II, VI, R) for the motor rotation is important. The synchronization needs to be temporally precisely controlled. Here, as an example of the corresponding technique, position control based on a control pulse train for a servo motor is shown. If a step motor is used for position control, rapid acceleration or deceleration often easily causes a loss of synchronism and makes the step motor uncontrollable. Thus, here, pulse position control based on a servo motor will be described. In the present invention, the control of the motor feedback (FB) control characteristics and the motor control based the pulse position control method are separately performed to allow a large number of illusionary tactile force sense interface devices 101 to be utilized through synchronous control. Thus, the present invention ensures scalability which allows the following to be easily dealt with: the consistency of the motor control signal for different motors used, an increase in the velocity at which an illusionary tactile force sense evoking pattern is generated, and an increase in the number of control motors to be synchronously controlled. Furthermore, a possible individual difference can be easily corrected.
(151) The illusionary tactile force sense evoking function generator 1701 separates the illusionary tactile force sense evoking function into a control signal for the motor FB characteristics controller and a control signal for the motor control signal generator. The motor control signal generator generates a pulse signal train gi(t)=gi(f(t)) allowing the phase position of the motor to be controlled. Thus, the phase pattern θ(t) for the motor is controlled.
(152) According to the present scheme, the rotation phase of the motor is feedback-controlled based on the number of pulses. For example, one pulse allows the motor to rotate by 1.8°. Either forward rotation or backward rotation is selected depending on a direction control signal. This pulse control technique is used to control any acceleration and deceleration pattern (rotating speed and rotation acceleration) at any phase timing with the phase relationship between at least two motors maintained.
(153)
(154)
(155) (1) As shown in
(156) (2) As shown in
(157) (3) As shown in
(158) (4) As shown in
(159) In the illusionary tactile force sense interface device 101, if the rotating speed (angular velocity) and the phase synchronization are adequately controlled in accordance with the user's sensory characteristics as shown in
(160)
(161)
(162)
(163) As with
(164) As shown in
(165) Furthermore, as shown in
(166)
(167) Instead of the eccentric weights 814 of the eccentric rotor and the eccentric motor 815 configured to drive the eccentric weights 814, the eccentric weights 814 and eccentric motor 815 all being shown in
(168)
(169) As shown in
(170) For example, as shown in
(171)
(172) (1)
(173)
(174) As shown in
(175)
(176) In the technique that uses the viscoelastic material as shown in
(177) Repeated acceleration and deceleration of rotation of the motor as shown in
(178)
(179)
(180)
(181)
(182)
(183)
(184)
(185)
(186)
(187)
(188)
(189)
(190)
(191)
(192)
(193)
(194)
(195)
(196)
(197)
(198)
(199)
(200)
(201)
(202) If for example, the apparatus is applied to a fishing game as shown in
(203) The mechanism used to transform the shape is not limited to the shape transforming motor 3002. Any mechanism such as a shape memory alloy or a driving mechanism that uses piezoelectric elements may be utilized provided that the mechanism enables the shape to be changed.
(204)
(205) In terms of the visual sense, the virtual controller 3101 is generated by the content creating device 102 and formed into a video in the hand using the audiovisual display 105 such as a hologram, an autostereoscopic display, or a head mounted display. In terms of the tactile sense, the illusionary tactile force sense interface device 101 is used to create a virtual controller 3101 so as to present the presence and feel of the virtual controller and senses of button operations. The conventional method of using vibration fails to haptically express the shape of the virtual object. However, the use of the illusionary tactile force sense interface device allows expression of the presence of the virtual button 3102 and a reaction force corresponding to pushback exerted in response to the push of any button.
(206) Conventional game controllers allow the user to enjoy experience games by moving the user's own body. The conventional game controllers are thus of a “pseudo experience type” with no force feedback more than vibration. In contrast, the use of the illusionary tactile force sense interface device 101 allows a “full experience type controller” to be implemented which allows the user to haptically touch the virtual object 531 and game characters.
(207) The virtual controller 3101 that uses the illusionary tactile force sense interface device 101 is effective in allowing the shape of the controller and the arrangement of the buttons to be freely designed in accordance with the contents of the game. In particular, since the lengths of the hand and fingers vary depending on age and sex, the shape of the virtual controller 3101 can be designed and transformed in accordance with the user's hand. The shape of the virtual controller 3101 may be formed in accordance with the content or changed in accordance with the storyline of the content. For example, conventional available game controllers are commercially adapted for game contents. On the contrary, if a single game controller is used to manipulate various contents, since the game controller is not optimum for the contents, the following problem may occur. That is, the game controller may fail to be intuitively manipulated or the created contents may be limited in association with the game controller. In contrast, in the present implementation example, controllers adapted for contents can be virtually created. This eliminates the need to repurchase a dedicated controller and allows the controller to be freely transformed and changed in accordance with scenes in or the stories of the contents.
(208) In particular, when new game software comes onto the market, information on the virtual controller can be contained in the software. This allows the use of a virtual controller optimized for the content of the game. The virtual controller can be distributed via a network as an item. Thus, the virtual controller can be inexpensively and readily upgraded and sold.
(209) In actual game controllers, it is difficult to consecutively and quickly depress a plurality of buttons while gripping the housing with the ring finger and the little finger. However, the virtual controller eliminates the need to grip the housing. Furthermore, the virtual controller involves no inertia force otherwise resulting from the weight of the game controller. Thus, the controller can be quickly moved. On the contrary, the virtual controller 3101 based on the illusionary tactile force sense allows the weight of the controller and the inertia force to be generated as required.
(210) In the conventional game controller, all inputs are performed using the buttons on the controller. Thus, to manipulate a switch or a door knob in the VR space, the buttons on the controller are used to select and manipulate the switch or the door knob. Hence, users unfamiliar with the game require a long time to learn the functions assigned to the buttons on the game controller, a method for manipulating the buttons, and a method for manipulating the buttons for the particular game. However, in the virtual controller 3101, the functions of the game controller can be arranged in the respective virtual buttons 3102 in the VR space. Consequently, the user can directly manipulate the buttons in the VR space using a manipulation manner with which the user is familiar. This eliminates the need for the learning time and enables intuitive manipulations.
(211)
(212)
(213) Furthermore, as shown in
(214)
(215) Thus, the rotational vibration can be reduced by using two sets of units as shown in
(216) Here, if the phases θ1 and θ2 between the two sets of units are changed, the sensory intensity of the illusionary tactile force sense can be changed by adjusting the phase difference θ2-θ1 as shown in
(217) As shown in
(218) The use of a plurality of sets of units also enables an illusionary tactile force sense control apparatus with high energy efficiency to be provided. An example of this illusionary tactile force sense control apparatus is shown in
(219) If as shown in
(220) Here, in such a synthesis method, the rotating velocities of the two basic units are not limited to ω.sub.0 and 2ω.sub.0 but may have a natural number ratio relationship such as mω.sub.0 and nω.sub.0 ((m) and (n) are natural numbers).
(221) In contrast, as shown in
(222)
(223) As shown in
(224) Embodiments
(225)
(226) After all the devices in VR environment generating apparatuses A and B are calibrated, communication is enabled between the VR environment generating apparatuses. The users are present in different spaces corresponding to the respective VR environment generating apparatuses. Information on the two VR environments is shared by both VR environment generating apparatuses via communication devices.
(227) Sensing by the sensors will be described below with reference to
(228) As data of content data, initial model information (the position Po of the vertex of a model) on a virtual clay mass is read from the content data 104.
(229) Then, the plurality of position sensors 111 and acceleration sensors 108 measure an information vector group Mu′ (position Xu′, posture Pu′, velocity Vu′, angular velocity Ru′, acceleration Au′, and angular acceleration Tu′) on each site of the user's body. Here, the position sensors used can also measure posture information. The velocity, angular velocity, acceleration, and angular acceleration are determined by the differentiation and second-order derivative of the position information. For quicker motions, information from the acceleration sensors is used. Furthermore, the physical simulator 113 provides a memory space in which the following are stored, in the content creating device 102: a group of information vectors Mo (position Xo, velocity Vo, acceleration Ao, and force Fo applied to each vertex) relating to the vertex of a physical model of virtual clay, a group of virtual force vectors Fuo applied to the vertex by the user, sound source data, a group of information vectors Mu (position Xu, posture Pu, velocity Vu, angular velocity Ru, acceleration Au, and angular acceleration Tu) relating to a user model (virtual user), and a group of virtual force vectors Fou applied to the virtual user by the vertex of the virtual clay. Based on the constantly updated information vector groups in the memory space, physical simulation of the virtual clay, which is a content, and the virtual user is repeated to allow the information in the memory space to be updated.
(230) The physical simulation will be described below with reference to the model shown in
(231) The physical simulator expresses the virtual clay using a spring and damper model shown in
u1=Fou1/∥Fou1∥−Pu1
Similar calculations are carried out on the other measurement points pi.
(232) As shown in
(233) In the above description, a set of illusionary tactile force sense devices is used to preset a force in any direction in the cross section of the finger. This method may be expanded such that three sets of illusionary tactile force sense devices are used to present a force in any of all the directions.
(234) As a physical intensity to be presented, the physical intensity corresponding to the illusionary tactile force sense intensity II to be presented is referenced using a table of figures indicating the equal-sensory-level contour of the illusionary tactile force sense in
ω1=2π/f1, ω2=2π/f2, where f1=f+Δf/2 and f2=f−Δf/2.
(235) The phase pattern θ(t) is expressed using the initial phase θi and the angular velocities ω1 and ω2, as shown in
(236) For the response characteristics R of the motor, the P, I, and D parameters are selectively set so as to prevent possible vibration caused by overshooting, thus offering excellent convergence responses. The control method based on the P, I, and D parameters is a servo motor control method commonly utilized by those skilled in the art. The P, I, and D parameters are selectively set in accordance with a selection method provided by the motor maker. If the vibration sense intensity VI, which is indicative of the sense of roughness 1111, is to be emphasized, then, the motor FB characteristics controller sets the parameters in a feedback manner so as to increase the P and D parameters to cause vibration with the acceleration sensor monitoring.
(237) As described above, the phase pattern θ(t) is determined to be f(t)=F(u, II, VI, R) using the illusionary tactile force sense evoking function F.
(238) If the resolution of motor control is set to 1.8°, the above-described phase pattern is used to divide a phase of 360° on the axis of ordinate into 200 pieces each with a phase of 1.8°. Then, the points in time on the axis of abscissa corresponding to the 200 points are determined. The points in time correspond to timings when a control pulse train is generated. As described above, the control pulse train g(t) is determined based on the phase pattern θ(t).
(239) A modified damper model and the spring and damper model in
(240) Another difference is that the length L.sub.0 of the spring in a balanced state in
(241) If a grid point 1 is connected to grid points 2 to 4, a force vector f12 applied to the grid point 1 from the grid point 2 is expressed by:
f12=−k×(∥p2−p1∥−L12)×(p2−p1)/∥p2−p1∥−c×(v2−v1) (9)
where
(242) pi: position vector of a grid point pi,
(243) vi: velocity vector of the grip point pi,
(244) k: elastic coefficient of the spring,
(245) c: viscosity coefficient of the damper, and
(246) Lij: natural length of the spring between grid points (i) and (j).
(247) When the resultant force of forces applied to the grid point 1 with a mass m1 by the surrounding grid points 2 to 4 is defined as f1, the equation of motion for the grid point 1 is expressed by:
m1×d.sup.2p1/dt.sup.2=f1=f12+f13+f14 (10).
(248) When the fingertip with the illusionary tactile force sense interface device installed thereon touches the grid point 1 (p1) of the virtual-object physical model, the grip point 1 (p1) changes to the position 1 (p′1) of the fingertip. The reaction force (−f) applied to the fingertip is expressed by:
−f=(f12+f13+f14)−m1×d.sup.2p′1/dt.sup.2 (11).
The motion of the finger, based on which whether or not the finger has touched the model is determined, is sensed by the position sensor and the acceleration sensor.
(249) In actual numerical simulation, the position p′1, velocity v′1, and force f′1 of the grid point 1 at a point in time t′ is determined from variables p1, v1, and f1 at the preceding point in time t. That is, the following hold true.
Velocity vector:v′1=v1+(f1/m1)Δ×t (12)
Position vector:p′1=p1+v1×Δt (13)
(250) The position and velocity of the grid pint 2 with a mass m2 are similarly calculated.
Velocity vector:v′2=v2+(f2/m2)×Δt (14)
Position vector:p′2=p2+v2×Δt (15)
(251) Finally, unlike in the case of
f′12=0 (16)
(252) The above-described physical simulation allows a force applied to the virtual clay by the virtual user's fingertip to be calculated. Thus, the virtual clay is transformed. Furthermore, a force applied to the virtual user's fingertip by the virtual clay is calculated. Based on the results of calculation of the stress, in the presentation, the illusionary tactile force sense interface device is controlled by the illusionary tactile force sense evoking device and the illusionary tactile force sense device driving control device. Hence, the user (entity) experiences the feel of the virtual clay in accordance with three-dimensional videos and sound images on the audiovisual display. The user transforms the virtual clay while checking the shape of the virtual object based on the feel. The user thus completes a virtual flower base. At this time, if virtual objects A and B are identical on the VR space, the users cooperate in completing the virtual flower base.
(253) Real objects may be used instead of the virtual objects A and B. A peripheral device takes videos of the objects and measures the shapes thereof. The videos and measurement results are shared by the VR environment generating apparatuses A and B via the communication instruments as data. If a real object is used instead of the virtual object A, the user B shares the user A's experience in ceramic art.
(254) In every calculation, the position, velocity, and force of each grid point are calculated and saved to the memory. The saved values are used to calculate the position, velocity, and force at the succeeding point in time. The above-described operations allow the reaction force to the fingertip to be presented, enabling the haptization of the virtual object to be realized.
(255) As in the case with the above-described physical simulation of the virtual objects, both the real object in the real space and the user are modeled in the same VR environment based on motion information on the real object sensed by the peripheral device and the motion information on the user sensed by the position sensor and the acceleration sensor. The contact and grip force exerted on the content are calculated. Thus, a VR space is created into which the virtual space and the real space are merged.
(256) A virtual controller shown in
(257) The present apparatus is expected to be applied to various fields other than that of virtual reality.
(258) In the information presentation and expression based on the virtual reality technique, users who are unsusceptible to motion sickness in a real vehicle may get motion sickness with the simulator. Furthermore, not a few users fail to feel a sense of three-dimensionality in stereoscopic virtual reality. Even with the same virtual reality, the feel of the reality varies among users. In addition, in connection with physical differences such as the hand size and muscle strength, differences among interfaces in weight and shape, and the level at which the user has learned how to handle the interface, the easiness with which the user has phantom sensation by the virtual reality technique, that is, the way in which the user feels, varies greatly depending on age and sex and among individual users. Hence, the effects of learning and correction vary depending on the application.
(259) When the present apparatus is applied to an information terminal such as a cellular phone or a PDA, the amount of tactile force sense information, the easiness of understanding the information, and manipulability are improved by adapting the apparatus for individual characteristics.
(260) For example, if the present apparatus is used instead of a manner mode vibrator, then compared to the conventional vibration with no directional information, the illusionary tactile force sense allows efficient presentation of a traveling direction for navigation and alarms that are likely to be overlooked by the user.
(261) If the illusionary tactile force sense device and tactile force sense device are built into an information terminal, the intensity and feel of the tactile force sense vary depending on the relative relationship between both the weight and size of the terminal and both the hand size and muscle strength. Furthermore, with the non-base type, held in the hand and shaken, the same tactile force sense information presented may be felt differently owing to an inertia force resulting from a mass and inertia moment. Thus, the correction function of the present apparatus is effective for allowing the tactile force sense information to be properly presented.
(262) When the present apparatus is utilized for a cellular phone game that uses a motion sensor, a force output against to a force input is effectively obtained. This improves interactivity and reality, thus enhancing intuitive manipulability. When the present apparatus is utilized for a touch pen (stylus) or a tablet PC, a sense of click is improved which is felt when an icon is clicked with the finger or a touch pen. Furthermore, windows overlapping in the display can be differentiated from one another by applying different frictional resistances to the respective windows. This improves usability for visually impaired users.
(263) Additionally, when the present apparatus is applied to training apparatuses such as surgery simulators, the simulator can be adjusted in accordance with the user's individual characteristics and learning level. Moreover, information such as characteristic points to be learned and points that are likely to be overlooked can be emphatically expressed in terms of the force sense. Thus, manipulability, the easiness of understanding the information, and learning effects are improved.
(264) Since the present invention involves emphasis based on illusions, it is necessary to emphasize and correct the information based on the user's sensory characteristics instead of simply increasing the physical quantity or contrast as in the case of the presentation of force sense information. Furthermore, in order to express the diversity of operation tools as well as different sets of tools for beginners and experts, corrections are performed in accordance with a sense of reality, which varies depending on the use frequency and learning level, as well as the easiness with which the user has phantom sensation by the virtual reality technique.