Method, device and program for browsing information on a display
11301196 · 2022-04-12
Assignee
Inventors
Cpc classification
G06F3/1407
PHYSICS
G06F1/1694
PHYSICS
G06F2200/1614
PHYSICS
G06F2200/1637
PHYSICS
G09G2320/0261
PHYSICS
G06F1/1626
PHYSICS
International classification
Abstract
An electronic device, such as held device, having a display and camera, displays a user interface that includes a user interface object displayed at a first size; detects, via the camera, a change in distance between a user of the electronic device and the electronic device; and in response to detecting the change in distance between the user the electronic device and the electronic device, changes size of the displayed user interface object.
Claims
1. A method, comprising: at an electronic device with a display and a camera: displaying, on the display, a user interface that includes a user interface object displayed at a first size; detecting, via the camera, a change in distance between a user of the electronic device and the electronic device; and in response to detecting the change in distance between the user of the electronic device and the electronic device, changing size of the displayed user interface object.
2. The method of claim 1, including, in response to detecting the change in distance between the user of the electronic device and the electronic device, changing size of the displayed user interface object in accordance with the change in distance between the user of the electronic device and the electronic device.
3. The method of claim 1, including setting a zoom mode of the device to enable zooming, wherein changing size of the displayed user interface object comprises zooming in on a portion of the user interface.
4. The method of claim 1, wherein the change in distance between the user of the electronic device and the electronic device is a change in distance between the electronic device and eyes of the user, and the method includes changing size of the displayed user interface object by an amount that is based on the change in distance between the electronic device and the eyes of the user.
5. The method of claim 1, wherein changing size of the displayed user interface object includes changing a zoom factor based on rotation of the electronic device.
6. The method of claim 5, wherein rotation of the electronic device is determined, using the camera, based on orientation of the electronic device in relation to the user.
7. The method of claim 1, including changing a zoom factor for the user interface or user interface object based on a tilting of the device, wherein tilting the device in a first predefined direction increases the zoom factor and tilting the device in a second predefined direction decreases the zoom factor.
8. The method of claim 7, wherein tilt of the electronic device is determined, using the camera, based on orientation of the electronic device in relation to the user.
9. An electronic device, comprising: a processor; a camera; and a display in communication with the processor, wherein the processor is configured to: display, on the display, a user interface that includes a user interface object displayed at a first size; detect, via the camera, a change in distance between a user of the electronic device and the electronic device; and in response to detecting the change in distance between the user of the electronic device and the electronic device, change size of the displayed user interface object.
10. The electronic device of claim 9, wherein the processor is configured to, in response to detecting the change in distance between the user of the electronic device and the electronic device, change size of the displayed user interface object in accordance with the change in distance between the user of the electronic device and the electronic device.
11. The electronic device of claim 9, wherein the processor is configured to set a zoom mode of the device to enable zooming, and changing size of the displayed user interface object comprises zooming in on a portion of the user interface.
12. The electronic device of claim 9, wherein the change in distance between the user of the electronic device and the electronic device is a change in distance between the electronic device and eyes of the user, and the processor is configured to change size of the displayed user interface object by an amount that is based on the change in distance between the electronic device and the eyes of the user.
13. The electronic device of claim 9, wherein changing size of the displayed user interface object includes changing a zoom factor based on rotation of the electronic device.
14. The electronic device of claim 13, wherein rotation of the electronic device is determined, using the camera, based on orientation of the electronic device in relation to the user.
15. The electronic device of claim 9, wherein the processor is configured to change a zoom factor for the user interface or user interface object based on a tilting of the device, wherein tilting the device in a first predefined direction increases the zoom factor and tilting the device in a second predefined direction decreases the zoom factor.
16. The electronic device of claim 15, wherein tilt of the electronic device is determined, using the camera, based on orientation of the electronic device in relation to the user.
17. A non-transitory computer readable storage medium, storing one or more programs, which when executed by one or more processors of an electronic device having a display and a camera, cause the electronic device to perform operations comprising: displaying, on the display, a user interface that includes a user interface object displayed at a first size; detecting, via the camera, a change in distance between a user of the electronic device and the electronic device; and in response to detecting the change in distance between the user of the electronic device and the electronic device, changing size of the displayed user interface object.
18. The non-transitory computer readable storage medium of claim 17, wherein the one or more programs, when executed by the one or more processors, cause the device to, in response to detecting the change in distance between the user of the electronic device and the electronic device, change size of the displayed user interface object in accordance with the change in distance between the user of the electronic device and the electronic device.
19. The non-transitory computer readable storage medium of claim 17, wherein the one or more programs, when executed by the one or more processors, cause the electronic device to set a zoom mode of the device to enable zooming, and changing size of the displayed user interface object comprises zooming in on a portion of the user interface.
20. The non-transitory computer readable storage medium of claim 17, wherein the change in distance between the user of the electronic device and the electronic device is a change in distance between the electronic device and eyes of the user, and the one or more programs, when executed by the one or more processors, cause the electronic device to change size of the displayed user interface object by an amount that is based on the change in distance between the electronic device and the eyes of the user.
21. The non-transitory computer readable storage medium of claim 17, wherein changing size of the displayed user interface object includes changing a zoom factor based on rotation of the electronic device.
22. The non-transitory computer readable storage medium of claim 21, wherein rotation of the electronic device is determined, using the camera, based on orientation of the electronic device in relation to the user.
23. The non-transitory computer readable storage medium of claim 17, wherein the processor is configured to change a zoom factor for the user interface or user interface object based on a tilting of the device, wherein tilting the device in a first predefined direction increases the zoom factor and tilting the device in a second predefined direction decreases the zoom factor.
24. The non-transitory computer readable storage medium of claim 23, wherein tilt of the electronic device is determined, using the camera, based on orientation of the electronic device in relation to the user.
Description
DRAWINGS
(1) The above-mentioned features and objects of the present disclosure will become more apparent with reference to the following description taken in conjunction with the accompanying drawings wherein like reference numerals denote like elements and in which:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
DETAILED DESCRIPTION
(12)
(13)
(14) In
(15) In one embodiment, the angle (α) is a predetermined angle, and it is determined by the manufacturer of the hand-held device 40. In the determination process it is defined that the display view plane is based on axis x_VD and y_VD, which are perpendicular to each other. The hand-held device is then set to a certain position (a), and that position is set as a default xy-plane. In
(16) From that moment on, the hand-held device 40 is tilted in respective to this plane. When the default xy-plane is fixed, the user of the hand-held device is always capable of returning to a certain view by tilting the device back to the original orientation when the sensors measuring the orientation of the hand-held device do not cause any restrictions to the measured position. In another embodiment, the angle α can be readjusted to a desired value.
(17)
(18) For simplicity in the following the viewpoint VP is defined to be at point [0 0 0]. Furthermore, the middle point of the virtual display VD is defined to be at P_xyz wherein P_xyz=[P_xyz.sub.1 P_xyz.sub.2 P_xyz.sub.3].sup.T, and the virtual screen VS to be at plane x-kuva_shift.
(19) The orientation of the virtual display VD is defined by tilting angels α.sub.x, α.sub.y, α.sub.z indicating rotation angle over each coordinate axe. In
(20) It must be noted that
(21) In order to the determine the orientation of the VD, two orthogonal vectors (in the x-plane) are defined as follows:
L=[0,1,−1].sup.T
M=[0,1,1].sup.T
(22) Those vectors present the orthogonal direction vectors of the VD. Next, the orientation of the virtual display VD is defined using the rotation angles:
(23)
(24) Next the unit normal vector of the VD is calculated:
(25)
(26) where PN is the unit normal vector of the VD-plane. The PN defines the applicable orientation of the VD to be used in the projection calculation.
(27) Next, the “image” on the virtual display VD is calculated. Let's assume that there is a vector beginning from the VP and being reflected via the VD. The point where the reflected vector hits on the plane VS defines the projection of the point on the VS to the point on the VD-plane. Hence, if all points on VD are processed as described above, the image on the VD can be defined.
(28) The idea of calculation is presented using vectors in
(29) By repeating phases 1-7 for all points in the VD-plane the whole image of the virtual display VD is defined. Using vector calculation the same can be presented as follows:
(30) First the point P is defined:
P=P_xyz+R.sub.xR.sub.yR.sub.z[0 peili_y peili_z].sup.T
(31) where P_xyz is the coordinate of the middle point of the VD, peili_y is the y-coordinate on the VD plane-coordinate system and peili_z is the z-coordinate on the VD plane-coordinate system.
(32) Next, the projection on the normal vector is defined:
(33)
(34) Hence the point Q can be defined:
Q=P−proj*PN
(35) Further, the point R can be defined (the reason for the factor 2 is that in mirror the arriving and departing light beam have equal angles compared to the normal vector of the surface).
B=Q−VP
R=VP+2*B
(36) And finally the direction vector C is defined as follows:
C=R−P.
(37) Because the VS is located at plane x=kuva_shift, the vector C hits that plane at the point
(38)
(39) where P.sub.1 is the x-component of the point P and C.sub.1 is the x-component of the vector C. Note that in this calculation the VP was defined to the origin to simplify the presentation of the algorithm. However, in practice the VP can locate freely in the coordinate space. It must be noted that the image on the virtual screen VS is horizontally inversed when the virtual screen VS is viewed from the viewpoint VP direction.
(40) The system of
(41) The present invention does not have to implement all the aforementioned features, but the most appropriate ones can be chosen. The ideal mirror-like functionality means that the information on the display device changes when: a) the location or orientation of the hand-held device in proportion to the coordinates bound to the physical environment changes, b) the location of the user (VP) in proportion to the coordinates bound to the hand-held device changes, c) the virtual location of the data (virtual screen) displayed on the display device in proportion to the coordinates bound to the physical environment changes.
(42) In order to simulate the operation of a mirror to the user, the information on the display device is changed at least either according to a) or b). If only a) or b) is taken into consideration, the operation of the display is not so mirror-like as if both a) and b) were implemented. In one embodiment, the display device operates according to all a), b) and c).
(43)
(44) In
(45) The reference point VP is on the virtual surface 200. The x-axis (not shown) runs through the reference point VP and the middle point P of the display 201. After calculating the point S by the method presented with reference to
(46) In
(47) Initially (i.e. when the virtual surface 200 and the display surface 201 are parallel with respect to each other as shown in
(48)
(49) The display adapter 90 with the processor 30 controls the display device 10. In order to not to use the data memory 60 for storing display-related information, the display adapter 90 comprises a data buffer in which the information to be displayed on the display device 10 is stored.
(50) The hand-held device 40 comprises measuring means which in a preferred embodiment of the invention refer to acceleration sensor(s) 50. With the acceleration sensor(s) 50 it is possible to measure tilting movements of the hand-held device 40. The processor 30 receives the measurement results and interprets them. The acceleration sensor(s) 50 can be e.g., piezo-electric or capacitive producing an analog voltage which is proportional to the acceleration factor.
(51) With the acceleration sensor(s) 50 it is possible to measure one, two or three-dimensional accelerations. The measurement of tilting movements is based on the fact that the highest acceleration is parallel to the gravity of the earth. Therefore, the orientation of the hand-held device 40 can be defined in relation to the earth. It is also possible to use gyroscopes with its various forms to measure the orientation of the hand-held device 40. The quantities measured are e.g., tilting angle and accelerations.
(52) The relation information between the rotation degree of the hand-held device and the memory address corresponding to the displayed view is stored e.g., on the data memory 60. The processor 30 defines the orientation of the hand-held device 40 in relation to the user or a reference position. The processor 30 may also define the distance between the user and the hand-held device 40 or the user orientation in relation to the hand-held device 40.
(53) The most important point is not the way of how the aforementioned definitions are made but the fact that the orientation of the hand-held device 40 affects the information displayed on the display device 10. The memory space can be implemented logically, e.g., as a two-dimensional memory space. When browsing starts, the processor 30 starts the definition process of the new memory address from the current memory address so that displacement in the memory space corresponds to the direction and amount of change in orientation according to the relation information.
(54) The hand-held device 40 comprises also a browse lock 80 with which it is signaled when the browsing is executed. The orientation of the hand-held device 40 must remain in the same position in order to keep the view on the display device unchanged. In a preferred embodiment, the hand-held device 40 comprises a lock feature, e.g., a push-button, with which the browsing can be locked. The user can tilt the hand-held device back to an appropriate viewing orientation in order to view the information on the display device 10 properly. The browsing may then continue when the button is released.
(55) The hand-held device 40 in
(56) By analyzing the video image it is possible to define the orientation of the hand-held device 40 in proportion to the reference point and the distance of the hand-held device 40 to the reference point tens of times within a second. The browsing functionality can be implemented merely using the video camera, so that additional acceleration sensor(s) are not necessarily needed. The measuring of the distance can also be implemented with an ultrasonic radar connected through an analog-digital converter to the processor 30 of the hand-held device 40. In one embodiment, from the user's perspective the information on the display device 10 is essentially browsed in the same manner as when looking in a mirror. In other words, the view on the display 10 depends on the viewing angle in relation to the display device plane as the view in a mirror depends on the viewing angle to the mirror.
(57) In one embodiment of
(58) In a preferred embodiment of
(59) In one embodiment of
(60)
(61) The zoom factor can be modified with several different ways. In one embodiment, the zoom factor depends on the distance between the reference point (e.g., the eyes of the user) and the hand-held device. When the distance decreases, graphical
(62) In another embodiment, the zoom factor changes when rotating the hand-held device around the axis being essentially perpendicular to a predefined xy-plane. The xy-plane may be the present plane of the display device 10 or some other predetermined plane. Yet in another embodiment, the zoom factor is changed by tilting the hand-held device. Before this the display device must be set into a zoom mode. When the hand-held device is tilted, e.g., to the right the zoom factor increases, and when the hand-held device is tilted to the left, the zoom factor decreases. It is not important which predefined tilting directions are used but that the two directions can be separated sufficiently from each other. The aforementioned zoom mode is set on and off e.g., with a predetermined button of the hand-held device.
(63)
(64) In another embodiment, the information on the display device 10 remains in the same position with respect to the hand-held device 40 when the hand-held device 40 is rotated around the axis being perpendicular to the display surface plane, as depicted in
(65)
(66) The hand-held device is switched on, and it is ready for browsing information on the display device, as represented in phase 100. When the hand-held device is functional, the acceleration sensor 50 measures constantly acceleration readings. The processor 30 receives the acceleration readings and defines the orientation of the hand-held device and also the change in the orientation compared to the prior measurement(s), as represented in phases 101 and 102. In phase 103, it is tested whether the browsing is on or off. If the browsing is off, the processor 30 examines if a predetermined browsing startup condition is fulfilled (phase 104). If it is not fulfilled, the method returns back to phase 101. It means that the orientation of the hand-held device has not changed sufficiently, which would indicate that the user wishes to browse information on the display device of the hand-held device.
(67) If the predetermined browsing startup condition is fulfilled, the processor 30 sets the browsing as started (phase 106) and determines the browsing speed based on the current acceleration value (phase 108). The processor 30 also changes the information presented on the display device according to a relation between the rotation degree and the amount of the displacement of the portion on the virtual data object stored on the data memory 60 and the determined browsing speed (phase 108). A certain orientation of the hand-held device always causes the same view (the same portion on the virtual data object stored on the memory) on the display device. If it is observed in phase 103 that the browsing is already on, and the browsing stopping condition is fulfilled (phase 105), the processor 30 stops the browsing and sets the browsing as stopped (phases 107 and 109). If it is observed that the browsing stopping condition is not fulfilled (phase 105), the processor 30 returns back to phase 101.
(68) While the apparatus and method have been described in terms of what are presently considered to be the most practical and preferred embodiments, it is to be understood that the disclosure need not be limited to the disclosed embodiments. It is intended to cover various modifications and similar arrangements included within the spirit and scope of the claims, the scope of which should be accorded the broadest interpretation so as to encompass all such modifications and similar structures. The present disclosure includes any and all embodiments of the following claims.