CONTROL AND INPUT DEVICE FOR MEDICAL OBSERVATION DEVICE INCORPORATING POINTING DEVICE WITH CO-LOCATED SOFT AND HARD INPUTS AND GRAPHICAL USER INTERFACE
20210378483 ยท 2021-12-09
Assignee
Inventors
Cpc classification
G06F3/04842
PHYSICS
G06F3/04812
PHYSICS
G06F3/0213
PHYSICS
G06F3/04886
PHYSICS
G06F3/016
PHYSICS
G06F3/033
PHYSICS
International classification
A61B1/00
HUMAN NECESSITIES
G06F3/033
PHYSICS
G06F3/0481
PHYSICS
Abstract
An input device for operating a medical observation system having, arranged on an interface surface, a pointing device, hardware buttons in spaced apart areas, and a touch display panel including a graphical user interface with interchangeable software buttons. The hardware buttons and software buttons have specific functions, such as control of the pointing device, and are placed surrounding and adjacent to the periphery of the pointing device. This configuration helps the operator to simultaneously and singlehandedly operate the input device and without devoting substantial focus toward the input device itself. Various features to allow the operator to distinguish the hardware and software buttons having specific functions are presented, including tactile and haptic features, illumination effects, and other visual effects.
Claims
1. An input device for operating a medical observation device, comprising: an interface surface; a pointing device arranged on the interface surface, wherein the pointing device is configured to input an operation to the medical observation device; a first area of the interface surface, wherein the first area is located adjacent a first portion of a periphery of the pointing device and wherein the first area includes at least a first plurality of hardware buttons configured to control functions of the medical observation device; a second area of the interface surface, wherein the second area is located adjacent a second portion of the periphery of the pointing device and wherein the second area includes at least a second plurality of hardware buttons configured to control functions of the medical observation device; and a touch panel arranged on the interface surface, wherein the touch panel includes a graphical user interface including an input display region containing at least one software button configured to control a function of the medical observation device, wherein the input display region is located at a first edge portion of the graphical user interface and the first edge portion of the graphical user interface is located adjacent a third portion of the periphery of the pointing device, wherein, along the periphery of the pointing device, the third portion of the periphery is interposed between the first portion of the periphery and the second portion of the periphery.
2. The input device according to claim 1, wherein, along the periphery of the pointing device, a fourth portion of the periphery is interposed between the first portion of the periphery and the second portion of the periphery.
3. The input device according to claim 2, wherein an area adjacent the fourth portion of the periphery is free of hardware buttons configured to control functions of the medical observation device and is free of software buttons configured to control functions of the pointing device.
4. The input device according to claim 1, wherein the first portion of the periphery and the second portion of the periphery are at opposing sides of the pointing device.
5. The input device according to claim 1, wherein a number of hardware buttons in the first area is different from a number of hardware buttons in the second area.
6. The input device according to claim 1, wherein at least one of the first plurality of hardware buttons is configured to control the operation inputted to the medical observation device by the pointing device.
7. The input device according to claim 6, wherein at least one of the second plurality of hardware buttons is configured to control the operation inputted to the medical observation device by the pointing device.
8. The input device according to claim 7, wherein the operation of the pointing device inputted by the at least one hardware button is used in a plurality of modes of the medical observation device.
9. The input device according to claim 6, wherein the operation of the pointing device inputted by the at least one hardware button is used in a plurality of modes of the medical observation device.
10. The input device according to claim 1, wherein at least one of the second plurality of hardware buttons is configured to control the operation inputted to the medical observation device by the pointing device.
11. The input device according to claim 1, wherein the operation of the pointing device inputted by the hardware buttons is selected from the group consisting of a pointer function, a measurement function, a set function, a comment function, and a delete function.
12. The input device according to claim 1, wherein the function of the pointing device associated with at least one software button changes in accordance with a mode of the medical observation device.
13. The input device according to claim 12, wherein the mode of the medical observation device is selected using a mode button arranged in a mode display region of the graphical user interface or adjacent to the touch panel.
14. The input device according to claim 1, wherein the function of the pointing device associated with the at least one software button is selected from the group consisting of scroll, rotation, focus change depth, ROI position change, and ROI size change.
15. The input device according to claim 1, wherein the function of the pointing device associated with the at least one software button changes based on a type of a medical device connected to the medical observation device.
16. The input device according to claim 1, wherein the input display region of the graphical user interface is located at a twelve o'clock position relative to the periphery of the pointing device.
17. The input device according to claim 16, wherein the first area is located at a nine o'clock position relative to the periphery of the pointing device and the second area is located at a three o'clock position relative to the periphery of the pointing device.
18. The input device according to claim 1, wherein the interface surface associated with one or more of (i) the pointing device, (ii) at least one of the first plurality of hardware buttons, (iii) at least one of the second plurality of hardware buttons, and (iv) the at least one software button includes a tactile feature.
19. The input device according to claim 1, wherein the interface surface associated with one or more of (i) the pointing device, (ii) at least one of the first plurality of hardware buttons, (iii) at least one of the second plurality of hardware buttons, and (iv) the at least one software button includes a haptic feedback feature.
20. The input device according to claim 1, wherein the pointing device is one of a touch pad, a track ball, a pointing stick, or a joystick.
21. The input device according to claim 1, wherein the hardware and the software buttons controlling the function of the pointing device have the same shape.
22. The input device according to claim 1, wherein the first area of the interface surface, the second area of the interface surface, and the input display region of the graphical user interface are all at a position on the interface surface that is within 15 cm from a center of the pointing device.
23. The input device according to claim 1, wherein the first area of the interface surface, the second area of the interface surface, and the input display region of the graphical user interface are all co-located within an operating area of the interface surface operable by a hand of an operator without changing a location of the hand relative to the interface surface.
24. The input device according to claim 1, wherein the first area of the interface surface, the second area of the interface surface, and the input display region of the graphical user interface are all co-located within an operating area of the interface surface of less than 100 cm.sup.2.
25. The input device according to claim 24, wherein the operating area of the interface surface is less than 90 cm.sup.2.
26. The input device according to claim 24, wherein a shape of the operating area of the interface surface is selected from one of a quadrilateral, a circle and an oval.
27. A medical observation system, comprising: a medical device; a medical observation device connected to the medical device; and the input device according to claim 1 connected to the medical observation device.
28. The medical observation system according to claim 27, wherein the medical device includes an endoscope.
29. A method of operating the medical observation system according to claim 27, the method comprising: inputting an operation to the medical observation device via the pointing device arranged on the interface surface of the input device; controlling functions of the medical observation device via the first plurality of hardware buttons arranged in the first area of the interface surface located adjacent the first periphery of the pointing device; controlling functions of the medical observation device via the second plurality of hardware buttons arranged in the second area of the interface surface located adjacent the second periphery of the pointing device; and controlling functions of the medical observation device via the at least one software button contained in the input display region of the graphical user interface included in the touch panel arranged on the interface surface of the input device, where the input display region is located at the first edge portion of the graphical user interface and the first edge portion of the graphical user interface is located adjacent the third portion of the periphery of the pointing device, wherein the third portion of the periphery is interposed between the first portion of the periphery and the second portion of the periphery of the pointing device.
Description
BRIEF DESCRIPTION OF THE DRAWING
[0038] The following detailed description of preferred embodiments can be read in connection with the accompanying drawings in which like numerals designate like elements and in which:
[0039]
[0040]
[0041]
[0042]
[0043]
[0044]
[0045]
[0046]
[0047]
[0048]
[0049] For ease of viewing, in some instances only some of the named features in the figures are labeled with reference numerals.
DETAILED DESCRIPTION
[0050]
[0051] The ultrasound endoscope 102 has, at its distal end portion, an ultrasound transducer 121. The ultrasound transducer 121 converts an electrical pulse signal received from the ultrasonic observation device 103 into ultrasonic pulses (acoustic pulses) and irradiates the ultrasonic pulses to the subject. Then, the ultrasound transducer 121 receives ultrasonic echoes reflected by the subject, converts the received ultrasonic echoes into an electrical echo signal (ultrasonic wave signal) in which an ultrasonic echo is expressed by, e.g., voltage variations, and outputs the electrical echo signal to the ultrasonic observation device 103.
[0052] The ultrasonic observation device 103 includes a transceiving unit 131, a display control unit 132, an input unit 133, a control unit 134, and a storage unit 135.
[0053] The transceiving unit 131 performs transmission and reception of an electrical signal with the imaging device and the ultrasound transducer 121. The transceiving unit 131 is electrically connected to the imaging device, transmits imaging information about imaging timing and the like to the imaging device, and receives an imaging signal generated by the imaging device. Moreover, the transceiving unit 131 is electrically connected to the ultrasound transducer 121, transmits an electrical pulse signal to the ultrasound transducer 121, and receives an echo signal that is an electrical reception signal from the ultrasound transducer 121. Specifically, the transceiving unit 131 generates an electrical pulse signal based on a predetermined waveform and transmission timing, and transmits the generated pulse signal to the ultrasound transducer 121.
[0054] The transceiving unit 131 also performs sensitivity time control (STC) correction in which an echo signal having a larger reception depth is amplified at a higher amplification factor. The transceiving unit 131 subjects the amplified echo signal to filtering and the like, and then generates a digital high frequency (radio frequency (RF)) signal of time domain to output.
[0055] The display control unit 132 generates endoscopic image data based on an imaging signal and ultrasound image data that corresponds to an electrical echo signal. Furthermore, the display control unit 132 superimposes various kinds of information on the endoscopic image data and the ultrasound image data to output, and controls display of the display device 104. The display control unit 132 uses a central processing unit (CPU) having arithmetic and control functions, various kinds of arithmetic circuits, or the like.
[0056] The input unit 133 receives an instruction signal input by the input device 105, and accepts input of various kinds of information according to the received instruction signal. The various kinds of information include observation mode setting information, observation condition setting information (for example, a gain and switching of a display range, scroll specification information (for example, a sliding direction and a sliding amount of a B-mode image)), rotation specification information (for example, a rotation direction and a rotation amount of an ultrasound image), and the like.
[0057] The control unit 134 controls the entire ultrasonic diagnostic system 100. The control unit 134 uses a CPU having arithmetic and control functions, various kinds of arithmetic circuits, and the like. The control unit 134 reads information stored in the storage unit 135, and overall-controls the ultrasonic observation device 103 by performing various kinds of arithmetic processing related to an operating method of the ultrasonic observation device 103. Note that the control unit 134 may also be realized by the CPU or the like that constitutes the display control unit 132 (and vice versa).
[0058] The storage unit 135 stores various kinds of programs to operate the medical observation system 100, data including various kinds of parameters necessary for operation of the medical observation system 100, and the like. The storage unit 135 stores, for example, an initial position (sound ray number) of a write start position (transmission start position of ultrasonic wave) of an ultrasound image.
[0059] Moreover, the storage unit 135 stores various kinds of programs including an operating program to perform an operating method of the ultrasonic diagnostic system 100. It is possible to distribute the operating program by storing the program in a computer-readable recording medium, such as a hard disk, a flash memory, a compact-disk read-only memory (CD-ROM), a digital versatile disk read-only memory (DVD-ROM), and a flexible disk. It is also possible to acquire the various kinds of programs described above by downloading through a communication network. The communication network herein can be realized by, for example, an existing public line network, a local area network (LAN), a wide area network (WAN), or the like, and it does not matter whether the communication network is wired network or wireless network.
[0060] The storage unit 135 having the configuration as above can be realized by using a read-only memory (ROM) in which various kinds of programs are installed in advance, a random access memory (RAM) in which arithmetic parameters and data of respective processing are stored, and the like.
[0061] The display device 104 is connected to the ultrasonic observation device 103. The display device 104 is configured with a display panel constituted of a liquid crystal display, an organic electroluminescence (EL) display, or the like. In the illustrated embodiment, the display device 104 displays an ultrasound image output by the ultrasonic observation device 103, and various kinds of information relating to operation.
[0062]
[0063]
[0064] The input device 105 also includes, arranged on the interface surface 500, several areas arranged relative to the pointing device and each containing select input devices, whether in the form of buttons, controls, slides, switches, etc. For example, hardware buttons 510 can be included that allow operators to input instructions, change settings or modes, and otherwise configure and control the operation of the medical observation device. In exemplary embodiments, the input device 105 is connected to the ultrasonic observation device 103 through a cable or through wireless connection, and outputs a signal of instruction to the input unit 133 based on input to the input device 105 from the pointing device 502, touch panel 506, and hardware buttons 510.
[0065] For example, the input device 105 detects a contact position by a contact sensor when a contact object, such as a finger of an operator, contacts the pointing device 502, and outputs information on the contact position to the ultrasonic observation device 103. Moreover, when the contact object moves while staying in contact with the pointing device 502, the input device 105 detects a moving direction and a moving amount (such as distance), and outputs information on the moving direction and the moving amount to the ultrasonic observation device 103. The ultrasonic observation device 103 performs signal processing in accordance with the received information. Furthermore, the ultrasonic observation device 103 outputs, for example, an image for which a position of the image to be displayed on the display device 104 is translated or rotated based on the received information.
[0066]
[0067] The touch panel 506 includes a graphical user interface 508 including an input display region that is configurable, such as to include software buttons and keyboards. The graphical user interface 508 may also display other information (i.e. patient information, medical observation device status information, and information on medical equipment being used) in text and other graphical formats. The information displayed on the graphical user interface 508 may be changed by selecting the observation mode setting, the observation condition setting, and the like using the hardware button 510, or other input methods such as pointing device 502 and touch panel 506.
[0068] Each hardware button 510 is allocated a function. The allocated function may be described on the hardware button 510 itself, labeled near or adjacent to the hardware button 510, or may have a representative symbol (not shown). Example functions that may be assigned to the hardware button 510 include a measurement function, a set function, a comment function, and a delete function. The hardware buttons 510 may be comprised of a transparent material and a light source such as a LED, so as to provide illumination of the hardware button 510 in multiple or predesignated colors.
[0069]
[0070] As readily seen in
[0071] The input display region 522 includes software buttons 530. The hardware buttons 510 in first area 512 and second area 518, as well as software buttons 530 in the input display region 522, may be assigned functions related to operating the medical observation device, e.g., ultrasound endoscope 102, ultrasonic medical observation device 103, display device 104, or the pointing device 502 of the input device 105.
[0072] As noted above, the software buttons 530 included in the input display region 522 may be assigned functions related to controlling a function of the medical observation device 103. The functions associated with the control of the medical observation device 103 may include one of scroll, rotation, focus change depth, and region of interest (ROI) parameters, such as ROI position change, or ROI size change. Such control function of the medical observation device 103 may also optionally include controlling the pointing device 502. The software buttons 530 may be interchangeable according to design and observation mode of the medical observation device, allowing the operator to select from almost infinite number of software buttons to be displayed in the input display region 522.
[0073] The location of first area 512, second area 518, and input display region 522 may be replaced according to the design of the input device 105 or interface surface 500, granted that the three areas are adjacent to the pointing device 502, preferably within 15 cm from the center of the pointing device. The center of the pointing device is the operational center of the pointing device, which is preferably also the geometric center of the point device. More preferably, the three areas are directly adjacent to the pointing device 502, where directly adjacent means there are no control input devices interposed between the periphery 516 of the pointing device 502 and the respective areas, e.g., first area 512, second area 518, and input display region 522. Co-locating the three areas within an operating area of the interface surface 500 of the input device 105 allows the input devices in the areas, e.g., first area 512, second area 518, and input display region 522, to be operable by a single hand of an operator without changing a location of the hand relative to the input device 105. Said operating area of the interface surface 500 is preferably less than 100 cm.sup.2 or less than 90 cm.sup.2. The allocation of the three areas being near the pointing device 502 allows the operator to operate the pointing device 502, hardware buttons 510 in the first area 512, hardware buttons 510 in the second area 518, and software buttons 530 in input display region 522 singlehandedly.
[0074] The same configuration may also allow the operator to operate two of more of the pointing device 502, hardware buttons 510 in the first area 512, hardware buttons 510 in the second area 518, and software buttons 530 in input display region 522 simultaneously, for example, by placing one finger on the touch pad 504 and the other fingers on one of a hardware button 510 in the first area 512, a hardware button 510 in the second area 518, and a software button 530 in input display region 522.
[0075]
[0076] Another way to allow the operator to easily distinguish the software buttons 530 within input display region 522 from the other software buttons, such as other software buttons 530b within graphical user interface 508 is to assign a tactile feature to the portion of the graphical user interface 508 where the software buttons 530b and/or software buttons 530 within the input display region 522 is designed to appear. Tactile features can include one or more of a protrusion, a surface having different friction level compared to the other portions of the graphical user interface 508, or other type of irregularity that could be sensed by the operator's tactile sensation.
[0077] Yet another way to allow the operator to easily distinguish software buttons within input display region 522, e.g., software buttons 530, from other software buttons within the graphical user interface 508, e.g., software buttons 530b, is to assign haptic feedback features to the software buttons 530 within the input display region 522. Haptic feedback feature can include one or more of a vibration within the input device 105 in response to pushing the software buttons 530 or other type of feedback feature responding to the operator's touching of the software buttons 530. Of course, the haptic feedback can also be provided on other software buttons within the graphical user interface 508, e.g., software buttons 530b, instead of on software buttons within input display region 522, e.g., software buttons 530; or a different haptic feedback can be used to distinguish between the two software buttons.
[0078]
[0079]
[0080]
[0081]
[0082]
[0083]
[0084] Although the present invention has been described in connection with preferred embodiments thereof, it will be appreciated by those skilled in the art that additions, deletions, modifications, and substitutions not specifically described may be made without department from the spirit and scope of the invention as defined in the appended claims.