System and method of using multiple touch inputs for controller interaction in industrial control systems
11079915 · 2021-08-03
Assignee
Inventors
- Pavan Kumar Singh Thakur (Hyderabad, IN)
- Jagadeesh Jinka (Hyderabad, IN)
- Chaithanya Guttikonda (Hyderabad, IN)
- Vibhoosh Gupta (Schenectady, NY, US)
Cpc classification
G06F3/04847
PHYSICS
G06F3/0488
PHYSICS
G06F2203/04808
PHYSICS
G06F2203/04104
PHYSICS
International classification
G06F3/0484
PHYSICS
G05B19/409
PHYSICS
Abstract
Exemplified methods and systems provide a graphical HMI having an interface that mitigate or prevent touch errors and/or inadvertent touches through the use of multiple touch inputs, at a graphical user interface, of a touch-screen input device, to trigger an associated user interface command. In some embodiments, the multiple touch inputs comprise an input at two locations, one in relative association, with a displayed interface command, to trigger the command.
Claims
1. A method of receiving multiple touch inputs, via a touch-screen display, at a graphical user interface of a control application executing in an industrial automation system, to trigger an associated user interface command within the control application, the method comprising: presenting, by a processor, via the touch-screen display, a graphical depiction of the industrial automation system, or a portion of the industrial automation system including a plurality of physical components performing an industrial process; presenting, by the processor, via the touch-screen display, a plurality of graphical elements, wherein each graphical element of the plurality of graphical elements is linked to execution of an associated user interface command in the control application, the user interface command operable to cause a controller in the industrial automation system to effect a change to a set-point of one or more of the physical components performing the industrial process, and wherein each of the plurality of graphical elements linked to execution of the associated user interface command is depicted adjacent to the associated one or more of the physical components; receiving, via the touch-screen display, a first touch input at a first position, the first touch point enabling activation of at least one of the plurality of graphical elements; receiving, via the touch-screen display, a second touch input at a second position, the second position associated with a selected graphical element of the at least one of the plurality of graphical elements, the second touch input received after the first touch input; causing the processor to activate the selected graphical element associated with the second position if the first touch input is maintained during a period in which the second touch input is received, the activation of the selected graphical element causing execution of the user interface command associated with the selected graphical element and effecting a change in the set-point of the one or more physical components performing the industrial process, and causing the processor to maintain in an inactivated state the selected graphical element associated with the second position if the first touch input is not maintained during the period in which the second touch input is received, wherein the processor is configured to reject the first touch input at the first position if the first touch input is maintained for more than a predetermined maximum time value and to ignore the second touch input at the second position if the second touch input is received after the first touch input is rejected.
2. The method of claim 1, comprising: presenting, by the processor, via the touch-screen display, a second graphical element associated with the first position.
3. The method of claim 1, wherein the first touch input comprises a point-based input received at one or more pre-defined virtual regions located proximal to the determined graphical element linked to execution of the associated user interface command.
4. The method of claim 3, comprising: presenting, by the processor, via the touch-screen display, an additional graphical element associated with selection of a location for the one or more pre-defined virtual regions relative to the graphical element.
5. The method of claim 1, wherein the first touch input comprises a point-based input received for a minimum time parameter.
6. The method of claim 1, wherein the first touch input comprises a point-based input received between a minimum time parameter and a maximum time parameter, wherein receipt of inputs outside the minimum and maximum time parameters are ignored as a non-activated input.
7. The method of claim 1, comprising: presenting, by the processor, via the touch-screen display, a second graphical element for receipt of the first touch input at one of a lower or upper corner of the touch-screen display.
8. The method of claim 1, comprising: in response to a third touch input concurrently received with the first touch input and the second associated touch input, maintaining, by the processor, the selected graphical element linked to execution of the user interface command in the non-activated state.
9. A system in an industrial automation system, the system receiving, at a graphical user interface of a control application executing in the industrial automation system, via a touch-screen display, multiple concurrently-received touch inputs to trigger an associated user interface command, the system comprising: the touch-screen display; a processor operatively coupled to the touch-screen display and to a controller in the industrial automation system, the controller controlling a plurality of physical components performing an industrial process; and a memory operatively coupled to the processor, the memory having instructions stored thereon, wherein execution of the instructions, cause the processor to: present, via the touch-screen display, a graphical depiction of the industrial automation system, or a portion of the industrial automation system including a plurality of physical components performing an industrial process; presenting, by the processor, via the touch-screen display, a plurality of graphical elements, wherein each graphical element of the plurality of graphical elements linked to execution of an associated user interface command operable to cause the controller in the industrial automation system to effect a change to a set-point of one or more of the physical components performing the industrial process, and wherein each of the plurality of graphical elements linked to execution of the associated user interface command is depicted adjacent to the associated one or more of the physical components; receive, via the touch-screen display, a first touch input at a first position, the first touch point enabling activation of at least one of the plurality of graphical elements; receive, via the touch-screen display, a second touch input at a second position, the second position associated with a selected graphical element of the at least one of the plurality of graphical elements, the second touch input received after the first touch input; cause the processor to activate the selected graphical element associated with the second position if the first touch input is maintained during a period in which the second touch input is received, the activation of the selected graphical element causing execution of the user interface command associated with the selected graphical element and effecting a change in the set-point of the one or more physical components performing the industrial process, and cause the processor to maintain in an inactivated state the selected graphical element associated with the second position if the first touch input is not maintained during the period in which the second touch input is received, reject the first touch input at the first position if the first touch input is maintained for more than a predetermined maximum time value and to ignore the second touch input at the second position if the second touch input is received after the first touch input is rejected.
10. The system of claim 9, wherein the instructions, when executed by the processor, further cause the processor to: present via the touch-screen display, a second graphical element associated with the first position.
11. The system of claim 9, wherein the first touch input comprises a point-based input received at one or more pre-defined virtual regions located proximal to the determined graphical element associated with execution of the associated user interface command.
12. The system of claim 11, wherein the instructions, when executed by the processor, further cause the processor to: present, via the touch-screen display, an additional graphical element associated with selection of a location for the one or more pre-defined virtual regions relative to the graphical element.
13. The system of claim 9, wherein the first touch input comprises a point-based input received for a minimum time parameter.
14. The system of claim 9, wherein the first touch input comprises a point-based input received between a minimum time parameter and a maximum time parameter, wherein receipt of inputs outside the minimum and maximum time parameters are ignored as non-activated input.
15. The system of claim 9, wherein the instructions, when executed by the processor, further cause the processor to: present, via the touch-screen display, a second graphical element for receipt of the first touch input at one of a lower or upper corner of the touch-screen display.
16. The system of claim 9, wherein the instructions, when executed by the processor, further cause the processor to: in response to a third touch input concurrently received with the first touch input and the second associated touch input, maintain the determined graphical element linked to execution of the associated user interface command in the non-activated state.
17. A non-transitory computer readable medium to trigger, within a control application executing in an industrial automation system, an associated user interface command using multiple concurrently-received touch inputs received at a graphical user interface of a touch-screen display the computer readable medium having instructions stored thereon, wherein when executed by a processor, cause the processor to: present, via the touch-screen display associated with a computing device, a graphical depiction of the industrial automation system, or a portion of the industrial automation system including a plurality of physical components performing an industrial process; presenting, by the computing device, via the touch-screen display, a plurality of graphical elements, wherein each graphical element is linked to execution of an associated user interface command operable to cause a controller in the industrial automation system to effect a change to a set-point of one or more of the physical components performing the industrial process, and wherein each of the plurality of graphical elements linked to execution of the associated user interface command is depicted adjacent to the associated one or more of the physical components; receive, via the touch-screen display, a first touch input at a first position, the first touch point enabling activation of at least one of the plurality of graphical elements; receive, via the touch-screen display, a second touch input at a second position, the second position associated with a selected graphical element of the at least one of the plurality of graphical elements, the second touch input received after the first touch input; cause the processor to activate the selected graphical element associated with the second position if the first touch input is maintained during a period in which the second touch input is received, the activation of the selected graphical element causing execution of the user interface command associated with the selected graphical element and effecting a change in the set-point of the one or more physical components performing the industrial process, and cause the processor to maintain in an inactivated state the selected graphical element associated with the second position if the first touch input is not maintained during the period in which the second touch input is received, cause the processor to reject the first touch input at the first position if the first touch input is maintained for more than a predetermined maximum time value and to ignore the second touch input at the second position if the second touch input is received after the first touch input is rejected.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
DETAILED DESCRIPTION
(8) Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. Methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present disclosure.
(9) As used in the specification and the appended claims, the singular forms “a,” “an” and “the” include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from “about” one particular value, and/or to “about” another particular value. When such a range is expressed, another embodiment includes—from the one particular value and/or to the other particular value. Similarly, when values are expressed as approximations, by use of the antecedent “about,” it will be understood that the particular value forms another embodiment. It will be further understood that the endpoints of each of the ranges are significant both in relation to the other endpoint, and independently of the other endpoint.
(10) “Optional” or “optionally” means that the subsequently described event or circumstance may or may not occur, and that the description includes instances where said event or circumstance occurs and instances where it does not.
(11) Throughout the description and claims of this specification, the word “comprise” and variations of the word, such as “comprising” and “comprises,” means “including but not limited to,” and is not intended to exclude, for example, other additives, components, integers or steps. “Exemplary” means “an example of” and is not intended to convey an indication of a preferred or ideal embodiment. “Such as” is not used in a restrictive sense, but for explanatory purposes.
(12) Disclosed are components that can be used to perform the disclosed methods and systems. These and other components are disclosed herein, and it is understood that when combinations, subsets, interactions, groups, etc. of these components are disclosed that while specific reference of each various individual and collective combinations and permutation of these may not be explicitly disclosed, each is specifically contemplated and described herein, for all methods and systems. This applies to all aspects of this application including, but not limited to, steps in disclosed methods. Thus, if there are a variety of additional steps that can be performed it is understood that each of these additional steps can be performed with any specific embodiment or combination of embodiments of the disclosed methods.
(13) It is understood that throughout this specification the identifiers “first”, “second”, “third”, and such, are used solely to aid in distinguishing the various components and steps of the disclosed subject matter. The identifiers “first”, “second”, “third”, and such, are not intended to imply any particular order, sequence, amount, preference, or importance to the components or steps modified by these terms.
(14) The present methods and systems may be understood more readily by reference to the following detailed description of preferred embodiments and the Examples included therein and to the Figures and their previous and following description.
(15)
(16) Referring still to
(17)
(18)
(19) As illustrated in
(20) Because of the dense presentation of widgets on a HMI for an industrial controller, mistakes may occur by an operator when the wrong area of the HMI is touched. In addition, the HMI may be located in space confined areas that increase the risk of inadvertent touching of the graphical input of the HMI.
(21) To mitigate, errors in input or unintentional touches, the exemplified system and method uses multiple touch inputs that may be specified in a given sequence and for a given duration to activate an operation associated with a widget presented on a control screen of the HMI.
(22)
(23)
(24) As shown in
(25) In some embodiments, the widget associated with a control function is associated with a land area. To this end, the HMI would activate the widget when a first input is received at widget (e.g., button) and a second input is received at the HMI at a second location that enables the operation of the widget. In other embodiments, the widget is associated with a tap area. To this end, the HMI would activate the widget when a first input is received at a land area associated with enable in the operation of a tap area, which is associated with the widget.
(26) In some embodiments, the HMI may present a visual indicator to the operator that the control widget is touched and activated. In some embodiments, the screen may change color, or the HMI may generate sound, or provide other visual, tactile (e.g., vibration), or acoustic notification.
(27) Referring back to
(28) In some embodiments, the land area and tap area may have the same spatial size. In other embodiments, the tap area may have an area smaller than the land area. In another embodiment, the tap area may have an area larger than the land area. In some embodiments, the tap area may change based on a failed attempt and/or a presentation of instructions of the tap area to the operator.
(29) In some embodiments, the land area corresponds in spatial size to a presented widget associated with a control function.
(30) Referring still to
(31) Referring back to
(32) As shown in
(33) In other embodiments, the area to receive the second input (i.e., the tap area) is not presented on the HMI.
(34) In other embodiments, the HMI provides feedback to a user that the user has landed on a land area (e.g., a sound, a change in screen color, a touch area highlighted, etc.).
(35) In some embodiments, the land area may be enabled via touch gestures besides a tap. For example, the land area may be touched to activate gesture control of virtual knobs or sliders that may require movements of the finger on the touch screen (i.e., movement other than a tap).
(36)
(37) Referring still to
(38) Referring still to
(39) Thus, to activate the control widget, the land input 606 and the tap input (shown as input 602c) are entirely overlapping in which the land input 606 is received prior to the tap input 602c.
(40) In some embodiments, the land input 606 may be rejected if the duration time for the input exceeds a pre-defined maximum time value. The maximum time value may be modified via a configuration panel of the HMI. In some embodiments, the maximum time value may be between 10 and 30 seconds. In some embodiments, the maximum time value may be between 5 and 10 seconds.
(41) In some embodiments, the HMI may only cause activation of the control widget if the tap input 602c is received within a predefined time (shown as time 605) from the contact time 601 of the land input 606. In some embodiments, this activation time is between 1 and 5 seconds.
(42) In some embodiments, the GUI receives input via a touch class, e.g., the system.windows.input class in PresentationCore.dll (for Windows). In some embodiments, the GUI receives via libinput library in Linux. In some embodiments, the GUI may operate in conjunction with a multi-touch gesture program such as Touchegg, or other multi-touch gesture programs, that runs as a user in the background, and adds multi-touch support to the window managers.
(43) Example HMI
(44)
(45) Processor 721 may include one or more processors, each configured to execute instructions and process data to perform one or more functions associated with a computer for indexing images. Processor 721 may be communicatively coupled to RAM 722, ROM 723, storage 724, database 725, I/O devices 726, and interface 727. Processor 721 may be configured to execute sequences of computer program instructions to perform various processes. The computer program instructions may be loaded into RAM 722 for execution by processor 721. As used herein, processor refers to a physical hardware device that executes encoded instructions for performing functions on inputs and creating outputs.
(46) RAM 722 and ROM 723 may each include one or more devices for storing information associated with operation of processor 721. For example, ROM 723 may include a memory device configured to access and store information associated with HMI controller 720, including information for identifying, initializing, and monitoring the operation of one or more components and subsystems. RAM 722 may include a memory device for storing data associated with one or more operations of processor 721. For example, ROM 723 may load instructions into RAM 722 for execution by processor 721.
(47) Storage 724 may include any type of mass storage device configured to store information that processor 721 may need to perform processes consistent with the disclosed embodiments. For example, storage 724 may include one or more magnetic and/or optical disk devices, such as hard drives, CD-ROMs, DVD-ROMs, or any other type of mass media device.
(48) Database 725 may include one or more software and/or hardware components that cooperate to store, organize, sort, filter, and/or arrange data used by HMI controller 720 and/or processor 721. For example, database 725 may store hardware and/or software configuration data associated with input-output hardware devices and controllers, as described herein. It is contemplated that database 725 may store additional and/or different information than that listed above.
(49) I/O devices 726 may include one or more components configured to communicate information with a user associated with HMI controller 720. For example, I/O devices may include a console with an integrated keyboard and mouse to allow a user to maintain a database of images, update associations, and access digital content. I/O devices 726 may also include a display including a graphical user interface (GUI) for outputting information on a monitor. I/O devices 726 may also include peripheral devices such as, for example, a printer for printing information associated with HMI controller 720, a user-accessible disk drive (e.g., a USB port, a floppy, CD-ROM, or DVD-ROM drive, etc.) to allow a user to input data stored on a portable media device, a microphone, a speaker system, or any other suitable type of interface device.
(50) Interface 727 may include one or more components configured to transmit and receive data via a communication network, such as the Internet, a local area network, a workstation peer-to-peer network, a direct link network, a wireless network, or any other suitable communication platform. For example, interface 727 may include one or more modulators, demodulators, multiplexers, demultiplexers, network communication devices, wireless devices, antennas, modems, and any other type of device configured to enable data communication via a communication network.