SIMPLIFIED PLC PROGRAMMING WITH MOBILE DEVICE

Abstract

Aspects of the present disclosure can involve systems and methods for a mobile device connected to a local area network (LAN) facilitating connections between a programmable logic controller (PLC) and one or more apparatuses, the mobile device involving a camera and a microphone, which can involve receiving input from one or more of the camera and the microphone; generating one or more of executable code for the PLC and configuration information for the one or more apparatuses from parsing of the input; and transmitting, over the LAN, the one or more of the executable code for the PLC and the configuration information for the one or more apparatuses to the PLC.

Claims

1. A method for a mobile device connected to a local area network (LAN) facilitating connections between a programmable logic controller (PLC) and one or more apparatuses, the mobile device comprising a camera and a microphone, the method comprising: receiving input from one or more of the camera and the microphone; generating one or more of executable code for the PLC and configuration information for the one or more apparatuses from parsing of the input; and transmitting, over the LAN, the one or more of the executable code for the PLC and the configuration information for the one or more apparatuses to the P LC.

2. The method of claim 1, wherein the executable code is generated from speech recognition of the input to the microphone, wherein the PLC executes the executable code once received.

3. The method of claim 1, wherein generating the configuration information comprises: parsing an image from the camera to select an apparatus from the one or more apparatuses; retrieving the configuration information for the selected apparatus; receiving additional input to reconfigure the selected apparatus; and generating new configuration information from the additional input and the retrieved configuration information; wherein the selected apparatus is reconfigured from the new configuration information.

4. The method of claim 3, wherein the image comprises a barcode, and wherein the selection is conducted on the apparatus associated with the barcode.

5. The method of claim 3, wherein the selection is based on a Bluetooth device transmitting an identifier to the mobile device.

6. A non-transitory computer readable medium, storing instructions for a mobile device connected to a local area network (LAN) facilitating connections between a programmable logic controller (PLC) and one or more apparatuses, the mobile device comprising a camera and a microphone, the instructions comprising: receiving input from one or more of the camera and the microphone; generating one or more of executable code for the PLC and configuration information for the one or more apparatuses from parsing of the input; and transmitting, over the LAN, the one or more of the executable code for the PLC and the configuration information for the one or more apparatuses to the PLC.

7. The non-transitory computer readable medium of claim 6, wherein the executable code is generated from speech recognition of the input to the microphone, wherein the PLC executes the executable code once received.

8. The non-transitory computer readable medium of claim 6, wherein generating the configuration information comprises: parsing an image from the camera to select an apparatus from the one or more apparatuses; retrieving the configuration information for the selected apparatus; receiving additional input to reconfigure the selected apparatus; and generating new configuration information from the additional input and the retrieved configuration information; wherein the selected apparatus is reconfigured from the new configuration information.

9. The non-transitory computer readable medium of claim 8, wherein the image comprises a barcode, and wherein the selection is conducted on the apparatus associated with the barcode.

10. The non-transitory computer readable medium of claim 8, wherein the selection is based on a Bluetooth device transmitting an identifier to the mobile device.

11. A mobile device configured to be connected to a local area network (LAN) facilitating connections between a programmable logic controller (PLC) and one or more apparatuses, the mobile device comprising: a camera; a microphone; and a processor, configured to: receive input from one or more of the camera and the microphone; generate one or more of executable code for the PLC and configuration information for the one or more apparatuses from parsing of the input; and transmit, over the LAN, the one or more of the executable code for the PLC and the configuration information for the one or more apparatuses to the PLC.

12. The mobile device of claim 11, wherein the executable code is generated from speech recognition of the input to the microphone, wherein the PLC executes the executable code once received.

13. The mobile device of claim 11, wherein the processor is configured to generate the configuration information by: parsing an image from the camera to select an apparatus from the one or more apparatuses; retrieving the configuration information for the selected apparatus; receiving additional input to reconfigure the selected apparatus; and generating new configuration information from the additional input and the retrieved configuration information; wherein the selected apparatus is reconfigured from the new configuration information.

14. The mobile device of claim 13, wherein the image comprises a barcode, and wherein the selection is conducted on the apparatus associated with the barcode.

15. The mobile device of claim 13, wherein the selection is based on a Bluetooth device transmitting an identifier to the mobile device.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0019] FIG. 1 is an overview of traditional architecture of programming PLC and devices.

[0020] FIG. 2 is the detail plot of the traditional PLC programming architecture.

[0021] FIG. 3 is the workflow of procedure of traditional PLC programming method.

[0022] FIG. 4 is an illustration of the new PLC programming technique, including speech recognition (upper) and vision-based device configuration (bottom), in accordance with an example implementation.

[0023] FIG. 5 illustrates three cases of automatic PLC ladder generation via the function of speech recognition, in accordance with an example implementation.

[0024] FIG. 6 is three examples of using vision-based device configuration and setup with mobile device, in accordance with an example implementation.

[0025] FIG. 7 is the workflow of the speech-based automatic PLC ladder generation and the vision-based device configuration methods, in accordance with an example implementation.

[0026] FIG. 8 illustrates an example of a mobile device, in accordance with an example implementation.

[0027] FIG. 9 illustrates example management information which may be stored on the mobile device to facilitate the example implementations described herein.

DETAILED DESCRIPTION

[0028] The following detailed description provides details of the figures and example implementations of the present application. Reference numerals and descriptions of redundant elements between figures are omitted for clarity. Terms used throughout the description are provided as examples and are not intended to be limiting. For example, the use of the term “automatic” may involve fully automatic or semi-automatic implementations involving user or administrator control over certain aspects of the implementation, depending on the desired implementation of one of ordinary skill in the art practicing implementations of the present application. Selection can be conducted by a user through a user interface or other input means, or can be implemented through a desired algorithm. Example implementations as described herein can be utilized either singularly or in combination and the functionality of the example implementations can be implemented through any means according to the desired implementations.

[0029] FIG. 1 is an overview of the traditional programming method. A user on a PC 101 works on PLC ladder software 102 and then downloads to a PLC 103, which is connected to an I/O adapter 104, an industrial robot 105, and an industrial camera 106. The I/O adapter 104 is communicated with a motor 107 and a safety scanner 108.

[0030] FIG. 2 shows a clear plot of FIG. 1. User operates PC 101 and to do PLC programming software 102, and then downloads the software code to a PLC 103. This PLC is used to communicate with an I/O adapter 104, an industrial robot 105, and an industrial camera 106. The I/O adapter 104 is connected with a motor 107 and a safety scanner 108.

[0031] FIG. 3 is the workflow of the traditional PLC programming method. As illustrated in the flow, the user may need to configure each of the devices, including the setup on the device itself and in software. Once the configuration is set up correctly, the user uses a PC for PLC ladder programming.

[0032] FIG. 4 illustrates the two methods to simplify the traditional PLC programming method, in accordance with an example implementation. It contains a speech-recognition-system-based PLC ladder programming method, and a vision-based device configuration method. The former is to function as speech recognition to generate the PLC ladder automatically on a mobile device. The latter is to confige by taking photos of devices. Both of them utilize peripheral devices of a mobile device, such as built-in microphone and camera. This method provides more convenience for users and simplifies the programming procedure with a lightweight mobile device.

[0033] In the example of the speech recognition system, the user 201 provides speech input 202 into the microphone of the mobile device 203. The mobile device 203 is configured to process the speech input with related art speech recognition techniques and generates executable PLC code. The PLC code can then be downloaded to a PLC 103.

[0034] The speech recognition system can be implemented in accordance with the related art implementations based on PLC logic software provided by the PLC vendor, from which terms are provided as the library for the speech recognition system. Such PLC logic software can function to provide mapping of text to specific code fragments. In this manner, the mobile device can be configured to take input for programmable logic controller code, in which speech is converted to text based on the PLC logic software and text is then converted to code.

[0035] In the example of the vision-based device configuration, the user takes a picture 204 of one or more apparatuses 205 to be configured from the camera of the mobile device. The mobile device 206 is configured to process the image of the one or more apparatuses and identify them using related art image recognition techniques in accordance with the desired implementation. The mobile device 206 can receive a user input to select one of the apparatuses for providing code, and then once the apparatus is selected, executable PLC code can be provided to the PLC 103.

[0036] FIG. 5 is an example of speech-recognition-system-based PLC programming method, in accordance with an example implementation. The system generates the PLC ladder based on what the user says. The user 301 can add, edit, or delete any rung in the software by saying keywords 302 to the mobile device 303, and then download the final code to a PLC 103.

[0037] FIG. 6 is an illustration of three cases of the vision-based device configuration method, in accordance with an example implementation. In the first one, a user takes a picture 204-1 of an I/O adapter 104 and then the information is displayed on the mobile device 206-1 which can be edited by the user. The second case is the configuration of a safety scanner 108. After taking a photo 204-2, the user sets up a warning and alarm areas for the safety scanner through the mobile device 206-2. The last case is to configure an industrial robot 105. As with the other two cases, the information of that robot 105, such as I/O, communication, and 3D working area, which can be seen with the mobile device 206-3, can be set up with this mobile device at 204-3. Similar to the PLC ladder, after the configuration, the code will be downloaded to PLC. In addition, the above setup/configuration will involve variables that will can be made available in the PLC ladder for the users to conduct PLC programming.

[0038] Depending on the desired example implementation, the configuration information can be obtained from the Bluetooth connection. The Bluetooth connection can be selected based on timestamp in conjunction with the image, or in cases in which multiple apparatuses are found, then based on location information or sorted based on image recognition on the apparatuses. An interface can be provided from which the user can select one of the apparatuses and corresponding Bluetooth connection from the mobile device.

[0039] Input can then be received (e.g., via voice recognition, via touch screen implementation) to reconfigure the selected apparatus; and based on such a configuration, new new configuration information from the additional input and the retrieved configuration information through modification of the old configuration information. Accordingly, executable code can be provided to the PLC and the selected apparatus can be provided with new configuration information via the PLC.

[0040] FIG. 7 is the workflow of the proposed methods, in accordance with an example implementation. On the left-hand side, the programming device generates and lists available functions for users to choose after listening to the user. Then PLC code will be downloaded to PLC once program is verified. On the right-hand side, the user takes a photo or scans a barcode of the device for configuration, and then the information can be edited on the mobile device. Once done, the configuration file can be downloaded to the PLC.

[0041] In the flow for conducting the PLC programming with speech recognition 700, at 701, the flow starts from the user talking to the programming mobile device as shown at 201 and 202 of FIG. 4. At 702, the programming device lists available options of functions as illustrated at 203 and 303 of FIG. 5. Once the options of functions are provided on the screen of the mobile device, the user can select the desired options or revise manually 703. At 704, a determination is made as to whether the programming is complete. If so (Yes), then the flow proceeds to 705, otherwise (No), the flow reverts to 701.

[0042] In the flow for starting the device configuration with a visualization 710, at 711, the flow starts when the user takes a photo image of each device and/or scans a barcode as shown at 204 of FIG. 4 or FIG. 6. In example implementations, the mobile device captures a picture image of the bar code attached to the apparatus in question with the camera, from which software as known in the art can extract the bar code information from the image and obtain the device information from the barcode information. In another example implementation, the mobile device can be configured to simply scan the bar code to obtain such information if desired. Further, the present disclosure is not limited to barcode, but can be replaced by other indicia such as quick response (QR) codes, augmented reality (AR) markers, embedded media markers, and so on in accordance with the desired implementation.

[0043] In another example implementation of extracting code information at 711, the image can be associated with a Bluetooth transmission from a Bluetooth device based on the timestamp of the image and the Bluetooth connection requests obtained from the mobile device. The Bluetooth connection can be used to retrieve the code information from the apparatus itself or the configuration information in accordance with the desired implementation.

[0044] At 712, the position and information are shown on the photo as illustrated at 206 of FIG. 4 and FIG. 6. At 713, the mobile device configures the I/O and internal information as shown at 205 and 206 of FIG. 4 and FIG. 6. At 714, a determination is made as to whether the configuration is complete. If so (yes) then the flow proceeds to 705 otherwise (No) the flow reverts to 711.

[0045] Once the configuration is made, at 705, the flow provides the transmission of executable to be executed by the PLC. The PLC can then execute the executable code at 706.

[0046] The simplified PLC programming method as described herein can be applied to most PLC and device vendors as long as they provide barcodes and information for developing this software. In addition, this method can be used in any production line, such as automotive, food, and oil & gas industries.

[0047] FIG. 8 illustrates an example of a mobile device, in accordance with an example implementation. The mobile device 800 can be any type of mobile device in accordance with the desired implementation, such as, but not limited to, cell phones, smart phones, tablets, laptops, and so on. Mobile device 800 can include camera 801, microphone 802, processor 803, memory 804, display 805, and interface (I/F) 806. Camera 801 can include any type of camera that is configured to record any form of video in accordance with the desired implementation. Microphone 802 can involve any form of microphone that is configured to record any form of audio in accordance with the desired implementation. Display 805 can involve a touch screen display configured to receive touch input to facilitate instructions to execute the functions as described herein, or a normal display such as a liquid crystal display (LCD) or any other display in accordance with the desired implementation. I/F 806 can include network interfaces to facilitate connections of the mobile device 800 to external elements such as the programmable logic controller (PLC) and any other device in accordance with the desired implementations, which may involve a connection over local area network (LAN) to connect to the PLC and factory or industrial apparatuses for security purposes. Processor 803 can be in the form of hardware processors such as central processing units (CPUs) or in a combination of hardware and software units in accordance with the desired implementation.

[0048] Memory 804 can be configured to manage a portion of all of management information as illustrated in FIG. 9, in accordance with the desired implementation.

[0049] In a first aspect, processor(s) 803 can be configured to execute a method or computer instructions involving receiving input from one or more of the camera and the microphone; generating one or more of executable code for the PLC and configuration information for the one or more apparatuses from parsing of the input; and transmitting, over the LAN, the one or more of the executable code for the PLC and the configuration information for the one or more apparatuses to the PLC as illustrated in FIG. 4 and from information as illustrated in FIG. 9.

[0050] In a second aspect, processor(s) 803 can be configured to execute the method or computer instructions in accordance with the first aspect, wherein executable code is generated from speech recognition of the input to the microphone, wherein the PLC executes the executable code once received as illustrated in FIG. 4, FIG. 5, FIG. 7, and FIG. 9.

[0051] In a third aspect, processor(s) 803 can be configured to execute the method or computer instructions in accordance with any of the above aspects, wherein generating the configuration information involves parsing an image from the camera to select an apparatus from the one or more apparatuses; retrieving the configuration information for the selected apparatus; receiving additional input to reconfigure the selected apparatus; and generating new configuration information from the additional input and the retrieved configuration information; wherein the selected apparatus is reconfigured from new configuration information as illustrated in FIG. 4, FIG. 6, and FIG. 7, and FIG. 9.

[0052] In a fourth aspect, processor(s) 803 can be configured to execute the method or computer instructions in accordance with any of the above aspects, wherein the image involves a barcode, and wherein the selection is conducted on the apparatus associated with the barcode as illustrated in FIG. 7 and FIG. 9.

[0053] In a fifth aspect, processor(s) 803 can be configured to execute the method or computer instructions in accordance with any of the above aspects, wherein the selection is based on a Bluetooth device transmitting an identifier to the mobile device as described in FIG. 7 and FIG. 9.

[0054] FIG. 9 illustrates example management information which may be stored on the mobile device to facilitate the example implementations described herein. Management information can include a code identifier corresponding to information extracted from the bar code, from a Bluetooth connection, or other code in accordance with the desired implementation, the device identifier that corresponds to the code (e.g., PLC, apparatus, etc.), and configuration information of the device. Configuration information can involve configuration information as provided by the apparatus. The management information of FIG. 9 can be provided by the PLC over LAN, and/or can be managed either by memory 804 or by PLC depending on the desired implementation.

[0055] Some portions of the detailed description are presented in terms of algorithms and symbolic representations of operations within a computer. These algorithmic descriptions and symbolic representations are the means used by those skilled in the data processing arts to convey the essence of their innovations to others skilled in the art. An algorithm is a series of defined steps leading to a desired end state or result. In example implementations, the steps carried out require physical manipulations of tangible quantities for achieving a tangible result.

[0056] Unless specifically stated otherwise, as apparent from the discussion, it is appreciated that throughout the description, discussions utilizing terms such as “processing,” “computing,” “calculating,” “determining,” “displaying,” or the like, can include the actions and processes of a computer system or other information processing device that manipulates and transforms data represented as physical (electronic) quantities within the computer system’s registers and memories into other data similarly represented as physical quantities within the computer system’s memories or registers or other information storage, transmission or display devices.

[0057] Example implementations may also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may include one or more general-purpose computers selectively activated or reconfigured by one or more computer programs. Such computer programs may be stored in a computer readable medium, such as a computer-readable storage medium or a computer-readable signal medium. A computer-readable storage medium may involve tangible mediums such as, but not limited to optical disks, magnetic disks, read-only memories, random access memories, solid state devices and drives, or any other types of tangible or non-transitory media suitable for storing electronic information. A computer readable signal medium may include mediums such as carrier waves. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Computer programs can involve pure software implementations that involve instructions that perform the operations of the desired implementation.

[0058] Various general-purpose systems may be used with programs and modules in accordance with the examples herein, or it may prove convenient to construct a more specialized apparatus to perform desired method steps. In addition, the example implementations are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the techniques of the example implementations as described herein. The instructions of the programming language(s) may be executed by one or more processing devices, e.g., central processing units (CPUs), processors, or controllers.

[0059] As is known in the art, the operations described above can be performed by hardware, software, or some combination of software and hardware. Various aspects of the example implementations may be implemented using circuits and logic devices (hardware), while other aspects may be implemented using instructions stored on a machine-readable medium (software), which if executed by a processor, would cause the processor to perform a method to carry out implementations of the present application. Further, some example implementations of the present application may be performed solely in hardware, whereas other example implementations may be performed solely in software. Moreover, the various functions described can be performed in a single unit, or can be spread across a number of components in any number of ways. When performed by software, the methods may be executed by a processor, such as a general purpose computer, based on instructions stored on a computer-readable medium. If desired, the instructions can be stored on the medium in a compressed and/or encrypted format.

[0060] Moreover, other implementations of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the techniques of the present application. Various aspects and/or components of the described example implementations may be used singly or in any combination. It is intended that the specification and example implementations be considered as examples only, with the true scope and spirit of the present application being indicated by the following claims