INTELLIGENT MOBILE DEVICE TEST FIXTURE
20170052527 ยท 2017-02-23
Inventors
- Thomas Eugene Dougherty (Merrimack, NH, US)
- Hongyu Wang (Dalian, CN)
- Hongbing Gu (Dalian, CN)
- Bahram Omidfar (Summit, WI, US)
Cpc classification
G05B19/402
PHYSICS
G05B2219/31295
PHYSICS
Y02P90/02
GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
G05B2219/34221
PHYSICS
International classification
Abstract
Described are methods and apparatuses, including computer program products, for testing mobile devices using an intelligent mobile device test fixture. A server computing device generates programmatic instructions for testing an application on a mobile device coupled to the test fixture, where the programmatic instructions are based upon a test script, and include a first set of instructions and a second set of instructions. The server computing device transmits the first set of instructions to the application, and transmits the second set of instructions to a controller board coupled to the test fixture. The application executes the first set of instructions to activate functionality. The controller board executes the second set of instructions to activate one or more servo motors of the test fixture that control movement of the fixture. The server computing device determines an updated state of the application after execution of each instruction.
Claims
1. A computerized method for testing mobile devices using an intelligent mobile device test fixture, the method comprising: generating, by a server computing device, programmatic instructions for testing an application on a mobile device held within an intelligent mobile device test fixture, wherein the programmatic instructions are based upon a test script and include a first set of instructions for performing functionality of the application and a second set of instructions for controlling the intelligent mobile device test fixture; transmitting, by the server computing device, the first set of instructions to the application on the mobile device; transmitting, by the server computing device, the second set of instructions to a controller board coupled to the intelligent mobile device test fixture; executing, by the application, the first set of instructions to activate functionality in the application; executing, by the controller board, the second set of instructions to activate one or more servo motors of the intelligent mobile device test fixture that control movement of the fixture; and determining, by the server computing device, an updated state of the application after execution of each instruction in the first set of instructions and the second set of instructions.
2. The method of claim 1, wherein the execution of the first set of instructions by the application is synchronized with the execution of the second set of instructions by the controller board.
3. The method of claim 1, wherein the intelligent mobile device fixture comprises a platform with two expanding tracks arranged perpendicular to each other and connected at the center of the platform, wherein each expanding track includes two arms that move in opposite directions along the track.
4. The method of claim 3, wherein each arm includes a tab at the end of the arm, the tab being used to secure the mobile device.
5. The method of claim 3, wherein the one or more servo motors operate to rotate the platform of the intelligent mobile device fixture.
6. The method of claim 3, wherein one or more linear actuators are coupled to the platform, the actuators operable to slide the platform laterally in either direction.
7. The method of claim 1, wherein the intelligent mobile test fixture comprises an audio playback apparatus including a speaker.
8. The method of claim 7, wherein the second set of instructions includes an instruction that, when executed by the controller board, activates playback of an audio file on the speaker of the audio playback apparatus.
9. The method of claim 8, wherein the first set of instructions includes an instruction that, when executed by the application, activates a microphone of the mobile device to capture the playback of the audio file from the speaker.
10. The method of claim 1, wherein the intelligent mobile test fixture comprises a display screen.
11. The method of claim 10, wherein the second set of instructions includes an instruction that, when executed by the controller board, activates display of a digital image on the display screen.
12. The method of claim 11, wherein the first set of instructions includes an instruction that, when executed by the application, activates an image capture apparatus of the mobile device to capture an image of the display screen.
13. The method of claim 1, wherein the server computing device translates the test script into the programmatic instructions.
14. The method of claim 1, wherein the first set of instructions is written in a first programming language and the second set of instructions is written in a second programming language.
15. A system for testing mobile devices using an intelligent mobile device test fixture, the system comprising a server computing device coupled to the intelligent mobile device test fixture, the server computing device configured to: generate programmatic instructions for testing an application on a mobile device coupled to an intelligent mobile device test fixture, wherein the programmatic instructions are based upon a test script, and include a first set of instructions for performing functionality of the application and a second set of instructions for controlling the intelligent mobile device test fixture; transmit the first set of instructions to the application on the mobile device; transmit the second set of instructions to a controller board coupled to the intelligent mobile device test fixture; wherein the application is configured to execute the first set of instructions to activate functionality in the application; wherein the controller board is configured to execute the second set of instructions to activate one or more servo motors of the intelligent mobile device test fixture that control movement of the fixture; and wherein the server computing device is configured to determine an updated state of the application after execution of each instruction in the first set of instructions and the second set of instructions.
16. The system of claim 15, wherein the execution of the first set of instructions by the application is synchronized with the execution of the second set of instructions by the controller board.
17. The system of claim 15, wherein the intelligent mobile device fixture comprises a platform with two expanding tracks arranged perpendicular to each other and connected at the center of the platform, wherein each expanding track includes two arms that move in opposite directions along the track.
18. The system of claim 17, wherein each arm includes a tab at the end of the arm, the tab being used to secure the mobile device.
19. The system of claim 17, wherein the one or more servo motors operate to rotate the platform of the intelligent mobile device fixture.
20. The system of claim 17, wherein one or more linear actuators are coupled to the platform, the actuators operable to slide the platform laterally in either direction.
21. The system of claim 15, wherein the intelligent mobile test fixture comprises an audio playback apparatus including a speaker.
22. The system of claim 21, wherein the second set of instructions includes an instruction that, when executed by the controller board, activates playback of an audio file on the speaker of the audio playback apparatus.
23. The system of claim 22, wherein the first set of instructions includes an instruction that, when executed by the application, activates a microphone of the mobile device to capture the playback of the audio file from the speaker.
24. The system of claim 15, wherein the intelligent mobile test fixture comprises a display screen.
25. The system of claim 24, wherein the second set of instructions includes an instruction that, when executed by the controller board, activates display of a digital image on the display screen.
26. The system of claim 25, wherein the first set of instructions includes an instruction that, when executed by the application, activates an image capture apparatus of the mobile device to capture an image of the display screen.
27. The system of claim 15, wherein the server computing device translates the test script into the programmatic instructions.
28. The system of claim 15, wherein the first set of instructions is written in a first programming language and the second set of instructions is written in a second programming language.
29. A computer program product, tangibly embodied in a non-transitory computer readable storage medium, for testing mobile devices using an intelligent mobile device test fixture, the computer program product including instructions operable to cause a server computing device coupled to the intelligent mobile device test fixture to: generate programmatic instructions for testing an application on a mobile device coupled to an intelligent mobile device test fixture, wherein the programmatic instructions are based upon a test script, and include a first set of instructions for performing functionality of the application and a second set of instructions for controlling the intelligent mobile device test fixture; transmit the first set of instructions to the application on the mobile device; transmit the second set of instructions to a controller board coupled to the intelligent mobile device test fixture; wherein the application is configured to execute the first set of instructions to activate functionality in the application; wherein the controller board is configured to execute the second set of instructions to activate one or more servo motors of the intelligent mobile device test fixture that control movement of the fixture; and wherein the server computing device is configured to determine an updated state of the application after execution of each instruction in the first set of instructions and the second set of instructions.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The advantages of the invention described above, together with further advantages, may be better understood by referring to the following description taken in conjunction with the accompanying drawings. The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention.
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
[0021]
DETAILED DESCRIPTION
[0022]
[0023] The arms can move along their respective tracks in order to accommodate mobile devices of varying sizes and dimensions. For example, the arms can move closer to the center of the platform, thereby decreasing the distance between the tabs to fit smaller devices. The arms can also move further from the center of the platform, thereby increasing the distance between the tabs to fit larger devices. In some embodiments, the arms can move in concert with each other. In some embodiments, the arms can move independently of each other.
[0024]
[0025] The servo motors 204a-204d are operable to rotate the platform 102 (and mobile device contained within) along three different axes (e.g., X, Y, and Z) to position the mobile phone in different orientations for the purpose of device testing. For example, servo motor 204a can rotate the bottom frame (206a) within a 360 range of motion, causing the platform 102 and mobile device to similarly rotate. In an embodiment, the bottom frame 206a rotates only within a 180 range of motion so as to avoid twisting of support harnesses (e.g., power cabling) that connect the servo motors 204a-204d to their respective controllers. In another example, servo motors 204b and 204c can rotate the top frame 206b within a 180 range of motion (i.e., in an arc from the left side to the top to the right side of the servo assembly 202) the platform 102 and mobile device to similarly rotate. In addition, servo motor 204d is operable to move the platform 102 in a 90 range of motion (i.e., between a parallel plane and a perpendicular plane) with respect to the servo assembly, enabling the mobile device to move between a flat, lying down position and an upright position.
[0026]
[0027] In some embodiments, the intelligent mobile device test fixture 100 includes one or more alignment sensors that can be embedded in the platform 102 in order to enable automated positioning and alignment of the test fixture. For example, an alignment sensor can be positioned at either end, or at both ends, of the platform 102. The alignment sensor(s) can be coupled to the controller board in order to send a signal to the controller board regarding positioning and alignment of the platform 102. The apparatus can also include additional proximity sensors embedded in other components of the apparatus (e.g., the stations 506a, 506b of
[0028]
[0029] It should be appreciated that the server computing device 406 can generate programmatic instructions to control both the controller boards 404a-404d (and correspondingly, the servo motors of the test fixtures 402a-402d) and applications executing on the mobile devices in the fixtures 402a-402d in order to perform a test procedure. For example, a developer or tester may author a test script (e.g., written in Java), which delineates a sequence of commands or operations to be performed in a test routine. The server computing device 406 translates the test script into a set of programmatic instructions to control the application and/or the test fixture in performing the test routine. In some embodiments, the programmatic instructions include a set of instructions for the application (e.g., written as an XML configuration file) and another set of instructions for the test fixture (e.g., written in C++).
[0030] The respective sets of instructions can be synchronized so that the application and the test fixture are coordinated to perform the operations of the test routine. For example, the individual instructions in each set of instructions can be associated with a timestamp or other form of temporal indicator so that the application and the controller boards execute the respective instructions at appropriate times to achieve synchronization.
[0031] In some embodiments, the communications assembly 400 of
[0032] As many mobile applications utilize images from the camera as part of their functionality, the table apparatus 500 advantageously enables developers and testers to automate app testing using a variety of different images and mobile device orientations in rapid succession, increasing the speed and efficiency of the testing process instead of relying on a manual tester to swap between images and re-orient the mobile device each time. As shown in
[0033] An exemplary image display apparatus comprises a thin film transistor (TFT) LCD display assembly, including a SainSmart MEGA2560 R3 development board coupled to a seven-inch TFT LCD screen (resolution: 800480, aspect ratio: 15:9, color/transparent screen), with an SD card slot and a TFT shield. The image display apparatus station 506a is capable of communicating with the server computing device 406 (e.g., via wireless connection) to receive commands and images that approximate a wide variety of use conditions. For example, in testing a mobile barcode scanning app, the image display apparatus station 506a can receive many barcode images from the server computing device 406 for display to the mobile device, including barcode images that are incomplete, skewed, blurry, in different lighting conditions, and so forth to encompass many different environments in which the app may be used. Also, the test fixture 404a can be moved in conjunction with the display of images so as to determine whether the barcode can be read if the mobile device is in various orientations and positions with respect to the image display, thereby enabling developers to determine potential problem scenarios and build solutions to overcome them during the testing phase.
[0034] In addition, many mobile applications use the microphone and/or speaker embedded in a mobile device to capture audio and play back audio in the context of mobile application functionality. For example, a mobile app may allow a user to speak into the microphone and leverage voice recognition or speech-to-text functionality to enhance the usability of the application. As shown in
[0035] An exemplary audio capture/playback apparatus comprises a voice playback controller apparatus including a micro controller (e.g., Anduino UNO or equivalent) with an MP3 trigger (e.g., WIG-11029 or WIG-13720) and command code logic for sequencing voice prompts and lookup. The voice playback controller assembly is coupled to a speaker assembly for producing audio that is then captured by the microphone of the mobile device.
[0036] In one embodiment, the audio file can comprise a voice command for operation of the application on the mobile device such that, when the audio file is played back by the audio capture/playback apparatus 406 and is captured by a microphone of the mobile device, the application receives the voice command and executes functionality of the application that is responsive to the voice command. For example, the audio file can comprise a voice command of Buy one hundred shares of Apple stock. The microphone of the mobile device captures the voice command, and the application converts the voice command into text. Then, the application parses the text to extract actionable information and generate corresponding actions in the applicationsuch as, e.g., automatically navigating to a trading page, filling in a user input field for quantity as 100, filling in a user input field for transaction as Buy, and filling in a user input field for ticker symbol as AAPL. Then, the application can, e.g., activate a Preview button in the user interface of the application which displays a preview page and enables the application to verify that the above-identified information is correct.
[0037] The audio capture/playback apparatus 506b is capable of communicating with the server computing device 406 (e.g., via wireless connection) to receive commands and audio files that approximate a wide variety of use conditions. For example, in testing an interactive voice response app, the audio capture/playback apparatus 506b can receive audio files of a variety of different voices from the server computing device 406 for playback to the mobile device, including audio files that are incomplete, noisy, quiet, choppy, and so forth to encompass many different environments in which the app may be used. Also, the test fixture 404b can be moved in conjunction with the playback of audio so as to determine whether the audio files can be detected by the microphone of the mobile device if the mobile device is in various orientations and positions with respect to the audio capture/playback apparatus 506b, thereby enabling developers to determine potential problem scenarios and build solutions to overcome them during the testing phase.
[0038] It should be appreciated that although
[0039]
[0040] The application (608) executes the first set of instructions to activate functionality in the application. The application can execute the first set of instructions to activate functionality such as manipulation of user interface elements in the application, entry of data into the application, and so forth. One aspect of functionality that the application can activate is mimicking user operation of the mobile device by, e.g., simulating touchscreen operations. For example, an instruction in the first set of instructions can be to provide input simulating a touch of the screen at certain x-y coordinates. The application can execute this instruction and activate the input at the specified coordinates, which results in interaction with the user interface displayed on the screen.
[0041] The controller board 404a (608) executes the second set of instructions to activate one or more servo motors of the intelligent mobile device test fixture 402a that control movement of the fixture. For example, an instruction in the second set of instructions can be to rotate the mobile device to change the orientation of its screen (e.g., from portrait to landscape). The controller board 404a can execute this instruction to cause the servo motors to rotate the text fixture 402a in a manner that results in, e.g., a gyroscope of the mobile device triggering the display screen of the mobile device to change its orientation.
[0042] The server computing device 406 determines (610) an updated state of the application based upon execution of the first set of instructions and the second set of instructions. For example, after execution of each instruction in the first set of instructions and the second set of instructions, the server computing device can receive a current state of the application from the mobile devicewhich may be comprised of a plurality of data elements relating to the application (e.g., workflow state, screen state, input state, mobile device orientation, and the like). The server computing device 406 compares the current state as received from the application with the prior state of the application that was received from the application after execution of the preceding instruction and determines how the state has changed. In some embodiments, the server computing device 406 can further compare the current state of the application with an expected state of the application to determine whether any errors have occurred in the application as a result of the testing procedure. In some embodiments, the server computing device 406 is network addressable, allowing for remote users to operate the intelligent mobile device test fixture from remote locations. Similar to the core instruction set on both the server computing device 406 and intelligent mobile device test fixture, multiple server computing units can be combined to allow parallel testing activities in an orchestrated manner dictated by the test scripts and intelligent mobile test fixture combinations.
[0043]
[0044] It should be appreciated that although the above techniques are described in the context of testing an application on a mobile device, the same techniques are applicable to a wide variety of devices and applications without departing from the scope of invention. For example, the intelligent mobile device test fixture described herein can be used to test other types of devices, e.g., Internet-of-Things (IoT) devices, smart appliances, wearable devices, and so forth. As one example, any device that includes voice-activated functionality (e.g., smart watches, smart TV devices) can be tested using the methods and systems described herein.
[0045] The above-described systems and methods can be implemented in digital electronic circuitry, in computer hardware, firmware, and/or software. The implementation can be as a computer program product (i.e., a computer program tangibly embodied in a computer readable storage medium). The implementation can, for example, be in a machine-readable storage device and/or include a propagated signal, for execution by, or to control the operation of, data processing apparatus. The implementation can, for example, be a programmable processor, a computer, and/or multiple computers.
[0046] A computer program can be written in any form of programming language, including compiled and/or interpreted languages, and the computer program can be deployed in any form, including as a stand-alone program or as a subroutine, element, and/or other unit suitable for use in a computing environment. A computer program can be deployed to be executed on one computer or on multiple computers at one site.
[0047] Method steps can be performed by one or more programmable processors executing a computer program to perform functions of the invention by operating on input data and generating output. Method steps can also be performed by and an apparatus can be implemented as special purpose logic circuitry. The circuitry can, for example, be a FPGA (field programmable gate array), an ASIC (application-specific integrated circuit), a DSP (digital signal processor), and/or any other discrete circuitry that is configured to implement the required functions. Modules, subroutines, and software agents can refer to portions of the computer program, the processor, the special circuitry, software, and/or hardware that implements that functionality.
[0048] Processors suitable for the execution of a computer program include, by way of example, special purpose microprocessors specifically programmed with instructions to carry out the methods described herein. Generally, a processor receives instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer can include, can be operatively coupled to receive data from and/or transfer data to one or more mass storage devices for storing data (e.g., magnetic, magneto-optical disks, or optical disks).
[0049] Data transmission and instructions can also occur over a communications network. Computer readable mediums suitable for embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices. The computer readable mediums can, for example, be EPROM, EEPROM, flash memory devices, magnetic disks, internal hard disks, removable disks, magneto-optical disks, CD-ROM, and/or DVD-ROM disks. The processor and the memory can be supplemented by, and/or incorporated in special purpose logic circuitry.
[0050] To provide for interaction with a user, the above described techniques can be implemented on a computer having a display device or a transmitting device. The display device can be, for example, a cathode ray tube (CRT) and/or a liquid crystal display (LCD) monitor. The interaction with a user can be, for example, a display of information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer (e.g., interact with a user interface element). Other kinds of devices can be used to provide for interaction with a user. Other devices can be, for example, feedback provided to the user in any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback). Input from the user can be, for example, received in any form, including acoustic, speech, and/or tactile input.
[0051] The remote/client device and the computing device can include, for example, a computer, a computer with a browser device, a telephone, an IP phone, a mobile device (e.g., cellular phone, personal digital assistant (PDA) device, smart phone, laptop computer, electronic mail device), and/or other communication devices. The browser device includes, for example, a computer (e.g., desktop computer, laptop computer) with a World Wide Web browser (e.g., Microsoft Internet Explorer available from Microsoft Corporation, Mozilla Firefox available from Mozilla Corporation). The mobile computing device includes, for example, a Blackberry or iPhone.
[0052] The web servers can be, for example, a computer with a server module (e.g., Microsoft Internet Information Services available from Microsoft Corporation, Apache Web Server available from Apache Software Foundation, Apache Tomcat Web Server available from Apache Software Foundation).
[0053] The above described techniques can be implemented in a distributed computing system that includes a back-end component. The back-end component can, for example, be a data server, a middleware component, and/or an application server. The above described techniques can be implemented in a distributing computing system that includes a front-end component. The front-end component can, for example, be a client computer having a graphical user interface, a Web browser through which a user can interact with an example implementation, and/or other graphical user interfaces for a transmitting device. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network).
[0054] The system can include clients and servers. A client and a server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
[0055] The above described communication networks can be implemented in a packet-based network, a circuit-based network, and/or a combination of a packet-based network and a circuit-based network. Packet-based networks can include, for example, the Internet, a carrier internet protocol (IP) network (e.g., local area network (LAN), wide area network (WAN), campus area network (CAN), metropolitan area network (MAN), home area network (HAN)), a private IP network, an IP private branch exchange (IPBX), a wireless network (e.g., radio access network (RAN), 802.11 network, 802.16 network, general packet radio service (GPRS) network, HiperLAN), and/or other packet-based networks. Circuit-based networks can include, for example, the public switched telephone network (PSTN), a private branch exchange (PBX), a wireless network (e.g., RAN, bluetooth, code-division multiple access (CDMA) network, time division multiple access (TDMA) network, global system for mobile communications (GSM) network), and/or other circuit-based networks.
[0056] Comprise, include, and/or plural forms of each are open ended and include the listed parts and can include additional parts that are not listed. And/or is open ended and includes one or more of the listed parts and combinations of the listed parts.
[0057] One skilled in the art will realize the invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The foregoing embodiments are therefore to be considered in all respects illustrative rather than limiting of the invention described herein.