INFERRING USER INTERACTION WITH AN IFRAME
20180052809 ยท 2018-02-22
Inventors
- John Nguyen (Redmond, WA, US)
- Peter Blair Gonzalez del Solar (Mercer Island, WA, US)
- Moein Pahlavan Yali (Bellevue, WA, US)
Cpc classification
G06F3/04842
PHYSICS
G06F9/44552
PHYSICS
G06F16/958
PHYSICS
G06F9/44505
PHYSICS
G06F9/44521
PHYSICS
G06F3/0481
PHYSICS
International classification
Abstract
Computer systems, devices, and associated methods of inferring user interaction with a web part comprising an <iframe> element are disclosed herein. In one embodiment, a method includes observing a location of a user input target on a web page rendered by a web browser. When the location of the user input target is over a web part comprising the host container and the web page detects that the web page has lost focus and a new active element is an <iframe>, the method includes inferring user interaction with respect to the <iframe> element and emphasizing the web part.
Claims
1. A method of inferring interaction by a user with a host container in a web page having a host document being rendered by a web browser, the host container hosting embedded content, wherein communication of user interactions with the embedded content to the host document is not permitted, the method comprising: tracking a location of a user input target on the document, the location of the user input target corresponding to a document element that most recently received user input; and when the location of the user input target is over the host container, detecting whether the host document has lost focus and an active element of the host document has changed to the host container when the host document lost focus; and in response to detecting that the host document has lost focus and the active element of the host document has changed to the host container, inferring user interaction with the host container in the web page; and based on the inferred user interaction, emphasizing the host container as displayed on the web page, thereby allowing user interactions with the host container without introducing additional programming to the embedded content in the host container.
2. The method of claim 1 wherein observing the location of the user input target includes tracking a mouse cursor movement into and out of the host container.
3. The method of claim 1, further comprising: periodically checking whether the active element of the document is the active element as determined when the host container was emphasized; and in response to determining that the active element is not the active element of the document as determined when the host container was emphasized, determining whether the active element is of the host container type; and in response to determining that the active element is not of the host container type, deemphasizing the host container.
4. The method of claim 1, further comprising: periodically checking whether the active element of the document is the active element as determined when the host container was emphasized; and in response to determining that the active element is not the active element of the document as determined when the host container was emphasized, determining whether a new location of the user input target is over the host container, and in response to determining that the new location of the user input target is not over the host container, deemphasizing the host container.
5. The method of claim 1, further comprising: periodically checking whether the active element of the document is the active element as determined when the host container was emphasized; and in response to determining that the active element is not the active element of the document as determined when the host container was emphasized, determining whether a new location of the user input target is over the host container, and in response to determining that the new location of the user input target is over the host container, maintaining emphasizing the host container.
6. The method of claim 1 wherein emphasizing the host container comprises modifying a visual appearance detail associated with the host container.
7. The method of claim 1 wherein emphasizing the host container comprises displaying tools associated with the host container.
8. The method of claim 1 wherein emphasizing the host container comprises displaying properties of a visual object associated with the host container.
9. The method of claim 1 wherein the host container is an <iframe> element, and wherein observing the location of the user input target includes tracking a mouse cursor movement into and out of a document element in the host document that corresponds to the <iframe>.
10. A method of inferring interaction by a user with content of a document being rendered by a web browser, the content comprising a host container hosting embedded content, wherein communication of user interaction with the embedded content to a parent element of the document is not permitted, the method comprising: tracking a location of a user input target on the document, the location of the user input target corresponding to a document element that most recently received user input; when the location of the user input target is over the content comprising the host container: detecting whether another element of the document different from the host container has lost focus; and in response to detecting that the element of the document different from the host container has lost focus while the input target is over the content comprising the host container, determining whether an active element of the document is of a host container type; and in response to determining that the active element of the document is of the host container type, emphasizing the content as displayed in the web browser.
11. The method of claim 10 wherein emphasizing the content includes emphasizing the content in response to both: that the element of the document different from the host container has lost focus, and that the active element of the document is of the host container type.
12. The method of claim 11, further comprising: periodically checking whether the active element of the document is the active element as determined when the content was emphasized; and in response to determining that the active element is not the active element of the document as determined when the content was emphasized, determining whether the active element is of the host container type; and when the active element is not of the host container type, deemphasizing the content.
13. The method of claim 11, further comprising: periodically checking whether the active element of the document is the active element as determined when the host container was emphasized; and in response to determining that the active element is not the active element of the document as determined when the host container was emphasized, determining whether a new location of the user input target is over the content, and when the new location of the user input target is not over the content, deemphasizing the host container.
14. The method of claim 10, wherein the host container is an <iframe> element.
15. The method of claim 10, wherein emphasizing the content comprises modifying a visual appearance detail associated with the content, displaying tools associated with the host container, or displaying properties of the host container.
16. A computing device, comprising: a processor; and a memory containing instructions executable by the processor to infer user interaction with respect to content of a document being rendered by a web browser, the content of the document comprising a host container, wherein information associated with user interaction with respect to the host container does not bubble up to parent elements, the memory containing additional instructions executable by the processor to cause the computing device to: track a location of a user input target on the document, the location of the user input target corresponding to an input location for receiving user input; and when the location of the user input target is over the content comprising the host container: detect whether an element of the document different from the host container has lost focus; and in response to detecting that the element of the document different from the host container has lost focus while the input target is over the content comprising the host container, emphasize the content as displayed by the web browser.
17. The computing device of claim 16 wherein the memory contains further instructions executable by the processor to cause the computing device to: determine whether an active element of the document is of a host container type, wherein the content comprising the host container is emphasized in response to both: detect that the element of the document different from the host container has lost focus, and determine that the active element of the document is of the host container type.
18. The computing device of claim 17, wherein the memory contains further instructions executable by the processor to cause the computing device to: periodically determine whether the active element of the document is the active element as determined when the content was emphasized; in response to determining that the active element is not the active element of the document as determined when the content was emphasized, determine whether the active element is of the host container type; and when the active element is not of the host container type, deemphasizing the content.
19. The computing device of claim 17, wherein the memory contains further instructions executable by the processor to cause the computing device to: periodically determine whether the active element of the document is the active element as determined when the content was emphasized; in response to determining that the active element is the active element of the document as determined when the content was emphasized, continue to emphasize the content as displayed in the web browser.
20. The method of claim 17, wherein the host container comprises an <iframe>.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
DETAILED DESCRIPTION
[0021] Certain embodiments of systems, devices, components, modules, routines, data structures, and processes for inferring user interaction within a host container in a document are described below. In the following description, specific details of components are included to provide a thorough understanding of certain embodiments of the disclosed technology. A person skilled in the relevant art will also understand that the technology can have additional embodiments. The technology can also be practiced without several of the details of the embodiments described below with reference to
[0022] As used herein, the term element generally refers to an individual component of a document, for example, an HTML document, after the document is parsed into a document model for rendering the document by a web browser. The document model can comprise an API, such as the Document Object Model. The document model can organize a document into a tree structure with each node of the tree structure representing an element. For example, an element in an HTML document can comprise a <title> element, for defining a title of the document.
[0023] Also used herein, the term parent element generally refers to an element in the tree structure of the document that is at a node higher up the tree structure than a child element to the parent element. A parent element of all elements of a web page can be a root element. Generally, events, such as user interaction (e.g., a mouse click) with respect to a child element, bubble up from the child element to one or more corresponding parent elements. In other words, information associated with the event (e.g., that the mouse was clicked) is made available to scripts that monitor corresponding parent nodes.
[0024] As used herein, the term host container generally refers to an element of a host document that hosts embedded content in the host document. The embedded content can be another document or a portion thereof different than the host document. In some embodiments, notification of events, such as user interactions with the embedded content, is not communicated outside the host container to other elements of the host document. For example, a web browser rendering the web page based on the host document can prevent scripts included in the hosted content from communicating with scripts of parent elements of the host container. Accordingly, parent elements of the host container can be restricted from detecting user interactions with the embedded content. In some embodiments, a host container comprises an <iframe> of an HTML host document. In other embodiments, a host container can include other suitable types of sandboxes in a host document.
[0025] As used herein, the term focused generally refers to a state for an element in a document, under which the element is selected by the web browser rendering the document. The web browser can be configured to focus on only one element at a time. When an element stops being the focused element, the element is said to lose focus or blur. In some embodiments, the web browser changes focus to an element when the element is interacted with by a user. For example, when user input (e.g., keyboard input or a finger tap) is received by the element, the element is focused in the web browser. As used herein, the term active used in relation to an element generally refers to an element that has most recently received user input. For example, under the DOM, the web browser can indicate the active element to scripts by utilizing the document.activeElement API.
[0026] As used herein, the term web part generally refers to a web page object or model that can be an extension to a web page. Developers can create web parts that an author of a web page can add to the web page. Example web parts include a calendar, a voting poll, a weather widget, a sales dashboard, and so forth. As used herein, the term emphasized used in relation to a web part generally refers to a highlighted state of the web part, as rendered by the web browser. In some embodiments, a web part is emphasized by changing a visual appearance of the web part. For example, the web application can generate a glow around a web part that is emphasized. In other embodiments, a web part is emphasized by generating controls and/or providing information associated with the web part. Also used herein, the term client-side generally refers to program codes that run as a script inside a web page on a web browser executing on a client device. By contrast, server-side code runs on a web server in, for example, a datacenter.
[0027]
[0028] Upon receiving a web page request 108 from the user 103 via the computer network, the web server 106 provides the page model 110 to the client device 102 instead of a fully or partially rendered web page. Upon receiving the page model 110, in certain embodiments, a web browser 112 on the client device 102 generates a content request 116 for retrieving various static elements 118 from the CDN servers 105 and executes the list of script modules to, for example, transmit a data request 120 to content servers 107 for retrieving dynamic data 122 from the content server 107 and render a web page 114 on the client device 102. In other embodiments, the web browser 112 can request and retrieve the static elements 118 from other suitable sources.
[0029] In some embodiments, the web page 114 can comprise an HTML document rendered by the web browser 112, including a graphical user interface for the web page 114. In the illustrated embodiment, the web page 114 can comprise functional elements, such as a graphical user interface GUI component 126 and a user input inference component 128. In other embodiments, the web page 114 can include additional or different components in some embodiments. Example implementations of the computing system 100 are described below with reference to
[0030] In some embodiments, the web servers 106 comprise a content management system (CMS), which can host a customizable database for storing web sites, document libraries, web pages, and other content. The CMS enables web pages 114 to be created by the user 103 or other suitable entities. As the author, the user 103 can be an information worker who works directly within the web browser to compose text, graphic, etc., and design visual appearances of the web page 114. Being the author, the user 103 is unlike a graphic designer or developer, who use program codes such as HTML or JavaScript for building the web page. Examples of CMS products include WordPress, Drupal, and SharePoint provided by Microsoft Corporation of Redmond, Wash.
[0031] Several embodiments of the disclosed technology are directed to a page creation engine that enables client-side workflow. Information workers or other users can use the page creation engine to create web pages 114 with rich content with improved productivity. Such content can include, for example, a page that shows a list of documents a user has been working on recently. The page creation engine can be configured to provide a web page that allows a user to create or edit a web page having web parts. Several embodiments of the disclosed technology are directed to inferring user interactions with a web part having an embedded host container from which user interactions are not directly observable from outside the web part. In response to inferring user interaction with the web part, the document or web page, as rendered by the web browser, can emphasize the web part, output additional web page elements, or perform other suitable operations, as outlined below with reference to
[0032] For an author, such as an information worker, the web page lifecycle starts when the author creates a new web page 114.
[0033] As shown in
[0034] As shown in
[0035] In response to inferring that user interaction has been received with respect to the <iframe> element 232, the web page 114 can be configured to perform an action. In some embodiments, the action is for the GUI component 126 to emphasize the web part 222 that includes the <iframe> element 232, as shown in
[0036] In some embodiments, the GUI component 126 of the web page 114 can be configured to continue performing the action (e.g., emphasizing the web part 222) until the user input inference component 128 detects user interaction outside the web part 222. For example, the user input inference component 128 can be configured to periodically check the value of the document.activeElement API provided by the web browser. The user input inference component 128 can be configured to determine whether the <iframe> element identified when it was inferred that user interaction was received is still the value of document.activeElement. When the value of document.activeElement is not the same <iframe> element as identified earlier, the user input inference component 128 can be configured to determine whether the value of document.activeElement is a different <iframe> element or a non-<iframe> element. In response to determining that the value of document.activeElement is a non-<iframe> element, the web page 114 can be configured to cease performing the action (e.g., the GUI component 126 can remove the emphasis on the web part 222).
[0037] In response to determining that the value of document.activeElement is an <iframe> element, the user input inference component 114 can be configured to examine a location of the input target for user interaction, and if the location of the input target is over the web part 222 when the value of document.activeElement changed to the <iframe>, the web page can be configured to determine that user interaction has been received with respect to another <iframe> element of the web part 222. If the location of the input target is not over the web part 222 when the value of document.activeElement changed, user input inference component 128 can be configured to infer that an <iframe> element of another object has been interacted with. In response to inferring that another <iframe> element of another object has been interacted with, the web page 114 can be configured to cease performing the action.
[0038] The web page 114 can also be configured to modify a web part 222 that is emphasized in response to the author's input. For example, the web page 114 can receive user input with respect to an edit button 228 displayed by a toolbox 226, and generate an interface for receiving modifications to the web part. For example, in certain embodiments, a web part can have properties that can be customized by the author using a property pane generated by the web page. Each web part can have corresponding properties that are relevant to that web part. In the illustrated example, the weather widget web part 222 can have properties such as a geographic location, as specified by the author, for identifying relevant weather information. In other embodiments, the web part can have other suitable properties that can be modified by using command lines, or other suitable editing means. The web page 114 can also be configured to receive other suitable modifications to an emphasized object, such as to delete the object or resize or move the object on the web page. The property pane and editing tools can be viewed by the author while the web page is in an edit mode. Visitors to the web page 114 can view the page in read mode, as is shown by the web page 114 in
[0039]
[0040]
[0041] As shown in
[0042] The operations include a decision stage 274 for determining whether the user input target was above a web part being displayed on the web page by the browser when the change in focus of the web page was observed at stage 272. For example, it can be determined whether a cursor was most recently moved within a border of the web part when the web page lost focus. When the user input target is not over the web part when the change in focus is observed, the operations include responding to user interaction at stage 276. Responding to user interactions can be handled in conventional ways when it is detected outside of the <iframe> element.
[0043] When the user input target is over the web part when the change in focus is observed, the operations include inferring user interaction with the web part at stage 278. In some embodiments, user interaction is inferred with respect to the web part when, in addition to the user input target being observed over the web part, the new value of document.activeElement is an <iframe> element. The operations then include emphasizing the web part at stage 280.
[0044] As shown in
[0045] When the value of document.activeElement has changed, operations include a decision stage 290 for determining whether the value of document.activeElement is an <iframe> element. When the value of document.activeElement is not an <iframe> element, operations include removing emphasis on the web part at stage 292. When the value of document.activeElement is an <iframe> element, operations include a decision stage 294 for determining whether the user input target was over the web part when focus shifted to the new <iframe> element. When the user input target is determined to have not been over the web part, operations include removing emphasis on the web part at stage 292. When the user input target is determined to have been over the web part, the operations include inferring user interaction associated with different <iframe> element of the web part. The emphasis can remain on the web part.
[0046]
[0047] After the author edits the web page, the author can save the web page. In some embodiments, a web part instance's property data, which can be provided by the author, is saved into a database (e.g., an SQL database) alongside the web page data. In certain embodiments, the web pages can be saved as HTML files or in other suitable manners. In other embodiments, web pages can be stored in a CMS system as a collection of page data fields in a database record. The following shows one such example:
[0048] Page Title: Bird watching club
[0049] Page Filename: bir.aspx
[0050] Page Author: John Doe
[0051] Page Id: 12345
[0052] Last Saved: 3:02 pm, January 3rd
[0053] Article Text: [formatted text codes]
[0054] Header Image: N/A
[0055] Show In Menu: Yes
[0056] Page Template: Standard Article Page
[0057] . . . .
[0058] When a person views the web page 114, the CMS system can use the stored data associated with the web pages to render the HTML output that constitutes the web page as presented by the web browser. This runtime rendering has many advantages. For example, if a page template is redesigned, or if a navigation menu is improved, every existing page can automatically reflect these changes. Thus, when a web part developer deploys a new program code to the server, the property data can be rendered with a new appearance, causing all web part instances to have the new look. In contrast, if a web page is stored as a simple HTML file, an author would have to manually edit each page separately to synchronize appearances.
[0059]
[0060] Depending on the desired configuration, the processor 304 can be of any type including but not limited to a microprocessor (P), a microcontroller (C), a digital signal processor (DSP), or any combination thereof. The processor 304 can include one more levels of caching, such as a level-one cache 310 and a level-two cache 312, a processor core 314, and registers 316. An example processor core 314 can include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP Core), or any combination thereof. An example memory controller 318 can also be used with processor 304, or in some implementations memory controller 318 can be an internal part of processor 304.
[0061] Depending on the desired configuration, the system memory 306 can be of any type including but not limited to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.) or any combination thereof. The system memory 306 can include an operating system 320, one or more applications 322, and program data 324. This described basic configuration 302 is illustrated in
[0062] The computing device 300 can have additional features or functionality, and additional interfaces to facilitate communications between basic configuration 302 and any other devices and interfaces. For example, a bus/interface controller 330 can be used to facilitate communications between the basic configuration 302 and one or more data storage devices 332 via a storage interface bus 334. The data storage devices 332 can be removable storage devices 336, non-removable storage devices 338, or a combination thereof. Examples of removable storage and non-removable storage devices include magnetic disk devices such as flexible disk drives and hard-disk drives (HDD), optical disk drives such as compact disk (CD) drives or digital versatile disk (DVD) drives, solid state drives (SSD), and tape drives to name a few. Example computer storage media can include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. The term computer readable storage media or computer readable storage device excludes propagated signals and communication media.
[0063] The system memory 306, removable storage devices 336, and non-removable storage devices 338 are examples of computer readable storage media. Computer readable storage media include, but not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other media which can be used to store the desired information and which can be accessed by computing device 300. Any such computer readable storage media can be a part of computing device 300. The term computer readable storage medium excludes propagated signals and communication media.
[0064] The computing device 300 can also include an interface bus 340 for facilitating communication from various interface devices (e.g., output devices 342, peripheral interfaces 344, and communication devices 346) to the basic configuration 302 via bus/interface controller 330. Example output devices 342 include a graphics processing unit 348 and an audio processing unit 350, which can be configured to communicate to various external devices such as a display or speakers via one or more A/V ports 352. Example peripheral interfaces 344 include a serial interface controller 354 or a parallel interface controller 356, which can be configured to communicate with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device, etc.) or other peripheral devices (e.g., printer, scanner, etc.) via one or more I/O ports 358. An example communication device 346 includes a network controller 360, which can be arranged to facilitate communications with one or more other computing devices 362 over a network communication link via one or more communication ports 364.
[0065] The network communication link can be one example of a communication media. Communication media can typically be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and can include any information delivery media. A modulated data signal can be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media can include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), microwave, infrared (IR) and other wireless media. The term computer readable media as used herein can include both storage media and communication media.
[0066] The computing device 300 can be implemented as a portion of a small-form factor portable (or mobile) electronic device such as a cell phone, a personal data assistant (PDA), a personal media player device, a wireless web-watch device, a personal headset device, an application specific device, or a hybrid device that include any of the above functions. The computing device 300 can also be implemented as a personal computer including both laptop computer and non-laptop computer configurations.
[0067] From the foregoing, it will be appreciated that specific embodiments of the disclosure have been described herein for purposes of illustration, but that various modifications can be made without deviating from the disclosure. In addition, many of the elements of one embodiment can be combined with other embodiments in addition to or in lieu of the elements of the other embodiments. Accordingly, the technology is not limited except as by the appended claims.