TOY CONSTRUCTION SYSTEM FOR AUGMENTED REALITY

Abstract

A toy construction system comprising toy construction elements, comprising coupling means for releasably interconnecting the toy construction elements, one or more marker construction elements comprising such coupling means and each having a visual appearance recognizable by an image processing means, and a data processing system adapted to process a captured image of a toy construction model constructed from the toy construction elements to detect at least a presence of at least one of the marker construction elements within the captured image, and responsive to the detected marker construction element, generate a computer-generated image, wherein the marker construction element comprises a visually detectable feature and a movable element movable between a first and a second position, wherein the movable element, when positioned in the first position, causes the visually detectable feature to be visible and, when positioned in the second position, causes the visually detectable feature to be obstructed from view.

Claims

1. A toy construction system comprising toy construction elements, the toy construction system comprising: at least one marker construction element comprising coupling means for releasably interconnecting the at least one marker construction element with at least one of the toy construction elements and each having a visual appearance recognisable by an image processing means, and a data processing system comprising image capturing means, image processing means, and display means, wherein the data processing system is adapted to: capture an image of a toy construction model constructed from the toy construction elements, the toy construction elements comprising coupling means for releasably interconnecting the toy construction elements, wherein at least a first toy construction element of the toy construction model is interpreted by the data processing system as a virtual button; process the captured image to detect a presence of the at least one marker construction element within the captured image responsive to a user connecting the at least one marker construction element to the toy construction model; trigger a virtual action responsive to the detected at least one marker construction element being connected to the first toy construction element; and display on said display means a composite image comprising the captured image having superimposed a generated computer-generated image.

2. The toy construction system according to claim 1, wherein the first toy construction element has a top surface, the top surface being detectable by the data processing system and functioning as the virtual button.

3. The toy construction system according to claim 1, wherein the coupling means define a regular grid restricting a position and/or orientation of the toy constructing elements in a toy construction model to discrete positions and/or discrete orientations relative to said regular grid.

4. The toy construction system according to claim 3, comprising at least two marker construction elements, and wherein the data processing system is adapted to detect respective positions and/or orientations of the at least two marker construction elements in a toy construction model relative to said regular grid, and to generate the computer-generated image responsive to the detected respective positions and/or orientations.

5. The toy construction system according to claim 4, wherein the data processing system is adapted to determine grid locations of a regular image grid from the detected locations of the at least two marker construction elements, and to generate an image feature of the computer-generated image at a location relative to the detected image grid.

6. The toy construction system according to claim 1, wherein the coupling means comprise one or more protrusions and one or more cavities, each cavity adapted to receive at least one of the protrusions in a frictional engagement.

7. The toy construction system according to claim 1, wherein the coupling elements define a 3D grid, wherein the locations of the top surfaces of the construction elements of the toy construction model are restricted to predetermined heights relative to an origin of a coordinate system.

8. The toy construction system according to claim 1, comprising displaying a computer-generated image element at the location of the virtual button.

9. The toy construction system according to claim 1, wherein the triggered virtual action further depends on one or more additional conditions.

10. The toy construction system according to claim 9, wherein the triggered virtual action depends on an insignia of the at least one marker construction element.

11. The toy construction system according to claim 10, wherein the triggered virtual action depends on a condition that the at least one marker construction element is connected to two virtual buttons.

12. A computer-implemented method for generating an image of a toy construction model constructed from toy construction elements of a toy construction system comprising: capturing an image of a toy construction model constructed from the toy construction elements, the toy construction elements comprising coupling means for releasably interconnecting the toy construction elements, wherein at least a first toy construction element of the toy construction model is interpreted as a virtual button; processing the captured image to detect a presence of at least one marker construction element within the captured image responsive to a user connecting the at least one marker construction element to the toy construction model, the at least one marker construction element comprising coupling means for releasably interconnecting the at least one marker construction element with at least one of the toy construction elements; triggering a virtual action responsive to the detected at least one marker construction element being connected to the first toy construction element; and displaying a composite image comprising the captured image having superimposed a generated computer-generated image.

13. A data processing system comprising image capturing means, processing means, and display means, wherein the data processing system comprises computer program means configured to cause the data processing system, when executed by the data processing system, to perform the steps of the method according to claim 12.

14. A computer program comprising program code means adapted to cause, when executed by a data processing system, said data processing system to perform the steps of the method according to claim 12.

15. A toy construction set comprising toy construction elements, the toy construction set comprising: at least one marker construction element comprising coupling means for releasably interconnecting the being connected to the first toy construction element toy construction elements and each having a visual appearance recognisable by an image processing system; and user instructions for installing, on a data processing system comprising image capturing means, processing means, and display means, the computer program according to claim 14.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0039] FIGS. 1a-1d each show a prior art toy building brick;

[0040] FIG. 2 shows an embodiment of a toy construction system as disclosed herein;

[0041] FIGS. 3-4B schematically illustrate examples of marker construction elements;

[0042] FIGS. 5a-5g illustrate examples of composite marker construction elements;

[0043] FIGS. 6a-6d and 7a-7d illustrate examples of uses of composite marker construction elements;

[0044] FIGS. 8a-8d illustrate an example of a toy construction system defining a regular grid;

[0045] FIGS. 9a-9b and 10 illustrate examples of uses of examples of a toy construction system as described herein; and

[0046] FIGS. 11a-11b illustrate an example of a marker construction element comprising two selectively visible AR markers.

DETAILED DESCRIPTION

[0047] Various aspects and embodiments of toy construction systems disclosed herein will now be described with reference to toy construction elements in the form of bricks. However, the invention may be applied to other forms of construction elements used in toy construction sets.

[0048] FIGS. 1a-1d each show a prior art toy building brick with coupling studs 105 on their top surface and a cavity 102 extending into the brick from the bottom. FIGS. 1a-1b show a top side of a toy building brick, while FIG. 1 b shows the bottom side of the same toy building brick. FIGS. 1 c-d show examples of similar toy building bricks of different sizes. Such construction elements are widely available under the trade name LEGO. The cavity has a central tube 103, and coupling studs of another brick can be received in the cavity in a frictional engagement as disclosed in U.S. Pat. No. 3,005,282. The building bricks shown in the remaining FIGS. may have this known type of coupling means in the form of cooperating studs and cavities. However, other types of coupling means may also be used. The coupling studs are arranged in a square planar grid, i.e. defining orthogonal directions along which sequences of coupling studs are arranged. Generally, such an arrangement of coupling elements allows the toy bricks to be interconnected in a discrete number of orientations relative two each other, in particular at right angles with respect to each other. It will be appreciated that other geometric arrangements of coupling elements may result in different orientational constraints. For example, the coupling elements may be arranged in a triangular, regular grid allowing a building element to be placed on another building element in three different orientations.

[0049] FIG. 2 shows an embodiment of a toy construction system as disclosed herein. The system comprises a computer 15, a display 1, a video camera 5, a toy construction model 10 constructed from at least one toy construction element and at least one marker construction element 11 of a toy construction system as described herein. The computer may be a personal computer, a desktop computer, a laptop computer, a handheld computer, a game console, a handheld entertainment device, or any other suitably programmable computer. The display 1 is operatively coupled to (e.g. integrated into) the computer 15, and operable to display, under the control of the computer 15, a video image. It will be appreciated that the system may comprise further peripheral computer devices operatively coupled to the computer 15, such as a keyboard, a mouse 2 or other pointing device, and/or the like. The video camera 5 is operable to capture video images of the environment in which the video camera is situated, e.g. of a field of view 7 including a visible surface area 8 of a surface 4, e.g. a desktop, a floor, or the like. Hence the visible surface area 8 is the projection of the field of view of the video camera on the surface 4. The video camera is operatively coupled to (e.g. integrated into) the computer 15 and operable to forward the captured video image to a processing unit of the computer 15, e.g. via a suitable input interface of the computer 15. For example, the video camera may be a webcam connected to or integrated into computer 15. In the example of FIG. 2, the video camera is positioned on a camera support 6, such as a tripod, supporting the camera at a predetermined height above the surface 4.

[0050] Here, the video camera 5 captures video images of the environment 8 in which the video camera is situated and forwards the captured video images to the computer 15. For example, the environment in which the video camera is situated may comprise the toy construction model 10. In addition to the toy construction model 10, the environment may comprise further objects such other household objects, toys, or the like.

[0051] The computer images are then rendered by the computer 15 on the display 1. Therefore, a user may move the toy construction model 10 around and/or otherwise manipulate the toy construction model within the field of view of the video camera 5 and view live video from the video camera 5 of the toy construction model. Alternatively or additionally, the user my change the position and/or orientation of the video camera so as to capture images of a (e.g. stationary) toy construction model from different positions. Additionally, the computer may be operable to store the captured video images on a storage device such as a hard disk of the computer, and/or forward the captured video to another computer, e.g. via a computer network. For example, the computer may be operable to upload the captured video images to a website.

The computer 15 is suitably programmed to operate in an augmented reality mode in which the computer performs image processing on the captured video image so as to detect one or more predetermined augmented reality markers or tags within the captured video image. Responsive to the detected AR marker(s), the computer may be programmed to generate a modified video image, e.g. a video image formed as the captured video image having a computer-generated image superimposed, or a video image wherein at least a part of the captured video image is replaced by a computer-generated image. The computer 15 is operable to display the modified video image on the display 1_ For the purpose of the present description, a computer operable to implement AR functionality operatively connected to a video camera and a display will also be referred to as an AR system.

[0052] Image processing methods for detecting AR markers and for generating modified video images responsive to detected AR markers are known as such in the art (see e.g. Daniel Wagner and Dieter Schmalstieg, “ARToolKitPlus for Pose Tracking on Mobile Devices”, Computer Vision Winter Workshop 2007, Michael Grabner, Helmut Grabner (eds.), St. Lambrecht, Austria, February 6-8, Graz Technical University).

[0053] In the example of FIG. 2, the physical toy construction model 10 comprises a toy construction element 9 having connected to it a marker construction element 11. The marker construction element 11 has an insignia in the form of a two-dimensional machine readable code on its top surface. The computer 15 is operable to detect the presence of the two-dimensional machine readable code in the captured image. Furthermore, the computer 15 may determine a relative position and orientation of the two-dimensional machine readable code relative to the position of the video camera 15.

[0054] Accordingly, the computer 15 may modify the captured image of the toy construction model resulting in a modified video image 12 displayed on the display 1. in this example, the modified image shows the captured image 13 of the toy construction model and computer-generated image elements 14 superimposed the captured image. The computer generates the computer-generated image elements at a position and orientation within the video image 12 as determined from the detected position and orientation of the AR marker of the marker construction element 11.

As the user manipulates the physical toy construction model 10 within the projection area 8 of the video camera, e.g. by moving and/or rotating the physical model, the computer 15 tracks the position and orientation of the insignia of the marker construction element 11 of the physical toy. The computer 15 displays the live video feed of the video camera (mirrored mode) on the display 1 and adds, responsive to the detected position and orientation of the insignia, augmented reality special effects to the live video feed.

[0055] FIGS. 3 and 4 schematically illustrate examples of marker construction elements.

[0056] FIG. 3 shows a toy construction element in the form of a toy building brick 311 similar to the building brick of FIG. 1c, i.e. a building brick comprising coupling studs 305 arranged in a planar square grid on the top surface of the toy building brick. The toy building brick also comprises one or more cavities on its bottom surface (not shown in FIG. 3) for receiving and frictionally engaging coupling studs of another, similar toy construction brick. The toy building brick comprises, on one of its side faces, an augmented reality marker 321 in the form of a 2D bar code.

[0057] It will be appreciated that other forms of AR markers may be used instead of a 2D bar code. Generally, the AR marker may be any object that is relatively easy to automatically distinguish within the captured video images using known image recognition methods. Where the AR marker is detected using image recognition methods, the AR marker may be, for example, a three dimensional object such as a cube or a cylinder or it may be a two dimensional marker such as a square or a circle. Typically, the AR marker comprises an easily distinguishable pattern such as a black and white square although other methods may be used for marker recognition such as using a particular colour or pattern of colours and the like.

The AR marker may include one or more elements allowing the computer to detect a position and/or an orientation of the AR marker. For example, the AR marker may comprise two or more intersecting lines. Additionally or alternatively, the AR marker may comprise visual elements for encoding information, thus allowing the computer to identify and distinguish different types of markers and to generate computer-generated images selectively responsive to specific types of markers.

[0058] The AR marker may be arranged in a uniform manner relative to the coupling means, i.e. to the coupling studs on the top surface and/or to the coupling cavity in the bottom. For example, the AR marker may define a direction parallel or normal to the planar grid defined by the coupling means. This makes the marker bricks interchangeable, and in a toy structure built from bricks as in FIGS. 1-3, several marker bricks can be used interchangeably, and a particular marker brick can be used in several constructions. A toy construction system may comprise several of such marker bricks having different insignia applied to it and causing the computer system to generate different computer-generated images. Nevertheless, if all marker bricks include the insignia at uniform positions, such marker bricks may easily be interchanged within a toy construction built from the building bricks described herein.

[0059] When the insignia is located on a part of the surface of the marker construction element such that the surface part comprising the insignia does not have any coupling means, the insignia is less likely to be unintentionally obstructed by other construction elements coupled to the marker construction element. A marker construction element may comprise a plurality of insignia, e.g. on different faces (or otherwise on different portions of the surface) of the toy construction element.

[0060] FIG. 4 illustrates how a marker construction element may be coupled to another toy construction element of a toy construction model. In this example, the toy construction model comprises toy construction elements 410 and 412 and a marker construction element 411. FIG. 4a shows the individual constructions elements, while FIG. 4b shows the marker construction element 411 releasably coupled to the construction element 410. To this end, the toy construction element 410 comprises coupling studs 405 on its top surface as described in connection with FIG. 1. The marker construction element 404 comprises cavities on its bottom surface (not shown in FIG. 4), allowing it to frictionally engage the coupling studs 404.

[0061] Hence, the user may connect a marker construction element comprising an AR marker to a constructed toy model so as to facilitate tracking the toy model in 3D space and to superimpose virtual graphics and effects, thereby providing the physical toy with virtual attributes and allow the constructed toy model to become a part of an AR application. As in the example of FIG. 3, the insignia of the marker construction element 411 is located on a surface of the marker construction element that does not comprise coupling elements.

[0062] FIGS. 5a-g illustrate examples of composite marker construction elements, i.e. a plurality of marker construction elements directly or indirectly connected with each other by means of the coupling means of the toy construction system, e.g. in a predetermined spatial relationship with each other. Generally, composite marker construction elements allow the user to create a large set of distinct objects, each identifiable by one of a large number of unique AR markers. In particular, the large number of unique AR markers may be created from a relatively limited set of individual marker construction elements, because combining a smaller set of markers through a building system dramatically increases the total number of combinatorial possibilities.

[0063] FIG. 5a shows an example of a human-like figure constructed from three marker construction elements, namely an element 511 resembling a head of the figure, an element 512 resembling the torso of the figure, and an element 513 resembling the legs of the figure. The construction elements are provided with coupling elements allowing a head and legs to be releasably coupled to the torso. The head, torso and legs may include insignia, e.g. in the form of facial features of the head and closing features of the torso and/or the legs. When presented to the video camera of an AR system, the AR system may thus detect the features of the head, the torso and the legs, e.g. as schematically illustrated in FIG. 5b by detected features 521, 522, 523 of the head, torso and legs, respectively. Hence, by combining different heads, torsos, and legs, a vast variety of combinations of AR markers can be created. Depending on which figure is detected by the AR system, the AR system may respond with different computer-created images, thereby allowing for a vast variety of interactions.

[0064] FIGS. 5c and d illustrate a similar composite marker construction element, constructed from toy building bricks as shown in FIG. 1c. The toy building bricks 531, 532, and 533 have different colors, e.g. red, white and blue, respectively, When stacked on top of each other and interconnected by their respective coupling means, the building bricks define a sequence of colors that may thus be detected by an AR system. By changing the order of the colors, as illustrated in FIG. 5d, different unique composite AR markers may be constructed from only three different marker construction elements.

[0065] FIG. 5e shows a similar example of a composite marker element, including toy building bricks 541, 542, 543 similar to the one shown in FIG. 3 having AR markers in the form of insignia on their side faces.

[0066] FIGS. 5f-5g show yet another example of a composite marker element In this example, the composite marker element comprises a base building brick 550, e.g. a brick similar to the brick of 1 c, but with a larger number of coupling elements on its top surface. The composite marker element further comprises marker construction elements 551, 552, 553 connected to the base building brick 550. Hence, in this example, the marker construction elements 551, 552, and 553 are indirectly coupled to each other in a rigid configuration so as to form a composite marker construction element. To this end, the marker construction elements have coupling elements, e.g. cavities, on their bottom surface that can engage corresponding coupling elements, e.g. studs, on the top surface of the base building brick 550. Each marker construction element has an insignia on its top surface, in this example letters of the Roman alphabet. As the marker construction elements are arranged side by side, they define a sequence of insignia detectable by the AR system.

[0067] As illustrated by FIG. 6g, the insignia of the marker construction elements may define an orientation in a plane, and individual marker elements may be rotated with respect to each other, thus further increasing the degrees of freedom of defining multiple codes, as the AR system may detect the orientation of the respective AR markers relative to each other and/or relative to another feature of the toy construction model.

[0068] Hence, in the examples of FIG. 5, individual marker construction elements, each comprising a visually detectable feature distinct from the other marker construction elements, are interconnected by means of their coupling elements, so as to define a sequence of visually detectable features. Each distinct sequence of visually detectable features may thus define a composite AR marker. For example, the individual visually detectable features may be detected by the AR system as symbols of a code, and a sequence of detected symbol may be decoded by the AR system so as to obtain information encoded by the sequence of symbols. In some embodiments, the system may use known error detection and/or error correction techniques for making the code more robust against detection errors.

[0069] FIGS. 6 and 7 illustrate examples of uses of composite marker construction elements. FIGS. 6a-6b illustrate a use of the marker construction elements of FIGS. 5a-5b.

[0070] FIG. 6a shows the video camera 5 and the display 1 of an AR system.

[0071] Initially, the display shows a scene 612. When the user presents the humanlike FIG. 613 to the video camera 5 of the AR system, the AR system may detect the combination of head, torso, and legs, determine the corresponding combined AR marker, and modify the displayed image responsive to the detected combined marker. For example, the AR system may generate an image of a FIG. 623 within the scene 612. The figure may resemble the physical FIG. 613 and may be animated to show a behavior (e.g. friendly, angry, sad, etc.) determined by the AR system responsive to the detected combined AR marker. FIGS. 6c-d illustrate a use of the marker construction elements of FIG. 5c or 5e. FIG. 6c shows the video camera 5 and the display 1 of an AR system.

When the user presents a composite marker element 633 to the video camera 5 of the AR system, the AR system may detect the combination of marker construction elements. Optionally, the AR system may determine the relative position and/or orientation of the individual markers forming the composite marker element relative to each other. The AR system may then respond responsive to the detected composite marker element. For example, as illustrated in FIG. 6d, the AR system may interpret the combined marker element as an access code, e.g. for granting access to advanced features of the AR system, to a predetermined level of a gaming application, etc. If the presented combined marker element matches a stored access code, the user is granted access; otherwise access may be denied. FIGS. 7a-d illustrate another example of a use of a combined marker construction elements, e.g. the marker construction elements of the type shown in FIGS. 5 f-g. FIG. 7a schematically illustrates coupling elements 705 of a toy construction element, e.g. coupling studs as described in connection with FIG. 1. The coupling elements are arranged in a planar square grid. FIG. 7b illustrates a marker construction element 711 coupled to the toy construction element by means of the coupling elements. The arrangement of the coupling elements allow the marker construction elements to be connected to another toy construction element in four distinct orientations. In the example of FIG. 7, the marker construction element 711 has an insignia
defining a direction. In this specific example, the insignia resembles an image of a cloud and a sun, i.e. the image defines an “up” direction. FIG. 7c illustrates a marker construction element 713 coupled to the coupling elements 705 adjacent to the marker construction element 711. Marker construction element 713 also has an insignia defining a direction, in this example resembling an arrow. The marker construction element 712 may be placed relative to the marker construction element 711 with its direction pointing to four different orientations relative to the direction defined by the construction element 711. Three of these orientations are illustrated in FIG. 7 by dotted circles 712. Similarly, FIG. 7d illustrates another marker construction element 715 coupled to the coupling elements 705 adjacent to the marker construction element 711. Marker construction element 715 also has an insignia defining a direction, in this example resembling an arrow. The marker construction element 715 may be placed relative to the marker construction element 711 with its direction pointing to four different orientations relative to the direction defined by the construction element 711. Three of these orientations are illustrated in FIG. 7 by dotted circles 714. The combined marker construction element comprises elements 711, 713, and 715, thus defining a sequence of AR markers arranged in a sequential order and having detectable orientations relative to each other. When presented to the camera of the AR system, the AR system may detect the three marker construction elements, their relative positions and their relative locations. Consequently, with only two different types of marker construction elements (the “cloud and sun” type of marker and the “arrow” marker), a large number of different composite markers may be constructed by coupling the marker elements to a toy construction element in different relative orientations and/or positions (e.g. different sequential orders).

[0072] For example, when generating computer generated image elements, the AR system may generate image elements resembling weather phenomena, e.g. clouds, rain, sun, snow, etc. The AR system may thus be controllable to display weather phenomena responsive to the orientation of the marker construction elements 711, 713, and 715. For example, when the AR system detects that a user has connected marker construction element 711 to a toy construction model, the AR system may initiate a function for controlling the display of image elements resembling weather phenomena.

[0073] Placement of the marker construction element 713 with its arrow pointing up (relative to the marker construction element 711) may cause the AR system to initiate a function for controlling display of image elements resembling a sun. A placement with the arrow pointing left or down may initiate display functions for displaying ‘rain’ and ‘snow,’ respectively.

[0074] Placement of the marker construction element 715 with its arrow pointing right (relative to the marker construction element 711) may cause the AR system to initiate a function for controlling display of the selected image element (in this example “sun”) with degree “medium” (e.g. partly covered by clouds). A placement with the arrow pointing up or down may initiate display functions for selected image element to a higher or lower degree (e.g. a lot of sun or a little sun), respectively.

[0075] Generally, when the coupling means of the marker construction elements only allow coupling of a marker construction element to another toy construction element (e.g. another marker construction element) of the toy construction system in a predetermined number of discrete relative positions and/or discrete relative orientations with respect to the other toy construction system, the AR system may be configured to detect the relative position and/or orientation of at least two marker construction elements with respect to each other, and to create a computer-generated image feature responsive to the detected relative position and/or orientation. For example, a limited number of discrete relative positions and orientations may be provided when coupling elements of the marker construction elements define a planar, regular grid, and when the coupling elements allow coupling of toy construction elements at a predetermined number of angles relative to each other within one of said planes.

[0076] FIGS. 8a-8d illustrate an example of a toy construction system defining a regular grid. FIG. 8a illustrates a 3D regular grid where each grid point of the grid defines a valid position of a connection element within a toy construction model. It will generally be appreciated that not all grid points of the grid actually need to be populated by coupling elements; however, in some embodiments all coupling elements are positioned on grid points of the regular grid. In particular, in the example of FIG. 8a, the grid points are arranged in parallel planes 801a-d wherein grid points within a plane define positions of connection elements on a surface of one or more toy construction element, e.g. the coupling studs of the building bricks shown in FIG. 1. The distance between the parallel planes defines a distance between surfaces of construction elements having coupling elements. For example, in some embodiments, construction elements have a height that is equal to (or an integer multiple of) the distance between two neighboring planes of the grid.

[0077] When the toy construction system defines a regular grid, the position and/or orientation of the AR markers may be accurately determined by the AR system relative to the grid. Furthermore, tracking markers that are attached to building elements in the grid of the building system provides more information about how the tagged elements are used in the model. Events in the application may selectively be triggered if the tags are built correctly, i.e. fit in the grid.

An AR marker may thus cause the AR system to initiate creation of computer
generated image elements responsive to a predetermined position and/or orientation (and/or other positional constraint) of the AR marker relative to the grid. The AR application can thus react to how the physical elements are used in the building system.
FIGS. 8b-d illustrate an example of such a position-sensitive trigger function. FIG. 8b schematically illustrates an example of a toy construction model constructed from toy construction elements with coupling elements defining a 3D grid as shown in FIG. 8a. The top surfaces of the construction elements may thus only be located at predetermined heights relative to an origin of a suitable coordinate system. For example, construction element 810a of FIG. 8b is located at a first height, while construction element 810b is located at a second height, and construction element 8100 is located at a third height.

[0078] The top surfaces of construction elements 810a-c may be detectable by the AR system and function as virtual buttons. If a user, as illustrated in FIG. 8c, connects a marker construction element 811 to one the construction elements being interpreted by the AR system as virtual buttons, detection of the marker construction element may cause the AR system to trigger a virtual action, e.g. displaying a computer-generated image element 812 at the location of the virtual button as illustrated in FIG. 8d. It will be appreciated that, in some embodiments, the triggered action may also depend on an insignia of the marker element 812 and/or on other conditions, e.g. that marker elements are connected to two of the virtual buttons.

[0079] It will be appreciated that the AR system may be adapted to determine the locations of grid points of the intrinsic discrete coordinate system defined by a toy construction model positioned in the field of view of the camera of the AR system. For example, the AR system may detect positions of coupling elements and/or positions of AR markers located in a predetermined spatial relationship to coupling elements of a corresponding marker construction element. Based on the determined grid locations, the AR system may determine the position and/or orientation of the regular grid relative to the camera position. The AR system may even create a regular image grid from the detected physical grid, e.g. such that the image grid has grid points coinciding with the images of the physical coupling elements. The AR system may then use the image grid to generate an image feature of the computer-generated image at a location relative to the detected image grid.

[0080] FIGS. 9 and 10 illustrate examples of uses of examples of a toy construction system as described herein.

[0081] FIG. 9 shows toy construction elements 911a-911b of the type shown in FIG. 1. In this embodiment, the different construction elements have respective insignia, different color, and/or another distinguishable visual feature. Each of the toy construction element, when detected by the AR system within a toy construction model, triggers a predetermined action. For example, in the example of FIG. 9, each of the construction elements 911a-911d represents a different type of landscape or environmental phenomena, such as grass, water, snow, fire, etc. When detected by the AR system, each construction element may thus be augmented by the AR system by displaying corresponding image elements representing grass, water, fire, snow, respectively. This allows a user to build a landscape by means of the construction elements, e.g. as illustrated in FIG. 9b.

[0082] It will be appreciated that the marker construction elements may also trigger further actions within the AR system. For example, the landscape of FIG. 9b may be used in the AR system as a landscape within which a game is performed, i.e. the user may build his/her own levels of a computer game by using physical building elements, FIG. 10 shows a gaming dice comprising a body member 2 with six identical sides, and wherein each of these sides is configured to be essentially square to the effect that the probability of the gaming dice coming to rest on a given side following throw of the gaming dice is equally high for all the sides of the gaming dice.

[0083] The sides of the body member 2 are essentially identical and thus a throw of such gaming dice will not in itself be meaningful, it not being possible for the user or an AR system to distinguish between the outcomes of such throw of the dice. According to this example of a dice, four couplings studs 3 are therefore provided on each of the sides of the body member 2 and are arranged in a square pattern to the effect that the coupling studs 3 form corners in a square, and the coupling studs 3 are configured such that it is possible to mount building elements from a toy building system on the coupling studs 3 to the effect that the sides of the gaming dice are able to change nature upon mounting of different such building elements on the sides of the gaming dice.

On the undersides of the building elements 4a, 4b, 4c, coupling flanges are configured that, following mounting of the building element 4a, 4b, 4c on the gaming dice, is in frictional abutment on the outer side of the coupling studs 3 on the gaming dice. There being ample opportunities for configuring such coupling flanges in different ways, and the person skilled in the art being able to freely point to such option.

[0084] By mounting different building elements on the gaming dice, it is thus possible to impart to each side of the gaming dice its own nature if the elements that are mounted on the gaming dice vary in number, shape or appearance as such.

FIG. 10 shows the same gaming dice wherein at least on the three visible sides of the gaming dice, building plates 4a, 4b, 4c from a toy building system are mounted. On the one side that faces upwards, a square building plate 4a is thus mounted that covers all four coupling studs on the side to the effect that the building plate does not leave space for mounting of further building elements on that side. On the two other visible sides of the gaming dice, a single square building plate 4b is mounted that covers precisely one single one of the coupling studs 3 on the one side, and, on the other side, two rectangular building plates 4c are mounted that each covers precisely two coupling studs 3 on that side, respectively, to the effect that the two rectangular building plates 4c do not leave space for further mounting of building plates on that side.

[0085] Hence, different AR markers may be associated with different faces of the gaming dice. Throwing of the dice within the field of view of the video camera of an AR system as described herein may thus cause the AR system to detect the AR marker e.g. on a face of the dice, and to present a corresponding outcome of a virtual game. As different AR markers may be associated with the faces of the dice, the AR system may implement different games without the need for reprogramming or otherwise reconfiguring the AR system, but simply by replacing the AR marker construction elements connected to respective faces of the dice.

[0086] FIGS. 11a-11b illustrate an example of a marker construction element comprising two selectively visible AR markers. FIGS. 11 a-b schematically show a toy construction model 1101 comprising a door 1103 hinged to a frame 1102 to allow the door 1103 to pivot between a closed position as shown in FIG. 11a and an open position as shown in FIG. ‘11 b. The frame and the door may be releasably interconnectable with each other, e.g. by snapping the hinges into place. Alternatively or additionally, the frame 1102 may comprise one or more coupling elements (not explicitly shown in FIG. 11) allowing the frame and door structure to be connected to a larger toy construction model, e.g. so as to provide a movable barrier covering an opening in a toy structure. The door 1103 has coupling elements (not explicitly shown in FIG. 11) on both sides allowing different marker construction elements 1111a and 1111b to be connected to the respective sides of the door. For example, the construction marker elements 1111a and 1111b may be elements similar to element 411 of FIG. 4a. The marker construction elements 1111a and 1111b comprise different insignia on their respective faces pointing away from the door. In the example of FIG. 11, marker construction element 1111a comprises a square frame and the letter ‘A’ inside the frame, while marker construction element 1111b comprises a square frame and the letter ‘B’ inside the frame. Hence, insignia 1111a is visible when the door is in its closed position, while insignia 1111b is visible when the door is in its open position. When the structure 1101 is positioned in the field of view of a video camera 1105 of an AR system as described herein, the AR system may thus detect which one of the insignia is visible, thereby detecting whether the door is in the open or in the closed position, and to generate computer-generated images responsive to the detected state of the door. For example, when in the closed position, the AR system may augment a live view of a toy structure comprising the door with computer-generated images of movable objects (e.g. human- or animal like figures) that move around the environment defined by the toy structure. When the AR system detects the door to be closed, the movement of the movable objects may constrained to only a portion of the environment (or no objects may be visible at all), while the movable objects may be shown to pass through an opening covered by the door, when the door is detected to be in the open position.

[0087] Hence, when two AR markers are connected to a functional element of a toy construction model such that either one of the markers is visible depending on the state of the functional element, the state of the functional element is detectable by the AR system. As the AR marker construction elements 1111a and 1111b may be releasably connected to different functional toy construction elements, a toy construction system may comprise a first marker construction element indicative of a first state and a second marker construction element indicative of a second state, thus enabling the AR system to detect different states of different functional construction elements.

[0088] In the claims enumerating several means, several of these means can be embodied by one and the same element, component or item of hardware. The mere fact that certain measures are recited in mutually different dependent claims or described in different embodiments does not indicate that a combination of these measures cannot be used to advantage.

[0089] It should be emphasized that the term “comprises/comprising” when used in this specification is taken to specify the presence of stated features, elements, steps or components but does not preclude the presence or addition of one or more other features, elements, steps, components or groups thereof.