Presentation system with movable display devices

Abstract

The present system and method provides for a new digital media paradigm enabling tight choreography of motion, content and, time able to be presented on a variety of hardware platforms consisting of robotic control of a multiplicity of display screens in the form of a movable array of 2 or more LCDs, LEDs, OLEDs, etc., with the movement and placement of each display achieved by one multi-axis manipulator arm mechanism. Motion control is achieved through software programmed onto one or more controller systems, and the corresponding tools necessary for creative visual designers to produce content meeting this new paradigm are also proposed. Each arm/display screen combination is kept aware of its positioning in physical space, relative to the positioning of each and every other arm/display screen at all times, in order to prevent collisions. The pre-programmed software control takes the form of a choreographed playlist of movements, content, and time that match the desired positioning of the array of display screens, in order to achieve the desired dynamic presentation of custom-produced digital content that will be presented across the array, in a fully coordinated fashion.

Claims

1. A visual display system comprising: a plurality of electronic display devices, a separate support arrangement for each electronic display device connected to a motor drive arrangement that accommodates movement of each electronic display device to change the orientation thereof and to accommodate substantial movement of the respective display device along first and second axes from a neutral position to any of a series of positions including stacked positions relative to the second axis; said separate support arrangement for each electronic display device further accommodating substantial movement of the respective display device along a third axis; said plurality of display devices, via said motor drive arrangement, are each controlled to move within a three dimensional display space in a coordinated manner and to accommodate a change in order of said display devices within said display space.

2. A visual display system as claimed in claim 1 wherein each separate support arrangement accommodates pivotal movement of each display device.

3. A visual display system as claimed in claim 1 wherein said three dimensional display space has a dominate plane of movement defined by said first and second axes.

4. A visual display system as claimed in claim 3 wherein each separate support arrangement allows rotation of said display devices within said display space such that a viewing direction perpendicular to said display devices can be altered as part of the movement of each display device.

5. A visual display system as claimed in claim 4 wherein each separate support arrangement accommodates angling of the display device for multi-plane viewing of said display devices in said display space.

6. A visual display system as claimed in claim 1 wherein said plurality of display devices is at least 4 display devices provided in pairs with the display devices of a pair being secured in a back to back manner either side of a common support arrangement.

7. A visual display system as claimed in claim 1 including a camera and an associated image processing arrangement, said camera being directed outwardly from said display space for recognition of at least one potential viewer; said associated image processing arrangement including identification and tracking of at least one recognized potential viewer and coordinating movement of said display devices to improve the likelihood of the at least one recognized potential viewer viewing the display devices of the visual display system.

8. A visual display system as claimed in claim 7 wherein said image recognition processing arrangement identifies head movement of viewers.

9. A visual display system as claimed in claim 1 including a camera directed outwardly from said display space in combination with an image processing arrangement used to identify motion and coordinating movement of said display devices at least partially based on the identified motion.

10. A visual display system as claimed in claim 1 wherein motion of said display devices is partially based on motion depicted in images shown on said display devices.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Preferred embodiments of the invention are shown in the drawings, wherein:

(2) FIG. 1 is an exploded view of the compact planar manipulator.

(3) FIG. 2A-D are a set of orthogonal views of the compact planar manipulator including A) a front view, B) a side view, C) a top view and D) a perspective view.

(4) FIG. 3A-B depicts the compact planar manipulator mounted onto a railing system.

(5) FIG. 4 depicts the compact planar manipulator mounted onto a scissor lift-type mechanism.

(6) FIG. 5A-C are a set of orthogonal views of the three degree-of-freedom parallel manipulator including A) a side view, B) a top view, and C) a perspective view.

(7) FIG. 6A-D are a set of frontal views of the three degree-of-freedom parallel manipulator, where the screens remain stationary and the hardware behind the screens is moved. FIG. 6E shows the beginning of the motion of the screens in order to end up in a plus-pattern, as depicted in FIG. 6F.

(8) FIG. 7 depicts the screens of the three degree-of-freedom parallel manipulator rotating about their geometric centres.

(9) FIG. 8 is the first 3 in a collection of 6 screen shots of the three degree-of-freedom parallel manipulator being used in a golf commercial storyboard.

(10) FIG. 9 is the second 3 in a collection of 6 screen shots of the three degree-of-freedom parallel manipulator being used in a golf commercial storyboard.

(11) FIG. 10 depicts the screens of the three degree-of-freedom parallel manipulator travelling on a large polymer surface, with the remaining mechanical components behind the surface.

(12) FIG. 11 outlines measurements of the three degree-of-freedom parallel manipulator.

(13) FIG. 12 shows the limits of the inner arms of the three degree-of-freedom parallel manipulator.

(14) FIG. 13 shows the limits of the outer arms of the three degree-of-freedom parallel manipulator.

(15) FIG. 14 is an exploded view of the six degree-of-freedom robotic arm manipulator.

(16) FIG. 15A-D are a set of orthogonal views of the six-degree-of-freedom robotic arm manipulator including A) a front view, B) a side view, C) a top view and D) a perspective view.

(17) FIG. 16 depicts the ability of the six-degree-of-freedom robotic arm manipulator to follow/track a viewer loosely.

(18) FIG. 17 depicts the ability of the six-degree-of-freedom robotic arm manipulator to follow/track a viewer with greater accuracy.

(19) FIG. 18 is an exploded view of the six degree-of-freedom robotic arm manipulator with dual-screens.

(20) FIG. 19A-D are sets of orthogonal views of the six degree-of-freedom robotic arm manipulator with dual-screens including A) a front view, B) a side view, C) a top view and D) a perspective view.

(21) FIG. 20 is a diagram of the six degree-of-freedom robotic arm manipulator mounted onto the ceiling.

(22) FIG. 21 is a flowchart of the content creation process.

(23) FIG. 22A-B depict an example planar robot and its assigned coordinate frames for the D-H analysis.

(24) FIG. 23 depicts the orientation of a planar robot used as a numeric example for the D-H analysis.

(25) FIG. 24 is a storyboard depiction of screen movement for a planar robot.

(26) FIG. 25 is a storyboard depiction of screen movement plus the content for the screens.

(27) FIG. 26 is a screen shot of the CDWA depicting the new project dialogue box.

(28) FIG. 27 is a screen shot of the CDWA depicting how the designer manipulates an MVST on-screen.

(29) FIG. 28 is a screen shot of the CDWA depicting the insertion of a new media file.

(30) FIG. 29 is a screen shot of the CDWA depicting the designer setting the properties for a newly inserted media file.

(31) FIG. 30 is a screen shot of the CDWA depicting the designer adjusting the entry/exit points of an MVST event.

(32) FIG. 31 is a screen shot of the CDWA depicting the working space of the designer.

(33) FIG. 32 is a screen shot of the CDWA depicting the export dialogue box.

(34) FIG. 33 is a screen shot of the CDWA depicting the generated heat profile for an MVST.

(35) FIG. 34 is a screen shot of the CDWA depicting the generated acceleration profile for an MVST

(36) FIG. 35 is a screen shot of the CDWA depicting the dialogue box asking for the MVST to be specified for the compilation process.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

(37) The presentation system will be described with respect to the hardware of the system. The system will subsequently be described in combination with preferred software component for controlling the moving display devices.

(38) Hardware

(39) There are several ways to mount the display devices (preferably electronic display screens) to accommodate various degrees of mobility and a variety of different visual effects. Three preferred embodiments, including the compact planar manipulator, the 3 degree of freedom planar manipulator and the 6 degree of freedom planar manipulator will be described. Each of these embodiments include two or more display screens. It should be noted that although the use of a screen as a display device is the preferred embodiment, other possible display devices could include LCD, Plasma, OLED, and 3D lenticular screens or other technologies such as projectors, flexible display panels to created shaped surfaces (for example convex, concave, tubular, conical and spherical) or LED's on a plurality of rods which can move in the z-axis to create a 3-D image. Display devices could also consist of combinations of the above devices or alternative display devices. In the example of mobile projectors a screen is provided behind the device and the projected image moves with movement of the projector. It is also possible to use different display devices in combination.

(40) The compact planar manipulator system enables the motion of a screen in a single plane. It can be used independently to form a single screen display or multiple compact planar manipulators can be used together to have a multiple screen display. Each compact planar manipulator (as seen in FIG. 1 and FIG. 2A-D) is driven by 4 DC-motors 1. Each motor is responsible for driving either an inside lower arm 50 or an outside lower-arm 51. Each lower-arm, both inside 50 and outside, has two ends including a proximal and distal end. Since the rotation of the motor gear-shaft and the rotation of the lower-arms (50 and 51) take place in perpendicular planes, the motion of the lower-arms (50 and 51) is controlled by the motor with a worm-gear apparatus 3. The worm-gear apparatus 3 connects the motor gear-shaft to the proximal end of a lower-arm. The distal end of the inside lower-arm 50 is connected to the proximal end of an inside upper-arm 52. The distal end of the outside lower arm 51 is connected to the proximal end of an outside upper-arm 53. The connection between the lower-arms and upper-arms is not actuated. The distal ends of both the inside 52 upper-arms then connect to one of the circular faces of a spacer 5. The other face of the spacer 5 attaches to one end of a crank shaft 7. The other end of the crank shaft 7 connects to a second spacer 6. The two outside arms 51 connect together and to spacer 6.

(41) A flat-panel mount 8 is attached to one of the outer-arm systems at spacer 6. The mount supports a rotary (pancake) motor 9, which in turn is attached to the flat-panel screen 10 or television which may be of any commercially-available technology including LCD, Plasma, LED, OLED, and 3D lenticular or other technologies. The robotic mechanism, excluding the panel and its accessories, allows for intricate planar motion through the precise control of the 4 motors. The limits of the planar motion are set by the lengths of the lower-arms and upper-arms, as well as the length of the crankshaft. Finally, the rotary (pancake) motor 9 adds extra rotational freedom to the panel, enabling the designer to achieve more elaborate and attention-grabbing motions.

(42) The compact planar manipulator is ideal for advertising areas with limited space. The system can act as a standalone unit, or can be linked up with other systems like it, to create a chain of moving advertisements. Other possibilities include, mounting the manipulator on a railing system, allowing the screen to follow a viewer along a path, as well as incorporating a scissor-lift type mechanism to allow the manipulator to travel towards and away from the viewer. These depictions can be seen in FIG. 3A-B and FIG. 4 respectively.

(43) The three degree-of-freedom planar manipulator (as seen in FIG. 5A-C) allows more maneuverability compared to the compact planar manipulator. This configuration allows for more complex interactions with a plurality of screens, preferably 4 screens. The system is attached to the wall, ceiling or other support structure via a wall mount 11. A high-torque rotary (pancake) motor 12 is responsible for rotating the entire system. The mounting bar 13, which attaches to the rotary motor 12, provides a mounting surface for four smaller high-torque motors 14. Each small high-torque motor 14 is attached to an arm system. Each arm system has one of 2 configurations. The outer arm systems 35 share the same configuration, while the inner two arm systems 36 have a second configuration. Each configuration will be described separately.

(44) In the outer arm configuration, a spacer 15 is attached to the high-torque motor 14. The spacer 15 is used to allow the outer arms 65 to rotate over the inner arms 66 without colliding, while keeping all the television screens (16) in the same plane. A motor-track system (17) is attached to the spacer 15. The outer arm 65 is slidably attached to the motor track 17 thereby allowing the outer arm to slide along its length. A second rotary motor 67 is attached to the end of the outer arm 65 and is fixed to a television screen 16, which allows for controlled rotation of the screen.

(45) The inner arm systems 36 have a similar configuration to the outer arm system 35 with the exception of the location of the spacer. The high-torque motor 14 has a motor-track 17 fixed to it. This is slidably attached to the inner arm 66 and allows for sliding of the arm along its length. A spacer 37 is attached between the end of the inner arm and a second rotary motor 68. The second rotary motor is then fixed to a television screen 16.

(46) The television screens 16 at the end of both the inner 65 and outer arms 66 can be mounted to the arms at any position on the back of the screens; however the preferred mounting position is a corner of the screen.

(47) This configuration allows for the system to keep the positions of the screens fixed, while re-orienting the hardware behind the screens. This is critical as the closing position of one choreography sequence can be held steady while the system is setting up for the next scene, where the subsequent motion requires a certain hardware starting point. Various hardware configurations for the same screen configuration are shown in FIGS. 6A-F. The versatility and flexibility of the hardware, as well as the ranges of motion possible when orienting the screens allows designers to create either the most efficient path-plans or the most visually appealing motions, depending on their applications and preferences.

(48) Referring to FIG. 7, this configuration allows for the appearance of the screens to be rotating about their geometric centres (point A), even though they are, in fact, actuated at their corners (point B).

(49) An example of an application of the three degree-of-freedom parallel manipulator can be seen in FIGS. 8 and 9. These are screenshots of a golf advertisement depicting the contribution of the screens throughout the video. As seen in the figures, the screens can be employed to introduce the scene by showing a fly-over view of the golf course. Then, 2 screens are used to show the golfer hitting the ball, while the other 2 screens stay fixed onto the flag, giving the viewer a sense of the breadth of the scene. As the ball soars through the air, a screen tracks its trajectory, until the ball drops onto the final 2 screens, rolling into the hole. The motion of the screens adds another dimension to the video graphics content, keeping the audience interested and engaged to the demonstration.

(50) Planar systems may also be configured to completely hide their manipulator structure by suspending a large low-friction polymer surface panel between the screens and the manipulator mechanics. Ends of each manipulator and the backs of the screens would be affixed with low-friction, polymer-encased, attracting magnets so that although each screen would magnetically track the motion of its corresponding manipulator behind the large polymer surface, viewers would simply see the screens floating in front of such a surface (FIG. 10). Although points where the polymer panel is suspended cannot be traversed, through artistic choreography, viewers will believe that there is unlimited motion. Inductive coils transfer energy to power the screens, or the screens can include rechargeable batteries which charge when the screens move into docking stations with charging contacts located towards the outsides of the polymer panel. Data can either be transferred using inductive coils or traditional wireless data protocols. A further aspect of this design is that in harsh climates, protecting the manipulators from salt, water, debris, and other forms of wear and corrosion can significantly lengthen the life of the now-enclosed robotic mechanisms. The screens themselves could also be moved up, under, or into overhanging shelters in times of inclement weather if the rear apparatus was mounted on a track enabling appropriate motion.

(51) Another key feature of this robot is its high screen area to workspace area ratio. Screen area is defined as the total usable area of the screens; for 55 screens, the screen area is approx. 6,532 inch.sup.2. The workspace area is defined as the total usable area by the robot. Using the dimensions from FIG. 11-13 the workspace area is calculated to be 482,686 inch.sup.2. Thus, the screen to workspace area ratio is 1:73. With this arrangement the workspace area is very large relative to the total screen area. Even with a ratio of 1:10 an improved system is realized.

(52) The six degree-of-freedom robotic arm manipulator (as seen in FIG. 14 and FIG. 15A-D) provides the most diverse 3D advertising/displaying experience of the systems presented in that the arms provide maneuverability that none of the previously mentioned systems can. A high-torque rotary (pancake) motor 20 is mounted directly to the wall, ceiling or another support structure, depending on the application of the system. A mounting plate 21 is attached to the rotary part of the high torque rotary motor 20. This mounting plate 21 provides a base to which one or more 3D robotic arms can be mounted. The preferred number of arms is 4. Unlike the 3 degree of freedom manipulator, each arm in the 6 degree of freedom manipulator is the same.

(53) A 3D robotic arm 55 is attached to the mounting plate 21 by a shoulder socket 22. The shoulder joint 23 attaches to the shoulder socket and the rotation between the shoulder socket and joint can either be fixed or controlled by a motor or gear. The arm support 24 attaches to the shoulder joint 23, either directly or through a motor, thus allowing axial rotation of the first arm member 25 which connects to the arm support 24. A rotary socket 26 is attached to a rotary joint 27 which is essentially a rotary motor with housing attached to it to facilitate connection to the rotary socket 26 and a spacer 28. This spacer is then connected to a second arm support 40, to which the second arm member 38 is fixed. It is possible to connect a rotary motor between the space and the arm support to allow axial rotation of the second arm member, therefore is possible for the second arm member to rotate axially as well as in the vertical plane. A second rotary socket 42 is fixed to the distal end of second arm member 38. The second rotary socket 42 is connected to a third arm support 41 via a second rotary joint 43. Once again this linkage could be modified to allow for controlled axial rotation of the third arm member 39, which is connected to the spacer 45 and then to third arm support 41. The distal end of the third arm member 39 has a wrist socket 29 attached to it, which facilitates connection to the wrist joint 30. The wrist joint 30 is connected to rotary support 31. The rotary support 31 is attached to rotary motor (not shown) which is mounted directly on the television screen 32.

(54) The arrangement of the six degree-of-freedom robotic-arm manipulator can be adopted to include dual-screens (as seen in FIG. 18 and FIG. 19A-D). This arrangement is very similar to the 6 degree of freedom manipulator previously described. All parts are identical to those in the single screen design and the part numbers have been kept constant in the figures. There is a difference in the linkage structure distal to the wrist socket 29. A wrist joint 30 attaches directly to the wrist socket 29 on one end and to 2 television screens, arranged back-to-back, on the other end. This results in the reduction of the last rotational degree of freedom in the previously described model, but enables designers to use the dual-screen setup in creative ways. The entire system can be installed on top of a reflective backdrop 34, which allows the designers to use this reflective surface in creative ways. Backdrops could include fog/mist walls, waterfalls, and smoke walls.

(55) One use for the six degree-of-freedom robotic-arm manipulator with dual-screen heads is exemplified in FIG. 20, where the robot is mounted onto a ceiling such that the upper-facing display can project patterns onto the ceilingcreating an ever-changing halo for various moods and effects. Note that it is also possible to place digital projectors on the rear of such dual-display heads which may also be coupled with 30-degree servomotors to rock such heads 15 degrees to the left and right. In a further implementation, such dual-screen systems can be mounted on a cross-bar in front of (in the case of wall-mounting), or below (in the case of ceiling mounting) a mirrored surface. Such surface may be one mirror or a tiled pattern of many mirrors (concave, flat, convex, spherical, or any combination thereof). By projecting images onto this surface in conjunction with the choreography, an entire room can be filled with moving projections.

(56) It is possible to adapt any of the three robotic displays described above to include choreography which stacks the displays perpendicular to the mounted wall to first attract the attention of side viewers, and then follow such viewers as they move around the robotic display system. This can be accomplished by the addition of one or more stereo cameras 47 and/or one or more smaller screen cameras 48 mounted directly onto each of the screens. This setup allows for creative and realistic interaction of the system with people. Such a system could be designed so that as people walk by the mechanism, the stereo cameras 47 can pinpoint the exact location of an individual within the vicinity of the robot, allowing the screens to interact with the individual. The screen cameras 48 could be used to capture details about the individual, which can then be manipulated by intelligent image-processing software to pick out specific details about the individual (hat colour, hair length, type of clothing, etc.) in order to provide a more realistic interaction with the robot. Furthermore, the cameras can be used to show the other individuals in the area, exactly what the robot sees on a larger screen set up to the side of the choreography or as a dynamic part of the choreography itself.

(57) With many of the embodiments already described, it is possible to accommodate partial overlap of screens if desired. This allows partial reduction of the viewing area and expansion thereof as the extent of overlap is reduced. This arrangement provides a number of advantages including visual opportunities for more realistic depiction of colliding objects or near miss of objects.

(58) As can be appreciated, the individual display screens can be very large commercial display screens, although smaller screens can also be used for other applications.

(59) Software Component

(60) A choreographed presentation involves integration of several elements: motion, visual-content render, sound and lighting effects, and time. Referred to as an MVST (Motion, Visual render, Sound/lighting and Time) display system, it defines a new media paradigm.

(61) The MVST-display systems consist of screens that are driven by robotic arms with two or more degree-of-freedom, controlled by a set of connected motors. The motion for the displays is expressed as a series of MVST events, described shortly. Creating a presentation in the Creative Designer's Workbench Application (CDWA) is a revolutionary experience, unlike anything seen before. Essentially, designers create MVST events, which are critical points in time during the choreography where the screens, content on the screens, music, and lighting all precisely line-up as required.

(62) MVST Events

(63) The content on the screens can be shared between multiple screens, or there can be individual content that targets one or more screens. Thus, all of the screens can be playing back a single video or looking at a single image so that the screens are windows into a large virtual area. The content can also be anchored to a screen or it can follow a custom path. For each MVST event in the motion path, a new video can begin playing, or a new image can be displayed. The location of the video or image can be relative to the background, relative to one of the screens, or can be specified as a zoom and rotation relative to the centre of the robot.

(64) Designers are required to create a minimum number of MVST events for their presentations, essentially outlining key points in the motion, such as abrupt stops, cusp motions, etc. The CDWA then interpolates and creates motion plans between MVST events, saving the designer the work of having to create the entire motion plan manually. This interpolation process takes into account the model of the specific MVST display being used and creates the optimal motion plan keeping motor acceleration limits and heat profiles in mind. The benefit of this is that the CDWA creates a motion plan that is specific to the current hardware, using the generic MVST events outlined by the designer.

(65) As an example, consider the following scenario. A designer is working on an advertisement involving a large-scale MVST display with 4 screens; this display will be installed in a large space such as an airport, or a shopping mall. The advertisement involves all 4 screens starting off together in a square pattern, then breaking away diagonally while all of the screens are spinning, and finally coming back to end in a square pattern. The designer uses the CDWA to create critical MVST events for the advertisement and the CDWA creates the motion paths between the MVST events for the large-scale MVST display being used. Now, consider a small-scale MVST display installed at a retail store-front. The designer can run the same presentation on the smaller MVST display without having to change the presentation. The MVST events are still the same, except now the CDWA interpolates a motion path for the smaller MVST display. As long as the smaller system is capable of performing the required motions, the presentation will be similar to the one seen on the large-scale system.

(66) MVST Model File

(67) Creating a presentation requires several steps (FIG. 21). Initially, the designer needs a model of the MVST display to be used (acquired externally). This is shown as Sign robot 59. This model file includes all of the following necessary parameters (and their units) to completely characterize the MVST system: Denavit-Hartenberg (D-H) variables link offset, d (m) link length, a (m) twist angle, (rad) joint angle, (rad) link masses (kg) link centres of gravity (coordinates) link envelopes motor locations (coordinates) motor limits peak torques (Nm) rated speeds (m/s, rad/s) peak Force (N) stroke (m) thermal resistivity (deg. C./Watt) rated voltage (V) rated current (A)

(68) The four D-H variables are required to completely characterize the forward-kinematics formulation of the MVST display. Treating the MVST display as a set of links and actuation points, one can easily determine the position and orientation of the end-effector (the screens in this case) based on the link lengths and the angles between consecutive links. The D-H characterization is not unique and one must make sure to account for all the degrees-of-freedom of the robot. Table 1 is an example D-H table for one of the arms of the 4-screen planar robot. FIGS. 22A and B show the planar arm and its coordinate-frame assignments.

(69) TABLE-US-00001 TABLE 1 D-H parameters for one manipulator of the planar robot Joint d a 1 0 1 0 0 2 0 0 0 m 3 0 3 0 0 4 0 0 0 p 5 0 5 0 0

(70) These parameters are then used in homogenous transformation matrices, in order to solve the forward kinematics problem for each specific arm. The planar robot is a combination of four such arms. In order to solve the forward kinematics problem for the entire robot, a D-H table for each arm is needed. Since each arm has a fixed reference frame at the centre of the planar robot (x0-y0 frame), a table with all the D-H parameters for the entire robot, Table 2, can be created.

(71) TABLE-US-00002 TABLE 2 D-H parameters for planar robot d a Arm 1 Joint 1 0 1 0 0 2 0 0 0 m 3 0 3 0 0 4 0 0 0 p 5 0 5 0 0 Arm 2 Joint 1 0 1 0 0 2 0 0 0 n 3 0 3 0 0 4 0 0 0 p 5 0 5 0 0 Arm 3 Joint 1 0 1 0 0 2 0 0 0 n 3 0 3 0 0 4 0 0 0 P 5 0 5 0 0 Arm 4 Joint 1 0 1 0 0 2 0 0 0 N 3 0 3 0 0 4 0 0 0 p 5 0 5 0 0

(72) This table gives information about the joints in each arm as well as the position of the base of each with respect to the fixed centre of the robot, which is enough to fully solve the forward kinematics problem for the robot with the help of the homogeneous transformation matrices. Table 3 provides an example of the D-H parameters for a specific orientation of the planar robot in FIG. 23.

(73) TABLE-US-00003 TABLE 3 Example D-H parameters for planar robot d a Arm 1 Joint 1 0 43 0 0 2 0 0 0 84 3 0 64 0 0 4 0 0 0 54 5 0 21 0 0 Arm 2 Joint 1 0 43 0 0 2 0 0 0 29 3 0 37 0 0 4 0 0 0 60 5 0 44 0 0 Arm 3 Joint 1 0 43 0 0 2 0 0 0 29 3 0 31 0 0 4 0 0 0 32 5 0 5 0 0 Arm 4 Joint 1 0 43 0 0 2 0 0 0 84 3 0 100 0 0 4 0 0 0 105 5 0 50 0 0

(74) The link masses and centres of gravity (CGs) are used by the CDWA to calculate the moments acting on the pivot-points, and thus the motors, in order to assess the loads on the motor. The link masses are multiplied by the acceleration due to gravity (9.807 m/s.sup.2) to obtain the forces (F) in Newtons (N). The CG is used as the moment-arm for the moment calculation. The moments are calculated using the following vector cross-product:
M=F*d
where, M is the calculated moment (Nm), d is the moment-arm (m), and F is the force acting at d (N)

(75) The calculated moment can be compared to the rated torque (Nm) of the respective motor to ensure that said motor is capable of driving the load.

(76) Link envelopes are used to assist the CDWA in visualizing the links of the MVST display. For example, stating the mass, CG, and length of a link are not sufficient because the shape of the link is not entirely categorized by these parameters. The link envelope provides this missing information in the form of a set of coordinates, or nodes, and connection relationships of these nodes to other adjacent nodes. These nodes are used as vertices, and the connection relationships as edges, to create a box-like structure that encompasses the link as accurately as possible. This box-envelope is then used by the CDWA for collision checks during the simulation stage.

(77) Peak torques and speeds are used to calculate the power consumption of DC motors using the following relationships:
P=w*t(for rotary motors)
where, P is the power (Watts), w is the angular velocity (rad/s), and t is the torque (Nm)
P=F*v(for linear motors)
where, P is the power (Watts), F is the force (N), and v is the speed (m/s)

(78) For 3-phase motors, instantaneous power is calculated as follows:
P=3*V_phase*I_phase*cos(theta)
where, P is the power (Watts), V_phase is the phase voltage (V), I_phase is the phase current (A), and theta is the phase angle (rad)

(79) Additional information such as stroke is provided for linear actuators, so that the CDWA always knows what the limits of the actuator are. In order to calculate the heat produced, the thermal resistivity and power are required.
T_m=TR*P
where, T_m is the motor temperature (deg. C.), TR is the thermal resistivity (deg. C./Watt), and P is the power (Watts)
T_total=T_m+T_a
where, T_total is the combined temperature (deg. C.), T_m is the motor temperature (deg. C.), and H_a is the ambient temperature (deg. C.)

(80) This information is used by the CDWA to calculate heat production during simulations and compare these values with predetermined heat profile curves to ensure the motor is operating in a safe zone.

(81) Presentation Creation Process

(82) Referring to FIG. 21, initially, a sequence of illustrations, or storyboard 57, is developed from a concept 58 in order to visualize the presentation. This storyboard is essentially a compilation of the series of events at designated intervals within the overall choreography. FIGS. 24-25 provide examples of storyboarding for a golfing presentation. In FIG. 24, shows a storyboard of just the screen movements, where the arrows indicate the direction of movement of the screens. The screen B will move up and continue to rotate horizontally. FIG. 25 shows both the image content and the screen movement a split screen of a golfer on the left and a pin on the green on the right. Here screen B would follow the golf ball while screen A moves horizontally from showing the golf club to showing the golfers feet. After the storyboard has been completed, the designer moves to the animatics 60 stage. At this stage, more details such as musical backgrounds, camera movements, and test motions of the robotic screens are added to the storyboard. It is anticipated that other lighting effects, produced either by rear-facing projectors or displays, optionally reflecting off back mirror surfaces, or by independent light arrays, will be included in certain choreography designs.

(83) Next, the actual MVST events 61 are created. The designer starts by creating a new project in the CDWA. The application allows the designer to choose the type of MVST display to work with, as shown in FIG. 26. After a selection has been made, the CDWA automatically loads the MVST display using the model parameters described above. To begin, the designer moves and orients the MVST display and all of its screens to an initial position. An example starting point is shown in FIG. 27. At this point, the starting time is selected, as well as one or more screens depending on how the presentation will start. A classical piece of media is then imported or linked by using the media menu (refer to FIG. 28), whether it is a picture, animation, video clip, video stream, data feed, or audio track or stream. This is referred to as Generate media for inclusion 62 in FIG. 21. The designer specifies the source of the media file, stream or feed, the start time, and the duration, as exemplified in FIG. 29, as applicable. The first MVST event 61 can be created at the starting point of the presentation.

(84) Subsequent MVST events 61 are created based on important points in the presentation, such as points of high/low acceleration, cusp-motions, shaking-effects, etc. Intricate motions can be described using shape-forming tools such as Bezier curves, linear paths, or high-order splines. Referring to FIG. 30, the designer can use these tools on any motor in the arm assembly as well as to create to synchronize the media for the presentation accordingly. For each MVST event 61, the designer specifies the location and orientation of the screens, as well as the location, orientation, and zoom-level of the content to be rendered on the screens, played through the speakers, or in the case of lighting effects, channeled to light sequencers. All these features can be edited at any time, by selecting the screens or the media and then changing the necessary parameters, an example of which is shown in FIG. 31. As the designer creates more MVST events, the CDWA connects the MVST events 61 with a motion plan based on its own interpolation algorithms. If the designer wants to change the interpolated motion plan, they can change the entry/exit points of the motion at the MVST event 61 in question, using the control points for a Bezier curve for example. The CDWA then produces an updated interpolated motion plan corresponding to these changes.

(85) It is important that designers not add more MVST events 61 than absolutely necessary to achieve the desired choreography if they intend the finished product to run on multiple different hardware configurations. Movement over time between various MVST events 61 is important to enable elasticity so that the choreography can be adapted to differing hardware configurations.

(86) After the completion of all the MVST events 61 and the generation of the motion plans, the designer plays back the presentation to ensure everything is working as desired. This is called Simulations 63 in FIG. 21. If the presentation satisfies the requirements visually and audibly, the presentation is ready to be exported 64.

(87) The export function in the CDWA allows the designer to do a final test before completing the presentation. The export screen is shown as FIG. 32. Test runs display information regarding energy requirements and heat dissipation for a certain targeted hardware model. Heat profile graphs are provided for each motor and examples can be seen in FIG. 33. This enables the designer to visualize heat levels during the presentation relative to each motor's critical region (determined from the model file imported at the beginning of the process). Acceleration curves are also provided (examples are shown in FIG. 34) so that the designer can ensure that each motor is capable of delivering the performance characteristics that are dictated by the target choreography. Furthermore, the CDWA reports on any collisions between MVST display screens or linkages. All issues with heat profiles, acceleration profiles, or collisions, if any, are displayed as warnings after test runs are completed. At this point, the designer can also choose to compile the presentation for another MVST system, if the presentation is desired to be run on differing MVST systems. After all the issues are resolved, the designer exports compiled presentations for each target hardware model.

(88) The CDWA exports the presentation as a package containing all the required media files, as well as an XML file containing links to all the required media and, if included, external video and audio streams. An example of the structure of the resulting XML file is attached following the detailed description of the preferred embodiment.

(89) This package is then uploaded to online servers to be used with various MVST displays. Each MVST display has a unique IP address, allowing for content managers to connect to systems remotely to check on operation and perform administrative tasks such as setting schedules for various choreography showings by time of day (or to pre-empt normal playlists for a live event) and to set the frequency of a given show within a repeating mix of shows. The benefit of managing the system online is that it enables the content managers to access and deploy their content to multiple MVST systems from a single location. This is referred to as Internet Deployment 65 in FIG. 21.

(90) In the event where links to real-time streams for live shows are included in a choreography package, the manager must ensure that such streams are available and live at the time the MVST showing is scheduled. For example, a MVST display normally presenting a series of promotional choreography, could be programmed to re-orient screens for a live telecast such as election night, where screens would group together to show contest maps, then break apart to simultaneously show data feeds of riding results while also displaying candidate interviews and other live video feeds. These systems might find their way into homes by including special TV or IPTV channels designed to deliver multiple streamse.g. a hockey game, having the score, stats, and other camera views on different screens.

(91) Another example of the invention is dynamically choreographed segments here for example, a camera mounted above the stage in a theatre or a camera integrated into one of the screens is fed directly into the choreography, optionally with some image recognition, enabling possible computer generated comments about the audience or other events. For example, the computer-audience-attraction-system reads data from audience RFID tags and then modifies part of the presentation accordingly. It could also use camera inpute.g. when recognizing a pointed hat, the computer asks: Any witches in the house, pauses, then displays the video of the witch on one or more of the displays and says Youyes you, the Witchcould you please take off your hat?.

(92) Multiple MVST display systems could also be configured as choreographed dustersfor example, along the wall of a building or around the circumference of an arena. In clusters, content can be rendered in conjunction with the choreography to appear to make the discrete breaks between MVST display systems within the cluster vanish.

(93) To those skilled in the art, many modifications and variations of the present invention are possible in light of the above teachings. It is therefore to be understood that the present invention can be practiced otherwise than as specifically described herein and still be within the spirit and scope of the appended claims.

(94) Example of Structure of XML File

(95) TABLE-US-00004 <?xml version=1.0 encoding=UTF-8 ?> <!-- Start XML File --> <!DOCTYPE choreography SYSTEM choreography.dtd> <choreography> <mvstdisplay model=CWMSVT4x60 screens=4> <mvstbase min=512,0,512 max=512,0,512/> <mvstscreen id=0 aspect=16:9 width=1280 height=720/> <mvstscreen id=1 aspect=16:9 width=1280 height=720/> <mvstscreen id=2 aspect=16:9 width=1280 height=720/> <mvstscreen id=3 aspect=16:9 width=1280 height=720/> </mvstdisplay> <title>Golf Green Number 4</title> <duration>10.0</duration> <events> <mvst time=0.0> <sound href=soundtrack.mp3/> <base angle=0/> <screen id=0> <media href=scene1golfer.avi width=640 height=480> <motion crop=50%,50% center=240,0 angle=0> <outcurve curve=averaging curl=0.5/> </motion> </media> <motion position=50,0,50 angle=0> <outcurve curve=averaging curl=0.5/> </motion> </screen> <screen id=1> <media href=scene1golfer.avi width=640 height=480> <motion crop=50%,50% center=240,0 angle=0> <outcurve curve=averaging curl=0.5/> </motion> </media> <motion position=50,0,50 angle=0> <outcurve curve=averaging curl=0.5/> </motion> </screen> <screen id=2> <media href=scene1pin.avi width=640 height=480> <motion crop=50%,50% center=240,0 angle=90> <outcurve curve=averaging curl=0.5/> </motion> </media> <motion position=50,0,50 angle=0> <outcurve curve=averaging curl=0.5/> </motion> </screen> <screen id=3> <media href=scene1pin.avi width=640 height=480> <motion crop=50%,50% center=240,0 angle=90> <outcurve curve=averaging curl=0.5/> </motion> </media> <motion position=50,0,50 angle=0> <outcurve curve=averaging curl=0.5/> </motion> </screen> </mvst> <mvst time=5.0> <screen id=0> <media> <motion crop=50%,50% center=240,0 angle=45> <incurve curve=averaging curl=0.5/> <outcurve curve=averaging curl=0.5/> </motion> </media> <motion position=50,0,150 angle=0> <incurve curve=averaging curl=0.5/> <outcurve curve=averaging curl=0.5/> </motion> </screen> <screen id=1> <media> <motion crop=50%,50% center=240,0 angle=0> <incurve curve=averaging curl=0.5/> <outcurve curve=averaging curl=0.5/> </motion> </media> <motion position=50,0,50 angle=0> <incurve curve=averaging curl=0.5/> <outcurve curve=averaging curl=0.5/> </motion> </screen> <screen id=2> <media> <motion crop=50%,50% center=240,0 angle=90> <incurve curve=averaging curl=0.5/> <outcurve curve=averaging curl=0.5/> </motion> </media> <motion position=125,0,50 angle=0> <incurve curve=averaging curl=0.5/> <outcurve curve=averaging curl=0.5/> </motion> </screen> <screen id=3> <media> <motion crop=50%,50% center=240,0 angle=90> <incurve curve=averaging curl=0.5/> <outcurve curve=averaging curl=0.5/> </motion> </media> <motion position=125,0,50 angle=0> <incurve curve=averaging curl=0.5/> <outcurve curve=averaging curl=0.5/> </motion> </screen> </mvst> <mvst time=7.5> <screen id=2> <media href=message2top.png width=1024 height=768> <motion crop=50%,50% center=240,0 angle=90> <incurve curve=averaging curl=0.5/> <outcurve curve=averaging curl=0.5/> </motion> </media> <motion position=125,0,50 angle=0> <incurve curve=averaging curl=0.5/> <outcurve curve=averaging curl=0.5/> </motion> </screen> <screen id=3> <media href=message2bottom.png width=1024 height=768> <motion crop=50%,50% center=240,0 angle=90> <incurve curve=averaging curl=0.5/> <outcurve curve=averaging curl=0.5/> </motion> </media> <motion position=125,0,50 angle=0> <incurve curve=avaraging curl=0.5/> <outcurve curve=averaging curl=0.5/> </motion> </screen> </mvst> <mvst time=10.0> <screen id=0> <media> <motion crop=50%,50% center=240,0 angle=45> <incurve curve=averaging curl=0.5/> </motion> </media> <motion position=50,0,150 angle=0> <incurve curve=averaging curl=0.5/> </motion> </screen> <screen id=1> <media> <motion crop=50%,50% center=240,0 angle=0> <incurve curve=averaging curl=0.5/> </motion> </media> <motion position=50,0,50 angle=0> <incurve curve=averaging curl=0.5/> </motion> </screen> <screen id=2> <media> <motion crop=50%,50% center=240,0 angle=90> <incurve curve=averaging curl=0.5/> </motion> </media> <motion position=125,0,50 angle=0> <incurve curve=averaging curl=0.5/> </motion> </screen> <screen id=3> <media> <motion crop=50%,50% center=240,0 angle=90> <incurve curve=averaging curl=0.5/> </motion> </media> <motion position=125,0,50 angle=0> <incurve curve=averaging curl=0.5/> </motion> </screen> </mvst> </events> </choreography> <!-- END XML File -->