METHOD FOR THE CREATION OF INTERACTIVE AUDIO-VIDEO CONTENTS OF THE LIVE-ACTION TYPE FOR MOBILE TERMINALS, ELECTRONIC SYSTEM FOR THE IMPLEMENTATION OF THE METHOD, AND MOBILE TERMINAL FOR USING THE METHOD
20210264950 · 2021-08-26
Assignee
Inventors
Cpc classification
G11B27/007
PHYSICS
G06F3/04847
PHYSICS
H04N5/783
ELECTRICITY
G11B27/031
PHYSICS
International classification
G06F3/0484
PHYSICS
G06F3/0488
PHYSICS
G11B27/10
PHYSICS
Abstract
A method is described for the creation of interactive audio-video contents, through visualization on an interactive display, comprising the following steps:—providing at least one set of interconnected video segments (101), adapted to evolve in time, comprising an initial video segment (START) and one or more final video segments (END1, . . . ENDn), each video segment comprising a non-interactive video clip (102) having a given, or even null, duration, and an interactive node (103), said interactive node acting as an interconnection with one or more other video segments, each interactive node comprising a loop interval (TLi, TLf) subject to continuous cyclical forward and backward time evolution, —providing at least one set of commands (202) adapted to determine connection choices among said video segments, wherein said commands can be activated during at least one of said loop intervals, —in at least one of said loop intervals, association between said commands and video segments of said one or more other video segments, —upon activation of a command during said loop interval, exiting said interval and continuing in that video segment of said one or more other video segments in which, said interactive audio-video content continues, depending on the command chosen from said set of commands, starting from said initial video segment (START) and ending in one of said final video segments (END1, . . . ENDn), so as to create an interactive audio-video content on said interactive display as a sequence of said video segments.
Claims
1. Method for the creation of interactive audio-video contents, through visualization on an interactive display, comprising the following steps: providing at least one set of interconnected video segments (101), adapted to evolve in time, comprising an initial video segment (START) and one or more final video segments (END1, . . . ENDn), each video segment comprising a non-interactive video clip (102) having a given, or even null, duration, and an interactive node (103), said interactive node acting as an interconnection with one or more other video segments, each interactive node comprising a loop interval (TLi, TLf) subject to continuous cyclical forward and backward time evolution, providing at least one set of commands (202) adapted to determine connection choices among said video segments, wherein said commands can be activated during at least one of said loop intervals, in at least one of said loop intervals, association between said commands and video segments of said one or more other video segments, upon activation of a command during said loop interval, exiting said interval and continuing in that video segment of said one or more other video segments in which said interactive audio-video content continues, depending on the command chosen from said set of commands, starting from said initial video segment (START) and ending in one of said final video segments (END1, . . . ENDn), so as to create an interactive audio-video content on said interactive display as a sequence of said video segments.
2. Method according to claim 1, wherein said interactive node (103) comprises a node sequence of video content comprising said loop interval and further forward and backward time continuation intervals around said loop interval, time markers being set: in said loop interval as start instants (TLi) and end instants (TLf) of the continuous cyclical forward and backward time evolution; in said further forward time continuation intervals (Tf1 . . . Tfn); in said further backward time continuation intervals (Tb1 . . . Tbn); each time marker in said further forward and backward continuation intervals corresponding to the instant when the interactive audio-video content continues in one of said one or more other video segments, the start instant of which is associated with one of said time markers.
3. Method according to claim 2, wherein a command coming at different time instants within a loop interval determines the exit from the node sequence in different ways towards different further video segments.
4. Method according to claim 2, wherein said node sequence evolves at an increased or decreased frame-rate, if the loop interval has a frame-rate that is higher than the standard one.
5. Method according to claim 1 or 2 or 3 or 4, wherein more than one set of interconnected video segments (101) are provided, and said commands can determine the evolution of said interactive audio-video content in any one of said sets of interconnected video segments (101) or, in parallel, in each one of said sets of interconnected video segments (101).
6. Method according to claim 1 or 2 or 3 or 4 or 5, wherein, if no command is received within a maximum time of evolution of said loop interval, an automatic evolution towards another video segment or an end of said creation of an interactive audio-video content will be determined.
7. Method according to any one of the preceding claims, wherein said commands (202) are derived from corresponding commands available on a user terminal, which can be activated by means of: a plurality of sensors of the terminal; a plurality of gestures of the terminal; software commands (203) in the terminal; any combination of the above.
8. System for the creation of interactive audio-video contents, through visualization on an interactive display, comprising means for implementing the method according to any one of the preceding claims.
9. User terminal for the creation of interactive audio-video contents, comprising means for implementing the method according to any one of claims 1 to 7, and comprising: an interactive display and/or one or more sensors, from which at least some of said commands can be derived, and/or one or more motion detectors, from which at least some of said commands can be derived.
10. Method of association between commands of evolution in time of an interactive audio-video content and video segments composing said interactive audio-video content, comprising providing one or more loop intervals (TLi, TLf) subject to continuous cyclical forward and backward time evolution in said video segments, and executing at least one of said commands during said one or more loop intervals to determine said evolution in time of the interactive audio-video content as a path among said video segments.
11. User terminal comprising means for implementing an association between commands of evolution in time of an interactive audio-video content and video segments composing said interactive audio-video content, said means providing one or more loop intervals (TLi, TLf) subject to continuous cyclical forward and backward time evolution in said video segments, and for executing at least one of said commands during said one or more loop intervals to determine said evolution in time of the interactive audio-video content as a path among said video segments.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Further objects and advantages of the present invention will become apparent from the following detailed description of a preferred embodiment (and variants) thereof referring to the annexed drawings, which are only supplied by way of non-limiting example, wherein:
[0033]
[0034]
[0035]
[0036]
[0037]
[0038] In the drawings, the same reference numerals and letters identify the same items or components.
DETAILED DESCRIPTION OF SOME EMBODIMENTS
[0039] As aforesaid, it is one object of the invention to provide a method for the creation of controllable/manoeuvrable interactive audio-video contents of the live-action type for mobile user terminals, as a sequence of video clips, through the use of a plurality of sensors and commands for managing, controlling and manipulating a frame in the video clip, which affect the timeline (time evolution) and the frame-rate (speed) of the video clip. In the present context, the interactive audio-video contents of the “live-action” type are meant to be “live-action” or “real-action” contents, i.e. films played by “real” actors, as opposed to films created through animation (drawing, computer graphics, stop-motion, etc.)
[0040] Thanks to the invention described herein, it is possible to enjoy “live-action” video narration that can instantly show, without solution of continuity, the results of the video-clip composition actions with no pauses, loadings or interruptions, as a sequence of video segments not known a priori, i.e. a fluid and continuous filmic narration, modified and controlled in real time by the user/spectator, with no image jumps when switching between successive video segments in the nodes, as will be described hereinafter.
[0041] The basic idea of the invention is, therefore, to provide a method for the creation of controllable/manoeuvrable interactive audio-video contents of the live-action type and a video editor that allows creating independent audio-video contents encapsulated into suitable APPs or readable by a video player capable of recognizing and appropriately reacting to the controls and commands issued by the user through the use of any mode of interaction available in his/her mobile terminal, modifying in real time the succession of the video segments and hence the contents of the filmic narration.
[0042] MANOEUVRABLE INTERACTIVE VIDEO refers to a filmic narration wherein the time succession of the scenes (also called montage or direction) is not defined a priori by the author of the video, but is built in real time as a function of the interactions and selections INTERACTION COMMANDS made by the spectator (or user) during fruition.
[0043] The MANOEUVRABLE INTERACTIVE VIDEO (
[0044] VIDEO SEGMENTS are joined together at narrative points called NODES 103.
[0045] A VIDEO CLIP (
[0046] A NODE 103 is the point of interconnection among different VIDEO SEGMENTS.
[0047] The NODE is also the INTERACTIVE OR INTEROPERABLE or MANOEUVRABLE filmic narrative element of a MANOEUVRABLE INTERACTIVE VIDEO.
[0048] A VIDEO CLIP is the time-successive aggregation of video takes or contents, called SEQUENCES—see
Video clip=Sequence.sub.1+Sequence.sub.2+Sequence.sub.3+ . . . +Sequence.sub.n-1+Sequence.sub.n
[0049] At the end of each video clip there is a node sequence or Sequence.sub.n or NODE.
[0050] A node sequence is a filmic take characterized by a series of [time markers], defined as follows:—see
Wait interval or LOOP 821 refers to the narration between the markers TLf 822 and TLi 823.
[0055] The markers Tfi 826,827 and Tbi 824,825 are referred to as exit points.
[0056] To each exit point, the start 828 of a VIDEO SEGMENT is connected, see
[0057] A node sequence may be the termination of several VIDEO SEGMENTS 109.
[0058] From one node sequence, several VIDEO SEGMENTS 109 may start.
[0059] The node sequences without exit points are called narration endings 105-108.
[0060] Given the above definitions, it is assumed that it is per se known how each video clip can be created, which is made up of a sequence of known video and audio frames in a per se known digital format, e.g. 2D or 3D.
[0061] With reference to
[0062] The lines in the network symbolize the evolution in time of each clip, while the nodes symbolyze the transitions from one clip to another. Several video clips may meet at one node and/or several lines may start from one node, meaning that it is possible to switch from one clip to one or more other clips according to the specific mode of the invention described below.
[0063] Entry into the node or into the node sequence occurs at the instant TLi 822, i.e. the start point of the wait interval, in which the evolution of the clip occurs automatically, cyclically and continuously forwards and backwards (rewind) between TLI (822) and TLf (823), see
[0064] Within this automatic cyclic evolution (from instant TLi to instant TLf), the system is in Loop (201), waiting to receive a command for evolving towards another clip, through any one of the commands of interaction between the mobile terminal and the user, which command may arrive at any instant within the loop, according to the decision of the user, who chooses the instant for exiting the loop, thus obtaining a soft transition from one clip to the next one.
[0065] Optionally, the system may evolve automatically towards another clip, if no commands are received from the user within a MAXIMUM TIME.
[0066] Optionally, the wait interval of a node sequence can be reproduced only once (no loop). This permits the creation of situations where, if the user interacts with the correct “interactive instruction” within the wait interval, then the narration will continue following the main narrative flow; otherwise. i.e. if no command or a wrong command is issued, different video segments will be linked.
[0067] Optionally, the system may automatically handle the timeline and frame-rate of the loop (e.g. for slowing down the scene, . . . ) while waiting for a command from the user.
[0068] The types of available commands 202 are many and can be issued through specific user actions, such as the following: [0069] a plurality of sensors, such as touch-pad, microphone, gyroscope, camera, . . . [0070] a plurality of gestures, such as swipe, pinch, . . . [0071] a plurality of combinations of the above sensors and gestures, . . .
or issued through [software commands] 203, e.g. generated by timers (e.g. maximum time of permanence of the wait interval) or as a consequence of other commands generated by the user (e.g. parallel multistories, wherein a user command issued for a first multistory also results in a software command affecting the second multistory).
[0072] These commands are already known and available, for example, in some types of mobile telephone terminals, such as smartphones, being transformed in a known manner into electric and/or electronic control signals in the terminal.
[0073] The user terminal comprises an interactive display and/or one or more sensors, from which at least some of said commands can be derived, and/or one or more motion or voice or image or position detectors, from which at least some of said commands can be derived.
[0074] Based on the specific command received, issued within the wait interval, the system determines how the transition from one clip to another clip should evolve.
[0075] This means that, based on the type of command received in the loop (e.g. fast or slow shaking of the motion sensor), the time evolution of the node sequence will be modified by managing the flow speed and direction and the point where a jump to the next clip should occur within the node sequence.
[0076] Therefore, based on the type of command issued, the system will decide how the node sequence should evolve (forwards, backwards, fast, slow, . . . ) and hence also the point of the node sequence (301, 302, 305, 306) from which to go towards another clip, see
[0077] For every single node within the system, a matching table is defined between command types and evolution types.
[0078] There is a user interface that senses the command issued by the user and associates it with the type of reaction affecting the evolution of the clip.
[0079] The available commands may depend on the node, and may therefore be different for each node sequence.
[0080] Some node sequences may not be associated with any commands, and therefore may not contain a narration endings loop 105-108.
[0081] Some node sequences may consist of the loop only, so that it will be possible to jump from a loop directly to a subsequent segment or loop 110.
[0082] With reference to
[0083] If the exit time marker is placed before the start of the loop, then the system will move backwards, by appropriately adjusting the timeline and frame-rate, up to the exit point, thus linking to the next video segment 205.
[0084] If the exit time marker is placed after the end of the loop, then the system will move forwards, by appropriately adjusting the timeline and frame-rate, up to the exit point, thus linking to the next video segment 206.
[0085] For example, if during the wait interval (loop) a swipe right command 402 is issued (
[0086] If during the wait interval (loop) a swipe left command 401 is issued (
[0087] INTERACTION COMMANDS can only be issued, and hence interpreted, during the execution of the [Wait interval].
[0088] Management commands are, on the contrary, commands not related to interaction, and can be issued at any instant during multistory fruition or development, e.g. in order to impose a rewind action following a wrong or unpleasant selection or to jump to a previous clip.
[0089] According to a further variant, the same command issued at different time instants within the wait interval may execute the exit from the node sequence in different ways.
[0090] The wait segment is divided into n time intervals ΔT (304) and associated with an interactive instruction. One Clip(n) will be associated with each ΔT—see
[0091] According to a further variant, it is possible to assign different commands to the same time interval ΔT within the wait interval in order to develop the node sequence in different ways.
[0092] In a given time interval ΔT, a defined Clip(n) corresponds to each interactive instruction, see
[0093] According to a further variant, if the wait interval of a node sequence is a video taken at a frame-rate higher than 24 fps (e.g. 300 fps), the commands of the interactive instruction may increase or decrease the frame-rate of the node sequence.
[0094] For example (see
[0095] For example (see
[0096] As an alternative to the given interactive instruction (e.g. tap), in a given time interval the frame-rate increases (accelerates) to allow increasing the elevation of the jump to reach the opposite bank (1104); should the given interactive instruction be executed in a wrong manner or out of sync, the player will not take sufficient run-up and will fall into the void, thus not reaching the opposite bank (1107).
[0097] Within the same wait segment there may be several interactive instructions, in different time intervals (1101-1105).
[0098] According to a further variant, based on further types of commands (interaction and management commands) received, simultaneous side-by-side visualization of two or more MULTISTORIES is obtained, each one possibly having a timeline of its own subject to different commands, at different times.
[0099] With reference to
a) mutually independent
b) mutually interactive
[0100] In case of simultaneous vision of multiple multistories, a user command issued on one multistory may be associated with software commands capable of causing the parallel evolution of one or more node sequences of other multistories.
[0101] Example of combined node sequences with independent interactive instructions: with reference to
[0102] The combined node sequences can be manoeuvred either simultaneously (at the same instant) or separately (at distinc instants); they will need different interactive instructions in the former case or, in the latter case, indifferent ones.
[0103] Example of combined node sequences with complementary interactive instructions: with reference to
[0104] The following will explain in more detail the operating sequence of the system/method with reference to [0105]
[0108] With reference to the flow chart of
[0109] From a given App Store (e.g.: Apple Store or Google Play), the user downloads an .IPA file (or a file in an equivalent format) to his/her own device (smartphone or tablet) (block 61).
[0110] The .IPA file (or file in an equivalent format) downloads to the memory of the device a library of [VIDEO CLIPS] and layouts/templates coded in computer languages (e.g.: C++) compatible with iOS, Android and other operating systems (block 62).
[0111] By clicking on the icon of the .IPA file (or file in an equivalent format), the Application is executed (block 63).
[0112] The initial interface is the menu of the Application, which includes, among others, the “START” button (or equivalent commands, e.g.: BEGIN, START, etc.) (block 64 and
[0113] The video player displays the first [VIDEO CLIP] or [INITIAL CLIP] (block 65 and
[0114] The flow continues from
[0115] With reference to the flow chart of
[0116] The software by means of computer code (e.g.: C++) compatible with the operating system of the device (smartphone or tablet) executes the assigned VIDEO SEGMENT, linking in succession the SEQUENCES of the VIDEO CLIP (block 70).
[0117] The last sequence Sequence.sub.n or Node Sequence is connected to Sequence.sub.n-1 at the instant TLi (block 71), i.e. the frame of the Node Sequence identified by the time marker TLi will be linked—in succession—to the last frame of Sequence.sub.n-1.
[0118] If the Node Sequence is a final sequence or [Narration ending], the procedure will end (END) (block 72), otherwise it will continue.
[0119] If the procedure goes on, the video clip will move forwards and backwards in the time segment between the markers TLi and TLf [Wait interval], waiting for a command action from the user (block 73 and 303).
[0120] The software by means of computer code (e.g.: C++) compatible with the operating system of the device (smartphone or tablet) may also appropriately adjust the running speed of the [Wait interval], slowing down or accelerating the frame-rate in order to give more realism to the wait situation (block 74).
[0121] When the reception of a [command] is verified (block 75), the software by means of computer code (e.g.: C++) compatible with the operating system of the device (smartphone or tablet) associates a given gesture of the touchscreen (e.g.: swipe, tap, rotate, etc.) or a given input of the sensors of the device (e.g.: gyroscope, volume, etc.) or a given software command with a given time direction (backwards or forwards relative to TLi or TLf) and/or with a given frame-rate of the video clip (acceleration or slowing down) and/or with a given combination of both factors (time direction+frame-rate (block 77,
[0122] If absence of interactions is verified (block 75), then the loop between TLi and TLf will continue (block 76), and the operations will return to block 73 (303).
[0123] In the presence of a command from the user or from the software, the procedure will exit the loop of the wait interval, moving forwards or backwards to the time marker Exit point connected to that user action or command (block 78, 307-308, 311-312).
[0124] When the Exit point is arrived at, the software selects from the library (see point 2) the new VIDEO SEGMENT associated with the type of selection and/or command just executed (block 79).
[0125] The video player displays the new VIDEO CLIP (block 80).
[0126] The process starts again from the beginning (block 70).
[0127] The result is a succession of VIDEO SEGMENTS, the evolution of which—MANOEUVRED by the user's actions—produces a narrative experience—characterized by the choices of the user him/herself—that is unique, original and involving as a whole.
[0128] The present invention can advantageously be implemented through a computer program VIDEO EDITOR, which comprises coding means for implementing one or more steps of the method when said program is executed by a computer.
[0129] The following will list the steps of the process of using the method through the VIDEO
Editor:
[0130] a) Given a library of (n), composed of all video sequences (including, therefore, all possible branches of the Multistory), the computer expert [OPERATOR] selects the sequences for composing the [VIDEO SEGMENTS], including the sequences transformed into node sequences.
b) On the timeline of the node segment, the computer expert sets two time markers that delimit the backward and forward loop Wait interval of the node sequence. In this way, the node sequence will only be executed in backward and forward loop within the two time markers set on the timeline.
c) On the timeline within the Wait interval, the computer expert may set other additional time markers, as a function of the interaction gestures expected by the narrative development of the Multistory.
d) On the timeline of a video segment, the computer expert also sets any [exit time markers] and connection markers towards the next video segments, in accordance with the narrative construction of the Multistory.
e) The computer expert selects a given command readable by the mobile device (smartphone and/or tablet) relating to gestures and sensors of the device capable of sending executable inputs (e.g.: gesture on the touchscreen, voice command through the microphone, rotation of the gyroscope, etc.).
f) At each time marker set within the wait interval, the computer expert associates the previously selected command, so that upon that given command the node sequence will be executed past the markers delimiting the wait interval [TLi, TLf], up to the time markers connected with the associated command.
g) The computer expert selects from the library the video segments that will follow the executed node sequence based on the associated command; in this way, a given video segment(n) will correspond to the given command associated with the time marker and to the relevant “unlocked” part of the node sequence.
h) The computer expert repeats the same process using all the n sequences in the library, alternating video clips and node sequences so as to create the plurality of possible narrative directions of the Multistory [or “quest tree” ].
i) Once the quest tree has been formed and closed, the expert exports the project as an .IPA or equivalent file readable by the App Stores (e.g.: Apple Store, Google Play, etc.).
[0131] It is therefore understood that the protection scope extends to said computer program VIDEO EDITOR as well as to computer-readable means that comprise a recorded message, said computer-readable means comprising program coding means for implementing one or more steps of the method when said program is executed by a computer.
[0132] The above-described non-limiting example of embodiment may be subject to variations without departing from the protection scope of the present invention, comprising all equivalent designs known to a man skilled in the art.
[0133] The elements and features shown in the various preferred embodiments may be combined together without however departing from the protection scope of the present invention.
[0134] The advantages deriving from the application of the present invention are apparent, as described below by way of example.
[0135] Soft switching from one clip to the next is obtained.
[0136] In prior-art systems, in order to obtain different types of clip evolution, different clips are created, among which the user makes a selection. According to the present invention, on the contrary, the evolution of the clip itself is modified.
[0137] In prior-art systems, overlays or hyperlinks are added to obtain interactions, which however distract from pure fruition of the video clip (the term “pure” referring herein to viewing the video clip with no additional elements). According to the present invention, on the contrary, the video clip is directly acted upon without requiring the use of any additional elements on the video clip.
[0138] From the above description, those skilled in the art will be able to produce the object of the invention without introducing any further construction details.