Systems and methods are disclosed for generating a content structure that can then be used as a content design building block. Specifically, new content can be automatically generated by combining, replacing or mixing and matching objects, actions, settings, effects, audio, etc., from any number of previously stored content structures to create a new content structure, which is then rendered as a new content segment. Content segments are then stitched together to generate a finished content stream. In this way, new and original content can be created without the traditionally expensive and labor-intensive filming or computer animation processes.

A multi-dimensional database and indexes and operations on the multi-dimensional database are described which include video search applications or other similar sequence or structure searches. Traversal indexes utilize highly discriminative information about images and video sequences or about object shapes. Global and local signatures around keypoints are used for compact and robust retrieval and discriminative information content of images or video sequences of interest. For other objects or structures relevant signature of pattern or structure are used for traversal indexes. Traversal indexes are stored in leaf nodes along with distance measures and occurrence of similar images in the database. During a sequence query, correlation scores are calculated for single frame, for frame sequence, and video clips, or for other objects or structures.

Systems and methods for full motion video search are provided. In one aspect, a method includes receiving one or more search terms. The search terms include one or more of a characterization of the amount of man-made features in a video image and a characterization of the amount of natural features in the video image. The method further includes searching a full motion video database based on the one or more search terms.

The overall architecture and details of a scalable video fingerprinting and identification system that is robust with respect to many classes of video distortions is described. In this system, a fingerprint for a piece of multimedia content is composed of a number of compact signatures, along with traversal hash signatures and associated metadata. Numerical descriptors are generated for features found in a multimedia clip, signatures are generated from these descriptors, and a reference signature database is constructed from these signatures. Query signatures are also generated for a query multimedia clip. These query signatures are searched against the reference database using a fast similarity search procedure, to produce a candidate list of matching signatures. This candidate list is further analyzed to find the most likely reference matches. Signature correlation is performed between the likely reference matches and the query clip to improve detection accuracy.

A software system of some embodiments allows automatic logging, organization, transcription, and/or subtitling/captioning of audio/video content. User selected audio/video content is automatically transcribed. Instances and recurrences of events are automatically identified within transcripts and their corresponding clips. Recurrences of events can be automatically identified based on user identification of instances. Identifications within transcripts/clips can be automatically demarcated and logically or systematically arranged, categorized, classified, labelled, marked, grouped, and/or organized into audio/video sequences by the software system. Categorization, classification, labelling, and/or marking can be applied from transcript data, file names, order of identification occurrence, custom text, and/or analysis of audio and/or video content using AI. Transcripts can be displayed in arrangement and/or amalgamation to match the arrangement of their corresponding clips in an audio/video sequence. Subtitles/captions are automatically generated from transcripts or amalgamated arrangements of transcripts.

A method and apparatus for characteristic-based video processing include: in response to receiving a region of a picture of a video sequence, determining a characteristic in the region, the region being independent of other regions of the picture for video coding; determining a class associated with the region based on the characteristic, the class being selected from a plurality of classes; and encoding the region using a parameter set associated with the class, the parameter set being selected from a plurality of parameter sets for video coding at different quality levels.

At least one aspect of the present disclosure is directed to systems and methods of extracting media segments based on fingerprint matching. The method can include receiving a media stream comprising a plurality of frames and generating a plurality of fingerprints corresponding to each frames. The method can receive a target timestamp and determine a target fingerprint from the plurality that corresponds to the target timestamp. The method can retrieve candidate fingerprints, each of the candidate fingerprints corresponding to a frame in a candidate media stream. The method can compare the target fingerprint to the candidate fingerprints to determine a matching candidate fingerprint. The method can match fingerprints that correspond to media frames before and after the target fingerprint to determine the upper and lower bounds of a segment of interest. The method can extract the segment of interest based on the bounds and provide it to a respective party.

A system includes a display, and a database including surgical videos, images of organs in a human body obtained from a medical imaging device, and images of disease in a human body obtained from the medical imaging device. A controller including a processor is coupled to memory, the database, and the display, and the memory stores information that when executed by the processor causes the system to perform operations. For example, the processor may determine first organ information from the images of the organs, and first disease information from the images of the disease. The processor my calculate a similarity score between the first organ information and the first disease information and second disease information and second organ information indexed to the surgical videos. The processor selects one or more of the surgical videos based on the similarity score, and displays the surgical videos on the display.

Provided are a transition type determination method, an electronic device and a storage medium. The method includes: acquiring a picture matching degree of a candidate transition type with a transition position between two adjacent video clips, and acquiring a music matching degree of the candidate transition type; determining, based on the acquired image matching degree and the acquired music matching degree, a matching degree of the candidate transition type at the transition position; and determining, according to the matching degree, whether to determine the candidate transition type as a target transition type, where the target transition type is used for a transition effect between the two adjacent video clip.

A method of generating congruous metadata is provided. The method includes receiving a similarity measure between at least two multimedia objects. Each multimedia object has associated metadata. If the at least two multimedia objects are similar based on the similarity measure and a similarity threshold, the associated metadata of each of the multimedia objects are compared. Then, based on the comparison of the associated metadata of each of the at least two multimedia objects, the method further includes generating congruous metadata. Metadata may be tags, for example.