Method and apparatus for high bandwidth data transmission in content delivery networks

Abstract

Methods and apparatus for delivering data over extant infrastructure within a content-based network. In one embodiment, the network comprises a cable network, and the infrastructure comprises that nominally used for on-demand (OD) services such as VOD. The method includes the allocation of dedicated end-to-end network resources via a session request, as well as data flow control and packet size adaptation, by a data server based on feedback from the requesting/receiving client device (e.g., DSTB) within the network. Mechanisms for retransmission requests for error recovery are also provided.

Claims

1. An on-demand server apparatus configured to provide data over a content delivery network, comprising: an interface for communication with at least one client device; and a processor configured to run at least one computer program thereon, said computer program comprising a plurality of instructions which are configured to, when executed, cause said on-demand server apparatus to: receive at least one non-content related data structure at a head-end distribution server from at least one data source; process data contained within said non-content related data structure into a packet stream; create a descriptive structure configured to describe said non-content related data structure; establish a dedicated on-demand session on an in-band downstream transmission pathway from said head-end distribution server to said at least one client device based on a request received from software configured to operate on said at least one client device; transmit a plurality of time-shifted copies of said non-content related data structure over said in-band downstream transmission pathway within a multiplexed transport stream and as part of said dedicated on-demand session; monitor one or more retransmission requests in order to determine a transmission efficiency; and when said transmission efficiency is at or above a determined threshold, incrementally increase a rate at which said plurality of time-shifted copies of said non-content related data structure are transmitted until said transmission efficiency falls below said determined threshold.

2. The on-demand server apparatus of claim 1, wherein said transmission of said non-content related data structure occurs only during prescribed periods of time.

3. The on-demand server apparatus of claim 2, wherein said prescribed periods of time are selected based at least in part on bandwidth considerations relating to said network.

4. The on-demand server apparatus of claim 1, wherein said plurality of instructions are further configured to send tuning information for said dedicated on-demand session to at least a second client device of said network so as to enable said second client device to access at least one of said time-shifted copies of said non-content related data structure via said dedicated on-demand session.

5. The on-demand server apparatus of claim 1, wherein said descriptive structure comprises a metadata file, and said packet stream comprises a Moving Pictures Experts Group (MPEG) based packet stream.

6. The on-demand server apparatus of claim 5, wherein said transmission of said plurality of time-shifted copies of said non-content related data structure comprises a software package that includes at least portions of said MPEG-based packet stream and said metadata file.

7. The on-demand server apparatus of claim 1, wherein said non-content related structure comprises a gaming-related data structure.

8. The on-demand server apparatus of claim 1, wherein said transmission of said plurality of time-shifted copies of said non-content related data structure is initiated only upon an expectation that multiple users within a local service area will download one or more of said plurality of time-shifted copies of said non-content related data structure in a substantially simultaneous fashion.

9. A server apparatus configured to provide data over a content delivery network, comprising: a data interface configured to enable communication with a first client device; and a processor apparatus in data communication with said data interface and configured to run at least one computer program thereon, said at least one computer program comprising a plurality of instructions which are configured to, when executed, cause said server apparatus to: receive at least one non-content related data structure from at least one data source; process data contained within said non-content related data structure into a packet stream; create a descriptive structure configured to describe said non-content related data structure; establish a session on a downstream transmission pathway from said distribution server to said at least one client device based on a request received from software configured to operate on said first client device; transmit a plurality of time-shifted copies of said non-content related data structure over said downstream transmission pathway; monitor one or more retransmission requests in order to determine a transmission efficiency; and when said transmission efficiency is at or below a determined threshold, reduce a rate at which said plurality of time-shifted copies of said non-content related data structure are transmitted by a dynamically determined amount; wherein said transmission efficiency is inversely proportional to errors produced by said rate.

10. The server apparatus of claim 9, wherein said plurality of instructions are further configured to cause said first client device to download at least a portion of a first application via a wireless interface of said first device, said portion of said first application configured to enable said first client device to access said plurality of time-shifted copies of said non-content related data structure via said session.

11. The server apparatus of claim 10, wherein said portion of said first application is integrated with a second application residing on a second device, and use portion of said first application and said second application configured to enable transmission of said plurality of time-shifted copies of said non-content related data structure to said second device via wireless interface of said first device.

12. The server apparatus of claim 9, wherein said transmission is based at least in part on one or more rules, said one or more rules being configured to said one or more rules being configured to assign bandwidth within multiple quadrature amplitude modulation (QAM) channels to a plurality of OD session requests, including said request, so as to maximize granting of OD requests for a second level of service among a plurality of OD requests for a first level of service and said second level of service.

13. The server apparatus of claim 12, wherein said first level of service comprises a standard definition (SD) data encoding, and said second level of service comprises a high-definition (HD) data encoding.

14. The server apparatus of claim 9, when said transmission efficiency is above said determined threshold, incrementally increase said rate at which said plurality of time-shifted copies of said non-content related data structure are transmitted until said transmission efficiency begins to decrease.

15. A method for providing data over a content delivery network, comprising: receiving at least one non-content related data structure at a network distribution server from at least one data source; generating a packet stream by at least processing data contained within said non-content related data structure; creating metadata describing said non-content related data structure; receiving a request generated by software operating on at least one client device in data communication with said content delivery network; based at least in part on said request, establishing a session on a dedicated in-band downstream transmission pathway from said network distribution server to said at least one client device; and transmitting a plurality of time-shifted copies of said non-content related data structure over said dedicated in-band downstream transmission pathway within a multiplexed transport stream and as part of said session; monitoring one or more retransmission requests in order to determine a transmission efficiency; and when said transmission efficiency is at or below a determined threshold, reducing by a dynamically determined amount a rate at which said plurality of time-shifted copies of said non-content related data structure are transmitted; wherein said transmission efficiency is inversely proportional to errors produced by said rate.

16. The method of claim 15, wherein said transmitting occurs only during prescribed periods of time, said prescribed periods of time being selected based at least in part on bandwidth considerations relating to said content delivery network.

17. The method of claim 15, further comprising sending tuning information for said on-demand session to at least a second client device of said content delivery network so as to enable said second client device to access at least one of said time-shifted copies of said non-content related data structure via said dedicated on-demand session.

18. The method of claim 15, wherein: said generating comprises disposing at least one or more portions of said data within Moving Pictures Experts Group (MPEG) packets; and said metadata is configured to describe said at least one or more portions of said data of said MPEG packets.

19. The method of claim 15, wherein said metadata is utilized to disable a user-performed trick-mode functionality of said multiplexed transport stream.

20. The method of claim 15, wherein said non-content related data structure comprises a gaming-related software application computer program.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 is a functional block diagram illustrating an exemplary HFC network configuration useful with the present invention.

(2) FIG. 1a is a functional block diagram illustrating one exemplary head-end configuration of the HFC network of FIG. 1.

(3) FIG. 2 is a logical flow diagram illustrating one embodiment of the generalized methodology of transmitting a non-content data stream over a network according to the invention.

(4) FIG. 2a is a graphical representation of an exemplary embodiment of the session and data stream establishment process, and the various entities utilized therein, according to the invention.

(5) FIG. 2b is a logical flow diagram illustrating one embodiment of the method of ingesting packetized (e.g., MPEG-encoded) data at a server from a remote packaging source according to the invention.

(6) FIG. 2c is a logical flow diagram illustrating one embodiment of the method of ingesting raw data from a remote packaging source according to the invention.

(7) FIG. 2d is a logical flow diagram illustrating one embodiment of the method of ingesting data (packetized or raw) from a local packaging source according to the invention.

(8) FIG. 2e is a logical flow diagram illustrating one embodiment of the method of provisioning ISA packages according to the invention.

(9) FIG. 3 is a graphical representation of an exemplary embodiment of the session and data stream establishment process, and the various entities utilized therein, according to the invention.

(10) FIG. 3a is a logical flow diagram illustrating an exemplary embodiment of the method of session and stream establishment according to the invention.

(11) FIG. 4 is a graphical representation of an exemplary embodiment of the client end flow initiation and pause process according to the invention.

(12) FIG. 4a is a logical flow diagram illustrating an exemplary embodiment of the method of starting the flow of an established data stream according to the invention.

(13) FIG. 5a is a graphical representation of one exemplary basic catalog structure useful with the present invention.

(14) FIG. 5b is a graphical representation of one exemplary group catalog structure useful with the present invention.

(15) FIG. 5c is a graphical representation of one exemplary on-demand menu (Menu3) catalog structure useful with the present invention.

(16) FIG. 5d is a graphical representation of one exemplary on-demand selection catalog structure useful with the present invention.

(17) FIG. 6 is a functional block diagram of one exemplary embodiment of network server adapted for high-speed data download.

(18) FIG. 7 is a functional block diagram of one exemplary embodiment of network CPE adapted for high-speed data download.

(19) FIG. 7a is a graphical representation of an exemplary protocol stack useful with the CPE of FIG. 7.

(20) FIG. 7b is a logical flow diagram illustrating an exemplary embodiment of the client side data download processing conducted according to the invention.

DETAILED DESCRIPTION OF THE INVENTION

(21) Reference is now made to the drawings wherein like numerals refer to like parts throughout.

(22) As used herein, the terms network and bearer network refer generally to any type of telecommunications or data network including, without limitation, hybrid fiber coax (HFC) networks, satellite networks, telco networks, and data networks (including MANs, WANs, LANs, WLANs, internets, and intranets). Such networks or portions thereof may utilize any one or more different topologies (e.g., ring, bus, star, loop, etc.), transmission media (e.g., wired/RF cable, RF wireless, millimeter wave, optical, etc.) and/or communications or networking protocols (e.g., SONET, DOCSIS, IEEE Std. 802.3, ATM, X.25, Frame Relay, 3GPP, 3GPP2, WAP, SIP, UDP, FTP, RTP/RTCP, H.323, etc.).

(23) As used herein, the term head-end refers generally to a networked system controlled by an operator (e.g., an MSO or multiple-systems operator) that distributes programming to MSO clientele using client devices. Such programming may include literally any information source/receiver including, inter alia, free-to-air TV channels, pay TV channels, interactive TV, and the Internet. DSTBs may literally take on any configuration, and can be retail devices meaning that customers may or may not obtain their DSTBs from the MSO exclusively. Accordingly, it is anticipated that MSO networks may have client devices from multiple vendors, and these client devices will have widely varying hardware capabilities. Multiple regional head-ends may be in the same or different cities.

(24) As used herein, the terms client device and end user device include, but are not limited to, personal computers (PCs) and minicomputers, whether desktop, laptop, or otherwise, set-top boxes such as the Motorola DCT2XXX/5XXX and Scientific Atlanta Explorer 2XXX/3XXX/4XXX/8XXX series digital devices, personal digital assistants (PDAs) such as the Apple Newton, Palm family of devices, handheld computers, personal communicators such as the Motorola Accompli or MPx 220 devices, J2ME equipped devices, cellular telephones, or literally any other device capable of interchanging data with a network.

(25) Similarly, the terms Customer Premises Equipment (CPE) and host device refer to any type of electronic equipment located within a customer's or user's premises and connected to a network. The term host device refers generally to a terminal device that has access to digital television content via a satellite, cable, or terrestrial network. The host device functionality may be integrated into a digital television (DTV) set. The term customer premises equipment (CPE) includes such electronic equipment such as set-top boxes, televisions, Digital Video Recorders (DVR), gateway storage devices (Furnace), and ITV Personal Computers.

(26) As used herein, the term network agent refers to any network entity (whether software, firmware, and/or hardware based) adapted to perform one or more specific purposes. For example, a network agent may comprise a computer program running in server belonging to a network operator, which is in communication with one or more processes on a CPE or other device.

(27) As used herein, the term ISA refers to any of the existing or future variants of the Interactive Services Architecture Specification or related specifications, including without limitation ISA versions 1.4 and 1.5, each incorporated herein by reference in its entirety.

(28) The term processor is meant to include any integrated circuit or other electronic device (or collection of devices) capable of performing an operation on at least one instruction including, without limitation, reduced instruction set core (RISC) processors, CISC microprocessors, microcontroller units (MCUs), CISC-based central processing units (CPUs), and digital signal processors (DSPs). The hardware of such devices may be integrated onto a single substrate (e.g., silicon die), or distributed among two or more substrates. Furthermore, various functional aspects of the processor may be implemented solely as software or firmware associated with the processor.

(29) As used herein, the term package refers to an arrangement of computer-readable data files or other data structures assembled to comply with a specific syntax or protocol.

(30) As used herein, the term provisioning refers generally to a process whereby a package, content title or other information is provided to a service (such as on-demand download service) so that the information is integrated with other functions and software modules within the service.

(31) As used herein, the terms computer program, routine, and subroutine are substantially synonymous, with computer program being used typically (but not exclusively) to describe collections or groups of the latter two elements. Such programs and routines/subroutines may be rendered in any language including, without limitation, C#, C/C++, Fortran, COBOL, PASCAL, assembly language, markup languages (e.g., HTML, SGML, XML, VoXML), and the like, as well as object-oriented environments such as the Common Object Request Broker Architecture (CORBA), Java and the like. In general, however, all of the aforementioned terms as used herein are meant to encompass any series of logical steps performed in a sequence to accomplish a given purpose.

(32) Overview

(33) The present invention provides, inter alia, apparatus and methods for downloading data (such as large binary objects or files) at accelerated rates over a communication channel within a network. In one embodiment, the network comprises a cable television network, and data delivery is accomplished via a point-to-point approach wherein a session is established between the receiving entity (such as a DSTB) and the distributing entity (e.g., an OD server) using one or more allocated QAMs, and a program identifier. Session establishment and data flow control are advantageously implemented using protocols and bandwidth that are typically used for delivery and control of video-on-demand (VOD) or similar services, thereby obviating any substantive modifications to the existing network infrastructure. Sessions can be established for the data transfer, and then immediately terminated when the transfer is completed, thereby rapidly freeing up bandwidth on the network as with a conventional OD session.

(34) In one variant, the data is compliant with the Interactive Services Architecture (ISA) specification, and is disposed within MPEG transport packets such that the data appears (and advantageously can be handled) much as any other MPEG transport stream. The exemplary client application receiving the MPEG packets checks the packets for integrity, manages flow control, and reassembles the packets at the client device to provide the user with data in its original form. A catalog entry is created for the data structure(s) so that the DSTB or other downstream device can be made aware of the availability of the data. This functionality can be accomplished using, e.g., a navigator application implemented within the DSTB.

(35) Applications where the accelerated data download capability of the present invention may be especially useful include downloading large binary executable files for software applications or games, downloading the results of network-based content searches or database queries, distributing data or files related to interactive television or television-based commerce, or any other use that requires significant volumes of data be delivered quickly and efficiently over the network.

(36) The data download methods of the invention are also completely agnostic to the type of payload data being transmitted, thereby allowing the transfer of literally any type of data or files over the network.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

(37) Exemplary embodiments of the apparatus and methods of the present invention are now described in detail. While these exemplary embodiments are described in the context of the aforementioned hybrid fiber coax (HFC) cable system architecture having an multi-systems operator (MSO), digital networking capability, and plurality of client devices/CPE, the general principles and advantages of the invention may be extended to other types of networks and architectures, whether broadband, narrowband, wired or wireless, or otherwise, the following therefore being merely exemplary in nature. For example, these techniques can be employed in the context of a broadband satellite network.

(38) It will also be appreciated that while described generally in the context of a network providing service to a customer (i.e., home) end user domain, the present invention may be readily adapted to other types of environments including, e.g., commercial/enterprise, and government/military applications. Myriad other applications are possible.

(39) System Architecture

(40) FIG. 1 illustrates a typical content-based network configuration with which the high-bandwidth data services apparatus and methodology of the present invention may be used. The various components of the network 100 include (i) one or more data and application origination points 102; (ii) one or more application distribution servers 104; (iii) one or more VOD servers 105, and (iv) customer premises equipment (CPE) 106. The distribution server(s) 104, VOD servers 105 and CPE(s) 106 are connected via a bearer (e.g., HFC) network 101. A simple architecture comprising one of each of the aforementioned components 102, 104, 105, 106 is shown in FIG. 1 for simplicity, although it will be recognized that comparable architectures with multiple origination points, distribution servers, VOD servers, and/or CPE devices (as well as different network topologies) may be utilized consistent with the invention. For example, the head-end architecture of FIG. 1a (described in greater detail below) may be used.

(41) The application origination point 102 comprises any medium that allows an application (such as a data download application or VOD-based application) to be transferred to a distribution server 104. This can include for example an application vendor website, CD-ROM, external network interface, mass storage device (e.g., RAID system), etc. Such transference may be automatic, initiated upon the occurrence of one or more specified events (such as the receipt of a request packet or ACK), performed manually, or accomplished in any number of other modes readily recognized by those of ordinary skill.

(42) The application distribution server 104 comprises a computer system where such applications can enter the network system. Distribution servers are well known in the networking arts, and accordingly not described further herein.

(43) The VOD server 105 a computer system where on-demand content, as well as the data discussed in greater detail below) can be received from one or more data sources 102 and enter the network system. These sources may generate the content/data locally, or alternatively act as a gateway or intermediary from a distant source. The VOD server 105 includes the Session Resource Manager (SRM) functionality, and asks the Digital Network Control System (DNCS) for resources. The DNCS responds with negative or positive response to the request, and the VOD server implements the appropriate resource allocation logic.

(44) The CPE 106 includes any equipment in the customers' premises (or other locations, whether local or remote to the distribution server 104) that can be accessed by a distribution server 104. Such CPEs 106 comprise processors and associated computer memory (and optionally mass storage) adapted to store and run the downloaded or resident application, as well as receive and store the streamed in-band content and data. In the present context, at least a portion of the CPE application necessary to facilitate high-speed data download can itself be downloaded to the CPE 106, wherein the latter executes the downloaded application(s)/components in order to enable the CPE to receive the high-speed data, although it will be recognized that the application(s) may also be resident on the CPE before download, received from another source (such as a third party Internet site, CD-ROM, etc.).

(45) Referring now to FIG. 1a, one exemplary embodiment of a head-end architecture useful with the present invention is described. As shown in FIG. 1a, the head-end architecture 150 comprises typical head-end components and services including billing module 152, subscriber management system (SMS) and CPE configuration management module 154, cable-modem termination system (CMTS) and OOB system 156, as well as LAN(s) 158, 160 placing the various components in data communication with one another. It will be appreciated that while a bar or bus LAN topology is illustrated, any number of other arrangements as previously referenced (e.g., ring, star, etc.) may be used consistent with the invention. It will also be appreciated that the head-end configuration depicted in FIG. 1a is high-level, conceptual architecture and that each MSO may have multiple head-ends deployed using custom architectures.

(46) The architecture 150 of FIG. 1a further includes a multiplexer/encrypter/modulator (MEM) 162 coupled to the HFC network 101 adapted to condition content for transmission over the network. In the present context, the distribution servers 104 are coupled to the LAN 160, which provides access to the MEM 162 and network 101 via one or more file servers 170. The VOD servers 105 are coupled to the LAN 160 as well, although other architectures may be employed (such as for example where the VOD servers are associated with a core switching device such as an 802.3z Gigabit Ethernet device). As previously described, information is carried across multiple channels. Thus, the head-end must be adapted to acquire the information for the carried channels from various sources. Typically, the channels being delivered from the head-end 150 to the CPE 106 (downstream) are multiplexed together in the head-end and sent to neighborhood hubs (not shown).

(47) Content (e.g., audio, video, etc.) is provided in each downstream (in-band) channel associated with the relevant service group. As will be discussed in greater detail subsequently herein, high-speed data is also provided over in-band channels, while associated metadata files are provided either in-band or out-of-band (OOB). To communicate with the head-end, the CPE 106 uses the OOB or DOCSIS channels and associated protocols. The OCAP 1.0 specification provides for networking protocols both downstream and upstream.

(48) It will also be recognized that the multiple servers (OD or otherwise) can be used, and disposed at two or more different locations if desired, such as being part of different server farms. These multiple servers can be used to feed one service group, or alternatively different service groups. In a simple architecture, a single server is used to feed one or more service groups. In another variant, multiple servers located at the same location are used to feed one or more service groups. In yet another variant, multiple servers disposed at different location are used to feed one or more service groups. One exemplary multi-server architecture particularly useful with the present invention is described in co-pending and co-owned United States Patent Application Publication No. 20020059619 to Lebar published May 16, 2002 and entitled Hybrid central/distributed VOD system with tiered content structure which is incorporated herein by reference in its entirety.

(49) Specifically, a hybrid central/distributed and tiered video on demand (VOD) service network with tiered content structure is disclosed. In particular, the system uses media servers located in both the head-end and hub stations. Set-top boxes generally would be supplied VOD services from the high-demand content media (and data) servers located in the hub station nearest to the user. The central media server located in the head-end would be used as an installed backup to the hub media servers; as the primary source for lower demand VOD services and as the source of the real time, centrally encoded programs with PVR (personal video recorder) capabilities. By distributing the servers to the hub stations, the size of the fiber transport network associated with delivering VOD services from the central head-end media server is reduced. Hence, each user has access to several server ports located on at least two servers. Multiple paths and channels are available for content and data distribution to each user, assuring high system reliability and enhanced asset availability. Substantial cost benefits are derived from the reduced need for a large content distribution network and the reduced storage capacity requirements for hub servers.

(50) Many other permutations of the foregoing system components and communication methods may also be used consistent with the present invention, as will be recognized by those of ordinary skill in the field.

(51) Methods

(52) Referring now to FIG. 2, one exemplary generalized methodology of providing high-speed data services over a network is described. It will be recognized that the steps shown in the embodiment of FIG. 2 are high-level logical steps applicable to literally any cable on-demand (e.g., VOD) architecture, and are not intended to require or imply any specific process flow that may occur within particular implementations of the method. In practical embodiments, some of these steps (or sub-steps within each step) may be implemented in parallel, on different hardware platforms or software environments, performed iteratively, and so forth.

(53) In the first step 202 of the method 200, data in the form of, e.g., one or more data files or structures is brought onto or ingested a head-end server, such as by executing the appropriate communication protocol between the source of data and a head-end server. Such ingestion of data and the supporting protocols to accomplish this function are well known in the art, and accordingly not described further herein. As used herein, the terms data files and data structures refer generally to literally any organized form or assembly of binary or other data such as, without limitation, binary executable files, graphics or audio files (compressed or otherwise), encryption-related files, zipped files, video files (e.g., AVIs or MPEGs), etc.

(54) Next, a catalog entry is optionally created for the loaded data file(s) per step 204. This action is performed so that user CPE 106 can be made aware of the availability of the data file(s). This can be accomplished via, e.g., a navigator application implemented on the CPE 106, or some other user interface (UI) mechanism including on-screen alerts, audible signals, periodic or regularly scheduled status functions, etc.

(55) Per step 206, the data file(s) or other structures is/are processed into a format or protocol suitable for transmission over the cable network (stream formation). For example, the data maybe formatted according to the well-known MPEG (e.g., MPEG2) packet data format such that the resulting data packets are effectively indistinguishable to the network infrastructure from other (i.e., content) packets.

(56) It will be appreciated that in various embodiments, the cataloging and stream formation processes (steps 204, 206 respectively) may be implemented concurrently, serially, or in another implementation-specific order as required.

(57) When the CPE 106 requests delivery of the data stream, a downstream data flow is established (typically involving allocation of server, network and client resources) per step 208. Based on optional feedback obtained (step 210) from the CPE 106 receiving the data stream, some characteristics of the stream (e.g. transmission rate, multiplexing parameters, etc.) may be modified, and/or re-transmissions of the data may take place in order to overcome any transmission errors (step 212).

(58) According to one exemplary protocol, the data transmission is conducted, and the software process at the head-end server (or other transmitting location) also monitors retransmission requests to evaluate their frequency, etc. in order to determine a transmission efficiency. For example, if transmission of a given data file at Rate A results in requests that 50% of the transmitted packets be re-transmitted (50% efficiency), then the server software process may reduce the rate by a predetermined or dynamically determined amount, to Rate B (B<A), measure efficiency, and adjust again if needed. Similarly, where Rate A produces no errors (100% efficiency), the rate can be incrementally increased until efficiency begins to drop, or another criterion (such as maximum channel rate) is reached.

(59) The foregoing steps are now described in detail with reference to FIGS. 2a-2c. It will be appreciated that while the exemplary method is described below primarily in the context of various logical or functional software processes or entities (which may include for example software objects), aspects of the invention may be realized in other forms as well, including hardware or firmware, or combinations of the foregoing.

(60) Data Ingestion and Package Creation

(61) As used herein, the term ingestion refers generally to the process by which a data file or other structure is transferred or loaded onto another entity, such as for example a head-end server. This ingestion may be accomplished by using appropriate interfaces for the data (and any associated metadata or other data structures).

(62) The details or particular implementation of the syntax used for the ingestion process of FIG. 2 may be maintained consistent, or alternatively vary, from network to network. For example, in cable networks that implement the aforementioned ISA architecture, such data transfer is performed in accordance with the CableLabs Asset Distribution Interface (ADI) Specification, Version 1.1, MD-SP-ADI1.1-I03-040107, dated Jan. 7, 2004, although it will be appreciated that compliance with this specification is not a requirement in practicing the broader invention disclosed herein. For such ADI-compliant cable networks, the ingestion process relies on valid Asset Distribution Interface (ADI)-compliant files, and a predetermined sequence of steps as defined by the ISA specification. ADI files minimally contain the ADI.XML, ADI.DTD, and any MPEG transport files contained with the ADI.XML file.

(63) When performing ingestion of data files in an existing VOD deployment, the steps taken under the exemplary embodiment of the invention are purposely selected to closely match the steps taken for corresponding VOD content ingestion, with the exception of adding/substituting data fields to indicate to the servers or other processing/distribution entities that the particular file being loaded is a data type (as opposed to content). These data fields are used by the head-end server(s) for a variety of functions including, e.g., to turn off or temporarily disable trick mode support (i.e., fast forward, rewind, or other PVR functions) for the data files, since such support is not desired or necessary. They may also be used by the CPE 106 to alert the CPE that the packets (e.g., MPEG packets) within which the data is encapsulated are data versus content-related.

(64) It will be recognized however that these data fields, or others included in the protocol, may also be used to turn on or enable functionality particular to, or which is to be selectively applied to, the data files if desired, such as e.g., additional encryption, error correction, coding, signal processing, compression/decompression, etc.

(65) FIG. 2a graphically illustrates an exemplary series of steps taken in performing the creation of a data package, as well as subsequent data ingestion (step 202 of FIG. 2), within a cable network implementing the ISA specification. As described in greater detail below, the sources of data for the ingestion process may be, e.g., either (i) remote (i.e., coming across the interface between a content provider or other third-party source and a cable head-end; see FIGS. 2b and 2c and their supporting discussion), or (ii) local (i.e., created locally within the head-end, data infrastructure, or associated components; see FIG. 2d). It will be recognized that the terms local and remote as used in the present context are arbitrary and relative in nature, and do not connote any particular location, proximity, relationship or the like.

(66) It will also be recognized that while the following description of the exemplary embodiments are rendered in terms of an object-oriented software environment (including various software objects and entities), other paradigms and architectures may be used consistent with the invention. For example, one or more aspects of the described functionality may be implemented in anon-object oriented environment, and/or in firmware or hardware as desired. Hence, the following discussion is merely illustrative of the broader principles.

(67) For enabling remote package ingestion of MPEG data (see FIGS. 2a and 2b), a content provider first receives the raw data (step 220), then creates an ADI-complaint package 270 including the ADI.XML, ADI.DTD, and data MPEG file(s) per step 222. For MPEG data, the metadata Type value is set to data_mpeg to indicate that the content file comprises valid MPEG sections encoded according to the relevant data encoding specification.

(68) The created package 270 is then ingested into the system using the Packager entity 271 (step 224). In the illustrated embodiment, receipt of the complete ADI package 270 triggers the ingestion of data into the system. An exemplary implementation of the interface used for this ingestion comprises one based on the Web-based Distributed Authoring and Versioning (WebDAV) method, although it will be appreciated that other paradigms may be used. As is well known, WebDAV is a set of extensions to the HTTP protocol which allows users to collaboratively edit and manage files on, e.g., remote servers.

(69) Referring now to FIGS. 2a and 2c, one exemplary method for ingestion of raw remote data is described. Per steps 230 and 232, the content provider receives the raw data and creates an ADI-compliant package including the ADI.XML, ADI.DTD, and the raw data file. The metadata for Type is set to data_raw to indicate that the content file requires MPEG packetization.

(70) In step 234, the raw content file is loaded onto the file system of the Packager entity 271.

(71) The Packager 271 user interface (UI) is optionally used by the operator to specify that the file requires encoding, and the MPEG packetizer 273 is triggered to convert the file(s) into an MPEG single program transport stream (SPTS). The raw file is then converted into MPEG format by the MPEG packetizer 273 (step 236) using any number of different techniques well known in the art. The converted MPEG file is stored on the Packager entity file system (step 237).

(72) Lastly, in step 238, the Packager entity 271 produces an ADI.XML and an ADI.DTD file describing the new package.

(73) Alternatively, for the local ingestion of the material (whether encoded, such as e.g. MPEG data, or raw) the exemplary process of FIG. 2d is performed as referenced to FIG. 2a. First, the data content is provided (step 239) and then locally ingested using, e.g., the Packager entity UI (step 241) and associated HTTP protocol. For data_raw types, the Packager may provide MPEG encapsulation to trans-code the file(s) to comply with a data encoding specification.

(74) In step 243, the raw content file is loaded onto the Packager file system. The Packager UI allows the operator to specify that the file requires encoding, and the MPEG packetizer is triggered to convert the file into an MPEG SPTS per step 245.

(75) In step 247, the converted MPEG file is stored on the Packager file system.

(76) Lastly, the packager produces an ADI.XML and an ADI.DTD file describing the new package (step 249).

(77) For provisioning of standard ISA packages, the exemplary process of FIG. 2e is employed (referenced to FIG. 2a). For data file provisioning, this process is optionally made identical to the standard ISA VOD provisioning, except for the differences as noted.

(78) In step 250, the Packager entity creates a new Package object on the PackageFactory entity 276. The new Package object provisioning includes passing a Uniform Resource Locator (URL) of the ADI files.

(79) In step 252, The PackageFactory entity transfers the ADI files from the Packager 271, and validates their content. The PackageFactory calls the Application associated with the Product found in the ADI to determine which ContentStore should be used to store the content (step 254).

(80) In step 256, the PackageFactory creates (Data) Assets within the AssetFactory 280.

(81) In step 258, The PackageFactory 276 creates metadata that corresponds to the metadata found in the ADI file.

(82) In step 260, the PackageFactory creates a Content object on the ContentStore returned from the Application. The Content object is provisioned with the URL of the content file on the Packager file system.

(83) Per step 262, the ContentStore entity 281 transfers (e.g., via FTP or other such protocol) the content file from the Packager file system. In the exemplary embodiment, the ContentStore does not generate a trick-mode or corresponding file from the Data Asset MPEG file. The Video Server will receive an indication of a data (versus content) file by either (i) parsing the metadata, or (ii) by the stream_type indication (0x05) located in the PMT.

(84) In step 264, the Application entity 290 creates a set of catalogs that contain the new Package object. This may be done automatically, based upon a condition precedent, at a specific time-of-day, by manual operator action, etc. Catalogs are placed on the Broadcast File System (BFS) 292 or suitable data carousel (e.g., the OCAP Object Carousel).

(85) In step 266, the client device 106 (and its resident application(s) 294) reads the new catalogs, and advertises the new titles, games, etc. to the viewer.

(86) Data Session and Stream Creation

(87) Referring now to FIGS. 3 and 3a, an exemplary process flow used in data session and stream creation is described in detail.

(88) Since it is often advantageous to introduce a new service into a cable network without having to add new software interfaces or hardware elements to the existing set up, the following exemplary embodiment of the invention uses session and stream creation methodologies which are completely analogous to a standard VOD flow, thereby minimizing or even obviating any such software and hardware additions. It will be appreciated, however, that such analogous methods are not required to practice the invention; hence, the following discussion is merely illustrative of the broader principles.

(89) As shown in FIGS. 3 and 3a, the first step of the process 300 comprises the relevant client application 294 (e.g., navigation application, Watch TV, dedicated data download application, etc.) on the CPE 106 creating a session request which is passed up to the SessionGW entity 380 using, e.g., the Session Setup Protocol (SSP) per step 302. The SessionGW entity 380 initiates a session by creating a Session object (step 304), and passes the Session object to the Service entity 382 (step 306). The Service entity 382 finds the relevant asset via the AssetFactory 280 (step 308).

(90) The Asset and Service entities then access the Stream Service 386 to access the Lightweight Stream Control Protocol (LSCP), and create the requested stream (step 310). As is well known, the LSCP allows, inter alia, VOD client sessions to communicate directly with a VOD server to control the content as well as streaming trick modes. However, it will be recognized that other protocols providing the desired functionality may be used consistent with the present invention.

(91) Lastly, in step 312, the session information is transmitted back to the initiating CPE 106 via, e.g., an in-band or OOB downstream channel (or other communications channel).

(92) Data Transport Control and Error Correction

(93) Referring now to FIGS. 4 and 4a, one embodiment of the process 400 of starting a stream that has been created according to the methodology of FIGS. 3 and 3a is described in detail.

(94) As shown in FIGS. 4 and 4a, the client application 294 uses LSCP over TCP (or a comparable protocol) to establish a stream control session with the servicing entity, e.g., VOD server 105 (step 402). This session establishment step includes sending an LSCP Play command starting at the beginning of the file (NPT: 0). In step 404, The StreamService entity 386 at the VOD server plays the MPEG or other content at the prescribed rate (e.g., 3.75 Mbps). In step 406, the client application 294 receives the MPEG sections, and validates these sections for integrity per step 408. If the client determines that the data receipt rate exceeds its effective processing rate such as, e.g., by determining that the CPE's buffers are filling faster than its application 294 can decode and validate the MPEG sections (step 410), it may pause the download of the file (step 412).

(95) Per step 412, the client sends an LSCP Pause command to allow it time to reduce its buffer levels. Once the CPE buffer returns to an acceptable level, the client can resume download by sending a LSCP Play command. Any number of different buffer management techniques may be applied consistent with the invention, including for example a high/low watermark approach, setting/expiration of a timeout value, etc. In one embodiment, an asymmetric high/low watermark approach is used, with the high water mark being offset or dead-banded from the low water mark (i.e., activation of the Pause function occurring at a higher buffer level than that at which the reset is activated), thereby mitigating cycling between the two states. In an alternate embodiment, a Pause command is issued at a certain buffer level (increasing), and a timer invoked that keeps the Pause state in effect for a prescribed period of time. It will be recognized by those of ordinary skill in the packet communication arts that myriad other approaches may be used as well.

(96) If the client discovers an error while processing its MPEG sections (per step 408), the client may request the VOD server 480 (or other servicing entity) to restart playback (step 414) from a previous section of the data stream. In one embodiment, the LSCP Play command is used, passing the NPT (normal play time) value calculated by the client application 294 based on the last known correct section. Exemplary methods utilized by the client application 294 for calculating the new NPT are described below with respect to FIG. 7b and Appendix I.

(97) In one embodiment of the data encoding methodology of the invention, the raw data stream is mapped into a format that is more suitable for transmission over the chosen bearer network. For example, in one variant, the network comprises an HFC cable television network, and the encoding of the raw data is performed using, e.g., the CableLabs Video-On-Demand Content Encoding Profiles Specification (MD-SP-VOD-CEP-I01-040107) dated Jan. 7, 2004 (with the exclusion of video and audio parameters) in order to maintain compatibility with the ISA specification. It will be appreciated by those of ordinary skill that other encoding approaches may be used as well, the foregoing being merely illustrative.

(98) The data encoded according to the chosen scheme is packetized in a format suitable for transportation of the data to the CPE 106. To maintain compatibility with current cable systems, this requires an initial packetization process in order to create a valid MPEG-2 Transport Stream. Several implementations of this packetization process are possible. For example, one implementation utilizes the MPEG-2 private_section field for encapsulating the data files (as described below). The syntax of MPEG-2 private section is shown in Appendix I hereto. Another implementation may use the well-known DSM-CC format or the like.

(99) Per the exemplary MPEG-2 standard, the Program Map Table (PMT) provides the mapping between the number of a program (PID) and the elements that comprise the program. In embodiments of the present invention based on ISA-compliant cable networks, packetization of the data files should be performed so as to create a single program comprised of MPEG-2 private sections. In such cases, the stream_type is set to 0x05 which identifies ITU-T Rec. H.222.0 ISO/IEC 13818-1 PES private_sections. The PMT can be located at the standard VOD stream location; i.e., PID 0x1E0 (480). The PID containing the private_section data is located at 0x1E1 (481). Other mapping schemes, including those not complaint with the aforementioned standards (such as in proprietary network applications), may also be utilized if desired.

(100) It is advantageous to have the aforementioned encapsulated data files formatted such that playback of the files will not require any modifications to the standard VOD server 105 or other infrastructure within the network. The CPE application 294 is also configured so as to be able to decode the transport stream and reassemble the data file at the CPE 106 at a desired rate. In one exemplary embodiment of the invention, Program Clock References (PCRs) located in the existing MPEG-2 transport stream infrastructure are used to manage the playback bit-rate of the data stream.

(101) In conventional cable VOD systems, program streams are assigned a fixed bit-rate. For example, 3.75 Megabits/second in a commonly used bit-rate for video streams in the conventional VOD deployments. In one exemplary embodiment of the invention, bandwidth equivalent to an integer (N) multiple of a baseline VOD program bandwidth is allocated to the data transfer according to the following relationship:
Data bandwidth=Baseline bandwidthNEqn. (1)

(102) For example, in a VOD system where video channels are allocated 3.75 Mbps each (SD), the data transfer of the invention can be allocated 3.75 Mbps, 7.5 Mbps (i.e., two times 3.75), 11.25 Mbps, (three times 3.75), and so forth. Alternatively, the full bandwidth of a channel (e.g., one 256-QAM), or roughly 38 Mbps (N=10), can be allocated to a given data session. Other baseline data rates may be used also, such as e.g., roughly 15 Mbps corresponding to an HD session. In one embodiment, the ADI file can be used to perform the selection, although it will be appreciated that other mechanisms may be employed.

(103) In another embodiment, two or more different data streams may share bandwidth, and hence rates that are non-integer multiples of the baseline video rate can be achieved if desired. This approach allows for a finer level of control and variability within the bandwidth allocation mechanisms, yet also makes the allocation process more complex as compared to the simple integer-based approach discussed above.

(104) Catalog Functions

(105) As noted above with respect to FIG. 2a, the exemplary process of the invention creates one or more catalog entries for the ingested data file(s) so that the user's CPE 106 can be made aware of the availability of the data file(s). These catalog entries can be made generic or standardized in nature, or alternatively application specific (e.g., with respect to the client application 294). Alerts or notice to the user can be accomplished via, e.g., a navigator application implemented on the CPE 106, or some other user interface (UI) mechanism including on-screen alerts, audible signals, periodic or regularly scheduled status functions, etc.

(106) The navigator deployed in cable networks operated by the Assignee hereof uses an exemplary catalog structure (see FIGS. 5a-5d herein) to carry all data to its CPE 106. Accordingly, one embodiment of the present invention utilizes the same catalog structure for consistency, although it will be recognized that other structures may be used. For existing on-demand (OD) services, three specific catalog structures are used (i.e., Group, OnDemand Menu3, and OnDemand Selection).

(107) FIG. 5a illustrates an exemplary basic construct of the catalog. All catalogs derive from this basic structure.

(108) FIG. 5b illustrates an exemplary group catalog structure useful with the present invention.

(109) FIG. 5c illustrates an exemplary OnDemand Menu3 Catalog Structure useful with the invention. See also Appendix II (OnDemand Menu3 Catalog MenuFlags Types) and Appendix III (OnDemand Selection Catalog Offering Types) hereto.

(110) FIG. 5d illustrates an exemplary OnDemand Selection Catalog Structure useful with the invention. See also Appendix IV (OnDemand Selection Catalog Selection Types).

(111) ADI Metadata Data Asset Type

(112) As will be recognized, it is important to provide consistent handling of the metadata (e.g., ADI Package metadata) for data MPEG types such as an ADI raw data file. This metadata is used, inter alia, to communicate with the VOD server 105 so as to avoid trick-mode generation (undesired during data operations), including disabling fast-forward and reverse play of the stream. Advantageously, this functionality can be readily implemented within existing architectures, such as e.g., through minor changes to the ADI specification previously referenced herein. Appendix V hereto provides an exemplary modification to the relevant ADI specification (i.e., CableLabs Video-On-Demand Content Specification Version 1.1, MD-SP-VOD-CONTENT1.1-I03-040107 dated Jan. 7, 2004).

(113) Specifically, in the illustrated embodiment, the Asset_Class and Type fields are used to contain non-video MPEG data. Asset_Class is an exemplary system-level type descriptor for the asset. This is intended for use with application mapping and routing, and is more general than the Type value for the content.

(114) The exemplary Type field determines how and where an asset is stored in the system. For example, values of this field might comprise data_mpeg or data_raw. If the type is data_raw, then the ingestion process will convert the data file into an MPEG file compliant with the relevant data encoding specification.

(115) Upon receipt of the encoded data packets, the CPE 106 removes the MPEG section packing, and reassembles the original data structure (e.g., file). As noted, these re-assembled data files may be for any number of different applications including, e.g., game download and video-to-go files that may be later downloaded or transferred to external devices. The ingesting VOD server 105 at the head-end interprets the Asset_class (e.g., data) and Type (e.g., data_mpeg or data_raw) metadata fields, and does not attempt to generate a trick-mode file used for fast-forward and reverse play. Also, the VOD server uses the Asset_class and Type fields to disable trick-mode commands initiated using the Lightweight Stream Control Protocol (LSCP).

(116) It will also be recognized that the exemplary scheme of Appendix V may be expanded or modified in order to provide further information about the content, such as for example data relating to encryption/keys (including, e.g., key lengths, residues or checksums), signatures or certificates, compression schemes, other applied data encoding schemes, CRC/FEC data, etc.

(117) Network Server

(118) Referring now to FIG. 6, one embodiment of the improved network electronic device with high-speed data download capability according to the present invention is described. As shown in FIG. 6, the device 601 generally comprises and OpenCable-compliant network server module adapted for interface with the HFC network 101 of FIG. 1 (e.g., the MEM 162 at the head-end, and/or the LAN 158, 160), digital processor(s) 604, storage device 606, and a plurality of interfaces 607 for use with other network apparatus such as IP routers and other packet network devices, network management and provisioning systems, local PCs, etc. Other components which may be utilized within the server device 601 include amplifiers, board level electronic components, as well as media processors and other specialized SoC or ASIC devices. Support for various processing layers and protocols (e.g., 802.3, DOCSIS MAC, OOB channels, DHCP, SNMP, H.323/RTP/RTCP, VoIP, SIP, etc.) may also be provided as required. A VOD application is also disposed to run on the server module 601 to provide a functional interface for VOD session requests received from network CPE 106, or other interposed entities. These additional components and functionalities are well known to those of ordinary skill in the cable and embedded system fields, and accordingly not described further herein.

(119) The server device 601 of FIG. 6 may take any number of physical forms, comprising for example one of a plurality of discrete modules or cards within a larger network head-end or edge device of the type well known in the art, including the MEM 162 itself. The server may also comprise firmware, either alone or in combination with other hardware/software components such as those previously described (e.g., disposed in the aforementioned edge device). Alternatively, the server module 601 may be a stand-alone device disposed at the head end or other location (such as a VOD server 105 or application server 104), and may even include its own RF front end (e.g., modulators, encryptors, etc.) or optical interface so as to interface directly with various portions of the HFC network 101. Numerous other configurations may be used. The server device 601 may also be integrated with other types of components (such as satellite transceivers, encoders/decoders, etc.) and form factors if desired.

(120) It can also be appreciated that the methods of the present invention may be practiced using any configuration or combination of hardware, firmware, or software, and may be disposed within one or any number of different physical or logical entities. For example, the data ingestion, packaging and delivery functionality described above may take the form of one or more computer programs running on a single device disposed within the network (e.g., the VOD server module 105), such as at a head-end, node, or hub. Alternatively, such computer programs may have one or more components distributed across various hardware environments at the same or different locations, such as the architecture shown in FIG. 2a, wherein various of the functions are distributed across the VOD servers 105, application servers 104 and Business Management System (BMS).

(121) As yet another example, portions of the functionality may be rendered as a dedicated or application specific IC having code running thereon. Myriad different configurations for practicing the invention will be recognized by those of ordinary skill in the network arts provided the present disclosure.

(122) CPE Architecture and Operation

(123) FIG. 7 illustrates a first embodiment of the improved client device (e.g., CPE 106) with data download capability according to the present invention. As shown in FIG. 7, the device 106 generally comprises and OpenCable-compliant embedded system having an RF front end 702 (including demodulator and decryption unit) for interface with the HFC network 101 of FIG. 1, digital processor(s) 704, RAM 705 and mass storage device 706, and a plurality of interfaces 708 (e.g., video/audio interfaces, IEEE-1394 Firewire, USB, serial/parallel ports, etc.) for interface with other end-user apparatus such as televisions, personal electronics, computers, WiFi/PAN or other network hubs/routers, etc. Other components which may be utilized within the device (deleted from FIG. 7 for simplicity) include RF tuner stages, buffer memory (which may be implemented in the RAM 705 or otherwise), various processing layers (e.g., DOCSIS MAC or DAVIC OOB channel, MPEG, etc.) as well as media processors and other specialized SoC or ASIC devices. These additional components and functionality are well known to those of ordinary skill in the cable and embedded system fields, and accordingly not described further herein.

(124) The device 106 of FIG. 7 is also provided with an OCAP-compliant monitor application and Java-based middleware which, inter alia, manages the operation of the device and applications running thereon. It will be recognized by those of ordinary skill that myriad different device and software architectures may be used consistent with the display element manager of the invention, the device of FIG. 7 being merely exemplary. For example, different middlewares (e.g., MHP, MHEG, or ACAP) may be used in place of the OCAP middleware of the illustrated embodiment.

(125) As described in greater detail subsequently herein, the processor 704 and internal bus and memory architecture of the CPE 106 of FIG. 7 is ideally adapted for high-speed data processing, at least sufficient to support the client-side processing tasks (see FIG. 7b) necessary to implement the high-speed data download functionality of the present invention effectively in real time. This may be accomplished, e.g., through a single high-speed multifunction digital processor, an array of smaller (e.g., RISC) cores, dedicated processors (such as a dedicated MPEG media processor, CPU, and interface controller), etc.

(126) FIG. 7a illustrates an exemplary configuration of the protocol stack 730 used on the CPE 106 of FIG. 7. Elements of this embodiment of the stack 730 include: (i) MPEG transport interface (demultiplexer) 732, (ii) 00B network interface 734, (iii) MPEG device driver 736, (iv) operating system (including aforementioned middleware) 737; (v) MPEG private data section access module 738, (vi) LSCP protocol module 740, and (vii) data download flow control module 742. As indicated in FIG. 7a, the local (or even remote) storage device 706, or alternatively RAM 705, is used to store data received by the CPE and extracted from the encoded packets as described in greater detail below. This data, which may comprise files (such as executables, compressed data files, etc.) or other data structures is utilized by the application layer 744 of the stack 730, or alternatively transmitted off-device for use by another processing entity such as a peripheral or client device to the CPE host.

(127) As part of the application layer 744 of the CPE 106, various different types of client applications 294 may be running (or operable to run) consistent with the present invention. In one embodiment, a separate (dedicated) client application adapted for high-speed data download may be used to interface with the lower layers of the stack 730 (including the data download flow control module 742). This may include, e.g., a separate GUI or other type of UI, and may operate substantially independent of other applications on the CPE 106. Alternatively, the data download functionality described herein may be integrated into one or more existing or downloadable applications (such as a VOD application, Watch TV application, navigator, or even EPG).

(128) As yet another option, the download functionality nay be completely transparent to the end user, such as where a gaming application running on the CPE 106 (or an associated device) makes data download calls as necessary to the other components of the stack in order to (i) initiate a session if not already established, (ii) download the data, including any necessary error correction and/or retransmission, and (iii) manage termination of the session; e.g., collapsing it if no further downloads are anticipated, or alternatively keeping it open while the parent (gaming) application is active. The CPE middleware and any other relevant components may also be modified in order to provide a universal software interface for the data download function, such that application developers can write their applications to make use of this capability. Similarly, the universal CPE described in co-owned U.S. patent application Ser. No. 10/782,680 filed Feb. 18, 2004, entitled MEDIA EXTENSION APPARATUS AND METHODS FOR USE IN AN INFORMATION NETWORK, and issued as U.S. Pat. No. 8,078,669 on Dec. 13, 2011, incorporated herein by reference in its entirety, may be used consistent with the present invention in order to allow specific features (including data download) to be configured by a particular MSO or other entity when the CPE is used in their network.

(129) In still another embodiment, the client application 294 can function in response to signals or communications provided by a device in communication with the CPE 106. For example, the CPE 106 may comprise a wireless interface (e.g., 802.11 a/g, Bluetooth, 802.15 PAN, 802.16 WiMAX, etc.) such that it can service data download requests from client devices of the CPE 106 itself. In one such variant, the client device comprises a PDA or similar handheld device which has a distributed portion of the client application 294 running thereon. This application may be stand-alone or integrated with another application, such as a gaming application. Hence, users operating the e.g., gaming application on the PDAs will utilize their wireless interface to the CPE 106 in order to remotely instigate a data download from the network via the CPE. The wireless forward channel(s) of the interface (e.g., CPE to PDA) can be used to transmit the downloaded data file after reassembly by the CPE, or even stream the raw unassembled data (or even the received and demultiplexed MPEG encoded packets) to the PDA(s) for use thereby. This approach advantageously allows users within, say, a home to pick up their respective PDAs or gaming devices, initiate the gaming application via these devices, and download any necessary games, data files, etc. without having to directly interact with the CPE 106 via its navigator or other application. The users need merely stay within sufficient proximity of the CPE 106 during the data acquisition process.

(130) Myriad other schemes for integrating the data download functions within the existing CPE software environment will be recognized by those of ordinary skill in the software arts when provided the present disclosure.

(131) Referring now to FIG. 7b, one exemplary embodiment of the client side data download process flow is described in detail. As previously discussed, a first step in performing the data download comprises the establishment of a session via which the data may be transferred (step 762). In the exemplary embodiment, the CPE 106 initiates an on-demand (OD) session for data download for a given asset ID. In cable networks compliant with the ISA specification, the session setup process is based on the well-known Session Setup Protocol (SSP), such as SSP Version 2.1. An asset ID (or offering ID) is used to identify the packetized data stream; this ID is accessed by the CPE 106 using, e.g., the OD catalog structures associated with an exemplary system. Alternatively, some other structure or mechanism adapted to transfer this information to the CPE 106 can be utilized.

(132) Upon successful setup of the session, the OD server (e.g., VOD server 105 of FIG. 1a) sends back information necessary for the CPE 106 to tune to the appropriate QAM (and MPEG program number that contains the data PID) used to transport the data (step 764). Since the CPE requires the exact MPEG data PID in order to access the MPEG private section, it must parse the Program Map Table (PMT) to determine the data PID value associated with the MPEG program to which it is tuning (step 766).

(133) After successful tuning to the physical channel(s) carrying the data payload, the CPE 106 must then (i) start the data stream, (ii) extract the data from the MPEG private data section, and (iii) reliably re-assemble the data to its original payload configuration (e.g., file or other data structure).

(134) In the exemplary implementation, the client application 294 running on the CPE 106 starts the data stream by issuing the LCSP Play command to play from the zero (0) location. It will be appreciated that an MPEG private section control approach is utilized in the illustrated embodiments, other approaches may be substituted with equal success such as, without limitation, a Digital Storage MediaCommand and Control (DSM-CC) based stream control in accordance with ISO/IEC 13818-6:1998(E).

(135) As the data stream begins flowing to the CPE, the client application 294 extracts the data from all packets having the appropriate PID using its MPEG demultiplexing hardware or a comparable process (step 768). Once the entire MPEG private_section carrying the data is available, the client application 294 computes the CRC on the payload to assure the integrity of the data (step 770), and verify that there are no transmission errors therein. If the CRC fails, the failing section(s) can be discarded (or otherwise segregated for later use or analysis).

(136) Once the CRC validates the data, the client application 294 evaluates whether the data received is in the proper sequence by, e.g., looking at the section numbers that are part of the packet headers in the payload (step 772). The exemplary scheme used comprises having the section numbers start from zero and increment in ascending order. By keeping track of the section number of the last valid payload that was received properly, the client application 294 can detect if the sequence number from the next section being processed is in proper sequence or not. If the section number falls within the expected sequence, the data from the payload is appended to the data previously received for re-assembly.

(137) If the CRC fails, or a given section received is out of order, then the client application sends an LSCP message to the server to reset the position in the stream to the NPT location that corresponds to the next valid section that was expected by the client (step 774). By sending the LSCP message to reset to a previous location in the stream, the server will re-transmit those sections that were lost or damaged in transmission. During re-transmission, the client application continues the re-assembly process when it receives the next expected data section, and the data stream flow is back in the proper order.

(138) It will be appreciated that a buffering scheme may also be applied to the evaluation of section numbers described above. For example, instead of applying a strict in-sequence or out-of-sequence criterion, a buffer mechanism can be used to allow a certain degree of variation in the sequence without invoking a retransmission. Specifically, in one variant, a simple plus or minus criterion is applied, wherein the section number of a received section will not trigger a retransmission if the sequence number falls within a predetermined range of the expected number. The application will then maintain the same range (and starting point) and await the next section.

(139) Alternatively, if the received sequence number exceeds the specified range, it will trigger a retransmission e.g., at the start or lowest sequence number of the range. In this fashion, the occasional out-of-sequence section will not cause a retransmission. Rather, the buffer maintains a small pool of sections whose order can be permuted to properly re-assemble (resequence) the sections.

(140) The payload headers in the exemplary packetization scheme also include the last section number in the sequence, and hence the client application can detect when it has received all of the data by matching the last section number received to the last section number in the header.

(141) Upon detecting of end of the sequence (step 780), the client application 294 terminates the on-demand session by sending a session release request or similar communication (per step 782), and passes back the fully assembled payload to the requesting entity (which may include the client application 294 itself, or another application as previously discussed).

(142) In one embodiment of the invention, the transmission rate for the data transfer stream is intended to be very high as compared to conventional data downloads to the CPE 106. For example, a data rate of 3.75 Mbps (corresponding to one VOD program) can be used. A typical implementation that uses MPEG private sections of length 4096 bytes will have a nominal rate of approximately 115 packets per second. This translates to (on average) about 8.7 milliseconds available to process each private section. Hence, fairly tight timing within the system must be observed in order to attain this high rate of data transfer to the CPE. As the data rate is increased, the timing constraints accordingly increase as well.

(143) Hence, the implementation of the client device 106 (and client application 294 running thereon) should be highly optimized; e.g., to consume as few CPU cycles as possible in order to process the data and pass it to the requesting application within these time constraints. The implementation should also transfer the data from the MPEG transport de-multiplexer to the application with as few data copies as possible.

(144) As one alternative, the data may be buffered to allow a slower CPE to still download at a higher rate. Since the data downloads are finite in size, the buffers may be sized accordingly. This approach allows legacy (slower) CPE to download at the faster rate, and subsequently collapse the OD session (thereby conserving network bandwidth), in exchange for longer processing time of the buffered data at the CPE. Optionally, a constraint may be imposed that any existing OD session cannot be terminated until at least the CRC/sequencing is performed on the buffered data, in case a retransmission was required. Myriad alternative schemes for managing the data download/CPE processing rates will be recognized by those of ordinary skill provided the present disclosure.

(145) In one exemplary embodiment, upon receipt of an invalid section within the stream, the CPE 106 utilizes the LSCP (or a similar mechanism) to request that the VOD server 105 resume playback at a point prior the beginning of the invalid section. Calculating the proper NPT for this function can be accomplished in a number of different ways, such as by multiplying the requested section_number by the section_number multiplier which is contained in the table_id_extension field of the private_section header. Upon restart, the CPE 106 may receive duplicate packets of a previous section before it can resume reassembly of the data file. Accordingly, the client application 294 is configured to disregard or discard these packets until it receives the packet associated with the start of the section_number it has requested, so as to avoid assembling the duplicate packets into the payload. Other approaches for culling out duplicate packets may also be employed, such as use of a separate re-assembly/compression algorithm after all of the received packets (whether duplicate or not) have been aggregated by the client application 294.

(146) Wideband and Multi-QAM Variants

(147) While the foregoing embodiments of the present invention are described primarily in terms of an OD infrastructure adapted to transmit data over a single physical channel (e.g., 256-QAM modulated carrier) at any given time, it will be recognized that this physical channel may actually comprise one or more carriers. For example, in one multi-carrier variant of the invention, the non-content data is streamed over multiple physical carriers according to a multiplexing algorithm such as that described in co-owned and co-pending U.S. patent application Ser. No. 11/013,671 filed on Dec. 15, 2004, and entitled Method And Apparatus For Wideband Distribution Of Content, incorporated herein by reference in its entirety. Under this approach, the data of a given TS can be multiplexed across a plurality of physical carriers, with the multiplexed signal being reassembled at the CPE 106 using a wideband tuner (or a plurality of related tuners). Information from the head-end as to the multiplexing scheme and channels used is provided to the CPE in order to enable it to de-multiplex (and decode) the multiplexed transport stream. Hence, for the purposes of the present invention, the aggregation of multiplexed channels acts like a single QAM.

(148) As yet another option, two or more QAMs within the network can be established simultaneously (as part of a single session, or alternatively two distinct but related sessions) to achieve one or more desired objectives, such as an increased download speed or statistical multiplex pool size. For example, where the download speed using a single on-demand session and QAM might be limited to a given value, that value can be increased through use of (i) two or more QAMs, and (ii) use of CPE which has the ability to simultaneously tune to the two or more QAMs and download data (e.g., MPEG packets with the data encapsulated therein). Depending on the extant processing capability within the CPE, this may require varying levels of modification to the CPE including e.g., the addition of a second or wideband tuner/demodulator stage, additional packet processing capability (such as an additional MPEG media processor), more RAM, etc. A common client application 294 can be used to perform data extraction, CRC, etc. as previously described herein for the multiple streams, and then recombine them into one unitary data structure (e.g., file).

(149) Stagger-Cast Variants

(150) It will be recognized that the apparatus and methods of the present invention can also be used to afford other benefits, including increased robustness and even a near data-on-demand (NDOD) capability. Specifically, in one embodiment, requested (or otherwise transmitted) data is stagger-cast for a period of time such that time-shifted copies of a given data file or the like are transmitted over one or more QAMs allocated to the session request issued by a given CPE. The term stagger-cast as used in the present context refers to a process wherein identical copies of the same data, with their start times staggered by some duration, are multiplexed with each other to form a transport stream.

(151) When a user tunes to the transport stream, the user can start downloading the data from the beginning as soon as the start of a next staggered copy of the data is received. This results in an OD-like functionality for the same data without having to establish a new session (and allocate a new dedicated channel). Hence, whereas one prior embodiment of the invention described herein would instantiate a new session, channel and transport stream based on each separate request received by the server (or other servicing entity) from different CPE within the network, the stagger-cast approach allows the second and subsequent users to be directed to an already existing QAM (or set of QAMs) to access one of the stagger-cast copies of the data file(s).

(152) The level of latency (i.e., how near the NDOD really is to true DOD for these second and subsequent users) can be set as granular as desired, this parameter being determined by the metrics of the time delay and multiplexing process.

(153) Similarly, the total duration of the staggered data transmissions (i.e., from the beginning of the first copy to the end of the last copy) can be controlled as desired so as not to monopolize too much bandwidth; e.g., to permit the established session to be torn down without too much delay. A window can be established (e.g., from the beginning of the first copy to the beginning of the last copy within the TS) such that any second and subsequent users requesting the download can either be serviced by the first or subsequent copy of the data if inside the window, or alternatively instantiate a new session (and QAM(s)) if they are outside the window.

(154) Hence, in one variant, the NDOD or stagger-cast data service provided to a user within a given local service area or node can be instigated based on a download (and session) request from that user. When the first user's session is established, and one or more QAMs allocated as previously described herein, the TS broadcast onto the allocated QAM(s) can comprise a plurality of stagger-cast copies of the requested data. As other users within the same service area/node request the same data download thereafter, they can access the beginning of the transmitted data file by waiting for one or more stagger or latency periods.

(155) Situations where this stagger-cast functionality may be useful include, e.g., where bandwidth available within a given local service node is extremely limited (i.e., the MSO does not want to penalize or rob bandwidth from potential VOD or other premium service users to service multiple data download requests, and the effective bandwidth utilized by the stagger-cast mode is less than that used by separate sessions). Another situation might comprise where multiple users within a local service area are expected to download the same data in a substantially simultaneous fashion, such as where multiple people in the same area will be playing a given (multi-user) game requiring the same files, or requiring an application to perform some sort of function in relation to a program broadcast into that specific area, or the like.

(156) Conversely, the MSO can constrain users to accessing the high-speed data download capability of the invention to only certain windows of time (for given files), these windows corresponding to the aforementioned windows for staggered access. That way, user data demands for the same data are funneled to a window where the minimum of bandwidth (or at least the minimum number of different QAMs and sessions) are required to service the requests.

(157) Similarly, the pause functionality previously described herein can be effected for at least a period of time by simply accessing the next successive packet in a latent copy of the file, such packet being a counterpart to the next successive packet in the file copy after that from where the pause was initiated. The user's CPE 106 (including any applications running thereon accessing the data download function) can be programmed if desired to determine the maximum allowable latency for the pause, and terminate the pause no later than this time in order to guarantee that all packets of the requested file will at least be received.

(158) The aforementioned staggered approach may also be used to increase download robustness, and obviate the retransmission processes previously described. For example, in one variant, the user's CPE 106 receives the data (e.g. in the form of the MPEG packets) and evaluates them for integrity, such as via a CRC. If the packets are corrupted, instead of issuing an upstream retransmission request, the CPE 106 can simply access the required packets within the next copy of the file that is present within the TS. The CRC or other evaluation process can be completed in a sufficiently short time so as to permit the CPE to grab one or more subsequent copies of the corrupted packets. Various TS multiplexing schemes well known in the art (e.g., those used within conventional stagger-cast applications) can be used to provide the desired data properties.

(159) The provision of staggered file transmissions within a given session can also be driven by other considerations such as, e.g., available bandwidth at the relevant local node, or the network as a whole. For example, an MSO might only allow stagger-casting of data where (i) there is a sufficient expectation of multiple different users downloading the same file at roughly the same time, and/or (ii) where remaining bandwidth so dictates. Hence, the decision to implement stagger-casting of data (or for that matter any other functionality described herein) can be driven by a higher level process such as a rules engine design to impose business-related and/or operational rules on the system. One exemplary rules engine compatible with the present invention is described in detail (in the context of an SD/HD bandwidth allocation system) in co-owned U.S. patent application Ser. No. 10/881,979 filed Jun. 29, 2004, entitled METHOD AND APPARATUS FOR NETWORK BANDWIDTH ALLOCATION, and issued as U.S. Pat. No. 8,843,978, on Sep. 23, 2014, incorporated herein by reference in its entirety.

(160) It will be recognized that while certain aspects of the invention are described in terms of a specific sequence of steps of a method, these descriptions are only illustrative of the broader methods of the invention, and may be modified as required by the particular application. Certain steps may be rendered unnecessary or optional under certain circumstances. Additionally, certain steps or functionality may be added to the disclosed embodiments, or the order of performance of two or more steps permuted. All such variations are considered to be encompassed within the invention disclosed and claimed herein.

(161) While the above detailed description has shown, described, and pointed out novel features of the invention as applied to various embodiments, it will be understood that various omissions, substitutions, and changes in the form and details of the device or process illustrated may be made by those skilled in the art without departing from the invention. The foregoing description is of the best mode presently contemplated of carrying out the invention. This description is in no way meant to be limiting, but rather should be taken as illustrative of the general principles of the invention. The scope of the invention should be determined with reference to the claims.

(162) TABLE-US-00001 APPENDIX I Copyright 2004 Time Warner Cable, Inc. All rights reserved. # of Field Name Bits Value Notes Private_section( ) { table_id 8 0x40 User private section_syntax_indicator 1 0b A value of 0b indicates the private_data_bytes immediately follow the private_section_length field. private_indicator 1 0b User definable reserved 2 11b private_section_length 12 Specifies the number of remaining bytes in the private section immediately following the private_section_length field. The value in this field shall not exceed 4093 (0XFFD). if (section_syntax_indicator) ==0) { for (i=0; i<N; i++) { table_id_extension 16 The table_id_extension contains a multiplier value to be used in error recovery. By using this multiplier value along with a requested section_number, the client will be able to specific calculate an NPT for retransmission of a stream at a section_number. reserved 2 11b version_number 5 00000b current_next_indicator 1 1b This field is set to 1b indicating the information in the private_section is always current. section_number 32 This value starts at 0 with the first section and shall be incremented by 1 with each additional section in this private table. This field has been modified from the generic section syntax to 32 bits in order to handle larger data file sizes. last_section_number 32 The number of the last section of the private table of which this section is a part. This field has been modified from the generic section syntax to 32 bits in order to handle larger data file sizes. (maximum data file size of 4 GB) for (i=0; i<private_section_length- 11; i++) { private_data_byte 8 The data file will be located at this field in 4078 byte segments. } CRC_32 32 32 bit CRC field } } } End of private_section

(163) TABLE-US-00002 APPENDIX II Copyright 2004 Time Warner Cable, Inc. All rights reserved Link Type Meaning How Used 0x00 Reserved Reserved 0x01 Service Tune Link Type Main Menu catalogs, OnDemand Service catalogs, Network Express catalogs 0x02 OnDemand Selection Main Menu catalogs, Catalog Link Type OnDemand Service catalogs, Network Express catalogs 0x03 OnDemand Menu3 Catalog Main Menu catalogs, Link Type OnDemand Service catalogs, Network Express catalogs 0x04 Index Catalog Link Type Alpha & Theme Menu catalogs 0x05 Index2 Catalog Link Type Alpha & Theme Menu3 catalogs 0x06 Genre Collection Catalog Main Menu catalogs Link Type (client-side synthetic) 0x07 Purchase List Catalog OnDemand Service catalogs Link Type (client-side synthetic) 0x08 Make Favorite Link Type Network Express catalogs (client-side synthetic) 0x09 Preview Selection Catalog Main Menu catalogs Link Type 0x0A Service Launch Link Type Network Express catalogs 0x0B Exit Menu Link Type Network Express catalogs 0x0C Network Express Link Type Network Express catalogs (client-side synthetic) 0x0D Record Link Type Network Express catalogs (client-side synthetic) 0x0E NDVR Lookback Link Type Network Express catalogs (client-side synthetic) (Compass only) 0x0E-0x0F Reserved Reserved

(164) TABLE-US-00003 APPENDIX III Copyright 2004 Time Warner Cable, Inc. All rights reserved OnDemand Selection Catalog Offering Types OfferingType Value Meaning 0x00 Reserved 0x01 Free. No charge for this offering type. 0x02 Per Use. A charge is applied each time this offering is used. 0x03 Subscription. A monthly charge is applied for unlimited offering usage. 0x04-0xFF Reserved

(165) TABLE-US-00004 APPENDIX IV Copyright 2004 Time Warner Cable, Inc. All rights reserved OnDemand Selection Catalog Selection Types Value Meaning 0x00 Reserved 0x01 OnDemand VOD/SVOD Selection. This is the typical OnDemand VOD/SVOD selection type. It uses rec_AssetNameRef to establish the session. Example: the client may allow the user to buy or preview trailer selections of this type. 0x02 Service Selection. This selection refers to a service selection. If selected, the service specified by rec_ServiceID is invoked. Example: the client may tune to a service specified by this selection. This could be a network boadcast service or perhaps a barker channel showing movie previews. 0x03 AssetName Auto-play Selection. When selected, this selection type is used to automatically play an asset, specified by rec_AssetNameRef. Example: the client may automatically play this selection without providing the user with choices. 0x04 AssetID Auto-play Selection. When selected, this selection type is used to automatically play an asset, specified by rec_AssetID. Example: the client may allow the user to play or reserve selections of this type. 0x05 Help Text Selection. When selected, these selection types display text instead of establishing a session. Example: the client may simply display the rec_DescriptionRef when selected. 0x06 AssetID Reserve Selection. This type is use to refer to a selection that represents a reservation of an NPVR show, specified by rec_AssetID. The client may allow the user to play or remove reserve selections of this type. 0x07 AssetID Season Pass Reserve Selection. This type is use to refer to a selection that represents a season pass reservation of an NPVR show, specified by rec_AssetID. The client may allow the user to play or remove reserve or remove season pass selections of this type. 0x08 AssetID Mystro Suggested Reserve Selection. This type is use to refer to a selection that represents a Mystro-suggested reservation of an NPVR show, specified by rec_AssetID. The client may allow the user to play or remove reserve or tell Mystro good suggestion or tell Mystro bad suggestion selections of this type. 0x09 AssetName OnDemand Resume Selection. This selection entry is used to denote an OnDemand purchase that is within the viewing window. 0x0A Season Pass Root Item. This is used in the Reserve Shows catalog to denote a higher level season pass entry. This entry does not point to a specific asset ID or asset name, and instead is use to consolidate settings related to a viewer's season pass. 0x0B OfferingID Auto-Play Selection When selected, this selection type is used to automatically play an asset, specified by rec_OfferingID. The client may automatically play this selection without providing the user with choices. 0x0C OfferingID OnDemand VOD/SVOD Selection. This is the typical OnDemand VOD/SVOD selection type. It uses rec_OfferingID to establish the session. If the Event flags denote that a Preview is available then the rec_OfferingID may be used with the preview Boolean set to true. 0x0D OfferingID OnDemand Resume Selection. This selection entry is used to denote an OnDemand purchase that is within the viewing window. 0x0E Data Download Asset Type This title refers to an applications ability to download a data file using the session protocol and the Data Encoding Spec. Unless the application knows how to download a data file using an InBand session, this type of title can be ignored. 0x0F- Reserved 0xFF

(166) TABLE-US-00005 APPENDIX V Copyright 2004 Time Warner Cable, Inc. All rights reserved Metadata Required vs Spec Name Description Type Optional AMS Provider A unique identifier for the provider String - ex. Req of the Asset Zodiac AMS Product An identifier for the product String (Max 20 Req offering Chars) AMS Asset_Name A string containing the identifying String (Max 50 Req name of the asset. Asset names Chars) must be unique within a product. AMS Version_Major An integer representing the major Integer Req version number. AMS Version_Minor An integer representing the minor Integer Req version number. AMS Description A human-readable string describing String Req the Asset. AMS Creation_Date A string representing the date on String - Req which the Asset was created. yyyy-mm-dd AMS Provider_ID A unique identifier for the provider String - Ex. Req of the asset. The Provider_ID must indemand.com be set to a registered internet domain name restricted to at most 20 characters. For example, CableLabs is cablelabs- films.com (19 chars). AMS Asset_Class A system-level type for the asset. String Req This is intended to be helpful for the application mapping and routing, and is expected to be more general than the Type value for the content. Expected value is data. AMS Verb A string containing an action to be String Opt performed on the asset. The only valid values for the Verb are the empty string (,) and DELETE. MOD or SVOD Rating MPAA or TV Rating. (See Annex String, one rating Req A, Ratings and Advisories) per element. MOD or SVOD MSORating MSO applied rating for content not String, one rating Opt otherwise rated. (See Annex A for per element. examples) MOD or SVOD Run_Time Run time for this data download. String - Req This value is determined based on hh:mm:ss how long it would take to play the data MPEG file at the standard VOD rate of 3.75 Mbps. MOD or SVOD Type The asset type that determines how data_mpeg or Req and where it is stored in the system. data_raw Value is expected to be data_mpeg or data_raw. If the type is data_raw then the ingest device must convert the data file into an MPEG file complaint with the Data Encoding Spec. MOD or SVOD Content_FileSize File Size (in bytes) of the included Integer - unsigned Req content for Quality Assurance & 64-bit value processing. MOD or SVOD Content_CheckSum A string containing a hex number String - Hex (32 Req representing a MD5 (RFC 1321) Chars) message digest value for Quality Assurance. MOD or SVOD Encryption A Y or N flag indicating String - Y or N Opt whether or not encryption is required MOD or SVOD Copy_Protection A Y or N flag indicating String - Y or N Opt whether or not copy protection is required. If Y then Encryption must be used. MOD or SVOD Analog_Protection_System An integer representing the value Integer - values 0-3 Optional of APS. Required if 0 - Analog protection off Copy_Protection 1 - AGC process on, split burst off is asserted 2 - AGC process on, 2-line split burst on 3 - AGC process on, 4-line split burst on MOD or SVOD Encryption_Mode_Indicator An integer representing the value Integer - values 0-3 Optional of EMI. Required if 0 - Copying is permitted Copy_Protection 1 - No further copying is permitted is asserted 2 - One generation copy is permitted 3 - Copying is prohibited MOD or SVOD Constrained_Image_Trigger An integer representing the value Integer - value 0 or Optional of CIT. 1 Required if 0 - No image constraint asserted Copy_Protection 1 - Image constraint required is asserted MOD or SVOD CGMS_A An integer representing the value Integer - values 0-3 Optional of the Copy Generation Required if Management System (Analog). Copy_Protection 0 - Copying is permitted without is asserted restriction 1 - No further copying permitted 2 - One generation of copies may be made 3 - No copying is permitted