SINGLE SERVICE RESOURCE CONFIGURATION METHOD AND APPARATUS, COMPUTER DEVICE AND MEDIUM

Abstract

The present disclosure provides a service resource configuration method, including: configuring resource parameters for a service to be configured according to an action policy, calculating a timely reward in a current state, performing IV analysis according to the action policy, and ending one episode after the IV analysis is completed; calculating and updating, according to the timely reward in each state, an optimization objective policy parameter in each state; iterating a preset number of episodes to calculate and update the optimization objective policy parameter in each state; determining, according to the optimization objective policy parameter in each state in the preset number of episodes, an optimal optimization objective policy parameter in each state; and updating the action policy according to the optimal optimization objective policy parameter in each state. The present disclosure further provides a single service resource configuration apparatus, a computer device and a computer-readable medium.

Claims

1. A single service resource configuration method, comprising: configuring resource parameters for a service to be configured according to an action policy, calculating a timely reward in a current state, performing, after all the resource parameters are configured, impairment verification analysis according to the action policy, and ending one episode after the impairment verification analysis is completed, wherein a next state is entered after an action in the action policy is completed, and the action comprises an action of configuring a resource parameter or an action of performing impairment verification analysis; calculating and updating, according to the timely reward in each state, an optimization objective policy parameter in each state; iterating a preset number of episodes to calculate and update the optimization objective policy parameter in each state; determining, according to the optimization objective policy parameter in each state in the preset number of episodes, an optimal optimization objective policy parameter in each state; and updating the action policy according to the optimal optimization objective policy parameter in each state.

2. The method according to claim 1, wherein the resource parameters comprise a route, a wavelength, a spectrum, and a modulation format, and the resource parameters are configured for the service to be configured within one episode in a sequence of: configuring the route, configuring the wavelength, configuring the spectrum, and configuring the modulation format.

3. The method according to claim 2, wherein the state comprises: a route to be configured state, a wavelength to be configured state, a spectrum to be configured state, a modulation format to be configured state, an impairment verification analysis to be performed state, and an end state, in the route to be configured state, the timely reward R.sub.0=0, and in other states, the timely rewards meet any one or combination of the following conditions: in the wavelength to be configured state, a timely reward R.sub.1 is a function of a working route cost, and R.sub.1 and the working route cost is monotonically decreasing; in the spectrum to be configured state, a timely reward R.sub.2 is a function of a wavelength resource utilization rate, and R.sub.2 and the wavelength resource utilization rate is monotonically increasing; in the modulation format to be configured state, a timely reward R.sub.3 is a function of service occupied bandwidth, and R.sub.3 and the service occupied bandwidth is monotonically decreasing; in the impairment verification analysis to be performed state, a timely reward R.sub.4 is a function of a service spectral efficiency, and R.sub.4 and the service spectral efficiency is monotonically increasing; or in the end state, a timely reward R.sub.5 is related to a result of the impairment verification analysis, wherein R.sub.5 is positive when the result of the impairment verification analysis is qualified, and R.sub.5 is negative when the result of the impairment verification analysis is unqualified.

4. The method according to claim 2, wherein the action policy comprises a random action policy and a deterministic action policy, and configuring resource parameters for the service to be configured according to the action policy comprises: configuring a route, a wavelength, a frequency spectrum and a modulation format for the service to be configured according to the random action policy, and performing impairment verification analysis according to the action policy comprises: performing impairment verification analysis according to the deterministic action policy.

5. The method according to claim 2, wherein in a case where the route of the service to be configured comprises a plurality of hops, configuring resource parameters for the service to be configured according to the action policy comprises: configuring resource parameters for the service to be configured according to the action policy in each hop, performing impairment verification analysis according to the action policy comprises: performing impairment verification analysis according to the action policy in each hop, and performing impairment verification analysis comprises: calculating a pre-calculated value of an optical signal noise ratio of each hop in the route of the service to be configured; determining, in response to the pre-calculated value of the optical signal noise ratio of each hop meeting a preset condition, that a result of the impairment verification analysis is qualified; and determining, in response to the pre-calculated value of the optical signal noise ratio of at least one hop not meeting the preset condition, that the result of the impairment verification analysis is unqualified.

6. The method according to claim 1, wherein performing impairment verification analysis comprises: calculating a pre-calculated value of an optical signal noise ratio of the service to be configured; determining, in response to the pre-calculated value of the optical signal noise ratio meeting a preset condition, that a result of the impairment verification analysis is qualified; and determining, in response to the pre-calculated value of the optical signal noise ratio not meeting the preset condition, that the result of the impairment verification analysis is unqualified.

7. The method according to claim 5, wherein a case where pre-calculated value of the optical signal noise ratio meets the preset condition comprises: b) OSNR.sub.pre-calculated valueOSNR.sub.flatnessOSNR.sub.transmission threshold, c) OSNR.sub.transmission threshold=OSNR.sub.B2B+OSNR.sub.nonlinear+OSNR.sub.CD+OSNR.sub.PMD+OSNR.sub.filter+OSNR.sub.PDL+OSNR.sub.fluctuation+OSNR.sub.net allowance, d) wherein OSNR.sub.pre-calculated value is the pre-calculated value of the optical signal noise ratio, OSNR.sub.flatness is a flatness of the optical signal noise ratio, OSNR.sub.B2B is a back-to-back optical signal noise ratio, OSNR.sub.nonlinear is a nonlinear cost of the optical signal noise ratio, OSNR.sub.CD is a chromatic dispersion cost of the optical signal noise ratio, OSNR.sub.PMD is a polarization mode dispersion cost of the optical signal noise ratio, OSNR.sub.filter is a filter membrane cost of the optical signal noise ratio, OSNR.sub.PDL is a polarization dependent loss cost of the optical signal noise ratio, OSNR.sub.fluctuation is a fluctuation of the optical signal noise ratio, OSNR.sub.net allowance is a net allowance required by the optical signal noise ratio, and OSNR.sub.flatness, OSNR.sub.B2B, OSNR.sub.nonlinear, OSNR.sub.CD, OSNR.sub.PMD, OSNR.sub.filter OSNR.sub.PDL, OSNR.sub.fluctuation and OSNR.sub.net allowance are preset values.

8. The method according to claim 1, wherein calculating and updating, according to the timely reward in each state, the optimization objective policy parameter in each state comprises: calculating, according to timely rewards in various states after a next state, an expected return in a current state; and calculating and updating, according to the expected return in the current state, the optimization objective policy parameter in the current state.

9. The method according to claim 8, wherein the expected return in the current state is calculated by: $G_{t} = {.Math.}_{k = 0}^{n - (t + 1)}^{k} R_{t + k + 1},$ wherein G.sub.t is an expected return of executing action .sub.t in state S.sub.t, is a discount coefficient, and 0<<1; R is the timely reward, t is the number of resource parameters already configured in state S.sub.t, t=(0, . . . , n1), and n1 is the total number of resource parameters.

10. The method according to claim 8, wherein the optimization objective policy parameter comprises a state behavior value $Q_{} (s, a) = E_{} [{.Math.}_{k = 0}^{n - (t + 1)}^{k} R_{t + k + 1} .Math. s = s_{t}, a = a_{t}],$ or the optimization objective policy parameter comprises a state value $V_{} (s), V_{} (s) = \underset{a A}{.Math.} (a .Math. s) Q_{} (s, a),$ wherein (|s) is a probability of taking action according to an action policy (s, ) in state S, and A is a set of actions executed in each state.

11. The method according to claim 10, wherein when the optimization objective policy parameter is the state behavior value Q.sub.(s,), the optimization objective policy parameter in each state is calculated and updated with a Monte Carlo Process algorithm, a TD-Error algorithm of different policies or a TD-Error algorithm of a same policy, and updating the action policy according to the optimal optimization objective policy parameter in each state comprises: updating the action policy according to the state behavior value Q.sub.(s,).

12. The method according to claim 10, wherein when the optimization objective policy parameter is the state value V.sub.(s), the optimization objective policy parameter is calculated with a dynamic programming algorithm; and updating the action policy according to the optimal optimization objective policy parameter in each state comprises: updating the action policy according to the state value V.sub.(s).

13. (canceled)

14. A computer device, comprising: one or more processors; and a storage means having one or more programs stored thereon, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement a single service resource configuration method comprising: configuring resource parameters for a service to be configured according to an action policy, calculating a timely reward in a current state, performing, after all the resource parameters are configured, impairment verification analysis according to the action policy, and ending one episode after the impairment verification analysis is completed, wherein a next state is entered after an action in the action policy is completed, and the action comprises an action of configuring a resource parameter or an action of performing impairment verification analysis; calculating and updating, according to the timely reward in each state, an optimization objective policy parameter in each state; iterating a preset number of episodes to calculate and update the optimization objective policy parameter in each state; determining, according to the optimization objective policy parameter in each state in the preset number of episodes, an optimal optimization objective policy parameter in each state; and updating the action policy according to the optimal optimization objective policy parameter in each state.

15. A non-transitory computer-readable medium storing a computer program thereon, wherein the program, when executed by a processor, causes the processor to implement a single service resource configuration method comprising: configuring resource parameters for a service to be configured according to an action policy, calculating a timely reward in a current state, performing, after all the resource parameters are configured, impairment verification analysis according to the action policy, and ending one episode after the impairment verification analysis is completed, wherein a next state is entered after an action in the action policy is completed, and the action comprises an action of configuring a resource parameter or an action of performing impairment verification analysis; calculating and updating, according to the timely reward in each state, an optimization objective policy parameter in each state; iterating a preset number of episodes to calculate and update the optimization objective policy parameter in each state; determining, according to the optimization objective policy parameter in each state in the preset number of episodes, an optimal optimization objective policy parameter in each state; and updating the action policy according to the optimal optimization objective policy parameter in each state.

16. The method according to claim 6, wherein a case where pre-calculated value of the optical signal noise ratio meets the preset condition comprises: e) OSNR.sub.pre-calculated valueOSNR.sub.flatnessOSNR.sub.transmission threshold, f) OSNR.sub.transmission threshold=OSNR.sub.B2B+OSNR.sub.nonlinear+OSNR.sub.CD+OSNR.sub.PMD+OSNR.sub.filter+OSNR.sub.PDL+OSNR.sub.fluctuation+OSNR.sub.net allowance, g) wherein OSNR.sub.pre-calculated value is the pre-calculated value of the optical signal noise ratio, OSNR.sub.flatness is a flatness of the optical signal noise ratio, OSNR.sub.B2B is a back-to-back optical signal noise ratio, OSNR.sub.nonlinear is a nonlinear cost of the optical signal noise ratio, OSNR.sub.CD is a chromatic dispersion cost of the optical signal noise ratio, OSNR.sub.PMD is a polarization mode dispersion cost of the optical signal noise ratio, OSNR.sub.filter is a filter membrane cost of the optical signal noise ratio, OSNR.sub.PDL is a polarization dependent loss cost of the optical signal noise ratio, OSNR.sub.fluctuation is a fluctuation of the optical signal noise ratio, OSNR.sub.net allowance is a net allowance required by the optical signal noise ratio, and OSNR.sub.flatness, OSNR.sub.B2B, OSNR.sub.nonlinear, OSNR.sub.CD, OSNR.sub.PMD, OSNR.sub.filter, OSNR.sub.PDL, OSNR.sub.fluctuation and OSNR.sub.net allowance are preset values, h)

17. The computer device according to claim 14, wherein the resource parameters comprise a route, a wavelength, a spectrum, and a modulation format, and the resource parameters are configured for the service to be configured within one episode in a sequence of: configuring the route, configuring the wavelength, configuring the spectrum, and configuring the modulation format.

18. The computer device according to claim 17, wherein the state comprises: a route to be configured state, a wavelength to be configured state, a spectrum to be configured state, a modulation format to be configured state, an impairment verification analysis to be performed state, and an end state, in the route to be configured state, the timely reward R.sub.0=0, and in other states, the timely rewards meet any one or combination of the following conditions: in the wavelength to be configured state, a timely reward R.sub.1 is a function of a working route cost, and R.sub.1 and the working route cost is monotonically decreasing; in the spectrum to be configured state, a timely reward R.sub.2 is a function of a wavelength resource utilization rate, and R.sub.2 and the wavelength resource utilization rate is monotonically increasing; in the modulation format to be configured state, a timely reward R.sub.3 is a function of service occupied bandwidth, and R.sub.3 and the service occupied bandwidth is monotonically decreasing; in the impairment verification analysis to be performed state, a timely reward R.sub.4 is a function of a service spectral efficiency, and R.sub.4 and the service spectral efficiency is monotonically increasing; or in the end state, a timely reward R.sub.5 is related to a result of the impairment verification analysis, wherein R.sub.5 is positive when the result of the impairment verification analysis is qualified, and R.sub.5 is negative when the result of the impairment verification analysis is unqualified.

19. The computer device according to claim 17, wherein the action policy comprises a random action policy and a deterministic action policy, and configuring resource parameters for the service to be configured according to the action policy comprises: configuring a route, a wavelength, a frequency spectrum and a modulation format for the service to be configured according to the random action policy, and performing impairment verification analysis according to the action policy comprises: performing impairment verification analysis according to the deterministic action policy.

20. The non-transitory computer-readable medium according to claim 15, wherein the resource parameters comprise a route, a wavelength, a spectrum, and a modulation format, and the resource parameters are configured for the service to be configured within one episode in a sequence of: configuring the route, configuring the wavelength, configuring the spectrum, and configuring the modulation format.

21. The non-transitory computer-readable medium according to claim 20, wherein the state comprises: a route to be configured state, a wavelength to be configured state, a spectrum to be configured state, a modulation format to be configured state, an impairment verification analysis to be performed state, and an end state, in the route to be configured state, the timely reward R.sub.0=0, and in other states, the timely rewards meet any one or combination of the following conditions: in the wavelength to be configured state, a timely reward R.sub.1 is a function of a working route cost, and R.sub.1 and the working route cost is monotonically decreasing; in the spectrum to be configured state, a timely reward R.sub.2 is a function of a wavelength resource utilization rate, and R.sub.2 and the wavelength resource utilization rate is monotonically increasing; in the modulation format to be configured state, a timely reward R.sub.3 is a function of service occupied bandwidth, and R.sub.3 and the service occupied bandwidth is monotonically decreasing; in the impairment verification analysis to be performed state, a timely reward R.sub.4 is a function of a service spectral efficiency, and R.sub.4 and the service spectral efficiency is monotonically increasing; or in the end state, a timely reward R.sub.5 is related to a result of the impairment verification analysis, wherein R.sub.5 is positive when the result of the impairment verification analysis is qualified, and R.sub.5 is negative when the result of the impairment verification analysis is unqualified.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0019] FIG. 1 is a schematic diagram of OTN single service resource configuration under a Software Defined Optical Network (SDON) architecture;

[0020] FIG. 2 is a flowchart of a single service resource configuration method according to the present disclosure;

[0021] FIG. 3 is a flowchart of a method for performing IV analysis according to the present disclosure;

[0022] FIG. 4 is a flowchart of a method for calculating an optimization objective policy parameter according to the present disclosure; and

[0023] FIG. 5 is a schematic structural diagram of a single service resource configuration apparatus according to the present disclosure.

DETAIL DESCRIPTION OF EMBODIMENTS

[0024] Example embodiments will be described more sufficiently below with reference to the accompanying drawings, but which may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0025] As used herein, the term and/or includes any and all combinations of one or more of the associated listed items.

[0026] The terminology used herein is for the purpose of describing specific embodiments only and is not intended to limit the present disclosure. As used herein, the singular forms a, an and the are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that as used herein, the terms comprise and/or consist of . . . specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

[0027] Embodiments described herein may be described with reference to plan and/or sectional views in idealized representations of the present disclosure. Accordingly, the example illustrations may be modified in accordance with manufacturing techniques and/or tolerances. Accordingly, the embodiments are not limited to the embodiments shown in the drawings, but include modifications of configurations formed based on a manufacturing process. Thus, the regions illustrated in the figures have schematic properties, and the shapes of the regions shown in the figures illustrate specific shapes of regions of elements, but are not intended to be limiting.

[0028] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the existing art and the present disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0029] FIG. 1 is a schematic diagram of OTN single service resource configuration under an SDON architecture.

[0030] In the SDON architecture, a path computation element (PCE) mainly undertakes routing computation and resource allocation functions for OTN services, provides optimized paths which meet costs and other target policies for the services, and performs resource configuration and evaluation, such as Routing and Wavelength Assignment (RWA), Routing and Spectrum Assignment (RSA), Software Defined Optics (SDO), Impairment Verification (IV) and the like, on the basis of the optimized paths, to finally obtain service resource paths meeting optimization conditions of comprehensive indicators. In a traditional resource configuration scheme for a single service in the OTN, calculation and configuration of path resources are implemented step by step, rather than completely completed and realized in a unified algorithm. As a result, the effect of optimized configuration of service resources, the degree of comprehensive optimization, the optimization efficiency, the strictness of the optimization algorithm theory, and the like all have certain disadvantages.

[0031] In the present disclosure, the reinforcement learning technology is adopted for creation and comprehensive optimization of an OTN single service. Since a design of the reinforcement learning algorithm model is close to flows and rules of an SDON management and control system for creation and resource allocation of a single service, the reinforcement learning technique can be ensured to comprehensively cover all flows and functional operations in comprehensive optimization of the OTN single service, and to be perfectly combined with the scene of comprehensive optimization of the OTN single service as much as possible, so that the reinforcement learning technology can be ensured to solve the problem of comprehensive optimization of the OTN single service in a manner suitable for local conditions and circumstances, and that an intelligent optimization effect of the reinforcement learning on the OTN single service can be fully exerted.

[0032] The OTN single service mentioned in the present disclosure refers to a wavelength level service of an optical channel layer (i.e., an OCh layer, also called an LO layer), and in an optimization calculation process of the OTN single service, the network environment (mainly including a network topology state, the number of other services in the OTN, a route, a resource allocation state, and the like) where the OTN service is located remains unchanged.

[0033] Relevant parameters in the reinforcement learning algorithm model are defined as follows:

[0034] 1. Definition of Episode

[0035] The whole process of adopting a certain action policy to complete routing determination, wavelength configuration, spectrum resource configuration, modulation format configuration (SDO) and impairment verification (IV) analysis of one OTN service is defined as a complete episode.

[0036] 2. Definitions of Action .sub.t and Action Policy (s,)

[0037] The action .sub.t includes an action of configuring a resource parameter or an action of performing IV analysis. The resource parameters may include: a route, a wavelength, a spectrum resource, and a modulation format, t=(0, . . . , n1), and n1 is the total number of resource parameters. In an embodiment of the present disclosure, n1=4, which means that one episode includes 5 actions, .sub.0 to .sub.4.

[0038] The action policy (s,) includes a random action policy and a deterministic action policy. The random action policy may be represented by (s,), and the deterministic action policy may be represented by (s,) (or (s)).

[0039] Action sets for resource configuration of a OTN single service are listed below: [0040] (1) Action set .sub.0: route configuration actions. Depending on routing constraint conditions (including mandatory nodes and links, mandatory avoided nodes and links, and the like) and differences in route selections, the action set .sub.0 includes a plurality of actions for route allocation, such as allocating route A, route B, route C . . . or the like to the service to be configured. [0041] (2) Action set .sub.1: wavelength configuration actions, including configuring a wavelength for the route of the service to be configured following the wavelength consistency and wavelength continuity principles. Depending on differences in wavelength values, the action set .sub.1, includes a plurality of actions for wavelength configuration, such as configuring wavelength L, wavelength M, wavelength N . . . or the like for the route of the service to be configured. It should be noted that if the route includes hops, different wavelengths may be used for different hops. [0042] (3) Action set .sub.2: spectrum configuration actions, including configuring a spectrum for the route of the service to be configured. Depending on differences in bandwidth values, the action set .sub.2 includes a plurality of actions for spectrum configuration, such as configuring bandwidth x, bandwidth y, bandwidth z . . . or the like for the route of the service to be configured. It should be noted that if the route includes hops, different bandwidths may be used for different hops. [0043] (4) Action set .sub.3: SDO (modulation format) configuration actions, including configuring a modulation format for the route of the service to be configured. Depending on differences in attribute values of the modulation format, the action set .sub.3 includes a plurality of actions for SDO configuration, such as configuring modulation format i, modulation format j, modulation format k . . . or the like for the route of the service to be configured. It should be noted that if the route includes hops, different modulation formats may be used for different hops. [0044] (5) Action set .sub.4: IV analysis actions, including based on the route of the service to be configured and network resources configured along the route, performing IV analysis along the route. It should be noted that if the route includes hops, different back-to-back optical signal noise ratios OSNR.sub.B2B are used for IV analysis of different hops.

[0045] In a multi-hop scenario, due to different resource configurations and IV analysis among the hops, on the premise that resource configuration constraints and attribute setting constraints of each service of the OTN are met, each action set may be further split into multiple action sets taking a hop as a unit. For example, an action set .sub.11 indicates actions for allocating wavelengths in a first hop of the route of the service to be configured, and an action set .sub.12 indicates actions for allocating wavelengths in a second hop of the route of the service to be configured.

[0046] 3. Definitions of States S.sub.t in Configuration of a Single Service

[0047] Each state in one episode is denoted as S.sub.t, where t=(0, . . . , n), and n is the total number of resource parameters+1. In an embodiment of the present disclosure, the total number of resource parameters is 4, and n=5, and therefore, one episode includes 6 states, i.e., S.sub.0 to S.sub.5. S.sub.0 is an initial state, i.e., a route to be configured state, in which no resource parameter is configured; S.sub.1 is a wavelength to be configured state, in which a route is configured (i.e., the route configuration action .sub.0 is executed), but no wavelength is configured; S.sub.2 is a spectrum to be configured state, in which a route and a wavelength are configured (i.e., the route configuration action .sub.0 and the wavelength configuration action .sub.1 are executed), but no spectrum is configured; S.sub.3 is a modulation format to be configured state, in which a route, a wavelength and a spectrum are configured (i.e., the route configuration action .sub.0, the wavelength configuration action .sub.1, and the spectrum configuration action .sub.2 are executed), but no modulation format is configured; S.sub.4 is an IV analysis state, in which a route, a wavelength, a spectrum and a modulation format are configured (i.e., the route configuration action .sub.0, the wavelength configuration action .sub.1, the spectrum configuration action .sub.2, and the modulation format configuration action .sub.3 are executed), but IV analysis is not performed; S.sub.5 is an end state, in which all resource parameters are configured and the IV analysis is performed (i.e., the route configuration action .sub.0, the wavelength configuration action .sub.1, the spectrum configuration action .sub.2, the modulation format configuration action .sub.3, and the IV analysis action .sub.4 are executed). Once the end state is reached, it indicates that an episode is ended.

[0048] Taking a service request from nod A to node D as an example, in state S.sub.0, a route configuration action .sub.0 is performed to select a working route from candidate routes; in state S.sub.1, a wavelength configuration action .sub.1 is performed to allocate a wavelength for the working route; in state S.sub.2, a spectrum configuration action .sub.2 is performed to allocate a spectrum for the route with the allocated wavelength; in state S.sub.3, an SDO configuration action .sub.3 is performed to configure a modulation format for the route with the allocated wavelength and spectrum; in state S.sub.4, an IV analysis action .sub.4 is performed to perform IV analysis on the route with the allocated wavelength, spectrum and SDO; and in state S.sub.5, the IV analysis is completed and the episode is ended.

[0049] In a multi-hop scenario, the next states S.sub.1, S.sub.2, S.sub.3, S.sub.4, and S.sub.5 corresponding to the actions .sub.0, .sub.1, .sub.2, .sub.3, and .sub.4 may be split into a plurality of states along with the splitting of actions in different hops. That is, the states may be divided in units of hops. In a multi-hop scenario, the entire episode is ended only when all the hops therein enter the end state S.sub.5.

[0050] FIG. 2 is a flowchart of a single service resource configuration method according to the present disclosure.

[0051] As shown in FIG. 2, the single service resource configuration method according to the present disclosure includes the following operations S11 to S15.

[0052] At operation S11, configuring resource parameters for a service to be configured according to an action policy, calculating a timely reward in a current state, performing, after all the resource parameters are configured, IV analysis according to the action policy, and ending one episode after the IV analysis is completed, where a next state is entered after an action in the action policy is completed, and the action includes an action of configuring a resource parameter or an action of performing IV analysis.

[0053] In this operation, within one episode, resource parameters are configured for a service to be configured according to an action policy (s,), and when one resource parameter is configured, a timely reward in that state is calculated. Then, the current state is ended and a next state is entered. According to the above operation, resource parameters are configured in one episode, respectively, and timely rewards under corresponding states are calculated, until all resource parameters are configured, and IV analysis is performed. Then, one episode is ended.

[0054] At operation S12, calculating and updating, according to the timely reward in each state, an optimization objective policy parameter in each state.

[0055] In this operation, different algorithms may be used to calculate and update the optimization objective policy parameter. However, it should be noted that different algorithms are used for different optimization objective policy parameters, and detailed description of the algorithms will be given later.

[0056] At operation S13, iterating a preset number of episodes to calculate and update the optimization objective policy parameter in each state.

[0057] In this operation, operations S11 to S12 are repeated to iterate the preset number of episodes, so as to calculate and update the optimization objective policy parameter in each state of each episode.

[0058] At operation S14, determining, according to the optimization objective policy parameter in each state in the preset number of episodes, an optimal optimization objective policy parameter in each state.

[0059] In this operation, for each state, the optimal optimization objective policy parameter in that state is determined from optimization objective policy parameters in different episodes. It should be noted that different algorithms may determine the optimal optimization objective policy parameter in different manners. After this operation, optimal optimization target policy parameters of the service to be configured in all states can be obtained.

[0060] At operation S15, updating the action policy according to the optimal optimization objective policy parameter in each state.

[0061] The optimization objective policy parameter is used for representing a state and an action. When the optimal optimization objective policy parameter in a certain state is determined, an optimal action .sub.t in that state, i.e., an action for configuring an optimal resource parameter in that state, can be determined, so that the optimal resource parameter in that state can be determined, thereby obtaining an action set of all optimal resource parameters, i.e., an optimized action policy (s,).

[0062] According to the single service resource configuration method and apparatus of the present disclosure, resource parameters are configured for a service to be configured according to an action policy, a timely reward in a current state is calculated, after all the resource parameters are configured, IV analysis is performed according to the action policy, and one episode is ended after the IV analysis is completed, where a next state is entered after an action is completed, and the action includes an action of configuring a resource parameter or an action of performing IV analysis; according to the timely reward in each state, an optimization objective policy parameter in each state is calculated and updated; a preset number of episodes is iterated to calculate and update the optimization objective policy parameter in each state; according to the optimization objective policy parameter in each state in the preset number of episodes, an optimal optimization objective policy parameter in each state is determined; and according to the optimal optimization objective policy parameter in each state, the action policy is updated. By means of a reward and punishment mechanism of the reinforcement learning algorithm, the present disclosure performs comprehensive optimization on various resources and performance indexes, optimizes resource configuration for an OTN single service, and thus provides an OTN service path with optimized comprehensive indexes for a user, and thereby obtaining an action policy with good convergence, high rigidness and high reliability.

[0063] In some embodiments, the resource parameters may include a route, a wavelength, a spectrum, and a modulation format, and the resource parameters are configured for the service to be configured within one episode in a sequence of: configuring the route, configuring the wavelength, configuring the spectrum, and configuring the modulation format. It should be noted that the present disclosure is described by taking the sequence of configuring a route, configuring a wavelength, configuring a spectrum, and configuring a modulation format as an example, but those skilled in the art will know that the sequence of resource parameter configuration and the types and number of resource parameters are not limited as long as it is ensured that the IV analysis is performed after all resource parameters are configured.

[0064] In some embodiments, the states include: a route to be configured state S.sub.0, a wavelength to be configured state S.sub.1, a spectrum to be configured state S.sub.2, a modulation format to be configured state S.sub.3, an IV analysis to be performed state S.sub.4, and an end state S.sub.5. R.sub.t represents a timely reward obtained in state S.sub.t, which is a timely reward obtained after performing action .sub.t1 in state S.sub.t1 and shifting to state S.sub.t, t is the number of resource parameters already configured in state S.sub.t, t=(0, . . . , n1), and n1 is the total number of resource parameters. It should be noted that in the route to be configured state S.sub.0, the timely reward R.sub.0=0, and in other states, the timely rewards meet any one or combination of the following conditions (1) to (5):

[0065] (1) In the wavelength to be configured state S.sub.1, a timely reward R.sub.1 is a function of a working route cost, and R.sub.1 and the working route cost is monotonically decreasing. In other words, R.sub.1 may be a function of the working route cost Svc Cost obtained after the service to be configured is subjected to action .sub.0, and there is a monotonically decreasing relationship between R.sub.1 and the working route cost Svc Cos t.

[0066] (2) In the spectrum to be configured state S.sub.2, a timely reward R.sub.2 is a function of a wavelength resource utilization rate, and R.sub.2 and the wavelength resource utilization rate is monotonically increasing. In other words, R.sub.2 may be a timely reward obtained after the service to be configured is subjected to action .sub.1 to obtain the working route wavelength resource, and under the condition that constraint conditions of wavelength consistency and continuity are met, R.sub.2 may be a function of the wavelength resource utilization rate U.sub. of the current network, and there is a monotonically increasing relationship between R.sub.2 and the wavelength resource utilization rate U.sub..

[0067] (3) In the modulation format to be configured state S.sub.3, a timely reward R.sub.3 is a function of service occupied bandwidth, and R.sub.3 and the service occupied bandwidth is monotonically decreasing. In other words, R.sub.3 may be a timely reward obtained after the service to be configured is subjected to action .sub.2 to obtain a spectrum resource of the working route, and under a constraint condition that a minimum bandwidth usage threshold of the service to be configured is met, R.sub.3 may be a function of a bandwidth F.sub.w currently occupied by the service to be configured, and there is a monotonically decreasing relationship between R.sub.3 and the bandwidth F.sub.w.

[0068] (4) In the IV analysis to be performed state S.sub.4, a timely reward R.sub.4 is a function of a service spectral efficiency, and R.sub.4 and the service spectral efficiency is monotonically increasing. In other words, R.sub.4 may be a timely reward obtained after the service to be configured is subjected to action .sub.3 to obtain an SDO (modulation format) of the working route, and under a constraint condition that a minimum bandwidth usage threshold of the service to be configured is met, R.sub.4 may be a function of a current spectral efficiency of the service to be configured, and there is a monotonically increasing relationship between R.sub.4 and the current spectral efficiency .

[0069] (5) In the end state S.sub.5, a timely reward R.sub.5 is related to a result of the IV analysis, where R.sub.5 is positive when the result of the IV analysis is qualified, and R.sub.5 is negative when the result of the IV analysis is unqualified. In other words, R.sub.5 may be a timely reward obtained after the service to be configured is subjected to action .sub.4 to complete the IV analysis. If the IV analysis result is qualified, which indicates that the working route meets performance requirements for service transmission, R.sub.5 is assigned with a positive reward which is greater than a sum of the above four timely rewards. If the IV analysis result is unqualified, which indicates that the working route does not meet performance requirements for service transmission, R.sub.5 is given with a negative reward as a penalty, which has an absolute value greater than the sum of the above four timely rewards.

[0070] In some embodiments, the action policy (s,) includes a random action policy (S,) and a deterministic action policy (S,), and configuring resource parameters for the service to be configured according to the action policy in operation S11 includes: configuring a route, a wavelength, a frequency spectrum and a modulation format for the service to be configured according to the random action policy (S,). Performing IV analysis according to the action policy in operation S11 includes: performing IV analysis according to the deterministic action policy (S,).

[0071] Actions .sub.0, .sub.1, .sub.2, .sub.3 correspond to four operations of route selection, wavelength allocation, spectrum allocation and SDO setting, respectively, each with a plurality of candidates. For example, one OTN service may select one of a plurality of candidate routes as a working route. If selecting one of the routes as the working route is regarded as a specific action, then action .sub.0 in state S.sub.0 actually corresponds to a set of actions, and a specific action may be performed according to an action policy to select a route. Therefore, an initial policy for action .sub.0 in state S.sub.0 may be a random policy (s.sub.0, .sub.0). Similarly, initial policies for actions .sub.1, .sub.2, .sub.3 in states S.sub.1, S.sub.2, and S.sub.3 are all random action policies, i.e., (s.sub.1, .sub.1), (s.sub.2, .sub.2), (s.sub.3, .sub.3). The action .sub.4 in state S.sub.4 corresponds to an IV analysis operation, and may adopt a deterministic action policy (S.sub.4, .sub.4).

[0072] The IV analysis is used for evaluating influences of factors related to OTN service transmission, such as back-to-back OSNR, optical fiber nonlinearity, optical fiber CD, optical fiber PMD, optical filtering, PDL introduced by an optical device, service multi-hop accumulated OSNR fluctuation, OSNR flatness and the like on the system performance, and on the basis, for evaluating and analyzing feasibility of the optical link resource performance passed by the OTN service according to an OSNR allowance configuration requirement of a customer and the manufacturer policy.

[0073] FIG. 3 is a flowchart of a method for performing IV analysis according to the present disclosure.

[0074] As shown in FIG. 3, the method for performing IV analysis includes the following operations S21 to S24.

[0075] At operation S21, calculating a pre-calculated value OSNR.sub.pre-calculated value of an optical signal noise ratio of the service to be configured.

[0076] In some embodiments, the pre-calculated value OSNR.sub.pre-calculated value of the optical signal noise ratio of the service to be configured may be calculated by the 58 equation, which is not described in detail here.

[0077] At operation S22, judging whether the pre-calculated value OSNR.sub.pre-calculated value of the optical signal noise ratio meets a preset condition, proceeding to operation S23 if the pre-calculated value OSNR.sub.pre-calculated value of the optical signal noise ratio meets the preset condition, and proceeding to operation S24 if the pre-calculated value OSNR.sub.pre-calculated value of the optical signal noise ratio does not meet the preset condition.

[0078] In some embodiments, a case where the pre-calculated value OSNR.sub.pre-calculated value of the optical signal noise ratio meets the preset condition includes:

OSNR.sub.pre-calculated valueOSNR.sub.flatnessOSNR.sub.transmission thresholdOSNR.sub.transmission threshold=OSNR.sub.B2B++OSNR.sub.nonlinear+OSNR.sub.CD+OSNR.sub.PMD+OSNR.sub.filter+OSNR.sub.PDL+OSNR.sub.fluctuation+OSNR.sub.net allowance

where OSNR.sub.flatness is a flatness of the optical signal noise ratio, i.e., a statistical empirical value in the OTN network; OSNR.sub.B2B is a back-to-back optical signal noise ratio, and may be found in an optical module manual; OSNR.sub.nonlinear is a nonlinear cost of the optical signal noise ratio, OSNR.sub.CD is a chromatic dispersion cost of the optical signal noise ratio, OSNR.sub.PMD is a polarization mode dispersion cost of the optical signal noise ratio, OSNR.sub.filter is a filter membrane cost of the optical signal noise ratio, OSNR.sub.PCL is a polarization dependent loss cost of the optical signal noise ratio, and OSNR.sub.fluctuation is a fluctuation of the optical signal noise ratio, i.e., a statistical empirical value in the OTN network; and OSNR.sub.net allowance is a net allowance required by the optical signal noise ratio and is determined based on actual requirements, and OSNR.sub.flatness, OSNR.sub.B2B, OSNR.sub.nonlinear, OSNR.sub.CD, OSNR.sub.PMD, OSNR.sub.filter, OSNR.sub.PCL, OSNR.sub.fluctuation and OSNR.sub.net allowance are preset values.

[0079] At operation S23, determining that a result of the IV analysis is qualified.

[0080] At operation S24, determining that the result of the IV analysis is unqualified.

[0081] In some embodiments, in a multi-hop scenario, the route of the service to be configured includes a plurality of hops. Accordingly, configuring resource parameters for the service to be configured according to the action policy in operation S11 of the single service resource configuration method described with respect to FIG. 1 includes: configuring resource parameters for the service to be configured according to the action policy in each hop. Performing IV analysis according to the action policy in operation S11 of the single service resource configuration method described with respect to FIG. 1 includes: performing IV analysis according to the action policy in each hop. Performing IV analysis in operation S11 of the single service resource configuration method described with respect to FIG. 1 includes: calculating a pre-calculated value of an optical signal noise ratio of each hop in the route of the service to be configured; determining, in response to the pre-calculated value of the optical signal noise ratio of each hop meeting a preset condition, that a result of the IV analysis is qualified; and determining, in response to the pre-calculated value of the optical signal noise ratio of at least one hop not meeting the preset condition, that the result of the IV analysis is unqualified.

[0082] FIG. 4 is a flowchart of a method for calculating an optimization objective policy parameter according to the present disclosure.

[0083] As shown in FIG. 4, calculating and updating, according to the timely reward in each state, the optimization objective policy parameter in each state includes the following operations S31 and S32.

[0084] At operation S31, calculating, according to timely rewards in various states after a next state, an expected return in a current state.

[0085] In some embodiments, the expected return in the current state may be calculated by:

[00004] $G_{t} = {.Math.}_{k = 0}^{n - (t + 1)}^{k} R_{t + k + 1};$

where G.sub.t is an expected return of executing action .sub.t in state S.sub.t, is a discount coefficient, and 0<<1; R is the timely reward, t is the number of resource parameters already configured in state S.sub.t, t=(0, . . . , n1), and n1 is the total number of resource parameters.

[0086] It should be noted that the expected return in a last state is the timely reward in that state.

[0087] At operation S32, calculating and updating, according to the expected return in the current state, the optimization objective policy parameter in the current state.

[0088] Through operations S31 and S32, optimization of the optimization objective policy parameter can be achieved through a reward and punishment mechanism of the reinforcement learning algorithm.

[0089] In some embodiments, the optimization objective policy parameter may be a state behavior value Q.sub.(s,), and

[00005] $Q_{} (s, a) = E_{} [{.Math.}_{k = 0}^{n - (t + 1)}^{k} R_{t + k + 1} .Math. s = s_{t}, a = a_{t}],$

which indicates an expected accumulative return obtained by an agent performing action according to action policy (s, ) from state S.sub.t.

[0090] In some embodiments, the optimization objective policy parameter may also be a state value V.sub.(s), and

[00006] $V_{} (s) = \underset{a A}{.Math.} (a .Math. s) Q_{} (s, a),$

which indicates a weighted sum of all state behavior values Q.sub.(s,) in state S, where (\s) is a probability of taking action according to action policy (s,) in state S, and A is a set of actions executed in each state. It should be noted that if action policy (s,) is a deterministic action policy, then V.sub.(s,)=Q.sub.(s,).

[0091] In some embodiments, when the optimization objective policy parameter is the state behavior value Q.sub.(s,), the optimization objective policy parameter in each state may be calculated and updated with a Monte Carlo Process (MCP) algorithm, a TD-Error algorithm of different policies or a TD-Error algorithm of a same policy. In some embodiments, a Q-Learning algorithm in the TD-Error algorithm of different policies may be used, or a State-Action-Reward-Action (SASA) algorithm in the TD-Error algorithm of a same policy may be used. Accordingly, updating the action policy according to the optimal optimization objective policy parameter in each state (i.e., operation S15 of the single service resource configuration method described with respect to FIG. 1) includes: updating the action policy according to the state behavior value Q.sub.(s,).

[0092] For example, if the Q-Learning algorithm or the SASA algorithm is adopted, determining the optimal optimization objective policy parameter in each state (i.e., operation S14 of the single service resource configuration method described with respect to FIG. 1) may include: determining, from the optimization objective policy parameter (i.e., state behavior value Q.sub.(s,)) in each state in the preset number of episodes, a maximum value of the optimal optimization objective policy parameter in each state.

[0093] In some embodiments, when the optimization objective policy parameter is the state value V.sub.(s), the optimization objective policy parameter may be calculated and updated with a dynamic programming algorithm. Accordingly, updating the action policy according to the optimal optimization objective policy parameter in each state (i.e., operation S15 of the single service resource configuration method described with respect to FIG. 1) includes: updating the action policy (s,) according to the state value V.sub.(s).

[0094] The process of implementing resource configuration of the OTN single service with the MCP algorithm, the Q-Learning algorithm, the SASA algorithm, and the dynamic programming algorithm will be described below.

[0095] (1) The process of implementing resource configuration of the OTN single service with an exploring initialization MCP algorithm includes:

TABLE-US-00001 initialize the entire network topology environment, for all s S, a A(s), Q(s,a) 0; where an initial value of the action policy is (s,a); returns(s,a) emptylist; repeat the following processing: { according to (s,a), select s.sub.0 S, a.sub.0 A(s), generate 1 new episode; for each pair (s,a) in the episode: G (s,a) returns after a first occurrence; add G to returns(s,a); let state behavior value Q(s,a) average(returns(s,a)) average the returns; for each s in the episode: (s) arg max.sub.a Q(s,a); }

[0096] (2) The process of implementing resource configuration of the OTN single service with the Q-Learning algorithm (i.e., TD-Error of different policies) includes: [0097] initialize the entire network topology environment, for all s S,A(s), Q(s,)0; the action policy is (s,); [0098] repeat the following processing in each episode: [0099] initialize a state space S; and [0100] repeat (the following processing in each operation of the episode): [0101] select action .sub.t in state s.sub.t according to a policy (s,); [0102] perform action .sub.t and obtain a timely reward R.sub.t+1 and a next state s.sub.t+1; [0103] let Q(s.sub.t,.sub.t)Q(s.sub.t, .sub.t)+[R.sub.t+1+ max.sub.Q (s.sub.t+1, )Q(s.sub.t, .sub.t)]; [0104] where is a learning rate; [0105] s.sub.ts.sub.t+1; [0106] until s.sub.t becomes an end state; [0107] until all Q(s,) are converged; and [0108] output a final policy: (s)arg max.sub.Q(s,).

[0109] (3) The process of implementing resource configuration of the OTN single service with the SARSA algorithm (i.e., TD-Error of a same policy) includes: [0110] initialize the entire network topology environment, for all s S,A(s), Q(s,)0; [0111] repeat the following processing in each episode: [0112] initialize state space S; [0113] give initial state s.sub.0, and select action .sub.0 according to greedy policy (taking an action with a maximum timely reward); and [0114] repeat (the following processing in each operation of the episode): [0115] select action .sub.t in state s.sub.t according to greedy policy , to obtain a timely reward R.sub.t+1 and a next state s.sub.t+1; [0116] obtain action .sub.t+1 according to greedy policy ; [0117] let Q(s.sub.t, .sub.t)Q(s.sub.t, .sub.t)+[R.sub.t+1,+Q(s.sub.t+1, .sub.t+1)Q(s.sub.t, .sub.t)]; [0118] where is a learning rate; [0119] s.sub.ts.sub.t+1; .sub.t.sub.t+1; [0120] until s.sub.t becomes an end state; [0121] until all Q(s,) are converged; and [0122] output a final policy: (s)arg max.sub.Q(s,).

[0123] (4) The process of implementing resource configuration of the OTN single service with the dynamic programming algorithm based on policy iteration includes the following operations 1 to 3.

[0124] At operation 1, initializing the entire network topology environment, in which [0125] for all s.sub.tS,A(s), V(s.sub.t)=0, let all P.sub.ss.sup.=1; and initialize the action policy to (s).

[0126] At operation 2, evaluating the policy, [0127] p(s.sub.t+1, R.sub.t+1|s.sub.t, (s)) and p(s.sub.t+1, R.sub.t+1|s.sub.t, ) represent probabilities of taking the corresponding action in state s.sub.t with the policy (s); [0128] repeat the following processing:

0; [0129] for each s.sub.tS:

vV(s.sub.t);

V(s.sub.t).sub.s.sub.t+1,.sub.R.sub.t+1p(s.sub.t+1,R.sub.t+1|s.sub.t+1,(2))[R.sub.t+1+V(s.sub.t+1)];

max(,|vV(s.sub.t)|); [0130] until < (where is a specified constant) is converged.

[0131] At operation 3, improving the policy [0132] for each s.sub.t S:

(s);

(s)arg max.sub..sub.s.sub.t+1,.sub.R.sub.t+1p(s.sub.t+1,R.sub.t+1|s.sub.t,)[R.sub.t+1+V(s.sub.t+1)]; [0133] if (s), it represents that the policy is not converged; otherwise, the policy is converged; and [0134] end the algorithm if the policy is converged and return to V(s) and (s); otherwise, return to the processing in operation 2.

[0135] The present disclosure can be applied to the field of intelligent management and control and operation and maintenance of optical networks, and comprehensively optimize various resources and performance indexes of a OTN single service through reinforcement learning, and thus provide an OTN service path with optimized comprehensive indexes for a user. The reinforcement learning algorithm makes path comprehensive optimization possible, and an ideal path optimization result can be intelligently obtained through iterative improvement of the action policy.

[0136] FIG. 5 is a schematic structural diagram of a single service resource configuration apparatus according to the present disclosure. As shown in FIG. 5, the single service resource configuration apparatus according to the present disclosure includes a first processing module 101, a second processing module 102 and an updating module 103. The first processing module 101 is configured to configure resource parameters for a service to be configured according to an action policy, calculate a timely reward in a current state, perform, after all the resource parameters are configured, IV analysis according to the action policy, and end one episode after the IV analysis is completed, where a next state is entered after an action in the action policy is completed, and the action includes an action of configuring a resource parameter or an action of performing IV analysis; calculate and update, according to the timely reward in each state, an optimization objective policy parameter in each state; and iterate a preset number of episodes to calculate and update the optimization objective policy parameter in each state.

[0137] The second processing module 102 is configured to determine, according to the optimization objective policy parameter in each state in the preset number of episodes, an optimal optimization objective policy parameter in each state.

[0138] The updating module 103 is configured to update the action policy according to the optimal optimization objective policy parameter in each state.

[0139] In some embodiments, the resource parameters include a route, a wavelength, a spectrum, and a modulation format, and the resource parameters are configured for the service to be configured within one episode in a sequence of: configuring the route, configuring the wavelength, configuring the spectrum, and configuring the modulation format.

[0140] In some embodiments, the states include: a route to be configured state, a wavelength to be configured state, a spectrum to be configured state, a modulation format to be configured state, an IV analysis to be performed state, and an end state. In the route to be configured state, the timely reward R.sub.0=0, and in other states, the timely rewards meet any one or combination of the following conditions: [0141] (1) in the wavelength to be configured state, a timely reward R.sub.1 is a function of a working route cost, and R.sub.1 and the working route cost is monotonically decreasing; [0142] (2) in the spectrum to be configured state, a timely reward R.sub.2 is a function of a wavelength resource utilization rate, and R.sub.2 and the wavelength resource utilization rate is monotonically increasing; [0143] (3) in the modulation format to be configured state, a timely reward R.sub.3 is a function of service occupied bandwidth, and R.sub.3 and the service occupied bandwidth is monotonically decreasing; [0144] (4) in the IV analysis to be performed state, a timely reward R.sub.4 is a function of a service spectral efficiency, and R.sub.4 and the service spectral efficiency is monotonically increasing; [0145] (5) in the end state, a timely reward R.sub.5 is related to a result of the IV analysis, wherein R.sub.5 is positive when the result of the IV analysis is qualified, and R.sub.5 is negative when the result of the IV analysis is unqualified.

[0146] In some embodiments, the action policy includes a random action policy and a deterministic action policy, and the first processing module 101 is configured to configure a route, a wavelength, a frequency spectrum and a modulation format for the service to be configured according to the random action policy; and perform IV analysis according to the deterministic action policy.

[0147] In some embodiments, when the route of the service to be configured includes a plurality of hops, the first processing module 101 is configured to configure resource parameters for the service to be configured according to the action policy in each hop; and perform IV analysis according to the action policy in each hop.

[0148] The first processing module 101 is configured to calculate a pre-calculated value of an optical signal noise ratio of each hop in the route of the service to be configured; determine, in response to the pre-calculated value of the optical signal noise ratio of each hop meeting a preset condition, that a result of the IV analysis is qualified; and determine, in response to the pre-calculated value of the optical signal noise ratio of at least one hop not meeting the preset condition, that the result of the IV analysis is unqualified.

[0149] In some embodiments, the first processing module 101 is configured to calculate a pre-calculated value of an optical signal noise ratio of the service to be configured; determine, in response to the pre-calculated value of the optical signal noise ratio meeting a preset condition, that a result of the IV analysis is qualified; and determine, in response to the pre-calculated value of the optical signal noise ratio not meeting the preset condition, that the result of the IV analysis is unqualified.

[0150] In some embodiments, a case where pre-calculated value of the optical signal noise ratio meets the preset condition includes:

OSNR.sub.pre-calculated valueOSNR.sub.flatnessOSNR.sub.transmission threshold,OSNR.sub.transmission threshold=OSNR.sub.B2B+OSNR.sub.nonlinear+OSNR.sub.CD+OSNR.sub.PMD+OSNR.sub.filter+OSNR.sub.PDL+OSNR.sub.fluctuation+OSNR.sub.net allowance,

where OSNR.sub.pre-calculated value is the pre-calculated value of the optical signal noise ratio, OSNR.sub.flatness is a flatness of the optical signal noise ratio, OSNR.sub.B2B is a back-to-back optical signal noise ratio, OSNR.sub.nonlinear is a nonlinear cost of the optical signal noise ratio, OSNR.sub.CD is a chromatic dispersion cost of the optical signal noise ratio, OSNR.sub.PMD is a polarization mode dispersion cost of the optical signal noise ratio, OSNR.sub.filter is a filter membrane cost of the optical signal noise ratio, OSNR.sub.PDL is a polarization dependent loss cost of the optical signal noise ratio, OSNR.sub.fluctuation is a fluctuation of the optical signal noise ratio, OSNR.sub.net allowance is a net allowance required by the optical signal noise ratio, and OSNR.sub.flatness, OSNR.sub.B2B, OSNR.sub.nonlinear, OSNR.sub.CCC, OSNR.sub.PMD, OSNR.sub.filter, OSNR.sub.PDL, OSNR.sub.fluctuation, and OSNR.sub.new allowance are preset values.

[0151] In some embodiments, the first processing module 101 is configured to calculate, according to timely rewards in various states after a next state, an expected return in a current state; and calculate and update, according to the expected return in the current state, the optimization objective policy parameter in the current state.

[0152] In some embodiments, the first processing module 101 is configured to calculate the expected return in the current state by:

[00007] $G_{t} = {.Math.}_{k = 0}^{n - (t + 1)}^{k} R_{t + k + 1};$

where G.sub.t is an expected return of executing action .sub.t, in state S.sub.t, is a discount coefficient, and 0<<1; R is the timely reward, t is the number of resource parameters already configured in state S.sub.t, t=(0, . . . , n1), and n1 is the total number of resource parameters.

[0153] In some embodiments, the optimization objective policy parameter includes a state behavior value Q.sub.(s, ), and

[00008] $Q_{} (s, a) = E_{} [{.Math.}_{k = 0}^{n - (t + 1)}^{k} R_{t + k + 1} .Math. s = s_{t}, a = a_{t}],$

or the optimization objective policy parameter includes a state value V.sub.(s),

[00009] $V_{} (s) = \underset{a A}{.Math.} (a .Math. s) Q_{} (s, a),$

where (|s) is a probability of taking action according to an action policy (s,) in state S, and A is a set of actions executed in each state.

[0154] In some embodiments, the first processing module 101 is configured to calculate and update, when the optimization objective policy parameter is the state behavior value Q.sub.(s,), the optimization objective policy parameter in each state with a Monte Carlo Process algorithm, a TD-Error algorithm of different policies or a TD-Error algorithm of a same policy.

[0155] The updating module 103 is configured to update the action policy according to the state behavior value Q.sub.(s, ).

[0156] In some embodiments, the first processing module 101 is configured to calculate, when the optimization objective policy parameter is the state value V.sub.(s), the optimization objective policy parameter with a dynamic programming algorithm.

[0157] The updating module 103 is configured to update the action policy according to the state value V.sub.(s).

[0158] The present disclosure provides a computer device, including: one or more processors and a storage means. The storage means has one or more programs stored thereon which, when executed by the one or more processors, cause the one or more processors to perform the single service resource configuration method described in any of the above embodiments.

[0159] The present disclosure further provides a computer-readable medium storing a computer program thereon which, when executed, causes the single service resource configuration method described in any of the above embodiments to be implemented.

[0160] Those of ordinary skill in the art will appreciate that all or some operations of the above described method, functional modules/units in the apparatus may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or operation may be performed cooperatively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a CPU, a digital signal processor or microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium which may include a computer storage medium (or non-transitory medium) and communication medium (or transitory medium). The term computer storage medium includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. The computer storage medium includes, but is not limited to, an RAM, an ROM, an EEPROM, a flash or any other memory technology, a CD-ROM, a digital versatile disc (DVD) or any other optical disc storage, a magnetic cartridge, a magnetic tape, a magnetic disk storage or any other magnetic storage device, or may be any other medium used for storing the desired information and accessible by a computer. Moreover, it is well known to those ordinary skilled in the art that a communication medium typically includes a computer-readable instruction, a data structure, a program module, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and may include any information delivery medium.

[0161] The present disclosure has disclosed exemplary embodiments, and although specific terms are employed, they are used and should be interpreted merely in a generic and descriptive sense, not for purposes of limitation. In some instances, as would be apparent to one skilled in the art, features, characteristics and/or elements described in connection with a particular embodiment may be used alone or in combination with features, characteristics and/or elements described in connection with another embodiment, unless expressly stated otherwise. It will, therefore, be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the present disclosure as set forth in the appended claims.

SINGLE SERVICE RESOURCE CONFIGURATION METHOD AND APPARATUS, COMPUTER DEVICE AND MEDIUM

Inventors

Cpc classification

Classification Explorer

H04B10/07953

ELECTRICITY

Classification Explorer

H04Q2011/0086

ELECTRICITY

Classification Explorer

H04J3/1652

ELECTRICITY

Classification Explorer

H04Q2011/0073

ELECTRICITY

Classification Explorer

H04Q2011/0084

ELECTRICITY

Classification Explorer

H04Q11/0062

ELECTRICITY

International classification

Classification Explorer

H04Q11/00

ELECTRICITY

Classification Explorer

H04J3/16

ELECTRICITY

Classification Explorer

H04B10/079

ELECTRICITY

Abstract

Claims

Description