MULTIDIMENSIONAL DATA GENERATION DEVICE, METHOD, AND COMPUTER-READABLE RECORDING MEDIUM

Abstract

The transforming means 72 transforms first multidimensional data in which the number of elements of dimension of channel is C and the number of elements of each dimension other than the dimension of channel is 1 into second multidimension of a predetermined form. The channel dimension element number increase means 73 generates third multidimensional data in which the number of elements of the dimension of channel is increased from 1 to N, by performing a convolution layer process with a filter size of 1×1. The transposition means 74 performs transposition on the third multidimensional data so that the number of elements of the dimension of channel becomes C. The generation means 75 generates multidimensional data in which the number of elements of the dimension of channel is C and the number of elements of each dimension other than the dimension of channel is predetermined number of elements.

Claims

1. A multidimensional data generation device comprising: a transformation unit, implemented by a processor, and that transforms first multidimensional data in which the number of elements of dimension of channel is C and the number of elements of each dimension other than the dimension of channel is 1 into second multidimensional data in which the number of elements of one dimension out of dimensions other than the dimension of channel is C and the number of elements of each dimension other than the one dimension is 1; a channel dimension element number increase unit, implemented by the processor, and that generates third multidimensional data in which the number of elements of the dimension of channel is increased from 1 to N, by performing a convolution layer process with a filter size of 1×1 with a common value of N weights on the second multidimensional data, when product of predetermined number of elements for each dimension other than the dimension of channel is N; a transposition unit, implemented by the processor, and that performs for predetermined transposition on the third multidimensional data so that the number of elements of the dimension of channel becomes C; and a generation unit, implemented by the processor, and that generates multidimensional data in which the number of elements of the dimension of channel is C and the number of elements of each dimension other than the dimension of channel is predetermined number of elements, based on the multidimensional data after the predetermined transposition.

2. The multidimensional data generation device according to claim 1, wherein the channel dimension element number increase unit generates the third multidimensional data, by performing the convolution layer process with a filter size of 1×1 with a common value 1 of N weights on the second multidimensional data.

3. The multidimensional data generation device according to claim 1, wherein the channel dimension element number increase unit generates the third multidimensional data, by performing the convolution layer process with a filter size of 1×1 with a common predetermined value of N weights on the second multidimensional data, and the generation unit divides a value of each element in the multidimensional data by the predetermined value, after generating the multidimensional data.

4. The multidimensional data generation device according to claim 1, wherein the first multidimensional data, the second multidimensional data, the third multidimensional data, the multidimensional data after the predetermined transposition, and the multidimensional data generated by the generation unit are 3 dimensional data.

5. A multidimensional data generation method comprising: transforming first multidimensional data in which the number of elements of dimension of channel is C and the number of elements of each dimension other than the dimension of channel is 1 into second multidimensional data in which the number of elements of one dimension out of dimensions other than the dimension of channel is C and the number of elements of each dimension other than the one dimension is 1; generating third multidimensional data in which the number of elements of the dimension of channel is increased from 1 to N, by performing a convolution layer process with a filter size of 1×1 with a common value of N weights on the second multidimensional data, when product of predetermined number of elements for each dimension other than the dimension of channel is N; performing predetermined transposition on the third multidimensional data so that the number of elements of the dimension of channel becomes C; and generating multidimensional data in which the number of elements of the dimension of channel is C and the number of elements of each dimension other than the dimension of channel is predetermined number of elements, based on the multidimensional data after the predetermined transposition.

6. A non-transitory computer-readable recording medium in which a multidimensional data generation program is recorded, wherein the multidimensional data generation program causes a computer to execute: a transformation process of transforming first multidimensional data in which the number of elements of dimension of channel is C and the number of elements of each dimension other than the dimension of channel is 1 into second multidimensional data in which the number of elements of one dimension out of dimensions other than the dimension of channel is C and the number of elements of each dimension other than the one dimension is 1; a channel dimension element number increase process of generating third multidimensional data in which the number of elements of the dimension of channel is increased from 1 to N, by performing a convolution layer process with a filter size of 1×1 with a common value of N weights on the second multidimensional data, when product of predetermined number of elements for each dimension other than the dimension of channel is N; a transposition process of performing predetermined transposition on the third multidimensional data so that the number of elements of the dimension of channel becomes C; and a generation process of generating multidimensional data in which the number of elements of the dimension of channel is C and the number of elements of each dimension other than the dimension of channel is predetermined number of elements, based on the multidimensional data after the predetermined transposition.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0029] FIG. 1 It depicts a block diagram showing an example configuration of a multidimensional data generation device of the example embodiment of the present invention.

[0030] FIG. 2 It depicts a schematic diagram showing an example of first 3 dimensional data.

[0031] FIG. 3 It depicts a schematic diagram showing an example of second 3 dimensional data.

[0032] FIG. 4 It depicts a schematic diagram showing second 3 dimensional data and third 3 dimensional data.

[0033] FIG. 5 It depicts a schematic diagram showing an example of 3 dimensional data after transposing.

[0034] FIG. 6 It depicts a schematic diagram showing a state in which H pieces of 3 dimensional data with size (1, W, C) are generated by dividing the 3 dimensional data shown in FIG. 5.

[0035] FIG. 7 It depicts a schematic diagram showing 3 dimensional data whose size is (H, W, C), generated by the generation unit 5

[0036] FIG. 8 It depicts a schematic diagram showing another example of 3 dimensional data after transposing.

[0037] FIG. 9 It depicts a flowchart showing an example of the processing flow of the example embodiment of the present invention.

[0038] FIG. 10 It depicts a schematic block diagram showing an example of computer configuration related to the multidimensional data generation device of the example embodiment of the present invention.

[0039] FIG. 11 It depicts a block diagram showing an overview of the multidimensional data generation device of the present invention.

[0040] FIG. 12 It depicts a schematic diagram showing the SE block.

[0041] FIG. 13 It depicts a schematic diagram showing the 3 dimensional data U.

[0042] FIG. 14 It depicts a schematic diagram showing the 1 dimensional data obtained in the Global Pooling layer.

[0043] FIG. 15 It depicts a schematic diagram showing the 1 dimensional data obtained in the FC layer in the SE block.

[0044] FIG. 16 It depicts a schematic diagram showing the process in the first FC layer in the SE block.

[0045] FIG. 17 It depicts a schematic diagram showing the process in the second FC layer in the SE block.

[0046] FIG. 18 It depicts a schematic diagram showing the 3 dimensional data X′ obtained by copying the 3 dimensional data of size 1×1×C, H×W times.

[0047] FIG. 19 It depicts a schematic diagram showing calculation of element-wise product of the 3 dimensional data U and the 3 dimensional data X′.

DESCRIPTION OF EMBODIMENTS

[0048] Example embodiment of the present invention is described below with reference to the drawings.

[0049] FIG. 1 is a block diagram showing an example configuration of a multidimensional data generation device of the example embodiment of the present invention. The multidimensional data generation device 1 of the present example embodiment includes a transformation unit 2, a channel dimension element number increase unit 3, a transposition unit 4, and a generation unit 5.

[0050] The data input to the multidimensional data generation device 1 of the present example embodiment will now be described. The output data of the Sigmoid layer (see FIG. 12) of the SE block is input to the multidimensional data generation device 1 of the present example embodiment. As already explained, the output data of the Sigmoid layer can be referred to as 3 dimensional data in which the number of elements of the H dimension is 1, the number of elements of the W dimension is 1, and the number of elements of the C dimension is C elements. In other words, the 3 dimensional data in which the number of elements of the H dimension is 1, the number of elements of the W dimension is 1, and the number of elements of the C dimension is C elements is input to the multidimensional data generation device 1. The multidimensional data input to the multidimensional data generation device 1 is referred to as the first multidimensional data (in the present example embodiment, the first 3 dimensional data).

[0051] In the first 3 dimensional data, the number of elements of the C dimension (the dimension of channel) is C and the number of elements of each dimension other than the C dimension is 1.

[0052] In the present example embodiment, the first 3 dimensional data is input to the multidimensional data generation device 1, and the multidimensional data generation device 1 generates 3 dimensional data in which the number of elements of the C dimension is C, the number of elements of the H dimension is H, and the number of elements of the W dimension is W. The number of elements of the C dimension “C” in the generated 3 dimensional data is the same as the number of elements of the C dimension (the dimension of channel) in the first 3 dimensional data. The number of elements of the H dimension “H” and the number of elements of the W dimension “W” in the generated 3 dimensional data are predetermined. In other words, the number of elements of each dimension in the generated 3 dimensional data is predetermined according to the 3 dimensional data U (see FIG. 13) that is an input to the SE block.

[0053] The first multidimensional data input to the multidimensional data generation device 1 may be 2 dimensional data or multidimensional data of 4 or more dimensions, if the number of elements of the C dimension is C and the number of elements of each dimension other than the C dimension is 1. The multidimensional data generation device 1 may generate 2 dimensional data or multidimensional data of 4 or more dimensions as multidimensional data. However, the multidimensional data generation device 1 generates multidimensional data of n dimension when multidimensional data of n dimension is input.

[0054] FIG. 2 is a schematic diagram showing an example of the first 3 dimensional data input to the multidimensional data generation device 1 in the present example embodiment.

[0055] When the first 3 dimensional data is input, the transformation unit 2 transforms the first 3 dimensional data into 3 dimensional data in which the number of elements of one dimension out of dimensions other than the C dimension (the dimension of channel) is C, and the number of elements of each dimension other than that one dimension is 1. In the present example embodiment, the case where “one dimension out of dimensions other than the C dimension” above is the W dimension will be used as an example, but it may also be the H dimension.

[0056] When “one dimension out of dimensions other than the C dimension” above is the W dimension, the transformation unit 2 transforms the first 3 dimensional data into 3 dimensional data in which the number of elements of the W dimension is C, and the number of elements of each of the other dimensions (the H dimension and the C dimension) is 1.

[0057] It is assumed that k is an integer from 0 to C−1. The transformation unit 2 transforms the first 3 dimensional data by replacing the element corresponding to the 0th of the H dimension, the 0th of the W dimension, and the kth of the C dimension in the first 3 dimensional data as the element corresponding to the 0th of the H dimension, the kth of the W dimension, and the 0th of the C dimension.

[0058] The multidimensional data after transformation by the transformation unit 2 is referred to as the second multidimensional data (in the present example embodiment, the second 3 dimensional data). FIG. 3 is a schematic diagram showing an example of the second 3 dimensional data.

[0059] The size of the first 3 dimensional data is (1, 1, C), while the size of the second 3 dimensional data is (1, C, 1) (see FIG. 2 and FIG. 3).

[0060] In the multidimensional data generated by the multidimensional data generation device 1 (3 dimensional data in the present example embodiment), the product of the predetermined number of elements for each dimension other than the C dimension (the dimension of channel) is N. In the present example embodiment, as mentioned above, the number of elements “H” in the H dimension and the number of elements “W” in the W dimension in the generated 3 dimensional data are predetermined. Therefore, N=H×W.

[0061] The channel dimension element number increase unit 3 generates 3 dimensional data in which the number of elements of the C dimension in the second 3 dimensional data is increased from 1 to N, by performing a convolution layer process with a filter size of 1×1 with a common value of N weights on the second 3 dimensional data. The multidimensional data generated by the channel dimension element number increase unit 3 is referred to as the third multidimensional data (in the present example embodiment, the third 3 dimensional data). The size of the third 3 dimensional data is (1, C, N). FIG. 4 is a schematic diagram showing the second 3 dimensional data and third 3 dimensional data.

[0062] The elements in the 3 dimensional data can be specified by their 3 dimension coordinates. Then, a value of the element specified by the H dimension coordinate h, the W dimension coordinate w, and the C dimension coordinate c in the second 3 dimensional data is expressed as (h, w, c).sub.before. Similarly, the value of the element specified by the H dimension coordinate h, the W dimension coordinate w, and the C dimension coordinate c in the third 3 dimensional data is expressed as (h, w, c).sub.after.

[0063] In the present example embodiment, there are N weights used in the convolution layer process with a filter size of 1×1, and the values of the N weights are all predetermined to be “ 1”. Therefore, the value of the N weights is 1 in common. The weights may be referred to as filter values.

[0064] It is assumed that i is an integer from 0 to N−1. The i-th weight is then written as t.sub.i. t.sub.0=t.sub.1=t.sub.2= . . . =t.sub.n−1=1. The weight t.sub.i is used to calculate the value of each element of the i-th channel in the third 3 dimensional data.

[0065] For example, the channel dimension element number increase unit 3 calculates the value of (0, 0, 0).sub.after by the formula (1) shown below.

(0, 0, 0).sub.after=(0, 0, 0).sub.before×t.sub.0  (1)

[0066] The channel dimension element number increase unit 3 also finds the values of the other elements of the 0th channel in the third 3 dimensional data by the same calculation, using the weights t.sub.0.

[0067] The channel dimension element number increase unit 3 calculates the value of (0, 0, i).sub.after by the formula (2) shown below.

(0, 0, i).sub.after=(0, 0, 0).sub.before×t.sub.i  (2)

[0068] The channel dimension element number increase unit 3 also finds the values of the other elements of the i-th channel in the third 3 dimensional data by the same calculation, using the weights t.sub.i.

[0069] Since t.sub.0=t.sub.1=t.sub.2= . . . ==1, as mentioned above, all of (0, 0, 0).sub.after, (0, 0, 1).sub.after, . . . , (0, 0, N−1).sub.after are equal to (0, 0, 0).sub.before.

[0070] The channel dimension element number increase unit 3 uses the above calculation to calculate the value of each element of the 0th channel, the value of each element of the 1st channel, . . . , the value of each element of the N−1th channel in the third 3 dimensional data. Then, the channel dimension element number increase unit 3 performs the same process at each position in the plane consisting of the H dimension and the W dimension in the second 3 dimensional data. In other words, the channel dimension element number increase unit 3 calculates the values of all elements in the third 3 dimensional data. Then, the channel dimension element number increase unit 3 derives the third 3 dimensional data. As a result, the third 3 dimensional data with size (1, C, N) is obtained.

[0071] It is assumed that j is an integer from 0 to C−1. As in the previous case, all of (0, j, 0).sub.after, (0, j, 1).sub.after, . . . , (0, j, N−1).sub.after are equal to (0, j, 0).sub.before.

[0072] The transposition unit 4 performs a predetermined transposition on the third 3 dimensional data generated by the channel dimension element number increase unit 3 so that the number of elements of the C dimension (the dimension of channel) becomes C.

[0073] Here, the transposition will be explained. The transposition is the operation of shifting the position of elements in multidimensional data by changing the order of coordinates in multidimension coordinates when the elements in multidimensional data are expressed in multidimension coordinates. The following is a specific explanation using 3 dimensional data as an example.

[0074] It is assumed that when h denotes the coordinates of the H dimension, w denotes the coordinates of the W dimension, and c denotes the coordinates of the C dimension, the element in the 3 dimensional data specified by the coordinates (h, w, c) is denoted as p(h, w, c). In addition, coordinates that rearrange the order of coordinates within (h, w, c) are considered to be, for example, (h, c, w). In this case, the operation of moving p(h, w, c) to p(h, c, w) is an example of transposition.

[0075] As a transposition such that the number of elements of the C dimension becomes C, the transposition unit 4 may perform a transposition on the third 3 dimensional data, moving p(h, w, c) to p(h, c, w) for the third 3 dimensional data. Alternatively, the transposition unit 4 may perform a transposition moving p(h, w, c) to p(c, h, w) for the third 3 dimensional data.

[0076] Here, first, the case where the transposition unit 4 performs a transposition on the third 3 dimensional data, moving p(h, w, c) to p(h, c, w) is shown. In this case, the size of the 3 dimensional data after the transposition is (1, N, C). In this case, the 3 dimensional data after the transposition is represented schematically as shown in FIG. 5. Note that N=H×W.

[0077] Based on the 3 dimensional data after the transposition, the generation unit 5 generates 3 dimensional data in which the number of elements of the C dimension is C and the number of elements of each dimension other than the C dimension is the predetermined number of elements. In the present example embodiment, based on the 3 dimensional data after the transposition, the generation unit 5 generates 3 dimensional data in which the number of elements of the C dimension is C, the number of elements of the H dimension is H, and the number of elements of the W dimension is W.

[0078] The generation unit 5, for example, generates H pieces of 3 dimensional data in which the number of elements of the H dimension is 1, the number of elements of the W dimension is W, and the number of elements of the C dimension is C (3 dimensional data whose size is (1, W, C)), by dividing the 3 dimensional data (3 dimensional data after transposing) shown schematically in FIG. 5 by W elements of the W dimension direction. FIG. 6 shows the state in which H pieces of 3 dimensional data with size (1, W, C) are generated by dividing the 3 dimensional data schematically shown in FIG. 5 as described above.

[0079] The generation unit 5 can generate the desired 3 dimensional data with size (H, W, C) by defining the H pieces of 3 dimensional data as the 0th to H−1st data in the H dimension, respectively. For example, the generation unit 5 may define the first 3 dimensional data obtained by dividing the 3 dimensional data after the transposition as described above as the 0th data of the H dimension, and the next 3 dimensional data as the 1st data of the H dimension, so that the obtained 3 dimensional data is sequentially defined as 0th to H−1st, thereby to generates the desired 3 dimensional data with size (H, W, C).

[0080] FIG. 7 is a schematic diagram showing the 3 dimensional data whose size is (H, W, C) generated by the generation unit 5.

[0081] The operation of generation unit 5 described above is an example of the operation to generate the 3 dimensional data shown in FIG. 7, and generation unit 5 may generate the 3 dimensional data shown in FIG. 7 by other operations based on the 3 dimensional data after the transposition is performed (see FIG. 5).

[0082] The above explanation describes the case where the transposition unit 4 transposes p(h, w, c) to p(h, c, w) for the third 3 dimensional data. Next, the case where the transposition unit 4 transposes p(h, w, c) to p(c, h, w) for the third 3 dimensional data is explained. In this case, the size of the 3 dimensional data after the transposition is (N, 1, C). In this case, the 3 dimensional data after the transposition is represented schematically as shown in FIG. 8. As mentioned above, N=H×W.

[0083] For example, the generation unit 5 generates W pieces of 3 dimensional data in which the number of elements of the H dimension is H, the number of elements of the W dimension is 1, and the number of elements of the C dimension is C (3 dimensional data whose size is (H, 1, C)), by dividing the 3 dimensional data (3 dimensional data after transposing) shown schematically in FIG. 8 by H elements of the H dimension direction.

[0084] The generation unit 5 can generate the desired 3 dimensional data with size (H, W, C) by defining the W pieces of 3 dimensional data as the 0th to W−1st data in the W dimension, respectively. For example, the generation unit 5 may define the first 3 dimensional data obtained by dividing the 3 dimensional data after the transposition as described above as the 0th data of the W dimension, and the next 3 dimensional data as the 1st data of the W dimension, so that the obtained 3 dimensional data is sequentially defined as 0th to W−1st, thereby to generates the desired 3 dimensional data with size (H, W, C). In this case, the 3 dimensional data expressed as shown in FIG. 7 is also obtained.

[0085] In this case, too, the operation of generation unit 5 described above is an example of the operation to generate the 3 dimensional data shown in FIG. 7, and generation unit 5 may generate the 3 dimensional data shown in FIG. 7 by other operations based on the 3 dimensional data after the transposition is performed (see FIG. 8).

[0086] As explained above, the transposition performed by transposition unit 4 on the third 3 dimensional data may be the transposition that moves p(h, w, c) to p(h, c, w) or the transposition that moves p(h, w, c) to p(c, h, w).

[0087] The generation unit 5 outputs the desired 3 dimensional data generated based on the 3 dimensional data after the transposition, whose size is (H, W, C) (see FIG. 7) to outside. For example, the generation unit 5 outputs the generated 3 dimensional data (see FIG. 7) to a device that executes element-wise product in the SE block (hereinafter referred to as an element-wise product calculation device. The figure is omitted.). The 3 dimensional data (see FIG. 7) generated by the generation unit 5 is the same 3 dimensional data as the aforementioned 3 dimensional data X′ (see FIG. 18). Therefore, in the calculation of the element-wise product with the 3 dimensional data U in the SE block, the 3 dimensional data generated by the generation unit 5 may be used instead of the 3 dimensional data X′ (see FIG. 18). In other words, the element-wise product calculation device may calculate the element-wise product of the 3 dimensional data U and the 3 dimensional data generated by the generation unit 5.

[0088] The transformation unit 2, the channel dimension element number increase unit 3, the transposition unit 4, and the generation unit 5 are realized, for example, by a CPU (Central Processing Unit) of a computer operating according to a multidimensional data generation program. For example, the CPU may read the multidimensional data generation program from a program storage medium such as a program storage device of the computer, and operate as the transformation unit 2, the channel dimension element number increase unit 3, the transposition unit 4, and the generation unit 5 according to the multidimensional data generation program. The channel dimension element number increase unit 3 may be realized by a dedicated circuit specialized for convolution layer process.

[0089] The transformation unit 2, the channel dimension element number increase unit 3, the transposition unit 4, and the generation unit 5 may each be realized by separate hardware. Moreover, as described above, the channel dimension element number increase unit 3 may be realized by a dedicated circuit specialized for convolution layer process.

[0090] Next, the processing flow will be described. FIG. 9 is a flowchart showing an example of the processing flow of the example embodiment of the present invention. Detailed explanations of matters already explained will be omitted.

[0091] When the first 3 dimensional data is input, the transformation unit 2 transforms the first 3 dimensional data into the second 3 dimensional data (step S1).

[0092] The first 3 dimensional data is the output data of the Sigmoid layer in the SE block. In the first 3 dimensional data, the number of elements of the C dimension is C, and the number of elements of each dimension other than the C dimension (H dimension, W dimension) is 1 (see FIG. 2). In the present example embodiment, in the second 3 dimensional data, the number of elements of the W dimension is C, and the number of elements of the other dimensions (H dimension and C dimension) is 1 (see FIG. 3).

[0093] The transformation unit 2 may transform the first 3 dimensional data into the second 3 dimensional data, by replacing the element corresponding to the 0th of the H dimension, the 0th of the W dimension, and the kth of the C dimension in the first 3 dimensional data as the element corresponding to the 0th of the H dimension, the kth of the W dimension, and the 0th of the C dimension. Here, k is an integer from 0 to C−1.

[0094] Next to step S1, the channel dimension element number increase unit 3 generates 3 dimensional data (the third 3 dimensional data. see FIG. 4.) in which the number of elements of the C dimension in the second 3 dimensional data is increased from 1 to N, by performing a convolution layer process with a filter size of 1×1 with a common value 1 of N weights on the second 3 dimensional data (step S2).

[0095] Next, the transposition unit 4 performs transposition on the third 3 dimensional data so that the number of elements of the C dimension is C (step S3). The transposition performed by transposition unit 4 on the third 3 dimensional data may be the transposition that moves p(h, w, c) to p(h, c, w) or the transposition that moves p(h, w, c) to p(c, h, w).

[0096] Next to step S3, based on the 3 dimensional data after the transposition (see FIG. 5 or FIG. 8), the generation unit 5 generates 3 dimensional data in which the number of elements of the C dimension is C and the number of elements of each dimension other than the C dimension (the H dimension, the W dimension) is the predetermined number of elements (step S4). In the present example embodiment, the generation unit 5 generates 3 dimensional data whose size is (H, W, C), as illustrated in FIG. 7.

[0097] The generation unit 5 outputs the generated 3 dimensional data to, for example, the element-wise product calculation device (The figure is omitted). The element-wise product calculation device may calculate the element-wise product of the 3 dimensional data U (see FIG. 13), which is input to the SE block, and the 3 dimensional data generated by the generation unit 5 (see FIG. 7), and define the 3 dimensional data obtained by the calculation of the element-wise product as the output data of the SE block.

[0098] According to the present example embodiment, when the first 3 dimensional data is input, the transformation unit 2 transforms the first 3 dimensional data into the second 3 dimensional data. Then, the channel dimension element number increase unit 3 generates 3 dimensional data (the third 3 dimensional data) in which the number of elements of the C dimension in the second 3 dimensional data is increased from 1 to N(=H×W), by performing a convolution layer process with a filter size of 1×1 on the second 3 dimensional data. The transposition unit 4 performs the transposition on the third 3 dimensional data, and the generation unit 5 generates 3 dimensional data whose size is (H, W, C), based on the 3 dimensional data after the transposition.

[0099] Thus, the multidimensional data generation device 1 in the present example embodiment generates the 3 dimensional data (see FIG. 7) which is similar to the aforementioned 3 dimensional data X′ (see FIG. 18), without executing the copy process. Therefore, no overhead is incurred by the copy process. Moreover, in the present example embodiment, the channel dimension element number increase unit 3 performs convolution layer process. The execution speed of the convolution layer process is very fast. Therefore, according to the present example embodiment, when the first 3 dimensional data is given, the 3 dimensional data (see FIG. 7), in which the number of elements of C dimension is same as the first 3 dimensional data, and the number of elements of each dimension other than the C dimension is predetermined, can be generated at high speed.

[0100] In other words, according to the present example embodiment, when the 3 dimensional data in which the number of elements of the C dimension is C and the number of elements of each dimension other than the C dimension (H dimension and W dimension) is 1 is given, the 3 dimensional data in which the number of elements of the C dimension is C and the number of elements of each dimension other than the C dimension (H dimension and W dimension) is predetermined, can be generated at high speed.

[0101] Therefore, by using the multidimensional data generation device 1 in the present example embodiment, the processing speed of the SE block can be increased.

[0102] The 3 dimensional data generated by the multidimensional data generation device 1 of the present example embodiment may not be intended to be used for the calculation of the element-wise product with the 3 dimensional data U (see FIG. 13). In other words, the 3 dimensional data generated by the multidimensional data generation device 1 of the present example embodiment may be applied to techniques other than SE blocks.

[0103] Next, a variation of the example embodiment of the invention will be described.

[0104] The above example embodiment describes the case where the values of the N weights in the convolution layer process with a filter size of 1×1 performed by the channel dimension element number increase unit 3 are “1”. The values of the N weights in the convolution layer process with a filter size 1×1 performed by the channel dimension element number increase unit 3 may be common with a predetermined value other than “1”. Hereafter, this predetermined value is referred to as α.

[0105] That is, the channel dimension element number increase unit 3 generates 3 dimensional data (the third 3 dimensional data) in which the number of elements of the C dimension in the second 3 dimensional data is increased from 1 to N, by performing a convolution layer process with a filter size of 1×1 with a common value α of N weights on the second 3 dimensional data. In this case, the value of each element in the third 3 dimensional data is α times the value of the corresponding element in the third 3 dimensional data in the aforementioned example embodiment.

[0106] Thus, in this case, for example, after generating the 3 dimensional data with size (H, W, C) based on the 3 dimensional data after the transposition, the generation unit 5 may divide the value of each element in the third 3 dimensional data by α. As a result, the same 3 dimensional data as in the aforementioned example embodiment (see FIG. 7) is obtained.

[0107] In the aforementioned example embodiment, the case where the multidimensional data generation device 1 generates 3 dimensional data was shown. The multidimensional data generation device 1 may generate multidimensional data other than 3 dimensional data.

[0108] For example, it is assumed that the multidimensional data generation device 1 generates 4 dimensional data. In this case, 4 dimensional data is input to the multidimensional data generation device 1 as the first multidimensional data. The dimensions in this case are the H dimension, the W dimension, the T dimension, and the C dimension. The H dimension, the W dimension, and the C dimension are the same as the H dimension, the W dimension, and the C dimension in the aforementioned example embodiment. In this case, 4 dimensional data in which the number of elements of the C dimension is C and the size is (1,1,1,C) is input to the multidimensional data generation device 1 as the first multidimensional data.

[0109] The number of elements of the H dimension “H”, the number of elements of the W dimension “W”, and the number of elements of the T dimension “T” in the 4 dimensional data to be generated are predetermined. In this case, N=H×W×T.

[0110] The transformation unit 2 may transform the first multidimensional data into the second multidimensional data in the same way as in the aforementioned example embodiment.

[0111] In this case, 4 dimensional data with size (1, C, 1, N) is generated by the channel dimension element number increase unit 3, for example. Then, the transposition unit 4 performs a transposition, for example, moving p(h, w, t, c) to p(h, c, t, w) to p(h, w, t, c). Note that t is a T dimension coordinate. In this example, as the transposition results, 4 dimensional data with size (1, N, 1, C) is obtained. Then, the generation unit 5 divides the multidimensional data after the transposition by W elements of the W dimension direction, arranges the resulting multidimensional data in the H dimension, divides the multidimensional data by H elements of the H dimension direction, and arranges the resulting multidimensional data in the T dimension. The generation unit 5 can generate 4 dimensional data with size (H, W, T, C) by such a process. However, the process by which the generation unit 5 generates 4 dimensional data with size (H, W, T, C) is not limited to the above example.

[0112] Thus, the multidimensional data generation device 1 can be applied to the generation of multidimensional data other than 3 dimensional data.

[0113] FIG. 10 is a schematic block diagram showing an example of computer configuration of the multidimensional data generation device 1 of the example embodiment of the present invention. The computer 1000 includes a CPU 1001, a main memory 1002, an auxiliary memory 1003, and an interface 1004.

[0114] The multidimensional data generation device 1 of the example embodiment of the present invention is realized by a computer 1000. The operation of the multidimensional data generation device 1 is stored in the auxiliary memory 1003 in the form of a multidimensional data generation program. The CPU 1001 reads the multidimensional data generation program from auxiliary memory 1003 and expands it to the main memory 1002, and executes the process described in the above example embodiment according to the multidimensional data generation program.

[0115] The auxiliary memory 1003 is an example of a non-transitory tangible medium. Other examples of non-transitory tangible media include magnetic disks connected via interface 1004, magneto-optical disks, CD -ROM (Compact Disk Read Only Memory), DVD-ROM (Digital Versatile Disk Read Only Memory), semiconductor memory, etc. When the program is delivered to the computer 1000 through a communication line, the computer 1000 may expand the program in the main memory 1002 and execute the process described in the above example embodiment according to the program.

[0116] Some or all of each of the components may be realized by general-purpose or dedicated circuitry, processor, or a combination of these. These may comprise a single chip or multiple chips connected via a bus. Some or all of each of the components may be realized by a combination of the above-mentioned circuitry, etc. and a program.

[0117] When some or all of each of components is realized by multiple information processing devices, circuits, etc., the multiple information processing devices, circuits, etc. may be centrally located or distributed. For example, the information processing devices and circuits may be realized as a client-and-server system, a cloud computing system, etc., each of which is connected via a communication network.

[0118] The following is an overview of the invention. FIG. 11 is a block diagram showing an overview of the multidimensional data generation device of the present invention. The multidimensional data generation device of the present invention includes transformation means 72, channel dimension element number increase means 73, transposition means 74, and generation means 75.

[0119] The transformation means 72 (e.g., the transformation unit 2) transforms first multidimensional data in which the number of elements of dimension of channel is C and the number of elements of each dimension other than the dimension of channel is 1 into second multidimensional data in which the number of elements of one dimension out of dimensions other than the dimension of channel is C and the number of elements of each dimension other than the one dimension is 1.

[0120] The channel dimension element number increase means 73 (e.g., the channel dimension element number increase unit 3) generates third multidimensional data in which the number of elements of the dimension of channel is increased from 1 to N, by performing a convolution layer process with a filter size of 1×1 with a common value of N weights on the second multidimensional data, when product of predetermined number of elements for each dimension other than the dimension of channel is N.

[0121] The transposition means 74 (e.g., the transposition unit 4) performs predetermined transposition on the third multidimensional data so that the number of elements of the dimension of channel becomes C.

[0122] The generation means 75 (e.g., the generation unit 5) generates multidimensional data in which the number of elements of the dimension of channel is C and the number of elements of each dimension other than the dimension of channel is predetermined number of elements, based on the multidimensional data after the predetermined transposition.

[0123] With such a configuration, it is possible to generate, when given multidimensional data in which the number of elements of the dimension of channel is C and the number of elements of each dimension other than the dimension of channel is 1, multidimensional data in which the number of elements of each dimension other than the dimension of channel is a predetermined number of elements rapidly.

[0124] The channel dimension element number increase means 73 may generate the third multidimensional data, by performing the convolution layer process with a filter size of 1×1 with a common value 1 of N weights on the second multidimensional data.

[0125] the channel dimension element number increase means 73 may generate the third multidimensional data, by performing the convolution layer process with a filter size of 1×1 with a common predetermined value of N weights on the second multidimensional data, and the generation means 75 may divides a value of each element in the multidimensional data by the predetermined value, after generating the multidimensional data.

[0126] The first multidimensional data, the second multidimensional data, the third multidimensional data, the multidimensional data after the predetermined transposition, and the multidimensional data generated by the generation means may be 3 dimensional data.

[0127] Although the present invention has been described above with reference to example embodiment, the present invention is not limited to the above example embodiment. Various changes may be made to the structure and details of the present invention, that may be understood by those skilled in the art within the scope of the present invention.

INDUSTRIAL APPLICABILITY

[0128] The present invention is suitably applied to a multidimensional data generation device that generates multidimensional data.

REFERENCE SIGNS LIST

[0129] 1 Multidimensional data generation device

[0130] 2 Transformation unit

[0131] 3 Channel dimension element number increase unit

[0132] 4 Transposition unit

[0133] 5 Generation unit