ONE-SUB-SYMBOL LINEAR REPAIR SCHEMES
20190158119 ยท 2019-05-23
Inventors
Cpc classification
H03M13/157
ELECTRICITY
H03M13/154
ELECTRICITY
H03M13/373
ELECTRICITY
G06F11/108
PHYSICS
International classification
H03M13/15
ELECTRICITY
Abstract
A method for repairing a single erasure in a Reed Solomon code in a system of a plurality of n storage nodes and a controller, wherein a content of each storage node is a codeword and each node stores a vector v. The method includes identifying a failed storage node; transmitting an index of the failed storage node to each surviving storage node; multiplying the content of each node i by a j-th component of a vector that is a permutation of elements of vector v that correspond to the surviving storage nodes; determining a trace map of the result and converting the result from an mr bit representation into a reduced representation of r bits; reconstructing the content of the failed storage node from the reduced representation of each surviving node's content; and outputting the reconstructed content of the failed storage node.
Claims
1. A method for repairing a single erasure in a Reed-Solomon (RS) code over a finite field F.sub.2.sup.mr with an evaluation set V={u.sub.0=0, u.sub.1, . . . , u.sub.n1} that is a subspace of size n=2.sup.d, for d<mr and divisible by m, wherein a normalized repair bandwidth .sub.i of r bits; reconstructing, by the controller, content of the failed storage node from the reduced representation of each surviving node's content; and outputting the reconstructed content of the failed storage node,
2. The method of claim 1, wherein j is calculated by the controller and transmitted to the surviving storage nodes,
3. The method of claim 1, wherein each surviving storage node calculates j based on an index of each surviving storage node and an index of the failed storage node.
4. The method of claim 1, wherein j is pre-calculated for each storage node.
5. The method of claim 1, wherein multiplying, by each surviving storage node i, the content c.sub.i of each said node i by j-th component of vector w(i) comprises calculating, by each node i, i j.sub.0, wherein j.sub.0 is the identifier of the failed node, out.sub.i:=Tr(w(i).sub.j(i,j.sub.
6. The method of claim 5, wherein converting the result from an in mr bit representation into a reduced representation of r bits comprises, when elements of field F.sub.2.sub.
7. The method of claim 5, wherein converting the result from an in mr bit representation into a reduced representation of r bits comprises, when elements of field F.sub.2.sub.
8. The method of claim 7, wherein matrix M F.sub.2.sup.rmr is r rows of a matrix T F.sub.2.sup.mrmr with indices supporting rr identity matrix in TM.sub.1, wherein M.sub.1 F.sub.2.sup.mrr is a matrix whose j-th column, j=0, . . . , m1 is a length-mr binary representation of .sup.j according to a basis {1, , . . . , .sup.mr1} of field E over field F.sub.2, E:=F.sub.2.sub.
9. The method of claim 1, wherein reconstructing content of the failed storage mode from the reduced representation of each surviving node's content comprises: calculating t.sup.T=(t.sub.1, . . . , t.sub.m).sup.T:=Az.sup.T (F).sup.m using a matrix A (F) .sup.m(n1), wherein F:=F.sub.2[X](p.sub.1(X)), wherein p.sub.1(X) is a minimal polynomial of , wherein vector z:=(z.sub.1, . . . , z.sub.n1) (F).sup.n1 wherein z.sub.a:=.sub.1 for all =1, . . . , n1 and for i j.sub.0 for which j=a, wherein n is a total number of storage nodes and
.sub.i is the reduced representation of surviving: node i's content; wherein matrix A is a matrix obtained by replacing each entry .sub.ij of A by M.sub.ij when .sub.ij is regarded as a column vector in F.sub.2.sup.mr according a basis {1, , . . . , .sup.mr1} of F.sub.2.sub.
10. The method of claim 9, wherein embedding elements of t (F).sup.m in E to obtain a vector w F.sup.m comprises calculating w.sub.:=M.sub.1.Math.t.sub. for =1, . . . , m, wherein each coordinate t.sub.a of t represented as a length-r binary column vector according to the basis 1, , . . . , ().sup.r1 of over F.sub.2, and outputting w:=(w.sub.1, . . . , w.sub.m), wherein each entry of w is a vector representation of an element of E, according to the basis {1, , . . . , .sup.m1}.
11. A non-transitory program storage device readable by a computer, tangibly embodying a program of instructions executed by the computer to perform the method steps for repairing a single erasure in a Reed-Solomon (RS) code over a finite field F.sub.2.sup.mr with an evaluation set V={u.sub.0=0, . . . , u.sub.n1} that is a subspace of size n=2.sup.d, for d<mr and divisible by m, wherein a normalized repair bandwidth .sub.i of r bits; reconstructing, by the controller, content of the failed storage node from the reduced representation of each surviving node's content; and outputting the reconstructed content of the failed storage node.
12. The computer readable program storage device of claim 11, wherein j is calculated by the controller and transmitted to the surviving storage nodes.
13. The computer readable program storage device of claim 11, wherein each surviving storage node calculates j based on an index of each surviving storage node and an index of the failed storage node.
14. The computer readable program storage device of claim 11, wherein j is pre-calculated for each storage node.
15. The computer readable program storage device of claim 11. wherein multiplying, by each surviving storage node i, the content c.sub.i of each said node i by a j-th component of vector w(i) comprises calculating, by each, node i, i k.sub.0, wherein j.sub.0 is the identifier of the failed node; out.sub.i:=Tr(w(i).sub.j(i,j.sub.
16. The computer readable program storage device of claim 15, wherein converting the result from an in mr hit representation into a reduced representation of r bits comprises, when elements of field F.sub.2.sub.
17. The computer readable program storage device of claim 15, wherein converting the result from an mr bit representation into a reduced representation of r bits comprises, when elements of field F.sub.2.sub.
18. The computer readable program storage device of claim 17, wherein matrix M F.sub.2.sup.rmr is r rows of a matrix T F.sub.2.sup.mrmr with indices supporting an rr identity matrix in TM.sub.1, wherein M.sub.1 F.sub.2.sup.mrmr is a matrix whose j-th column, j=0, . . . , m1, is a length-mr binary representation of .sup.j according to a basis {1, , . . . , .sup.mr1} of field E over field F.sub.2, E:=F.sub.2.sub.
19. The computer readable program storage device of claim 11, wherein reconstructing content of the failed storage node from the reduced representation of each surviving node's content comprises: calculating t.sup.T=(t.sub.1, . . . , t.sub.m).sup.T:=Az.sup.T (F).sup.m using a matrix A (F).sup.m(n1), wherein F:=F.sub.2[X]/(X)), wherein p.sub.1 is a minimal polynomial of , wherein vector z:=(z.sub.1, . . . , z.sub.n1) (F).sup.n1 wherein z.sub.:=.sub.i for all =1, . . . , n1 and for i j.sub.0 for which j=, wherein n is a total number of storage nodes and
.sub.i is the reduced representation of surviving node i's content; wherein matrix A is a matrix obtained by replacing each entry of .sub.ij of A by M.sub.ij when .sub.ij regarded as a column vector in F.sub.2.sup.mr according a basis {1, , . . . , .sup.mr1} of F.sub.2.sub.
20. The computer readable program storage device of claim 19, wherein embedding elements of t (F).sup.m in E to obtain a vector w F.sup.m comprises calculating w.sub.:=M.sub.1.Math.t.sub. for =1, . . . , m, wherein each coordinate t.sub.of t is represented as a length-r binary column vector according to the basis 1, , . . . , ().sup.r1 of F over F.sub.2, and outputting w:=(w.sub.1, . . . , w.sub.m), wherein each entry of w is a vector representation of an element of E, according to the basis {1, , . . . , .sup.mr1}.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0028]
[0029]
[0030]
[0031]
DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS
[0032] Exemplary embodiments of the invention as described herein generally provide systems and methods for repairing corrupted Reed-Solomon codes. While embodiments are susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that there is no intent to limit the invention to the particular forms disclosed, but on the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. Herein, when two or more elements are described as being about the same as each other, such as AB, it is to be understood that the elements are identical to each other, indistinguishable from each other, or distinguishable from each other but functionally the same as each other as would be understood by a person having, ordinary skill in the art.
1. Overview
[0033] It was recently shown by V. Guruswami and M. Wootters, Repairing Reed-Solomon. Codes, arXiv:1509.04764v1, the contents of which are herein incorporated by reference in their entirety, hereinafter Guruswami, that if C is a Reed-Solomon code of length n:=q.sup.m over .sub.q.sub.
, and ndim(C)n/q, then any erased coordinate in C can be repaired by transmitting only one
.sub.q-element from each one of the non-erased coordinates. Embodiments of the disclosure consider such simple repair schemes:
.sub.q-linear repair schemes in which repairing a single erased coordinate in an
.sub.q.sub.
.sub.q-symbol from each non-erased coordinate.
[0034] According to embodiments, a necessary and sufficient condition for a code to have a simple repair scheme is provided. A condition according to an embodiment is closely related to the task of finding the dimension of subfield-subcodes. In the case where the code to be repaired is a Reed-Solomon code, a condition according to an embodiment is closely related to the task of calculating the dimension of alterant codes. As a first application of the condition, it is used to re-prove the above result of Guruswami.
[0035] According to embodiments, the case of Reed-Solomon codes over .sub.2.sub.
.sub.2-subspace of
.sub.2.sub.
.sub.2-subspace U of
.sub.2.sub.
Reed-Solomon code defined on U has a simple repair scheme.
[0036] For the special case where d=4, it is shown that for all r2, there exists an appropriate .sub.2-subspace U .Math.
.sub.2.sub.
Reed-Solomon code defined on U has a simple repair scheme. Shortening such a
code, a
shortened Reed-Solomon code is obtained that requires reading a total of only 52 bits for repairing any single erased coordinate. Note that in the best existing repairing scheme for the
Hadoop distributed file system (HDFS) code, repairing a single erased coordinate requires reading a total of 64 or 60 bits, depending on the erased coordinate, from the non-erased coordinates. In addition, the Hitchhiker-nonXOR code of Rashmi et al., A Hitchhiker's Guide to Fast and Efficient Data Reconstruction in Erasure coded Data Centers, SIGCOMM'14, Aug. 17-22, 2014, Chicago, USA, the contents of which are herein incorporated by reference in their entirety, achieves the same saving in repair bandwidth as a code according to embodiments, but only for the 10 information coordinates, and at the cost of coupling pairs of bytes,
[0037] In addition to the above case of of Reed-Solomon codes over .sub.2.sub..sub.2.sub.
with evaluation set U, there is a simple .sub.2.sub.
2. Preliminaries
[0038] Throughout this disclosure, q is a prime power and m .sup.+. If F is a field and M F.sup.rn, for r, n
.sup.+, then ker(M) represents the kernel of the map F.sup.n.fwdarw.F.sup.r defined by x
Mx. If F is a finite field, then for k, n
, an [n, k].sub.F code is an F-linear code of dimension k in F.sup.n.
2,1. The Trace Map
[0039] Definition 2.1: The trace map, Tr=:
.sub.z.sub.
.sub.q, is defined by
where :xx.sup.q is the Frobenius automorphism of
.sub.q.sub.
.sub.1. For x, Y
.sub.q.sub.
.sub.q.sub.
.sub.q.sub.
.sup.+, write
x.Math..sub.Try;=(x.Math..sub.Try.sub.0, . . . , x.Math..sub.Try.sub.n1) .sub.q.sup.n,
and similarly for column vectors,
[0040] Since a polynomial of degree q.sup.m1 cannot have q.sup.m roots, the trace is not identically zero, and hence (x, y)Tr(xy)=x.Math..sub.Try is a non-degenerate symmetric
.sub.q-bilinear form. As usual in this case of finite dimension, the map :x
x.Math..sub.Try) is an isomorphism of
.sub.q-vector spaces between
.sub.q.sub.
[0041] Let b.sub.1, . . . , b.sub.m be a basis of .sub.q.sub.
.sub.q. The elements b.sub.1, . . . , b.sub.m that map under to the dual basis of b.sub.1, . . . , b.sub.m are referred to as the trace-dual basis of b.sub.1, . . . , b.sub.m. By definition, b.sub.1.Math..sub.Trb.sub.j=1.sub.(i=j), so that if x
.sub.q.sub.
.sub.q for all i, then for all i, a.sub.i=x.Math..sub.Trb.sub.i. [0042] Remark 2.2 Note that for all x,y,z
.sub.q.sub.
2.2, Linear Repair Schemes
[0043] Linear repair schemes are defined in Guruswami, incorporated by reference above, as follows. [0044] Definition 2.3 Let n , let C
.sub.q.sub.
.sub.q-linear repair scheme for coordinate i of C comprises: [0045] 1. For all j {0, . . . , n1}\{i}, a set L.sub.j of
.sub.q-linear functionals
.sub.q.sub.
.sub.q; [0046] 2. An
.sub.q-linear function f:
.sub.q.sup..sup.
.sub.q.sub.
The repair bandwidth b.sub.i of the above repair scheme for coordinate i of C is defined as b.sub.i:=log.sub.2(q).Math..sub.ji|L.sub.j|. This is the total number of bits that have to be read from the non-erased coordinates to restore the erased coordinate i. The normalized repair bandwidth
[0047] If for all ji, |L.sub.j|1, then the linear repair scheme for coordinate i is simple.
[0048] So, in a simple linear repair scheme for coordinate i, at most 1 sub-symbol, i.e., a symbol from the subfield .sub.q, has to be read from each non-erased coordinate to restore coordinate i. Hence, the repair bandwidth is not larger than (n1)log.sub.2(q). [0049] Remark 2.4 If a linear code C .Math.
.sub.q.sup.n has a transitive automorphism group, then the existence of a simple
.sub.q-linear repair scheme for some coordinate of C is equivalent to the existence of a simple
.sub.q-linear repair scheme for all coordinates. Recall that the automorphism group of C is the set Aut(C) of all permutations of {0, . . . , n1} such that .Math.C=C, where for c=(c.sub.0, . . . , c.sub.n1) C, the action of defined by .Math.c=(c.sub.(0), . . . , c.sub.(n1)), the group operation on Aut(C) is composition. Aut(C) is called transitive if for all i,j, there exists Aut(C) with (i)=j. More generally, if there exists a linear repair scheme for some coordinate i of C and C has a transitive automorphism group, then the repair scheme can be permuted to become a linear repair scheme for any coordinate in detail, let be an automorphism of C with (i)=i. By re-labeling the coordinates, the task of restoring coordinate i of (c.sub.0, . . . , c.sub.n1) C is transformed to the task of restoring coordinate i of d:=(c.sub.(0), . . . , c.sub.(i), . . . , c.sub.(n1) C. So, using the notation of Definition 2.3, to restore coordinate i, for all ji( (j)(i)=i), node (j) should use the functionals from L.sub.j to calculate the sub-packets it transmits. [0050] Remark 2.5 Suppose that a code C has a simple
.sub.q-linear repair scheme for some coordinate i. Then the same holds also for any subcode Consequently, if C has a simple repair scheme for all coordinates, so does any code obtained by shortening C.
3. The Repairing Theorem
[0051] According to an embodiment of the disclosure, a necessary and sufficient condition for the existence of a simple .sub.q-linear repair scheme is presented. A result according to an embodiment of the disclosure is Theorem 3.2, which specifies a condition in terms of the dimensions of some sub field subcodes.
[0052] For a vector v, write diag(v) for the diagonal matrix with v on its main diagonal. [0053] Lemma 3.1 For n , let C .Math.
.sub.q.sub.
Then the code C has a simple .sub.q-linear repair scheme for the first coordinate if and only if the following condition holds: [0054] Condition (*): There exists a vector v:=(v.sub.1, . . . , v.sub.n1)
.sub.q.sub.
.sub.q.sup.n1 for the code with parity-check matrix H:=Gdiag(v), where C is the
.sub.q-subfield subcode of ker(H), there exists an
.sub.q-linear subcode D .Math. C with [0055] (1) dim(D)=m and [0056] (2) D ker(g.sub.0diag(v))={0}. [0057] Proof: A general codeword of C has the form
c=(c.sub.0, c.sub.1, . . . , c.sub.n1):=(x.sub.0, x.sub.1, . . . , x.sub.k1)G
for some x.sub.0, . . . , x.sub.k1 .sub.q.sub.
A general .sub.q-linear functional of the form v.sub.j .Math..sub.Tr(), for some v.sub.i
.sub.q.sub.
v.sub.j.Math..sub.Trc.sub.j=(v.sub.jg.sub.0j).Math..sub.Trx.sub.0+.sub.i=1.sup.k1(v.sub.jg.sub.ij).Math..sub.Trx.sub.i. (2)
Collecting EQS. (2) for all j (1, . . . , n1):
{v.sub.j.Math..sub.Trc.sub.j}.sub.j=1.sup.n1=x.sub.0.Math..sub.Tr{v.sub.j.sup.g.sub.0j}.sub.j=1.sup.n=1+.sub.i=1.sup.k1x.sub.i.Math..sub.Tr{v.sub.jg.sub.ij}.sub.j=1.sup.n1. (3)
Because .sub.q.sub.
.sub.q.sup.m as an
.sub.q-vector space, the map f of Definition 2.3 might as well have a codomain of
.sub.q.sup.m. So, let A .sub.q.sup.m(n1), and consider the vector
A{v.sub.j.Math..sub.Trc.sub.j}.sub.j=1.sup.n1=x.sub.0.Math..sub.Tr(A{v.sub.jg.sub.0j}.sub.j=1.sup.n1)+.sub.i=1.sup.k1x.sub.i.Math..sub.Tr(A{v.sub.jg.sub.ij}.sub.j=1.sup.n1),
where the equality follows from EQ. (3) and the .sub.q-bilinearity of (-).Math..sub.Tr(-),which implies that, for all i and all l:
[0058] Because the x.sub.i are arbitrary, a necessary condition for reconstruction is that
[0059] (i) i {1, . . . , k1}: A{v.sub.jg.sub.ij}.sub.j=1.sup.n1=0, and
[0060] (ii) The m elements of the vector A{v.sub.jg.sub.0j}.sub.j=1.sup.n1=A(g.sub.0diag(v)).sup.T .sub.q.sub.
.sub.q-linearly independent, that is, these elements form a basis for
.sub.q.sub.
.sub.q.
But (i) and (ii) is clearly also a sufficient condition: given (i), then
A{v.sub.j.Math..sub.Trc.sub.j}.sub.j=1.sup.n1=x.sub.0.Math..sub.Tr(A{v.sub.jg.sub.0j}.sub.j=1 .sup.n1),
and given (ii), the elements of this vector are the coefficients in the description of x.sub.0 as a linear combination of the elements of the dual of the basis in (ii). So, there exists a simple repair scheme for the first coordinate of C if there exist v and A such that conditions (i) and (ii) bold.
[0061] Note that condition (i) is equivalent to
Recalling that H:=G diag(v), it can be seen that condition (i) is equivalent to HA.sup.T=0.
[0062] To, complete the proof, it will be verified that condition (*) there exists v and A such that conditions (i) and (ii) hold.
[0063] Suppose that condition (*) holds. Let the m rows of A be a basis of the code D. Then by the definition of D, it follows that HA.sup.T=0, so that condition (i) holds. For condition (ii), suppose that there exists a row vector t .sub.q.sup.m such that t(A(g.sub.0diag(v)).sup.T)=0. Then A.sup.Tt.sup.T is in D ker(g.sub.0diag (v)), and hence must be the zero vector. Since the rows of A are independent, this implies that t is the zero vector, as required.
[0064] Suppose that there exist v and A such that conditions (i) and (ii) hold. Because of condition (ii), the rows of A must be .sub.q-linearly independent. Let D be the
.sub.q-span of the rows of A. Then dim(D)=m. In addition, since by condition (i) HA.sup.T=0, it holds that D .Math. C. Finally, a vector in D is of the form tA for some row vector t
.sub.q.sup.m. If (t A).sup.T ker(g.sub.0diag (v)), then
(g.sub.0diag(v)A.sup.T)t.sup.T=0.
Because the elements of the vector g.sub.0diag(v)A.sup.T .sub.q-linearly independent by condition (ii), it can be seen that t is the zero vector, as required. [0065] Theorem 3.2 The code C of of Lemma 3.1 has a simple
.sub.q-linear repair scheme for the first coordinate if and only if the following condition holds: [0066] Condition (**): There exists a vector v:=(v.sub.1, . . . , v.sub.n1)
.sub.q.sub.
[0067] Writing C .Math. .sub.q.sup.n1 for the code with parity-check matrix H:=Gdiag(v) and C .Math.
.sub.q.sup.n1 for the code with parity-check matrix
it follows that dim(C)dim(C)m, which is equivalent to dim(C)=dim(C)m. Note that adding one line to a check matrix over .sub.q.sub.
.sub.q can decrease the dimension by at most m. [0068] Proof. It will be shown that Condition (**) of the theorem Condition (*) of Lemma 3.1.
[0069] Suppose that (**) holds. Then it is possible to choose c.sub.1, . . . , c.sub.m .sub.q.sup.n1 that complete a basis of C to a set of dim(C)+m linearly independent elements in C. Let D be the
.sub.q-span of c.sub.1, . . . , c.sub.m). Then by construction D C, and dim(D))=m. Also,
where the last equality follows front the way the basis of D as constructed.
[0070] Suppose that Condition (*) holds. Then
C (C ker(g.sub.0diag(v)))D.
But C ker(g.sub.0diag(v)) is just C, so that C C D. Since dim(D)=m by assumption, it follows that dim(C)dim(C)m.
4. Reed-Solomon Codes of Length q.sup.m over .sub.q.sub.
[0071] As a first example for an application of Theorem 3.2, Theorem 1 of Guruswami, incorporated above, will be re-proven. According to embodiments of the disclosure, from this to point on, a Reed-Solomon code of dimension k over .sub.q.sub.
.sub.q.sub.
.sub.q.sub.
be such that nkn/q, that is, such that kq.sup.mq.sup.m1. Then the Reed-Solomon code of length n and dimension k over
.sub.q.sub.
.sub.q-linear repair scheme for any coordinate. [0073] Proof. First, since a linear repair scheme, for at code is automatically also a linear repair scheme for any subcode, it suffices to consider the case k=q.sup.mq.sup.m1. Also, since the code has a (doubly) transitive automorphism group, it suffices to establish a simple linear repair scheme for the first coordinate (see Remark 2.4, above).
[0074] Now, suppose that k=q.sup.mq.sup.m1, and let .sub.q.sub.
where g.sub.0=1, the all ones row vector of length q.sup.m1.
[0075] For j {1, . . . , q.sup.m31 1}, let v.sub.j:=.sup.(j1). Then H=Gdiag(v) is a check matrix of the cyclic code over .sub.q of length q.sup.m1 and zeros the elements of Z:={1, , . . . , .sup.q.sup.
.sub.q, while
is a check matrix of the cyclic code over .sub.q with zeros the elements of Z:=Z {.sup.1} and their conjugates.
[0076] By Theorem 3.2, it is sufficient to, show that Z does, not contain an element of the orbit of .sup.1 under the action of Gal(.sub.q.sub.
.sub.q)
, and that the orbit of .sup.1 has m elements. In this regard, recall that for a cyclic code C of length n with generator polynomial g, it follows that dim(C)=ndeg(g). Translating to the language of cyclotomic cosets, it should he shown that, the cyclotomic cosets modulo q.sup.m1 the numbers in {0,1, . . . , q.sup.mq.sup.m12} do not include the cyclotomic coset of (1)q.sup.m2, and that this last cyclotomic coset has m elements.
[0077] According to embodiments of the disclosure, it will be convenient to represent numbers in {0, . . . , q.sup.m2} as their length-m base-q expansions. So, for i {0, . . . , q.sup.m2}, write [i].sub.q=(i.sub.m1, i.sub.m=2, . . . , i.sub.0) for its base-q expansion, so that i=.sub.j=0.sup.m1 i.sub.jq.sup.j.
[0078] Now,
[q.sup.m2].sub.q=(q1, q1, . . . , q1, q2). (4)
Since all m cyclic shifts of this vector are distinct, the cyclotomic coset of q.sup.m2 indeed has m elements.
[0079] Also, since q.sup.mq.sup.m11=(q.sup.m1)q.sup.m1, then
[q.sup.mq.sup.m11].sub.q=(q2, q1, . . . , q1, q1). (5)
It follows from EQS. (4) and (5) that q.sup.mq.sup.m11 is the smallest element in the cyclotomic coset of q.sup.m2. Hence this cyclotomic coset does not contain any number in {0,1, . . . , q.sup.mq.sup.m12}.
5. Reed-Solomon Codes on .sub.2-Subspaces of
.sub.2.sub.
[0080] According to embodiments, this section presents simple repair schemes for Reed-Solomon codes defined on an .sub.2-subspace of
.sub.q.sub.
.sub.2-subspace U .Math.
.sub.2.sub.
.sub.
Reed-Solomon code defined on U has a simple repair scheme. Moreover, the probability that a randomly selected subspace results in a simple repair scheme is 1O(1/2.sup.r).
[0081] In general, to check if Condition (**) holds, there is a need for an exact formula for the dimension of subfield subcodes, as lower bounds are insufficient. For this, some results from H. Stichtenoth, On the dimension of Subfield Subcodes, IEEE Trans. Inform. Theory, Vol. 36, No. 1, January 1990, the contents of which are herein incorporated by reference in their entirety, will be used, as follows. In what follows, let operate on vectors coordinate-wise. [0082] Proposition 5.1 (Lemmas 1, 2 and Corollary 1 of Stiehtenoth): For q a prime power and m
, let :x
x.sup.q be the Frobenius automorphism of
.sub.q.sub.
.sub.q. For n
, let D .Math.
.sub.q.sub.
)=dim(D.sup.0) [0084] 2. (D.sup.0).sup.=(D.sup.).sup. [0085] 3. dim(
)=Ndim (D.sup.).sup.
[0086] It will be convenient to state explicitly a special case of Proposition 5.1. [0087] Corollary 5.2 With the notation of Proposition 5.1, suppose that m=2. Then
[0088] Before considering Reed-Solomon codes defined on .sub.2-vector subspaces of
.sub.q.sub.
[0091] From this point on, q is always a power of 2. Turn now to Reed-Solomon codes defined on .sub.2-vector subspaces of
.sub.q.sub.
x+y permutes U , and is therefore an automorphism of the Reed-Solomon code defined on U, for the reason that the variable change X
X+y does not increase the degree of a polynomial.
[0092] Let U be an .sub.2-subspace of
.sub.q.sub.
, ln1, let C=C(U, l) be the Reed-Solomon code obtained by evaluating polynomials of degreel from
.sub.q.sub.
[0093] From this point on, according to embodiments, it will be assumed that the vector v of Theorem 3.2 is defined by v.sub.u:=1/u for all u U\{0}, assuming that coordinates are labeled by elements of U. With this choice of v, the rows of the matrix H of Theorem 3.2 can be considered as a basis for the space of polynomial functions U\{0}.fwdarw..sub.q.sub.
.sub.q.sub.
[0094] Let f.sub.U(X):=.sub.xU (Xx). Then by Theorem 5.4, there are some a.sub.0, . . . , a.sub.d1 .sub.q.sub.
f.sub.U(X)=X.sup.2.sup.
and with a.sub.00 (since f separable by definition). There is an isomorphism of .sub.q.sub.
:.sub.q.sub.
.sub.q.sub.
g+(f.sub.U/X)(x
g(x)).
For a polynomial g(X) .sub.q.sub.
.sub.q.sub.
[0095] Let D .Math. .sub.q.sub.
.sub.
[0096] Similarly, let D .sub.q.sub.
{
[0097] Recalling that f.sub.U(X)/X=X.sup.2.sup.
and
(
This implies the following lemma. [0098] Lemma 5.5 with the above notation, if lb2.sup.d12, then
Spa{
{
and
Spa{
{
, let wt.sub.2(j) be the number of 1's in the binary expansion of j: If j=.sub.i=0.sup.r a.sub.i2.sup.i with a.sub.i {0,1} for all i, then wt.sub.2(j):=.sub.i=0.sup.r a.sub.i. [0101] Lemma 5.7 For all j.sub.1, J.sub.2
, wt.sub.2(j.sub.1+j.sub.2)wt.sub.2(j.sub.1)+wt.sub.2(j.sub.2). [0102] Proof. Suppose first that j.sub.2=2.sup.s for some s. Write j.sub.1=.sub.i=o.sup.r a.sub.i2.sup.i, where
r:=max{([log.sub.2(j.sub.1)],s}+1,
so that r>s and a.sub.r=0. If a.sub.s=0, then clearly wt.sub.2(j.sub.1+j.sub.2)=wt.sub.2(j.sub.1)+wt.sub.2(j.sub.2) Otherwise, let s>s be the smallest such that a.sub.x=0. The single 1 from the binary expansion of j.sub.2 propagates as a carry, so that for all i {s,s+1, . . . , s1}, the i-th digit of the binary expansion of j.sub.i+j.sub.2 is 0, the s-th digit of this binary expansion is 1, and all other digits of this expansion are the same as those of the expansion of j.sub.1. All in all, the total, number of 1's is wt.sub.2(j.sub.1)+1(ss)wt.sub.2(j.sub.1)+wt.sub.2(j.sub.2)(ss)<wt.sub.2(j.sub.1)+wt.sub.2(j.sub.2), as required
[0103] For the general case, suppose that wt.sub.2(j.sub.2)2, and assume the assertion true for all j.sub.2 with smaller weight (note that the case j.sub.2=0 is obvious). Then, write j.sub.2=j.sub.2+j.sub.2 with wt.sub.2(j.sub.2)=1 and wt.sub.2(j.sub.2)=wt.sub.2(j.sub.2)1. Then
as required. [0104] Proposition 5.8 Using the terminology of the paragraphs receding EQ. (8), for all d2, for all j with wt.sub.2(j)d1, and for all s
.
({
.sub.q.sub.
{
for which induction on r will be used. If r<d the assertion is clear. Suppose, therefore, that rd and that there exist b.sub.0, . . . , b.sub.d1 such that
Squaring this equation:
But since f.sub.U(X)=X.sup.2.sup.
[0107] For the induction step, suppose that wt.sub.2(j)2 and the assertion is true for all j with wt.sub.2(j)<wt.sub.2(i). Write j=2.sup.j.sup.
for some .sub.0.sup.(1), . . . , .sub.d1.sup.(1), . . . , .sub.0.sup.(w) .sub.q.sub.
Two cases can be distinguished as follows,
Case I. If there are some l.sub.1, l.sub.2 such that i.sub.l.sub.
By Lemma 5.7, wt.sub.2(.sub.rl.sub.
for some coefficients {.sub.i}, {.sub.k}. This last expression is itself an .sub.q.sub.
Case 2. All the i.sub.l are distinct. Since by assumption w=wt.sub.2(j)d1 and wt.sub.2(2.sup.d1)=d, 2.sup.i.sup..sub.q.sub.
.sub.q.sub.
.sub.q.sub.
where the rightmost implication holds because f.sub.U, and hence also
is separable. Hence
[0110] For the second assertion, suppose that .sub.i=1.sup.r .sub.i.sub.q.sub.
(.sub.i=1.sup.r .sup.1(.sub.i)
so that .sub.i=1.sup.r .sup.1(.sub.i).sub.2-subspace U of
.sub.q.sub.
code. Suppose that the vector v of Theorem 3.2 is given by v.sub.u1/u for all u U\(0). Suppose also that
Then for C of Theorem 3.2, dim(C)2.
Remarks:
[0112] 1. Informally, this proposition states that half of Condition (**) is satisfied for any subspace of dimension d if ll.sub.0. Note that l.sub.0 depends on d, but not on q. For example, for even d and for all q with q.sup.22.sup.d, the proposition shows that half of Condition (**) is satisfied for codes of length n=2 and dimension up to n{square root over (n)} in .sub.q.sub.
Reed-Solomon codes.
[0113] 2. Note that for odd d, the codimension is the same as that for d+1. Hence, the parameters of an odd d can be achieved by shortening a code designed for the even dimension d+1. Therefore, apart from the current proposition, only even d will be considered.
[0114] 3. Note that for a choice according to an embodiment of v, there do exist some subspaces U .Math. .sub.q.sub.
.sub.q
.sub.q.sub.
.sub.q in the check matrix cannot decrease the
.sub.q-dimension by more than 1. Similarly, the condition does not hold for subspaces of the form z.Math.
.sub.q for z
*.sub.q.sub.
.sub.q.sub.
.sub.
where dim(D.sup..Math.)=l has been used, and recalling that that
dim(D.sup..Math. (D.sup..Math.))2l+32.sup.d.
In view of EQ. (8), it should be shown that
dim(Spa{
{
In other words, it should be shown that
Spa{
contains 2l+32.sup.d independent elements that can be written as linear combinations of .sub.q.sub.
[0116] Write
A:=|{i {0, . . . , l1}|wt.sub.2(i)>wt.sub.2(l1)}|
and
B:=|{i {l, . . . , 2.sup.d2}|wt.sub.2(i)wt.sub.3(l1)}|.
It follows from Proposition 5.8 that for each of the lA values of i in {0, . . . , l1} with wt.sub.2(i)wt.sub.2(l1), (
[0117] Theorem 5.11 Let d4 be even. Suppose that q=2.sup.r and that d elements x.sub.0, . . . , x.sub.d1 are drawn uniformly and independently from .sub.q.sub.
.sub.
code. Suppose that the vector v, is given by v.sub.u=1/u for all u U\{0}. Then the probability that both dim(U)=d and Condition (**) holds is at least
[0118] and is positive for rd1+log.sub.2(d). [0119] Remark 5.12
[0120] 1. The case of r=d/2 is a special case of Theorem 4.1, above.
[0121] 2. Some examples (see below) seem to suggest that the result should hold for all rd/2, and that the probability of drawing a good subspace U is much higher than in the theorem.
[0122] 3. For d=4, the theorem proves the existence of a good subspace for all r5, and Theorem 4.1 covers the case r=2. Since the two remaining cases of r=3, 4 are considered in the examples below, it can be seen that for d=4, a good subspace exists for all rd/2.
[0123] Before proving the theorem, some auxiliary results are needed. [0124] Proposition 5.13 Using the terminology of Proposition 5.10 and continue to assume that the vector v is given by v.sub.u=1/u for all u U\{0}. Suppose that l12.sup.d12, which holds for l=l.sub.0 if d2. Then Condition (**) for the [2.sup.d, l+ Reed-Solomon code defined on U is equivalent to the following condition: For all .sub.1, .sub.2
.sub.q.sub.
.sub.1{
or, equivalently, to
{
and
({
Since D.sup..Math.D.sup..Math.+Spa{v}, then
D.sup..Math.+(D.sup..Math.)=D.sup..Math.+(D.sup..Math.)+Spa{v,(v)}.
Hence Condition (**) holds iff for .sub.1, .sub.2 .sub.q.sub.
.sub.1v+.sub.2(v) D.Math.+(D.sup..Math.) .Math. .sub.1=.sub.2=0.
Applying the isomorphism .sup.1, it can be seen that Condition (**) holds iff for .sub.1, .sub.2 .sub.q.sub.
.sub.1{
which, from Lemma 5.5, is equivalent to
.sub.1{
This concludes the proof [0126] Definition 5.14 (The numbers i.sub.j) Recall that that d is assumed even, and note that the numbers i {0, . . . , 2.sup.d2) with wt.sub.2(i)=d1 are those numbers i of the form 2.sup.d12.sup.j with j {0, . . . , d1}. Hence, there are d such numbers, and d/21 of them are <l.sub.0=2.sup.d2.sup.d/21, namely, those with j {d/2+1, . . . , d1}. For j in the above range, write i.sub.j:=2.sup.d12.sup.d1j, so that
2.sup.d11=i.sub.0<i.sub.1< . . . <i.sub.d12.sup.d2
are all the numbers I {0, . . . , 2.sup.d2} with wt.sub.2(I)=d1, and i.sub.0, . . . , i.sub.d/22 are all the numbers i {0, . . . , l.sub.01} with wt.sub.2(i)=d1. [0127] Definition 5.15: (The matrices M.sub.r {tilde over (M)}.sub.r) for r0 and for i, j {0, . . . , 2.sup.d2}, define coefficients .sub.i,j.sup.(r) .sub.q.sub.
.sub.q.sub.
##STR00001##
Note that each column of M.sub.r includes some of the coefficients corresponding to (
[0128] Define also a matrix {tilde over (M)}.sub.r .sub.q.sub.
[0131] (i) .sub.q.sub.
[0132] (ii) to the projection of (.sub.q.sub.
[0133] Consider (i) above. By Proposition 5.8, if .sub.q.sub.
[0134] Similarly, by Proposition 5.8 if the projection of (
[0135] To continue, parametrize the problem. Instead of working in .sub.q.sub.
.sub.2[X, T.sub.0, . . . , T.sub.d1] for free variables T.sub.0, . . . , T.sub.d1. For this, let T:=(T.sub.0, . . . , T.sub.d1) be a vector of free variables over
.sub.2[X], and let f (X, T)
.sub.2[X, T] be defined by
f(X, T):=X.sup.2.sup.
Re-define (g.sub.2[X, T].fwdarw.
.sub.2[X, T]/(f/X). Since
.sub.2[T] .Math.
.sub.2[X, T].fwdarw.
.sub.2[X, T]/(f/X) is an injection, which follows from deg.sub.X(uv)=deg.sub.X(u)+deg.sub.X(v),
.sub.2[T] can be thought of as a subring of
.sub.2[X, T]/(f/X). In addition,
.sub.2[X, T]/(f/X) is a free
.sub.2[T]-module with basis
.sub.2[X, T]/(f/X) as an
.sub.2[T]-module, since the coefficient of X.sup.2.sup.
[0136] The idea is that for any finite dimensional .sub.2-subspace U of any extension field K of
.sub.2 (e.g., K=
.sub.q.sub.
.sub.r and
.sub.r). For integer r, define matrices
.sub.r
.sub.2[T].sup.dd/2 and
.sub.r
.sub.2[T].sup.d/2d/2 as follows. The matrix
.sub.r is defined as the right-band side of EQ. (18), where now .sub.i,j.sup.(r)
.sub.2[T] are the unique polynomials in (
.sub.r is defined as the last d/2 rows of
.sub.r. Note that by Lemma B.1, if for an
.sub.2-subspace U
.sub.q.sub.
.sub.q.sub.
.sub.r(a.sub.0, . . . , a.sub.d1) and {tilde over (M)}.sub.r=
.sub.r(a.sub.0, . . . , a.sub.d1). Consequently, det({tilde over (M)}.sub.r)=(det(
.sub.r)) (a.sub.0, . . . , a.sub.d1).
[0138] Next, it will be shown that for rd/2, det(.sub.r) is not the zero polynomial in
.sub.2[T]. To this end, begin with the following lemma. [0139] Lemma 5.18: (Update Lemma) Write
For all i and r, and for the i.sub.j's of Definition 5.14, the .sub.i,j.sup.(r)=.sub.i,j.sup.(r)(T) of Definition 5.17 satisfy
From Proposition 5.8, only j's with wt.sub.2(j)=d1 can contribute to .sub.i,k.sup.(r+1) for k With wt.sub.2(k)=d1. In the sum S.sub.1, there is only such j, namely 2.sup.d11. In the sum S.sub.2, the relevant j's are those of the form j=2.sup.d12.sup.j for j {0,1, . . . , d2}, from Definition 5.14, Hence, the partial sum
S.sub.3:=(.sub.i,2.sub.
includes all contributions to the .sub.i,k.sup.(r+1) for k with wt.sub.2(k)=d1.
[0141] Now
Substitute is S.SUB.3 .to get
[0142]
S.sub.3:=(.sub.i,2.sub.
Once again, only those exponents with wt.sub.2(.Math.)=d1 can contribute to .sub.i,k.sup.(r+1) for k with wt.sub.2(k)=d1 and should be kept. For j+1 {1, . . . , d1}, the binary representation of 2.sup.d22.sup.j+1=(2.sup.d1)12.sup.j+1 has exactly two zeros (the coefficients of 2.sup.0 and 2.sup.j+1), and so wt.sub.2(2.sup.l+2.sup.d22.sup.j+1)=d1 iff l {0, j+1}. It follows that all contributions to the .sub.i,k.sup.(r+1) for k with wt.sub.2(k)=d1 appear in
Recalling that 2.sup.d12.sup.j=i.sub.d1j and using a variable change j.Math.d1j in the last sum, the above expression reads
This concludes the proof. [0143] Definition 5.19: (The Matrices A.sub.i and .sub.2A). For the matrix A of Lemma 5.18, let A.sub.1 .sub.2[T].sup.dd/2 include the first d/2 columns of A, and let .sub.2A
.sub.2[T].sup.d/2d include the last d/2 rows of A. Explicitly,
in particular, using 0,1, . . . , to number rows, rows number d/21, . . . , d2 are zero, and
.sub.2[T], it holds that for all j
, (M.sub.1M.sub.2).sup.2.sup.
and for r1,
.sub.r=A.Math.A.sup.2.Math.A.sup.4 . . . A.sup.2.sup.
a total of r multiplied matrices. Consequently.
.sub.r=.sub.2A.Math.A.sup.2.Math.A.sup.4 . . . A.sup.2.sup.
.sub.1=A.sub.1,
.sub.2=A.Math.A.sub.1.sup.2,
.sub.3=A.Math.A.sup.2.Math.A.sub.1.sup.4. [0148] Proof of Lemma 5.23: EQ. (20) follows directly from the definition of
.sub.r, and then EQ. (24) follows from Lemma 5.18 by induction. Finally, EQ. (22) follows from EQ. (21) and the definition of
.sub.r as the last d/2 rows of
.sub.r.
[0149] It can now be proven that det(.sub.r) is not the zero polynomial. [0150] Lemma 5.24: For even d4 and rd/2, det(
.sub.r)
.sub.2[T] is a non-zero polynomial of total
.sub.r) is not the zero polynomial. For the basis, consider the case r=d/2. Note that B:=A(1,0, . . . , 0) represents the cyclic shift left operator. Hence, if d/23,
N:=B.sup.2 . . . B.sup.2.sup.
represents the cyclic shift left by d/22 operator. Therefore,
N:=N.Math.A.sub.1(1,0, . . . , 0).sup.2.sup.
is given by
It follows that .sub.d/2(1,0, . . . , 0)=.sub.2 A(1,0, . . . , 0)N=I.sub.d/2, and
(det(.sub.d/2)) (1,0, . . . , 0)=1. (23)
For d/2=2, where .sub.d/2=.sub.2A.Math.A.sub.1.sup.2, it can be verified directly that
.sub.d/2(1,0, . . . , 0)=I.sub.d/2, so that EQ. (23) holds also for this case. This shows that det(
.sub.d/2) 0
.sub.2[T] and establishes the induction basis.
[0152] For the induction step, assume that rd/2+1 and the assertion is true for r1. Write .sub.2A:=.sub.2 A/T.sub.0 .sub.2.sup.d/2d, and note that .sub.2A=(.sub.2A).sup.2. When by EQ. (22),
Now, using 0,1, . . . to index rows:
Cyclically shifting the rows results in the matrix
Write .sub.3A for the matrix in EQ. (25), and let
.sub.r:=.sub.3 A.Math.A.sup.2.Math.A.sup.2.sup.
It follows from EQ. (24) that det(.sub.r) 0 iff det(
.sub.r) 0. By the
.sub.2[T]-linearity of the determinant in the first row, it is shown that
Note that the following fact was used: for rows r.sub.0, r.sub.0, r.sub.1, r.sub.2, . . . and for a matrix. M,
[0153] Note also that the first determinant in the above sum equals det(.sub.r1)/T.sub.0, and is therefore non-zero by the induction hypothesis. In addition, the first summand (T.sub.d/2.Math.det( . . . )) is the only one in which T.sub.d/2 may appear in an odd degree, and as the first determinant is non-zero and T.sub.d/2 appears in all its monomials in an even degree, the first summand must contain monomials whose T.sub.d/2-degree is odd. As such monomials cannot be canceled by monomials in the second determinant, whose T.sub.d/2-degree is essentially even, it can be concluded that det(
.sub.r) 0, so that det(
.sub.r) 0. This completes the proof that det(
.sub.r) 0 for all rd/2.
[0154] As for the assertion regarding the total degree, note first that combining EQ. (22), the fact that all polynomials in A are either 1 or some T.sub.j, and the additivity of the total degree, it can be seen that the total degree of each entry in .sub.r is at most 1+2+ . . . +2.sup.r1=2.sup.r1. As the determinant of a d/2d/2 matrix is a sum of products of d/2 entries, it follows that tot. deg(
.sub.r)(d/2) (2.sup.r1), as required.
[0155] Theorem 5.11 can now be proven. [0156] Proof of Theorem 5.11: Take some x.sub.0, . . . , x.sub.d1 .sub.q.sub.
.sub.2-linearly independent can be expressed as g(x.sub.0, . . . , x.sub.d1) 0 for
g(X.sub.0, . . . , X.sub.d1):=(X.sub.0+X.sub.1) (X.sub.0+X.sub.2) . . . (X.sub.d2+X.sub.d1) . . .(X.sub.0+ . . . +X.sub.d1),
a product of 2.sup.dd1 linear polynomials. In addition, assuming that x.sub.0, . . . , x.sub.d1 are , .sub.2-linearly independent and writing U for the subspace that they span, the coefficients a.sub.0, . . . , a.sub.d1 of f.sub.U=.sub.xU (X+x)=X.sup.2.sup.
.sub.2[X.sub.0, . . . , X.sub.d1]. Substituting these polynomials from
.sub.2[X.sub.0, . . . , X.sub.d1] for the T.sub.i in det(
.sub.r) with r=log.sub.2(q), results in a polynomial h
.sub.2[X.sub.0, . . . , X.sub.d1] of total degree at most
where EQ. (26) follows from Lemma 5.24 and the fact that for a field K, a polynomial u K[Y.sub.0, . . . , Y.sub.n.sub.
tot. deg(u(v.sub.0, . . . , v.sub.n.sub.
To see the validity of the previous equation, take a monomial Y.sub.0.sup.i.sup.
[0157] Write X:=(X.sub.0, . . . , X.sub.d1). According to an embodiment, it should be shown that (gh) (X) has a non-zero in .sub.q.sub.
, and
), and if S .Math. K is a finite set, then the number of (y.sub.0, . . . , y.sub.m1) S.sup.m for which f (y.sub.0, . . . , y.sub.m1)=0 is at most .Math.|S|.sup.m1. However, it first needs to be shown that gh 0. Since g(X) is obviously non-zero, it only needs to be shown that h(X) 0. If h(X) is the zero polynomial in
.sub.2[X], then for any extension field K of
.sub.2 and for any d-dimensional
.sub.2-subspace U of K , the result of substituting the .sub.i coming from .sub.xU (Xx)=X.sup.2.sup.
.sub.r) is 0 K. Suppose that |K|=Q where Q2.sup.d is some power of 2. Noting that distinct subspaces result in distinct vectors (a.sub.0, . . . , a.sub.d1), this means that the non-zero polynomial det(
.sub.r) has at least
zeroes. However, since .sub.r 0 and tot. deg(det(
.sub.r))d/2) (q1), it follows from the Schwartz-Zippel lemma that the number of zeros of det(
.sub.r) in K.sup.d is at most Q.sup.d1(d/2)(q1), which is in O(Q.sup.d1) for fixed d and q. As N.sub.1=O(Q.sup.d), a contradiction is obtained for large enough Q. This shows, that h(X) 0.
[0158] As gh .sub.2[X] is a non-zero polynomial and
it follows from the Schwartz-Zippel lemma that the probability that a randomly drawn point (x.sub.0, . . . , x.sub.d1) from (.sub.q.sub.
For fixed d, p>0 if
Note that since L is monotonically decreasing with q while R is monotonically increasing with q, if L(2.sup.r)<R(2.sup.r) for some r, then the same is true for all rr.
[0159] Now, for even d4 and r=log.sub.2(d2.sup.d1), it follows that
as required.
6. The Case d=4 and Additional Examples
[0160] For d=4, it follows that l.sub.0=11, and the only integers j2.sup.d2=14 with wt.sub.2(j)=d1=3 are j=7, 11, 13, 14, of which only 7 is <l.sub.0. Hence, by definition
recalling that a.sub.0 0 is the free coefficient of f.sub.U/X, where (*) follows from Lemma 5.18 by noting that by definition, .sub.2i,j.sup.(r)=.sub.i,j.sup.(r+1), as (.sub.
.sub.2-linearized polynomial f(X):=X.sup.16+X.sup.4+X
.sub.2[X]. It can be verified by direct substitution that this polynomial splits in
.sub.2.sub.
.sub.q.sub.
.sub.2-subspace. It can also be verified directly that
(
so that with the above notation, .sub.13=0, .sub.11 0, and .sub.14 0, and it follows that .sub.13.sup.3+.sub.11.sub.14.sup.2 0. Hence, Condition (**) holds, and the [16,12].sub. Reed-Solomon code defined on U has a simple repair scheme. [0162] Example 6.2 Consider the case q=2.sup.r with r=4, represent
.sub.q.sub.
.sub.2.sub.
.sub.2[X]/(X.sup.8+X.sup.4+X.sup.3+X.sup.2+1), and let be the image of X. It can be verified that is a primitive element of
.sub.2.sub.
.sub.2-subspace U spanned by {1, , .sup.2, .sup.3} and the Reed-Solomon code C defined on U. Using any computer algebra software, it can be verified that .sub.11=.sup.249, .sub.13=.sup.171, .sub.14=.sup.176, and .sub.11.sub.14.sup.2+.sub.13.sup.3.sup.431 0. Hence, C has a simple repair scheme. Note that an alterative way to see this is to calculate the dimensions involved in Condition (**) directly, say, by Gaussian elimination.
[0163] C can be shortened to obtain a [14, shortened Reed-Solomon code. For this shortened code, repairing any single erased coordinate requires reading ,a total of only 13.Math.4=52 bits from the remaining coordinates. Note that in the best existing repairing scheme for the [14,
Facebook FUNS code, repairing a single erased coordinate requires reading a total of 64 or 60 bits, depending on the erased coordinate, from the non-erased coordinates. [0164] Example 6.3 The cases of q=2.sup.r with r {5,6,7} can be considered in a similar way. The following table presents for each r {5,6,7} the minimal polynomial of a primitive element of
.sub.2.sub.
.sub.2-subspace spanned by {1, , .sup.2, .sup.3}. It can be seen that .sub.11.sub.14.sup.2+.sub.13.sup.3 0 in all cases, and hence the Reed-Solomon code defined on U has a simple repair scheme,
TABLE-US-00001 minimal polynomial .sub.11.sub.14.sup.2 + r of .sub.11 .sub.13 .sub.14 .sub.13.sup.3 parameters 5 X.sup.10 + X.sup.3 + 1 .sup.699 .sup.872 .sup.583 .sup.431 [16, 6 X.sup.12 + X.sup.6 + .sup.2119 .sup.2676 .sup.1516 .sup.2855 [16, X.sup.4 + X + 1
7 X.sup.14 + X.sup.10 + .sup.12908 .sup.9078 .sup.15325 .sup.11665 [16, X.sup.6 + X + 1
[0165] Example 6.4 This example considers the case of dimension d=6. Let q=2.sup.4, so that q.sup.2=2.sup.8, and let U be
.sub.2-subspace spanned by 1, , . . , .sup.5, where a is as in Example 6.2. Then it can be verified by direct Gaussian elimination that Condition (**) holds for l=l.sub.0=55. This gives a [64,
Reed-Solomon code with a simple
.sub.2.sub.
7.Extensions of Degree>2
[0166] While the proofs given in the previous sections consider only the case of extensions .sub.2.sub.
.sub.2.sub.
.sub.2.sub.
.sub.2.sub.
.sub.2-subspaces U .Math.
.sub.2.sub.
.sub.2.sub.
Recall that in such a case, if a single coordinate is erased, then each of the non-erased cc ordinates has to transmit only 1/m of its content to reconstruct the erased coordinate. [0167] Example 7.1. Let the field .sub.2.sub.
.sub.2[X]/X.sup.12+X.sup.6+X.sup.4+X+1), and let be the image of X. Then it can be verified that is a primitive element. Let U .Math.
.sub.2.sub.
.sub.2-span of {1, , . . . , .sup.7}. Then d:=dim(U)=8. Consider the subfield
.sub.2.sub.
.sub.2.sub.
.sub.2.sub.
.sub.23]=4. Then it can be verified by Gaussian elimination that Condition (**) holds for the RS code of dimension
defined on U. This gives a [256,192].sub.
.sub.2.sub.
.sub.2.sub.
.sub.2.sub.
.sub.2.sub.
.sub.2.sub.
.sub.2.sub.
.sub.2-span of {1, , . . . , .sup.5}, with d:=dim(U)=6. Then it can be verified by Gaussian elimination that Condition (**) holds for the RS code of dimension
defined on U. This gives a [64, RS code with a simple
.sub.2.sub.
.sub.2.sub.
.sub.2[X]/(X.sup.9+X.sup.4+1), and let a he the image of X. Then it can be verified that is a primitive element. Let U
.sub.2.sub.
.sub.2-span of {1, , . . . , .sup.5}. Then d:=dim(U)=6. Consider the subfield
.sub.2.sub.
.sub.2.sub.
defined on U. This gives a [64, RS code with a simple
.sub.2.sub.
8. Low-Bandwidth Repair Algorithm For Reed-Solomon (RS) Codes
[0170] A repair algorithm according n an embodiment of the disclosure includes two parts: an offline pre-processing part, and an online part. In the offline part, the various blocks used by the online part are prepared, In the online part, data is encoded using the RS code constructed in the offline part, and erasures are decoded if necessary.
[0171] If there is a single erasure, then, using the blocks constructed in the of part, the single erased coordinate can, be corrected with a low repair bandwidth, in that only a small, part of the content of the non-erased coordinates is transmitted to reconstruct the erased coordinate, where small part will be defined below.
[0172] If there are two or more erasures, then standard erasure-decoding techniques for RS codes are used, and there is no gain in repair bandwidth. However, since single erasures are much more common in practice than multiple erasures, there is it in supporting low-bandwidth repairs for the case of a single erasure.
[0173] In what follows, for a field E to be defined below, 1:=(1, . . . , 1) E.sup.n1 is the vector of n1 ones, 0:=(0, . . . , 0) E.sup.n1 is the vector of n1 zeros, and (.Math.).sup.T stands for vector and matrix transposition. In addition, for a field L, a subset I .Math. L, and a positive integer k|l|, the RS code with evaluation set I and dimension k is defined as the set of length-|l| vectors in L.sup.|l| obtained by evaluating all polynomials f L[X] of degree up to k1 on the points of I. Explicitly, letting I={x.sub.1, . . . , x.sub.n}, with n:=|I|, then the above RS code can be defined as
{(f(x.sub.1), . . . , f(x.sub.n))|f L[X], deg(f)k1{.
In addition, a stripe according to an embodiment can have up to
symbols of information, and
symbols of parity, where d, m are as defined above. For example, when d=4 and m=2, there can be up to 12 symbols of information, and 4 symbols of parity.
Offline Part
[0174] The inputs of an offline part of an algorithm according to an embodiment are as follows, [0175] Integers r1, m2 defining finite fields F:=F.sub.2.sub.
[0181] Note that lengths 2.sup.d1<n.sub.0<2.sup.d can be obtained by shortening For simplicity, embodiments will only consider the case of n.sub.0=n=2.sup.d.
[0182] The outputs of an offline part of an algorithm according to an embodiment are as follows. [0183] An F.sub.2-vector subspace V .Math. E with dim.sub.F.sub.
[0190] With the inputs and outputs defined as above, an offline part of an, algorithm according to an embodiment is as follows, with reference to the steps of the flowchart of
[0210] Step 111: Let B.sub.F/E={x.sub.1, . . . , x.sub.m} E be the dual basis of {y.sub.1, . . . , y.sub.m} with respect to the trace map E.fwdarw.F, and output B.sub.F/E. The trace tap Tr:E.fwdarw.F, defined above in Def. 2.1, can equivalently be defined by u.sub.i=0.sup.m1 u.sup.|f|.sup.
[0215] Step 115: Obtain a matrix of the form (I.sub.k|G.sub.1) from matrix G E.sup.kn of Step 105, where G.sub.1 E.sup.k(nk). According to an embodiment, such a matrix can be obtained by e.g., Gaussian elimination. Output G.sub.1.
Online Part
[0216] According to embodiments, during online operation, incoming messages are encoded and stored in individual unreliable storage nodes. When it is found that one or more nodes have failed and its data is erased, erasure-correction algorithms are used to restore the erased data. In the common case of a single node failure, algorithms according to embodiments of the disclosure can be used. Otherwise, if more than one node fails, standard erasure-correction methods for RS codes can be used.
[0217] According to embodiments, a systematic encoding algorithm is presented, because it is the one most likely to be used in practice. Note however, that single-erasure-decoding algorithms according to other embodiments are not limited to systematic encoding, and any encoding for the code C.sub.RS with generator matrix (I.sub.k|G.sub.1) may be used. For example the generator matrix G from Step 105 of
i. Systematic Encoding [0218] Inputs: Row data vector x E.sup.k. [0219] Output: A codeword y C.sub.RS E.sup.n corresponding to x. [0220] Algorithm: Output y=(y.sub.0, . . . , y.sub.n1):=(x|xG.sub.1) where G.sub.1 was obtained in an offline stage according to an embodiment.
ii. Single-Erasure Decoding
[0221] According to an embodiments, it is assumed that the entries y.sub.j of y are stored in n different storage nodes, labeled 0, . . . , n1. If exactly one node j.sub.0 fails, a following algorithm according to an embodiment can be used for restoring y.sub.j.sub.
[0222] The inputs of an online part of a single erasure decoding algorithm according to an embodiment are the failed node index, j.sub.0. It should be noted that U can learn the failed node index j.sub.0, by, e.g., periodically scanning nodes and checking their status.
[0223] The outputs of an online part of a single-erasure decoding algorithm according to an embodiment are a restored value for y.sub.j.sub.
[0224] Each storage node can store a different vector of length n1, call it w(i) for node i. The j-th entry of w(i) is used by node i in the process of reconstructing the j.sub.0-th failed node in a predefined enumeration of all nodes but i. In such a setup, the controller can send surviving node i the index j.sub.0 of the failed node, and node i then multiplies its content by w(i).sub.j for
and then takes the trace of the result, converts the result to a compact representation and transmits the compact representation to the controller. The vector w(i) (F.sub.2.sub.
[0225] With the inputs and outputs defined as above, an online part of an algorithm according to an embodiment is as follows, with reference to the steps of the flowchart of .sub.i. According to embodiments, them are several possible ways for doing this, for example: [0231] 1. If elements of F.sub.2.sub.
.sub.i:= [0232] 2. If elements of F.sub.2.sub.
.sub.i:=M.Math.out.sub.i. According to an embodiment, this option will be assumed hereinafter, however, embodiments are not limited thereto. [0233] Each surviving node i, i j.sub.0, then returns its extracted part
.sub.i to U. Hence, each surviving node transmits only r bits instead of the total stored mr bits. [0234] Step 24: U reconstructs the erased symbol y.sub.j.sub.
.sub.i for the unique i j.sub.0 for which j=a, and defines z:=(z.sub.1, . . . , z.sub.n1) (F).sup.n1. Recall that j was defined above in Step 21. (Step 31) [0236] 2. U calculates t.sup.T=(t.sub.1, . . . , t.sub.m).sup.T:=Az.sup.T (F).sup.m using the matrix A (F).sup.m(n1) stored in an offline stage according to an embodiment. (Step 32) [0237] 3. U embeds the elements of t (F).sup.m in E to obtain a vector w F.sup.m. According to an embodiment, one possible way to do this is to use the matrix M.sub.1 F.sub.2.sup.mrr calculated in an offline stage according to an embodiment as follows. If each coordinate t.sub.a of t is represented as a length-r binary column vector according, to the basis 1, , . . . , ().sup.r1 of F over F.sub.2, then U can calculate w.sub.a:=M.sub.1.Math.t.sub.a for a=1, . . . , m and output w:=(w.sub.1, . . . , w.sub.m). Each entry of w is then a vector representation of an element of E, according to the basis {1, , . . . , .sup.mr1}. (Step 33) [0238] 4. Finally, U restores the erased symbol by calculating y.sub.i=.sub.=1.sup.m w.sub.x.sub. (multiplication and addition of E), recalling that B.sub.F/E={x.sub.1, . . . , x.sub.m} E was stored in an offline stage according to an embodiment. (Step 34) [0239] Step 25: U outputs the reconstructed data symbol y.sub.i.
[0240] Note that it is only possible to re-store the value in node j.sub.0 if the failed node had a temporary failure. According to embodiments, if the node is considered lost, then the reconstructed data can be stored on a different node. According to other embodiments, the reconstructed data is not re-stored at all, but is rather re-calculated (erasure decoding) and sent to the user each time the user wishes to access the data stored on the failed node. In such a case, each time the user wishes to access the data, much more data than the required lost block has to be read from the non-failed nodes for reconstruction.
[0241] Alternatively, according to other embodiments of the disclosure, the overall action of steps 22 and 23 can be represented by an rmr binary matrix V.sub.j F.sub.2.sup.rmr that can be pre-computed, depending on the basis of F.sub.2.sup.mr over F.sub.2 used for representing elements. In detail, one can pre-compute overall matrices V.sub.1, . . . , V.sub.n1 F.sub.2.sup.rmr, and instead of the current steps 22 and 23, use a single combined step in which node j multiplies its content, regarded as a length-mr binary vector according to the above basis, by the binary matrix V.sub.j to obtain the required length-r binary vector .sub.i.
[0242] In other embodiments of the disclosure instead of storing the vector v and using its j-th entry, v.sub.j, for the j defined by Step 21, above, it is possible to use a different method, in which node i stores a different vector w(i), depending on i, and uses its j=j(i,j.sub.0)-th entry, for
In an alternative embodiment, all nodes store the vector v. When node j.sub.0 fails, the controller tells surviving node i, for all i j.sub.0: node j.sub.0 failed, calculate j by yourself. Then, each surviving node calculates j, depending on its own index i and the failed node index j.sub.0, by using the above formula, and, as before, uses w(i).sub.j=v.sub.j. In another alternative embodiment, instead of letting node i calculate j, the right permutation of v is pre-calculated and stored: its first entry is the entry that has to be used when node 0 fails, . . . its (i1)-th entry is the entry that has to be used when node i1 fails, its i-th entry is the entry that has to be used when node i+1 fails, etc. Explicitly, define, for all j.sub.0 {0, . . . , i1, i+1, . . . , n}, w(i).sub.j(i,j.sub.
[0243] There are n=2.sup.d different vectors w(i), one for each node. These vectors are pre-computed in the offline stage, and stored in the respective n nodes, i.e., w(i) is stored on node i for all i. In some cases, this may be a favorable method, because it avoids the online permutation calculation by both the controller and the surviving nodes.
[0244] It is to be understood, however, that the above scenario, in which an algorithm according to an embodiment is performed by a controller U and of the n1 surviving nodes, is exemplary and non-limiting, and the situation can differ in actual practice. It is possible, for example, that one node coordinates the encoding and reconstruction processes but does not actually performs them, or performs only part of the encoding/reconstruction.
[0245] For example, in Hadoop File System (HDFS), developed by the Apache Software Foundation, which is a distributed file system there is a module called HDFS RAID (RAID stands for Redundant Array of independent Disks, and is typically, related to erasure coding) that works roughly as follows. In HDFS RAID, there is a node called RaidNode that coordinates the encoding and decoding related to the erasure code, but need not actually perform them.
[0246] For encoding the RaidNode periodically scans all paths that should be erasure encoded, and selects large enough files that have not been recently modified. Once it selects a source file, it iterates over all its stipes and creates the appropriate number of parity blocks for each stripe. The actual calculation of the parity blocks may be performed locally at the RaidNode (LocalRaidNode), or distributed to other machines (DistributedRaidNode). The LocalRaidNode configuration is similar to what is described in the present disclosure.
[0247] For one case of decoding, BlockFixer, a program that runs at the RaidNode, periodically inspects the health of all relevant paths. When a file with missing or corrupt blocks is encountered, these blocks are decoded after a (parallel) read of a sufficient amount of data from the surviving nodes. Once again, the actual computation of the missing blocks may be performed either locally at the RaidNode (LocalBlockFixer), or distributed to one or more other machines (DistBlockFixed). Embodiments of the disclosure can be incorporated into the distributed case.
[0248] Note that the above decoding is a result of a periodic scan. Recall that there is also a case where the user (client) itself finds out that there are erasures when trying to read a file. In HDFS RAID, the client implements the DRFS (Distributed RAID File System) client software, which can locally reconstruct such erased blocks, and acts as U. In this case, the client does not re-store the reconstructed blocks: these reconstructed blocks are sent to higher layers and discarded. In this case, the client has to re-calculate the missing blocks every time the file is read.
9 . System Implementations
[0249] It is to be understood that embodiments of the present disclosure can be implemented in various forms of hardware, software, firmware, special purpose processes, or a combination thereof. In one embodiment, the present disclosure can be implemented in hardware as an application-specific integrated circuit (ASIC), or as a field programmable gate array (FPGA). In another embodiment, the present disclosure can be implemented in software as an application program tangible embodied on a computer readable program storage device. The application program can be uploaded to, and executed by, a machine comprising any suitable architecture.
[0250]
[0251] The computer system 41 also includes an operating system and micro instruction code. The various processes and functions described herein can either be part of the micro instruction code or part of the application program (or combination thereof) which is executed via the operating system. In addition, various other peripheral devices can be connected to the computer platform such as an additional data storage device and a printing device.
[0252] It is to be further understood that, because some of the constituent system components and method steps depicted in the accompanying figures can be implemented in software, the actual connections between the systems components (or the process steps) may differ depending upon the manner in which the present invention is programmed. Given the teachings of the present invention provided herein, one of ordinary skill in the related art will be able to contemplate these and similar implementations or configurations of the present invention.
[0253] While the present invention has been described in detail with reference to exemplary embodiments, those skilled in the art will appreciate that various modifications and substitutions can be made thereto without departing from the spirit and scope of the invention as set forth in the appended claims.
APPENDIX
A. A Calculation for the Proof of Proposition 5.10
[0254]
it should be shown that
2.sup.d3l.sub.0A+B. (28)
[0255] Suppose first that d is even. Then the binary expansion of l.sub.01=2.sup.d2.sup.d/22 is, with the MSB on the left:
TABLE-US-00002 d 1 d 2 . . . d/2 + 1 d/2 d/2 1 d/2 2 . . . 1 0 1 1 . . . 1 0 1 1 . . . 1 0
In particular, wt.sub.2(l.sub.01)=d2.
[0256] Note that the usual ordering of the integers corresponds to the lexicographic ordering of the binary expansions. Hence an expansion represents a number kl.sub.01 with wt.sub.2(k)=d1, which is the only possibility of a weight>wt.sub.2(l.sub.01) for a numberl.sub.01, iff the single 0 in its binary expansion is to the left of the 0 in position d/2 in the expansion of l.sub.01. Hence, there are exactly d/21 such numbers k, so that. A=d/21.
[0257] Similarly, a number k is strictly larger than l.sub.01 iff its binary expansion has 1 in the first coordinate from the left in which it differs from the binary expansion of l.sub.01. Since only numbers k with wt.sub.2(k)wt.sub.2(l.sub.01) are of interest, k=l.sub.0 is excluded, and the remaining numbers are exactly the numbers k with binary expansion of the form
TABLE-US-00003 d 1 d 2 . . . d/2 + 1 d/2 d/2 1 d/2 2 . . . 0 1 1 . . . 1 1 ? ? . . . ?
[0258] where the sub-vector with question marks may be any binary vector of length d/2 and weightd2d/2=d/22. The total number of such vectors is B=2.sup.d/2d/21. There are now explicit expressions for both A and B, and it is straightforward to verify that EQ. (28) holds with equality.
[0259] The case where d is odd is similar: now the binary expansion of l.sub.01 is
TABLE-US-00004 (d + 1)/ (d + 1)/ (d 1)/ (d 1)/ d 1 d 2 . . . 2 + 1 2 2 2 1 . . . 1 0 1 1 . . . 1 0 1 1 . . . 1 0
[0260] The numbers A and B are determined as above: there are A=(d1)/21 ways to place a single zero to the left of the zero at index (d+1)/2 of the binary expansion of l.sub.01, and there are B=2.sup.(d+1)/2(d+1)/21 binary vectors of length (d+1)/2 with weightd2(d1)/2=(d+1)/22. So now the left-hand side of EQ. (28) equals
2.sup.d3(2.sup.d2.sup.(d+1)/21)=2.sup.(d+1)/22,
while the right-hand side equals A+B=2.sup.(d+1)/23.
B. A Lemma on Substitution
[0261] Lemma B.1 For a vector T:=(T.sub.0, . . . , T.sub.d1) of free variables distinct from X, let
f(X,T):=X.sup.2.sup..sub.2[X,T].
Let K be an extension field of .sub.2, let a.sub.0, . . . , a.sub.d1 K, and write a:=(a.sub.0, . . . , a.sub.d1). Then the map on the dashed arrow at the bottom of the following diagram is a well-defined homomorphism of rings, and the unique homomorphism making the diagram commutative.
##STR00002## [0262] Proof. Let A and B be commutative, unital rings and let a .Math. A and b .Math. B be ideals. Let :A.fwdarw.B be a homomorphism with (a) .Math. b. Consider the following diagram:
##STR00003##
[0263] Since (a) .Math. b, the kernel of the left vertical arrow, which is surjective, is contained in the kernel of the path .fwdarw.. Hence there exists a unique dashed homomorphism making the square commutative, and this homomorphism is given by a+a(a)+b. Now the assertion of the lemma follows as a special case. [0264] Remark B.2 Lemma B.1 is implicitly used in the main text in the following way. Instead of calculating directly the image of (X.sup.2.sup.
.sub.2[X] for X=(X.sub.0, . . . , X.sub.d1) are substituted for the T.sub.i, and then x.sub.0, . . . , x.sub.d1
.sub.q.sub.
.sub.2 (e.g., K=
.sub.q.sub.
##STR00004##