Image processing for diffuse-specular separation

Abstract

A method of image processing includes receiving a set of images of an object, the set of images including images corresponding to binary spherical gradient illumination along first, second and third mutually orthogonal axes and images corresponding to complementary binary spherical gradient illumination along the first, second and third axes. The method also includes determining a specular reflectance map of the object and a diffuse reflectance map of the object based on the set of images, and/or determining a diffuse photometric normal map and a specular photometric normal map of the object based on the set of images.

Claims

1. A method of image processing, comprising: receiving a set of images of an object, the set of images including images obtained using a camera without a polarising filter and corresponding to binary spherical gradient illumination along first, second and third mutually orthogonal axes and images corresponding to complementary binary spherical gradient illumination along the first, second and third axes; determining a specular reflectance map of the object and a diffuse reflectance map of the object based on the set of images; wherein each binary spherical gradient illumination corresponds to uniform illumination from a hemisphere, and the respective complementary binary spherical gradient illumination corresponds to uniform illumination from an opposite hemisphere.

2. A method according to claim 1, further comprising: receiving a viewpoint vector corresponding to a camera used to obtain the set of images; generating a set of local images by using a rotation matrix to rotate the illumination conditions of the set of images from the first, second and third mutually orthogonal axes to a set of rotated mutually orthogonal axes defined with respect to the viewpoint vector; determining a specular reflectance map of the object and a diffuse reflectance map of the object based on the set of local images.

3. A method according to claim 2, comprising receiving multiple sets of images, each set of images corresponding to a different viewpoint vector, and for each set of images: generating a set of local images by using a rotation matrix to rotate the set of images from the first, second and third mutually orthogonal axes to a set of rotated mutually orthogonal axes defined with respect to the respective viewpoint vector; determining a specular reflectance map of the object and a diffuse reflectance map of the object based on the set of local images.

4. A method according to claim 1, wherein the set of images comprises six images obtained using white light illumination and corresponding to: a first image of the object illuminated using binary spherical gradient illumination along the first axis; a second image of the object illuminated using the complement of the binary spherical gradient illumination along the first axis; a third image of the object illuminated using binary spherical gradient illumination along the second axis; a fourth image of the object illuminated using the complement of the binary spherical gradient illumination along the second axis; a fifth image of the object illuminated using binary spherical gradient illumination along the third axis; and a sixth image of the object illuminated using the complement of the binary spherical gradient illumination along the third axis.

5. A method according to claim 4, wherein determining a specular reflectance map of the object and a diffuse reflectance map of the object based on the set of images comprises, for each pixel: determining a first estimate of specular reflectance based on a difference in color saturation between the first and second images; determining a second estimate of specular reflectance based on a difference in color saturation between the third and fourth images; determining a third estimate of specular reflectance based on a difference in color saturation between the fifth and sixth images; and determining a specular reflectance for the pixel based on the first, second and third estimates.

6. A method according to claim 4, wherein the first to sixth images include data corresponding to two or more color channels, and wherein determining a specular reflectance map of the object and a diffuse reflectance map of the object based on the set of images comprises, for each pixel: determining an estimate of a diffuse normal; for one or more color channels of the two or more color channels: determining a first estimated diffuse reflectance value and a first estimated specular reflectance value by solving a linear system of equations relating the pixel values of the first and second images and a component of the estimated diffuse normal parallel to the first axis; determining a second estimated diffuse reflectance value and a second estimated specular reflectance value by solving a linear system of equations relating the pixel values of the third and fourth images and a component of the estimated diffuse normal parallel to the second axis; determining a third estimated diffuse reflectance value and a third estimated specular reflectance value by solving a linear system of equations relating the pixel values of the fifth and sixth images and a component of the estimated diffuse normal parallel to the third axis; determining the specular reflectance based on the first, second and third estimated specular reflectance values for the one or more color channels; and determining the diffuse reflection map by subtracting the specular reflectance map from a mixed reflectance map.

7. A method according to claim 1, wherein the set of images comprises: a seventh image of the object illuminated: along the first axis using either a first binary spherical gradient illumination or its complement, encoded in a first color channel; along the second axis using either a second binary spherical gradient illumination or its complement, encoded in a second color channel; along the third axis using either a third binary spherical gradient illumination or its complement, encoded in a third color channel; an eighth image of the object illuminated: along the first axis using the other of the first binary spherical gradient illumination and its complement, encoded in the first color channel; along the second axis using the other of the second binary spherical gradient illumination and its complement, encoded in the second color channel; along the third axis using the other of the third binary spherical gradient illumination and its complement, encoded in the third color channel.

8. A method according to claim 7, wherein determining a specular reflectance map of the object and a diffuse reflectance map of the object based on the set of images comprises, for each pixel: determining an estimate of a diffuse photometric normal; determining a first estimated diffuse reflectance value and a first estimated specular reflectance value by solving a linear system of equations relating the pixel values of the seventh and eighth images for the first color channel and a corresponding component of the estimated diffuse photometric normal parallel to the first axis; determining a second estimated diffuse reflectance value and a second estimated specular reflectance value by solving a linear system of equations relating the pixel values of the seventh and eighth images for the second color channel and a corresponding component of the estimated diffuse photometric normal parallel to the second axis; determining a third estimated diffuse reflectance value and a third estimated specular reflectance value by solving a linear system of equations relating the pixel values of the seventh and eighth images for the third color channel and a corresponding component of the estimated diffuse photometric normal parallel to the third axis; determining the specular reflectance based on the first, second and third estimated specular reflectance values, and determining the diffuse reflection map by subtracting the specular reflectance map from a mixed reflectance map.

9. A method according to claim 1, further comprising determining a diffuse photometric normal map and a specular photometric normal map of the object based on the set of images.

10. A method of image processing, comprising: receiving a set of images of an object, the set of images obtained using a camera without a polarising filter and including images corresponding to binary spherical gradient illumination along first, second and third mutually orthogonal axes and images corresponding to complementary binary spherical gradient illumination along the first, second and third axes; determining a diffuse photometric normal map and a specular photometric normal map of the object based on the set of images; wherein each binary spherical gradient illumination corresponds to uniform illumination from a hemisphere, and the respective complementary binary spherical gradient illumination corresponds to uniform illumination from an opposite hemisphere.

11. A method according to claim 10, further comprising: receiving a viewpoint vector corresponding to a camera used to obtain the set of images; generating a set of local images by using a rotation vector to rotate the illumination conditions of the set of images from the first, second and third mutually orthogonal axes to a set of rotated mutually orthogonal axes defined with respect to the viewpoint vector; determining a diffuse photometric normal map and a specular photometric normal map of the object based on the set of local images.

12. A method according to claim 11, comprising receiving multiple sets of images, each set of images corresponding to different viewpoint vector, and for each set of images: generating a set of local images by using a rotation vector to rotate the illumination conditions of each of the set of images from the first, second and third mutually orthogonal axes to a set of rotated mutually orthogonal axes defined with respect to the respective viewpoint vector; determining a diffuse photometric normal map and a specular photometric normal map of the object based on the set of local images.

13. A method according to claim 10, wherein the set of images comprises six images obtained using white light illumination and corresponding to: a first image of the object illuminated using binary spherical gradient illumination along the first axis; a second image of the object illuminated using the complement of the binary spherical gradient illumination along the first axis; a third image of the object illuminated using binary spherical gradient illumination along the second axis; a fourth image of the object illuminated using the complement of the binary spherical gradient illumination along the second axis; a fifth image of the object illuminated using binary spherical gradient illumination along the third axis; and a sixth image of the object illuminated using the complement of the binary spherical gradient illumination along the third axis.

14. A method according to claim 13, wherein determining a diffuse normal map and a specular normal map of the object based on the set of images comprises: determining a mixed photometric normal map; determining the diffuse photometric normal map; for each pixel: determining the specular photometric normal based on modelling the mixed photometric normal as a spatially-varying, a-priori unknown weighted sum of the determined diffuse photometric normal and an unknown specular photometric normal, and solving for the specular photometric normal by first determining the unknown weights of summation.

15. A method according to claim 14, wherein determining the diffuse photometric normal map comprises: transforming the first to sixth images into an suv color space; for each pixel, determining the diffuse photometric normal based on the u and v values of the first to sixth images in suv color space.

16. A method according to claim 14, wherein the first to sixth images include data corresponding to two or more color channels, and wherein determining the diffuse photometric normal map comprises: for each pixel, determining the diffuse photometric normal based on the values of the first to sixth images corresponding to a brightest color channel and a darkest color channel.

17. A method according to claim 10, wherein the set of images comprises: a seventh image of the object illuminated: along the first axis using either a first binary spherical gradient illumination or its complement, encoded in a first color channel; along the second axis using either a second binary spherical gradient illumination or its complement, encoded in a second color channel; along the third axis using either a third binary spherical gradient illumination or its complement, encoded in a third color channel; an eighth image of the object illuminated: along the first axis using the other of the first binary spherical gradient illumination and its complement, encoded in the first color channel; along the second axis using the other of the second binary spherical gradient illumination and its complement, encoded in the second color channel; along the third axis using the other of the third binary spherical gradient illumination and its complement, encoded in the third color channel.

18. A method according to claim 17, wherein determining a diffuse normal map and a specular normal map of the object based on the set of images comprises: determining a mixed photometric normal map; for each pixel: determining a component of a diffuse photometric normal parallel to the first axis based on the darkest pixel value of the seventh and eighth images in the first color channel; determining a component of a diffuse photometric normal parallel to the second axis based on the darkest pixel value of the seventh and eighth images in the second color channel; determining a component of a diffuse photometric normal parallel to the third axis based on the darkest pixel value of the seventh and eighth images in the third color channel; determining the specular photometric normal map based on subtracting the diffuse photometric map from the mixed photometric map.

19. A method according to claim 10, further comprising determining a specular reflectance map of the object and a diffuse reflectance map of the object based on the set of images.

20. A method comprising: illuminating an object: along the first axis using either a first binary spherical gradient illumination or its complement, encoded in a first color channel; along the second axis using either a second binary spherical gradient illumination or its complement, encoded in a second color channel; along the third axis using either a third binary spherical gradient illumination or its complement, encoded in a third color channel; acquiring a first image of the object using an image capturing device without a polarising filter; illuminating the object: along the first axis using the other of the first binary spherical gradient illumination and its complement, encoded in the first color channel; along the second axis using the other of the second binary spherical gradient illumination and its complement, encoded in the second color channel; along the third axis using the other of the third binary spherical gradient illumination and its complement, encoded in the third color channel; acquiring a second image of the object using the image capturing device; wherein each binary spherical gradient illumination corresponds to uniform illumination from a hemisphere, and the respective complementary binary spherical gradient illumination corresponds to uniform illumination from an opposite hemisphere.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

(2) Certain embodiments of the present invention will now be described, by way of example, with reference to the accompanying drawings, in which:

(3) FIGS. 1(a) to 1(f) present photographs obtained using white light binary spherical gradients and their complements;

(4) FIGS. 1(g) to 1(j) respectively present diffuse albedo, specular albedo, diffuse photometric normals and specular photometric normals;

(5) FIG. 1(k) presents a computer graphics rendering generated using the data shown in FIGS. 1(g) to 1(j);

(6) FIG. 2 illustrates an LED sphere used to provide white light binary spherical gradients and their complements;

(7) FIGS. 3(a) to 3(c) present a comparison of mixed albedo, specular albedo and diffuse albedo of an image;

(8) FIGS. 4(a) to 4(f) present a comparison of mixed photometric normals calculated using red, green and blue color channels;

(9) FIGS. 5(a) to 5(c) respectively present a comparison of diffuse photometric normals, initially estimated specular photometric normals, and synthesized specular photometric normals;

(10) FIGS. 6(a) and 6(b) present a comparison of specular photometric normals estimated for a simulated sphere and a real orange;

(11) FIGS. 7(a) and (7(b) present examples of images encoding spectrally multiplexed binary spherical RGB gradients;

(12) FIGS. 8(a) and 8(b) present a comparison of mixed albedo and mixed photometric normals obtained from images encoding spectrally multiplexed binary spherical RGB gradients;

(13) FIGS. 9(a) and 9(b) present a comparison of blurred diffuse photometric normal and synthesized photometric normals approximating the specular photometric normal;

(14) FIGS. 10(a) and 10(b) present separated specular reflectance and diffuse reflectance obtained from images encoding spectrally multiplexed binary spherical RGB gradients;

(15) FIGS. 11(a) to 11(d) respectively present diffuse reflectance, specular reflectance and photometric normals for a ball, comparing binary spherical gradient illumination with polarized linear spherical gradient illumination;

(16) FIGS. 11(e) to 11(h) respectively present diffuse reflectance, specular reflectance and photometric normals for a human subject, comparing binary spherical gradient illumination with polarized linear spherical gradient illumination;

(17) FIGS. 12(a) and 12(b) present a comparison of a hybrid computer graphics rendering in panel (a) and a photograph in panel (b); and

(18) FIGS. 13(a) to 13(c) respectively present a comparison between mixed photometric normal, diffuse photometric normals obtained from rotation to a suv color space, and diffuse photometric normals obtained by subtracting the darkest color channel from the brightest color channel.

DETAILED DESCRIPTION OF CERTAIN EMBODIMENTS

(19) In the following, like parts are denoted by like reference numerals.

(20) The inventors have developed a novel variant active illumination method for acquiring separated diffuse and specular reflectance, and additionally separated diffuse and specular photometric normals, using binary spherical gradient illumination. Similar to previous work (see Background section), the present specification relates to using an LED sphere to illuminate an object, for example a person, with binary spherical gradients and their complements. However, unlike previous work, the present specification employs unpolarized illumination and unpolarized imaging. Instead, in the present specification, an inherent property of the binary spherical gradients is recognized and utilised to enable diffuse-specular separation of acquired albedo and photometric normals.

(21) In this respect, the methods of the present specification may use a simpler setup (compared to previous work), which does not require polarizing multiple (usually hundreds of) light sources, or any polarizers in the optical path of the sensor. The methods of the present specification may also require fewer photographs. Advantageously, the methods of the present specification do not impose any restrictions on the acquisition viewpoint for diffuse-specular separation.

(22) Referring to FIG. 1, panels (a) to (f) show acquired images of a face lit with binary spherical gradient illumination and their complements. Panels (g) and (h) show estimated diffuse and specular albedo obtained according to methods of the present specification. Panels (i) and (j) show diffuse and specular photometric normal maps obtained from the acquired data using methods according to the present specification. The obtained normal maps and albedo data enables realistic relighting with hybrid normal rendering, as shown in panel (k). Details of how to obtain images as shown in FIG. 1 shall be apparent from the description hereinafter.

(23) In addition to achieving albedo separation using a novel color-space analysis on the binary spherical gradients, the present specification also demonstrates separation of photometric normals including robust estimation of a specular normal map from the acquired data (as shown in FIG. 1) and estimation of a pure diffuse normal map. This result of the methods according to the present specification is somewhat surprising given the binary nature of the incident illumination for specular reflectance.

(24) This specification also concerns methods for using spectral multiplexing in conjunction with binary spherical gradient illumination to enable a very efficient two-shot acquisition approach, with potential applications in, for example, dynamic capture applications.

(25) Although the unpolarized binary spherical gradient illumination employed is similar to Vlasic et al. (VPB-09), the present specification goes beyond the disclosure of Vlasic et al. (VPB-09) by describing diffuse-specular separation of albedo and surface normals suitable for dielectric materials such as skin.

(26) Image Acquisition

(27) In the following description, examples will be illustrated using images obtained using an LED sphere including 168 white LED lamps (ColorKinetics iW MR gen3) mounted on a 2.5 m diameter spherical structure. In general, any light sources that allow simple binary on-off control can be employed with the methods of the present specification, because the binary spherical gradients used do not require control of grey-level intensity of light emission.

(28) Referring also to FIG. 2, panel (a) shows an acquisition setup consisting of an LED sphere 1. Panels (b) through (g) respectively show binary spherical gradient illumination conditions termed X, X*, Y, Y*, Z and Z* as imaged on a mirror ball.

(29) The lamps are mounted on the LED sphere 1 structure along longitudes and latitudes. This arrangement helps to simplify partitioning of the lights for each of the binary spherical gradient illumination conditions X, Y, Z and complements X*, Y*, Z*. Exactly half of the lights, 84 in this example, are used for each lighting condition X, Y, Z, X*, Y*, Z*. Partitioning along two orthogonal longitudes is used for the X and Z binary spherical gradients and their complements X*, Z*, and partition along the equatorial latitude is used for the Y binary spherical gradient and its complement Y.

(30) The binary spherical gradient illumination conditions X, Y, Z and the complements X, Y, Z* may be further explained with reference to a system of spherical polar coordinates (r, , ). On an illumination sphere with constant radius r (in the example r=2.5 m), the binary spherical gradient X along a first, x-axis may correspond to 3o illumination of the LEDs within the region of the LED sphere 1 defined by the coordinate ranges 0 and 0<. Similarly, the complement X* along the first, x-axis may correspond to illumination of the LEDs within the region of the LED sphere 1 defined by the coordinate ranges 0 and <2.

(31) The binary spherical gradient Y illumination along a second, y-axis may correspond to illumination of the LEDs within the region of the LED sphere 1 defined by the coordinate ranges 0</2 and 0<2. The complement Y* along the second, y-axis may correspond to illumination of the LEDs within the region of the LED sphere 1 defined by the coordinate ranges /2 and 0<2. The reference to x- and y-axes is relative to x- and y-axes of an image acquired looking towards the coordinate system origin along the direction =/2, =0.

(32) The binary spherical gradient Z illumination along a third, z-axis may correspond to of the LEDs within the region of the LED sphere 1 defined by the coordinate ranges 0, 0</2, and 3/2<2 of the illumination sphere (equivalently 0, and /2</2 if is defined on <). The complement Z* along the third, z-axis may correspond to illumination of the LEDs within the region of the LED sphere 1 defined by the coordinate ranges 0, and /2<3/2 of the illumination. The reference to the z-axis is relative to distance towards/away from a viewpoint for an image acquired looking towards the coordinate system origin along the direction =/2, =0.

(33) In a first set of methods according to the present specification, six images of an object or subject are rapidly acquired, one under each of the binary spherical gradients X, Y, Z and the complements X*, Y*, Z*. The images were acquired using a consumer DSLR camera (Canon EOS 750D with a 18-135 mm lens) operating in burst mode. This camera was used to obtain the exemplary images shown in the figures. However, any suitable digital camera or equivalent image capturing apparatus may be used to obtain the images. Burst mode acquisition is not essential. Provided that the object being imaged is not moving, a delay between acquiring images may be seconds, minutes or longer.

(34) The LED sphere 1 includes an additional set of 168 RGB LED lamps (ColorKinetics iColor MR gen3), which are mounted adjacent to the white LED lamps. In a second set of methods according to the present specification, the RGB LED lamps may be used for spectral multiplexing of the binary spherical gradients X, Y, Z and the complements X*, Y*, Z* to enable a two-image acquisition of sufficient information to permit albedo separation and also calculation of diffuse and specular normals.

(35) Although examples described in this specification relate to images obtained using the hereinbefore described LED sphere 1, the methods of the present specification are equally applicable to binary spherical gradient images obtained using a LED sphere of arbitrary configuration. In particular, the precise number and distribution of LEDs is not important, provided that the binary spherical gradients X, Y, Z and complements X*, Y*, Z* may be provided.

(36) Although the examples described in this specification relate to images obtained using an LED sphere 1 including multiple, discrete LED lamps, the methods of the present specification are equally applicable to any other type of illumination arrangement which is capable of providing the binary spherical gradients X, Y, Z and complements X*, Y*, Z*. A continuous illumination sphere capable of generating the binary spherical gradients X, Y, Z and complements X*, Y*, Z* would be suitable for use with the methods of the present specification.

(37) Although the binary spherical gradients X, Y, Z and complements X*, Y*, Z* have been described in relation to a particular coordinate system, this is non-limiting and purely in the interests of clarity of the explanations hereinafter. The binary spherical gradients X, Y, Z and complements X*, Y*, Z* may be expressed in any suitable coordinate system(s).

(38) Although we refer to binary spherical gradients X, Y, Z and complements X*, Y*, Z*, the illumination conditions X*, Y*, Z* are also binary spherical gradients, to which the illumination conditions X, Y, Z are complementary.

(39) First Method of Albedo Separation

(40) First to sixth images of an object or subject (e.g. a person) are acquired corresponding to the binary spherical gradients X, Y, Z and the complements X*, Y*, Z*. Separation of albedo (reflectance) under uniform illumination into diffuse and specular reflectance may be formulated according to a first method according to the present specification.

(41) A first image I.sub.X corresponds to the binary spherical gradient X along the first, x-axis. If the first image I.sub.X is an N by M image, then an individual pixel of the first image I.sub.X may be denoted by I.sub.X(n,m), with n a first index 1nN and m a second index 1nM. In general, a pixel may include red, R, green, G and blue, B values, which may be denoted for the first image as R.sub.X(n,m), G.sub.X(n,m) and B.sub.X(n,m) respectively.

(42) Similarly, a second image I.sub.X corresponding to the complement X* may have pixel values denoted as I.sub.X*(n,m) or for the RGB components, R.sub.X*(n,m), G.sub.X*(n,m) and B.sub.X*(n,m). A third image I.sub.Y corresponding to the binary spherical gradient Y may have pixel values denoted as I.sub.Y(n,m), or for the RGB components, R.sub.Y(n,m), G.sub.Y(n,m) and B.sub.Y(n,m). A fourth image I.sub.Y* corresponding to the complement Y* may have pixel values denoted as I.sub.Z(n,m) or for the RGB components, R.sub.Y*(n,m), G.sub.Y*(n,m) and B.sub.Y*(n,m). A fifth image I.sub.Z corresponding to the binary spherical gradient Z may have pixel values denoted as I.sub.Z(n,m), or for the RGB components, R.sub.Z*(n,m), G.sub.Z*(n,m) and B.sub.Z*(n,m). A sixth image I.sub.Z* corresponding to the complement Z* may have pixel values denoted as I.sub.Z*(n,m) or for the RGB components, R.sub.Z*(n,m), G.sub.Z*(n,m) and B.sub.Z*(n,m).

(43) It should be noted that due to the nature of the binary spherical gradient X, Y, Z and complement X, Y, Z lighting conditions, the sum of images corresponding to any two complementary lighting conditions is equivalent to an image I.sub.U obtained using a uniform illumination condition in which all LEDs of the LED sphere 1 are turned on. Any one on of the combinations I.sub.U=I.sub.X+I.sub.X*, I.sub.U=I.sub.Y+I.sub.Y*, or I.sub.U=I.sub.Z+I.sub.Z* may provide a resulting image I.sub.U corresponding to uniform illumination. Unless otherwise stated, operations indicated to be performed on images should be presumed to be carried out pixelwise, i.e. I.sub.X+I.sub.X* corresponds to calculating I.sub.U(n,m)=I.sub.X(n,m)+I.sub.X*(n,m) for each combination of n and m. A uniform image I.sub.U may provide an estimate of the mixed albedo containing both the diffuse reflectance .sub.d and the specular reflectance .sub.s.

(44) The first method is based, at least in part, on the inventor's observation that using illumination according to the binary spherical gradients X, Y, Z and complements X*, Y*, Z*, a surface on a convex dielectric object exhibits pure diffuse reflectance when lit with the dark (off) hemisphere, while exhibiting both diffuse and specular reflectance when lit with the bright (on) hemisphere of the binary spherical gradients X, Y, Z and complements X*, Y*, Z*.

(45) Without wishing to be bound by theory, this is believed to be due to the wide hemispherical integration lobe of diffuse reflectance (centered around the surface normal) which continues to integrate some light from the bright hemisphere of a binary spherical gradient X, Y, Z or complement X*, Y*, Z* even when a surface faces away towards the dark hemisphere, while receiving no light only when facing exactly in the direction of the dark side. On the other hand, typical specular reflectance on a dielectric object is believed to have a much narrower lobe of reflection (centered around the reflection vector) which integrates light only when facing the bright side of a binary spherical gradient X, Y, Z or complement X*, Y*, Z*.

(46) This key observation is utilised herein to formulate a color-space separation for the albedo of an imaged object or subject under uniform illumination. In particular, for colored dielectric materials, specular reflectance .sub.s is white in color when a surface is lit with white light. Consequently, when a colored dielectric surface (of an imaged object or subject) is lit with a binary spherical gradient X, Y, Z and its complement X*, Y*, Z* the reflected RGB color of that surface is more saturated in the darker (pure diffuse) lighting condition while exhibiting reduced saturation under the brighter (diffuse plus specular) lighting condition due to mixing with some white reflection. As an example, a point on an object surface facing towards to the illuminated half in the binary spherical gradient X will be relatively more saturated in color (although darker) when not directly illuminated under the complement X.

(47) The RGB color values of the first to sixth images I.sub.X, I.sub.X*, I.sub.Y, I.sub.Y*, I.sub.Z, I.sub.Z* are converted into HSV space (representing Hue, Saturation, and Value: although the following color-space analysis also holds for other similar color-spaces such as HSI or HSL). For example, for the first, x-axis, the RGB values of the binary spherical gradient X image I.sub.X={R.sub.X(n,m), G.sub.X(n,m), B.sub.X(n,m)} are converted into equivalent HSV values H.sub.X(n,m), S.sub.X(n,m), V.sub.X(n,m), and the RGB values of the complement X* image I.sub.X*={R.sub.X*(n,m), G.sub.X*(n,m), B.sub.X*(n,m)} are converted into equivalent HSVvalues H.sub.X*(n,m), S.sub.X*(n,m), V.sub.X*(n,m).

(48) Considering a generic pixel n, m corresponding to an imaged object in relation to a binary spherical gradient and complement, for example the binary spherical gradient X and complement X. In one of the corresponding images I.sub.X, I.sub.X* the pixel n, m will face the illuminated half of the LED sphere 1, whilst in the other of the images I.sub.X, I.sub.X* the pixel n, m will not be directly illuminated. In other words, every pixel n, m is a light pixel I.sub.L=(H.sub.L, S.sub.L, V.sub.L) in one of the images I.sub.X, I.sub.X* and a dark pixel I.sub.D=(H.sub.D, S.sub.D, V.sub.D) in the other image I.sub.X, I.sub.X*.

(49) As an example, if S.sub.X(n,m)<S.sub.X*(n,m) then I.sub.L=I.sub.X and I.sub.D=I.sub.X*, in other words the pixel of the pair having the lower saturation is the brighter, light pixel I.sub.L. Conversely, if S.sub.X(n,m)>S.sub.X*(n,m) then I.sub.L=I.sub.X* and I.sub.D=I.sub.X. Although explained in relation to the first binary spherical gradient X and first complement X*, the pairing of the second binary spherical gradient Y and second complement Y* or the pair of the third binary spherical gradient Z and third complement Z* may be treated in the same way.

(50) As discussed hereinbefore, the inventors have appreciated that the dark pixel (H.sub.D, S.sub.D, V.sub.D) represents only diffuse reflectance .sub.d, whilst the light pixel I.sub.l=(I.sub.L, S.sub.L, V.sub.L) includes a mixture of diffuse reflectance .sub.d and specular reflectance .sub.s. Thus, the amount of specular reflectance .sub.s(n,m) for the pixel may be approximated as the difference which needs to be subtracted from the light pixel I.sub.L=(H.sub.L, S.sub.L, V.sub.L) such that its new saturation S.sub.L equals the saturation S.sub.D of the dark pixel I.sub.D=(H.sub.D, S.sub.D, V.sub.D). This can be expressed as:

(51) $\begin{array}{cc}{S}_D=S_L^{}=\frac{C_L}{V_L-}& \left(1\right)\end{array}$

(52) In which C.sub.L is the chroma (related to hue H.sub.L) of the brighter, light pixel I.sub.L=(H.sub.L, S.sub.L, V.sub.L). This leads to a closed-form solution for (Equation 2) which can be easily computed for every surface point imaged under a binary spherical gradient X, Y, Z and its complement X*, Y*, Z* and corresponding to a pixel position n, m.

(53) $\begin{array}{cc}=V_L-\frac{C_L}{S_D}& \left(2\right)\end{array}$

(54) As the first to sixth images I.sub.X, I.sub.X*, I.sub.Y, I.sub.Y*, I.sub.Z, I.sub.Z* corresponding to three pairings of binary spherical gradient and complement X, X*, Y, Y*, Z, Z*, three separate estimates .sub.X, .sub.Y, .sub.Z may be obtained for each pixel n, m:

(55) $\begin{array}{cc}{}_X\left(n,m\right)=\{\begin{array}{cc}{V}_X\left(n,m\right)-\frac{C_X\left(n,m\right)}{S_{X*}\left(n,m\right)}& \mathrm{if}{S}_X\left(n,m\right)S_{X*}\left(n,m\right)\\ V_{X*}\left(n,m\right)-\frac{C_{X*}\left(n,m\right)}{S_X\left(n,m\right)}& \mathrm{if}{S}_X\left(n,m\right)>S_{X*}\left(n,m\right)\end{array}& \left(3\right)\end{array}$
an equivalent expression may be written for the second binary spherical gradient Y and complement Y* pair:

(56) $\begin{array}{cc}{}_Y\left(n,m\right)=\{\begin{array}{cc}{V}_Y\left(n,m\right)-\frac{C_Y\left(n,m\right)}{S_{Y*}\left(n,m\right)}& \mathrm{if}{S}_Y\left(n,m\right)S_{Y*}\left(n,m\right)\\ V_{Y*}\left(n,m\right)-\frac{C_{Y*}\left(n,m\right)}{S_Y\left(n,m\right)}& \mathrm{if}{S}_Y\left(n,m\right)>S_{Y*}\left(n,m\right)\end{array}& \left(4\right)\end{array}$
and for the third binary spherical gradient Z and complement Z* pair:

(57) $\begin{array}{cc}{}_Z\left(n,m\right)=\{\begin{array}{cc}{V}_Z\left(n,m\right)-\frac{C_Z\left(n,m\right)}{S_{Z*}\left(n,m\right)}& \mathrm{if}{S}_Z\left(n,m\right)S_{Z*}\left(n,m\right)\\ V_{Z*}\left(n,m\right)-\frac{C_{Z*}\left(n,m\right)}{S_Z\left(n,m\right)}& \mathrm{if}{S}_Z\left(n,m\right)>S_{Z*}\left(n,m\right)\end{array}& \left(5\right)\end{array}$

(58) In this way, according to the first method of albedo separation, three separate estimates .sub.X, .sub.Y, .sub.Z may be obtained for each pixel n, m. These three separate estimates are useful to have because each estimate on its own exhibits a region of uncertainty at the boundary (terminator) between directly and indirectly illuminated portions of the object surface. In this region, the classification of a pixel n, m may become harder and the estimated differences .sub.X, .sub.Y, .sub.Z relatively small, which may cause the estimates .sub.X, .sub.Y, .sub.Z to become relatively more noisy and less precise. However, using three pairings of binary spherical gradient and complement X, X*, Y, Y*, Z, Z*, the regions of uncertainty occur in different regions over a sphere for each of the pairings.

(59) According to the first method, for each pixel n, m the specular reflectance .sub.s(n,m) is estimated to be the median of the three estimates .sub.X(n,m), S.sub.Y(n,m), S.sub.Z(n,m). In this way, a robust output estimate of specular reflectance .sub.s(n,m) may be obtained. In other examples, a mean, or weighted mean, of the three estimates .sub.X(n,m), .sub.Y(n,m), .sub.Z(n,m) may be used to provide the value of the specular reflectance .sub.S(n,m).

(60) In some sets of images, the estimate from one of the pairings of binary spherical gradient and complement X, X*, Y, Y*, Z, Z* can be unstable when one of the measurements is too dark. Such outliers may simply be ignored, for example by taking the median of the three estimates .sub.X(n,m), .sub.Y(n,m), .sub.Z(n,m) as the specular reflectance .sub.s.

(61) Given that the illuminated, bright side of a binary spherical gradient X, Y, Z or complement X*, Y*, Z* is lit with uniform constant illumination from the LED sphere 1 (or equivalent illumination arrangement), the separated specular reflectance .sub.X(n,m), .sub.Y(n,m), .sub.Z(n,m) is proportional to the specular albedo .sub.s(n,m), modulated by Fresnel gain at grazing viewing angles. An estimate of the separated diffuse albedo .sub.d may be obtained by subtract the estimated specular reflectance .sub.s(n,m) from the mixed albedo I.sub.U corresponding uniform illumination and obtained from the sum of a binary spherical gradient image I.sub.X, I.sub.Y, I.sub.Z and its corresponding complement I.sub.X*, I.sub.Y*, I.sub.Z*.

(62) Referring also to FIG. 3, a mixed albedo image I.sub.U is shown in panel (a), specular reflectance .sub.s is shown in panel (b) and diffuse reflectance .sub.d is shown in panel (c).

(63) The mixed albedo image I.sub.U shown in FIG. 3(a) is estimated from the sum of a binary spherical gradient X, Y, Z and its corresponding complement X*, Y*, Z*, and contains both diffuse reflectance .sub.d and specular reflectance .sub.s. The specular reflectance .sub.s shown in FIG. 3(b) is estimated from the difference in color saturation in reflectance under a binary spherical gradient X, Y, Z compared to its complement X*, Y*, Z* (as Equations (1) to (5)). The diffuse albedo .sub.d shown in FIG. 3(c) is obtained by subtracting the specular reflectance .sub.s shown in FIG. 3(b) from the mixed albedo image I.sub.U shown in FIG. 3(a).

(64) First Method of Separating Photometric Normals

(65) A first method of separating photometric normals utilises separated diffuse and specular albedo components .sub.d, .sub.d, which may be obtained according to the first method of albedo separation. However, the first method of separating photometric normals is equally applicable using specular albedo components .sub.d, .sub.d obtained in any other way such as, for example, the hereinafter described alternative methods of albedo separation.

(66) Once the albedo observed under uniform (full-on) illumination has been separated into a diffuse component .sub.d and a specular component .sub.s, photometric surface normals obtained using binary spherical gradients may be separated into pure diffuse and specular normals using the first method of separating photometric normals.

(67) Similar to the approach of Wilson et al. (WGP-10), the binary spherical gradients X, Y, Z and the corresponding complements X*, Y*, Z* can be used to compute photometric surface normals given as:

(68) $\begin{array}{cc}{N}_{\mathrm{mixed}}=\frac{\left(X-X^{*},Y-Y^{*},Z-Z^{*}\right)}{.Math.\left(X-X^{*},Y-Y^{*}Z-Z^{*}\right).Math.}& \left(6\right)\end{array}$

(69) In which N.sub.mixed is the mixed photometric normal, and X, X*, Y, Y*, Z, Z* represent pixel values in Equation (6). Unless indicated otherwise, the mixed photometric normal N.sub.mixed and other vector quantities (denoted in bold) referred to hereinafter should be assumed to be normalised unit vectors. The photometric normals N.sub.mixed are different for each pixel n, m, i.e N.sub.mixed(n,m) which may be written for the red R pixel values as:

(70) $\begin{array}{cc}{N}_{\mathrm{mixed}}\left(n,m\right)=\frac{\begin{array}{c}(R_X\left(n,m\right)-R_{X^{*}}\left(n,m\right),\\ R_Y\left(n,m\right)-R_{Y^{*}}\left(n,m\right),R_Z\left(n,m\right)-R_{Z^{*}}\left(n,m\right))\end{array}}{\begin{array}{c}.Math.(R_X\left(n,m\right)-R_{X^{*}}\left(n,m\right),\\ R_Y\left(n,m\right)-R_{Y^{*}}\left(n,m\right),R_Z\left(n,m\right)-R_{Z^{*}}\left(n,m\right)).Math.\end{array}}& \left(7\right)\end{array}$

(71) Analogous expressions may be written for the green G, and blue B pixel values.

(72) Consistent with the observations of Ma et al. (MHP-07), for human skin different amounts of surface detail may be observed in the photometric normals Nmixed computed from the red R, green G, and blue B channels respectively. Without wishing to be bound be theory, this is believed to be a result of different amounts of subsurface scattering of light in skin for these different wavelength ranges.

(73) Referring also to FIG. 4, mixed photometric normals N.sub.mixed obtained using red R, green G and blue B color channels are shown.

(74) Panel (a) shows mixed photometric normals N.sub.mixed obtained using the red R color channel. Panel (d) shows a cropped and zoomed view of the subject's left eye in panel (a). Panel (b) shows mixed photometric normals N.sub.mixed obtained using the green G color channel. Panel (e) shows a cropped and zoomed view of the subject's left eye in panel (b). Panel (c) shows mixed photometric normals N.sub.mixed obtained using the blue B color channel. Panel (f) shows a cropped and zoomed view of the subject's left eye in panel (c).

(75) It may be observed that for human skin, the surface details are increasingly visible for shorter wavelengths as seen in the crops (FIGS. 4(d) to 4(f)) around the subject's left eye. Thus, for imaging human subjects, the mixed photometric normals N.sub.mixed may be advantageously obtained using the blue B color channel. For application in imaging of other (non-human) subjects or objects, a different color channel may be selected and/or multiple color channels may be combined or averaged.

(76) The computed mixed photometric normals N.sub.mixed include a mix of both diffuse and specular reflectance .sub.d, .sub.s, and consequently are not ideal for direct usage in computer graphic rendering.

(77) Instead, the mixed photometric normal N.sub.mixed may be separated into a pure diffuse photometric normal and a pure specular photometric normal for shading with a diffuse and a specular Bidirectional Reflectance Distribution Function (BRDF) respectively.

(78) The first method of separating photometric normals utilises the suv color space analysis of Mallick et al. (MZKB05), which is hereby incorporated by reference. The acquired binary spherical gradient images I.sub.X, I.sub.Y, I.sub.Z and their complement images I.sub.X*, I.sub.Y*, I.sub.Z* are transformed from RGB color space to Mallick's proposed suv space. This transformation rotates the RGB color information to align any white component in the original RGB signal with the s component, while leaving the chroma in the u and v components. Once transformed to the suv space of Mallick et al., the transformed binary spherical gradient images I.sub.X, I.sub.Y, I.sub.Z and their complement images I.sub.X*, I.sub.Y*, I.sub.Z* may be used to estimate a pure diffuse photometric normal as described hereinafter.

(79) Without wishing to be bound be theory, it is believed that the chroma in colored dielectric materials is the result of absorption due to subsurface scattering, and that consequently the u and v components may be employed to recompute the photometric normals according to Equation (6) using only these purely diffuse components of the original RGB signal given as (u.sup.2+v.sup.2).sup.0.5. For example, if the RGB components R.sub.X(n,m), G.sub.X(n,m), B.sub.X(n,m) for the binary spherical gradient X are transformed to sub color space components s.sub.X(n,m), u.sub.X(n,m), v.sub.X(n,m), then a corresponding pure diffuse component P.sub.X(n,m) may be expressed as:
P.sub.X(n, m)={square root over (((u.sub.X(n, m)).sup.2+(v.sub.X(n, m)).sup.2))}(8)
and analogous expressions may be written for each other binary spherical gradient Y, Z and complement X*, Y*, Z*. An estimate of the pure diffuse photometric normal N.sub.u,v may be estimated using Equation (6) with the pure diffuse components P.sub.X(n,m):

(80) $\begin{array}{cc}{N}_{u,v}=\frac{\begin{array}{c}(P_X\left(n,m\right)-P_{X^{*}}\left(n,m\right),P_Y\left(n,m\right)-P_{Y^{*}}\left(n,m\right),\\ P_Z\left(n,m\right)-P_{Z^{*}}\left(n,m\right))\end{array}}{\begin{array}{c}.Math.(P_X\left(n,m\right)-P_{X^{*}}\left(n,m\right),P_Y\left(n,m\right)-P_{Y^{*}}\left(n,m\right),\\ P_Z\left(n,m\right)-P_{Z^{*}}\left(n,m\right)).Math.\end{array}}& \left(9\right)\end{array}$

(81) This estimate of photometric normal N.sub.u,v provides an estimate of a pure diffuse photometric normal which is very smooth in its appearance. Without wishing to be bound be theory, the smooth appearance is believed to be due to blurring of surface details with subsurface scattering.

(82) Referring also to FIG. 5, diffuse photometric normals N.sub.u,v, specular normals N.sub.spec and corrected specular normals N.sub.corr are shown.

(83) Diffuse photometric normals N.sub.u,v calculated according to Equation (9) are shown in panel (a). Panel (b) shows specular normals N.sub.spece calculated as described hereinafter and panel (c) shows corrected specular normals N.sub.corr calculated as described hereinafter.

(84) A specular photometric normal N.sub.spec may be estimated from the separated diffuse and specular albedo components .sub.d, .sub.s and the estimated diffuse photometric normals N.sub.u,v. This separation of specular photometric normals N.sub.spec is based, at least in part, on the inventors' observation that surface details are clearly visible in the acquired mixed photometric normals Nmixed. Surface details are particularly visible in the normals N.sub.mixed calculated using shorter green G and blue B wavelengths.

(85) Without wishing to be bound by theory, this is considered to imply that the mixed normals N.sub.mixed encode a mixture of a diffuse normal N.sub.u,v and some specular information. The contribution of the specular information to the mixed normals N.sub.mixed is strongest in the blue B channel for imaging of human skin. Based on these observations, the specular normals may be estimated according to a mixture model:
N.sub.mixed.N.sub.u,v+(1)R.sub.spec(10)

(86) In which R.sub.spec is an unknown reflection vector. In general, each normal N.sub.mixed, N.sub.u,v and the reflection vector R.sub.spec vary with pixel position n, m, i.e. N.sub.mixed(n,m), N.sub.u,v(n,m) and R.sub.spec(n,m), and the mixture weigh also varies with pixel position (n,m). Equation (10) approximates the acquired mixed photometric normal N.sub.mixed to be the result of a linear combination of the pure diffuse normal N.sub.u,v obtained using color-space separation, and the unknown reflection vector R.sub.spec.

(87) The mixture weights (n,m) for the linear combination are obtained from the relative strengths of the diffuse and specular reflectance .sub.d and .sub.s respectively, according to:

(88) $\begin{array}{cc}\left(n,m\right)=\frac{{}_d\left(n,m\right)}{{}_d\left(n,m\right)+{}_s\left(n,m\right)}& \left(11\right)\end{array}$

(89) The mixture weights (n,m) may be calculated using data from any of the red R, green G or blue B color channels. It may be noted that the specular reflectance .sub.s does not vary with color channel as the specular reflectance is white (R=G=B). For the specific application of imaging human skin, as already noted surface details are best resolved using the blue B color channel. Consequently, for calculating the unknown vector R.sub.spec for a human subject, the blue B color channel may used both for calculating the mixed normals N.sub.mixed and for calculating the mixture weights (n,m). However, the inventors have empirically determined that using the green G color channel to calculate the mixture weights (n,m) provides reduced noise compared to the blue B color channel, without sacrificing surface details. For other imaging applications, other color channels, or combinations of multiple color channels, may be used instead of the blue B channel.

(90) The unknown reflection vector R.sub.spec can be estimated by re-arranging Equation (10) for R.sub.spec and normalising to obtain:

(91) 0$\begin{array}{cc}{R}_{\mathrm{spec}}=\frac{N_{\mathrm{mixed}}-N_{u,v}}{.Math.N_{\mathrm{mixed}}-N_{u,v}.Math.}& \left(12\right)\end{array}$

(92) It may be note that in Equation (12), the division by a factor of (1) from Equation (10) need not be included due to the normalization. The inventors also noted that omitting the division by a factor of (1) may also be beneficial for numerical stability.

(93) The specular normal N.sub.spec is computed as the half-vector given the viewing direction V and the estimated reflection vector R.sub.spec:

(94) $\begin{array}{cc}{N}_{\mathrm{spec}}=\frac{V+R_{\mathrm{spec}}}{.Math.V+R_{\mathrm{spec}}.Math.}& \left(13\right)\end{array}$
in which the viewing direction V is the direction from which the images are obtained.

(95) Referring in particular to FIG. 5(b), it may be observed that the specular normals N.sub.spec estimated using Equation (13) exhibit very fine scale skin surface details such as, for example, fine scale wrinkles, pores, and scarring on the forehead. Such details are not visible in the pure diffuse normal N.sub.u,v (FIG. 5(a)). However, it may also be observed that the specular normals N.sub.spec directly obtained with Equation (13) appear to include a low-frequency bias. Without wishing to be bound by theory, the existence of low-frequency bias in the specular normal N.sub.spec believed to result from the use of N.sub.spec binary spherical gradient illumination for estimating the specular reflection vector N.sub.spec, which results in eight discrete quadrants over the sphere of directions.

(96) Referring also to FIG. 6, specular normals directly estimated with binary spherical gradients are shown for a simulated dielectric sphere in panel (a), and for a real orange in panel (b).

(97) In may be observed that the specular normals directly estimated with binary spherical gradients X, Y, Z and complements X*, Y*, Z* appear to exhibit eight discrete spherical quadrants.

(98) Employing Equations (12) and (13) to estimate the specular normals N.sub.spec appears to add a small amount of smooth variation (gradient) across the discrete quadrants, which can be more clearly seen on a real acquired spherical object, for example the orange shown in FIG. 6b. Without wishing to be bound by theory, it is believed that this small gradient across the quadrants is the result of non-linear Fresnel gain in the specular reflectance (as a function of the scalar product N.V) which is encoded both in the mixed normal N.sub.mixed and the mixture weight (due to Fresnel effects being incorporated into .sub.s) in the mixture model of Equation 12. A combination of such a Fresnel resultant gradient and surface specular roughness may help to create a desirably smooth variation in the high frequency of the estimated specular normal field N.sub.spec (n,m). However, since the specular normals N.sub.spec appear to include a low-frequency bias, the normals N.sub.spec are non-ideal for use as specular normals for computer graphics rendering purposes.

(99) With the aim of obtaining an estimate of the specular normals N.sub.spec which is better suited for use in computer graphics rendering, a method has been developed to synthesize a corrected specular normal N.sub.corr with improved low-frequency content, whilst also incorporating the high frequency surface details contained in the directly estimated specular normal vector N.sub.spec. A corrected specular normal N.sub.corr may be obtained by applying a high-pass filter to the directly estimated specular normal vector N.sub.spec, and adding the output of the high-pass filter to the low frequency diffuse normal N.sub.u,v obtained using color-space separation. The corrected specular normal N.sub.corr(n,m) for each pixel n, m is then re-normalised to unit length.

(100) Referring in particular to FIG. 5(c), the corrected specular normals N.sub.corr obtained using this procedure is shown. The corrected specular normals N.sub.corr do not include the low frequency bias of the directly estimated specular normals N.sub.spec, and may be more suitable for computer graphics rendering applications.

(101) Spectral Multiplexing

(102) The first method of albedo separation and the first method of separating photometric normals have been described based on diffuse-specular separation of the first to sixth images I.sub.X, I.sub.Y, I.sub.Z, I.sub.X*, I.sub.Y*, I.sub.Z* obtained using the binary spherical gradients X, Y, Z and complements X*, Y*, Z*. Although the first method of albedo separation and the first method of separating photometric normals only require six images, it would be preferable to further reduce the number of images required. This is because the fewer images are needed, the less a subject may move between images and the greater the correspondence between pixels n, m across the image set. This factor is not a concern when the imaged object is inanimate.

(103) Spectral multiplexing may be used to encode the binary spherical gradients X, Y, Z and complements X*, Y*, Z* into only two illumination conditions, enabling a two-image acquisition approach. The spectral multiplexing uses the RGB LED lamps of the LED sphere 1 to simultaneously emit the three principal binary spherical gradients along X, Y and Z multiplexed in individual RGB color channels.

(104) For example, a first multiplexed illumination condition may encode the first binary spherical gradient X in the green G color channel, the second binary spherical gradient Y in the blue B color channel and the third binary spherical gradient Z in the red R color channel. The first multiplexed illumination condition is used to obtain a seventh image I.sub.G(n,m)=(R.sub.Z(n,m), G.sub.X(n,m), B.sub.Y(n,m)) encoding all three binary spherical gradients X, Y, Z.

(105) A second illumination condition may encode the first complement X* in the green G color channel, the second complement Y* in the blue B color channel and the third complement Z* in the red R color channel. The second multiplexed illumination condition is used to obtain an eighth image I.sub.C(n,m)=(R.sub.Z*(n,m), G.sub.X*(n,m), B.sub.Y*(n,m)) encoding all three complements X*, Y*, Z*.

(106) Referring also to FIG. 7, examples of spectrally multiplexed seventh I.sub.G and eighth I.sub.C imaged encoding binary spherical RGB gradients are shown.

(107) Panels (a) on the right hand side of FIG. 7 show a seventh image I.sub.G of a mirror ball in the top row, and a seventh image I.sub.G of a subject in the bottom row. Panels (b) on the left had side of FIG. 7 show an eighth image I.sub.C of a mirror ball in the top row, and an eighth image I.sub.C of a subject in the bottom row.

(108) The proposed mapping of X and X* to green G, Y and Y* to blue B, and Z and Z* to red R has been developed because the inventors have observed that for imaging of human subjects, this mapping provides optimal surface details in the photometric normals with X and Y variation (left-right and up-down variation in the images I.sub.G, I.sub.C) encoded in shorter wavelengths (green G and blue B) having relatively lower amounts of subsurface scattering. The Z variations (towards-away from the camera which obtains the image I.sub.G, I.sub.C) are encoded in the longer wavelength (red R), which exhibits relatively higher amounts of scattering in skin.

(109) However, it should be noted that for different applications and/or for imaging of other objects and/or non-human subjects, the mapping of binary spherical gradient-complement pairs X-X*, Y-Y*, Z-Z* to RGB color channels may be different, without affecting the methods described hereinafter.

(110) Additionally, the seventh image I.sub.G need not encode all the binary spherical gradients X, Y, Z whilst the eight image I.sub.C encodes all the complements X*, Y*, Z*. In some examples, a seventh image I.sub.7 may encode either the binary spherical gradient X or the complement X*, whilst an eighth image I.sub.8 encodes the other of the binary spherical gradient X and complement X* in the same color channel. Similarly, the seventh image I.sub.7 may encode either the binary spherical gradient Y or the complement Y* and either the binary spherical gradient Z or complement X*, whilst the eighth image I.sub.8 encodes the other of the binary spherical gradient Y and complement Y* and the other of the binary spherical gradient Z and complement Z*.

(111) The acquired spectrally multiplexed data in the form of a seventh image I.sub.G(n,m)=(R.sub.Z(n,m), G.sub.X(n,m), B.sub.Y(n,m)) and an eighth image I.sub.C(n,m)=(R.sub.Z*(n,m), G.sub.X*(n,m), B.sub.Y*(n,m)) may be employed as input to a second method of albedo separation and a second method of separating photometric normals.

(112) Second Method of Separating Photometric Normals

(113) Adding together the seventh and eighth images I.sub.G+I.sub.C yields a uniform image I.sub.U which is equivalent to an image acquired under full-on white illumination. The uniform image I.sub.U provides an estimate of the mixed albedo.

(114) Referring also to FIG. 8, panel (a) shows the mixed albedo in the form of a uniform image I.sub.U and panel (b) shows mixed normals N.sub.mixed computed from the seventh and eighth images I.sub.G, I.sub.C acquired using spectrally multiplexed RGB binary spherical gradient illumination.

(115) The mixed normals N.sub.mixed may be calculated by applying Equation (6) to the available color components corresponding to the binary spherical gradients X, Y, Z and complements X*, Y*, Z*:

(116) $\begin{array}{cc}{N}_{\mathrm{mixed}}\left(n,m\right)=\frac{\begin{array}{c}(G_X\left(n,m\right)-G_{X^{*}}\left(n,m\right),B_Y\left(n,m\right)-B_{Y^{*}}\left(n,m\right),\\ R_Z\left(n,m\right)-R_{Z^{*}}\left(n,m\right))\end{array}}{\begin{array}{c}.Math.(G_X\left(n,m\right)-G_{X^{*}}\left(n,m\right),B_Y\left(n,m\right)-B_{Y^{*}}\left(n,m\right),\\ R_Z\left(n,m\right)-R_{Z^{*}}\left(n,m\right)).Math.\end{array}}& \left(14\right)\end{array}$

(117) Referring again to FIG. 8(b), an example of mixed normals N.sub.mixed is shown corresponding to the mixed albedo shown in FIG. 8(a).

(118) The first method of albedo separation and the first method of separating photometric normals used white illumination. The spectrally multiplexed second methods do not, and consequently the hereinbefore described color-space suv separation is not applicable to analysis of the seventh and eighth images I.sub.G, I.sub.C.

(119) Instead, the second method of separating photometric normals applies a heuristic approach for separation of the mixed photometric normals N.sub.mixed. An approximation to the diffuse normal map N.sub.blur is obtained by spatial blurring of the mixed photometric normals N.sub.mixed(n,m). In some simple examples, the blurred normal map N.sub.blur may be obtained using simple averaging over a local region:

(120) $\begin{array}{cc}{N}_{\mathrm{blur}}\left(n,m\right)=\underset{i=1}{\overset{2F+1}{.Math.}}\underset{j=1}{\overset{2K+1}{.Math.}}A\left(i,j\right)N_{\mathrm{mixed}}\left(n-F-1+i,m-k-1+j\right)& \left(15\right)\end{array}$
followed by re-normalisation to unit length. In Equation (15) A(i,j) denotes weighting coefficients used in the blurring, F and K are integers denoting the number of pixels to either side of pixel n, m to which the blurring region should extend. Collectively, A(i,j), F and K may be referred to as a blur kernel. In some examples the blur kernel may be flat, i.e. all A(i,j) constant, to provide a simple averaging. In other examples, the blur kernel may be based on, for example, Gaussian functions (or even a Bilateral filter kernel) on i,j centred at the pixel n, m. The size 2F+1, 2K+1 of the blur kernel is not particularly limited, larger widths will provide more significant spatial blurring.

(121) Referring also to FIG. 9, panel (a) shows an example of blurred diffuse normals N.sub.blur and panel (b) shows synthesized normals N.sub.synth approximating the specular normals N.sub.spec and estimated from the mixed photometric normals N.sub.mixed obtained using seventh and eighth images I.sub.G, I.sub.C.

(122) For the example shown in FIG. 9(a), the blur kernel A(i,j) was empirically selected (for the example of the images shown in FIG. 9, the blur kernel A(i,j) was a Gaussian of width 30 pixels, images of other objects or obtained using a different camera may use alternative blur kernels A(i,j)) to produce a smooth normal map N.sub.blur that is qualitatively similar to the diffuse normals N.sub.u,v obtained using color-space separation according to the first method of separating photometric normals.

(123) The specular normal map N.sub.spec is then estimated by first subtracting the blurred diffuse normals N.sub.blur from the mixed normals N.sub.mixed:
N.sub.(n, m)=N.sub.mixed(n, m)N.sub.blur(n, m)(16)
followed by re-normalisation to unit length, and in which N.sub. is an intermediate difference map providing a first approximation to the specular normal map N.sub.spec. The intermediate difference map N.sub.may be used as a rough estimate of the specular normal map N.sub.specN.sub.. The intermediate difference map N.sub. isolates the high spatial frequencies present in the mixed normals such as, for example, features due to skin pores, fine wrinkles, scars and so forth.

(124) Alternatively, the inventors have empirically established that a synthesized estimate of the specular normal map N.sub.synth having improved properties for computer graphic rendering may be obtained by scaling up the intermediate difference map N.sub. (without renormalizing) and adding it back to the blurred normal map N.sub.blur according to:
N.sub.syn(n, m)=N.sub.(n, m)+N.sub.blur(n, m)(17)
followed by re-normalisation to unit length, and in which is a scaling factor. For example, the synthesized normals N.sub.synth shown in FIG. 9(b) were determined using a scaling factor of =2.

(125) Referring again to FIG. 9(b), the synthesized normals N.sub.synth may be observed to be qualitatively comparable to the more directly calculated corrected specular normals N.sub.corr shown in FIG. 5(c) and obtained using data from white light illumination.

(126) Second Method of Albedo Separation

(127) A second method of separating the mixed albedo I.sub.U, for example as shown in FIG. 8(a), into diffuse and specular components .sub.d, .sub.s requires an estimate of diffuse normals to have been calculated previously. The blurred normals N.sub.blur may be used for the purpose of the second method of albedo separation.

(128) According to the second method of separating the albedo, instead of formulating the albedo separation with color-space analysis as in the first method, the diffuse and specular components of albedo are formulated as a linear system. The data from each color channel are treated independently in the acquired seventh and eighth images I.sub.G, I.sub.C, in order to obtain two observations under every principal binary spherical gradient X, Y, Z and the corresponding complement X*, Y*, Z*.

(129) In the first method, the concepts of light and dark pixels I.sub.L, I.sub.D was employed. Similar considerations apply to the spectrally multiplexed seventh and eighth images I.sub.G, I.sub.C, except that because of the spectral multiplexing a particular pixel n, m for, for example the seventh image I.sub.G(n,m)=(R.sub.Z(n,m), G.sub.X(n,m), B.sub.Y(n,m)), may be a light pixel in one color channel and a dark pixel in a different channel.

(130) Treating each color channel separately, for each pixel n, m, a light value R.sub.L, G.sub.L, B.sub.L and a dark value R.sub.D, G.sub.D, B.sub.D may be assigned. For example, for the red R channel the assignment is according to:

(131) $\begin{array}{cc}{R}_L\left(n,m\right)=\{\begin{array}{ccc}{R}_Z\left(n,m\right)& \mathrm{if}& R_Z\left(n,m\right)R_{Z^{*}}\left(n,m\right)\\ R_{Z^{*}}\left(n,m\right)& \mathrm{if}& R_Z\left(n,m\right)<R_{Z^{*}}\left(n,m\right)\end{array}& \left(18\right)\end{array}$

(132) The light and dark values G.sub.L, B.sub.L, G.sub.D, B.sub.D for the green G and blue B color channels are similarly defined based on the values of the seventh image I.sub.G(n,m)=(R.sub.Z(n,m), G.sub.X(n,m), B.sub.Y(n,m)) and the eighth image I.sub.C(n,m)=(R.sub.Z*(n,m), G.sub.X*(n,m), B.sub.Y*(n,m)).

(133) The diffuse-specular albedo separation according to the second method is formulated as a solution to a linear system. As an example, for the red channel:

(134) $\begin{array}{cc}\left[\begin{array}{c}{R}_L\left(n,m\right)\\ R_D\left(n,m\right)\end{array}\right]=\left[\begin{array}{cc}{N}_d\left(n,m\right)& 1\\ \left(1-N_d\left(n,m\right)\right)& 0\end{array}\right]\left[\begin{array}{c}{D}_R\left(n,m\right)\\ S_R\left(n,m\right)\end{array}\right]& \left(19\right)\end{array}$

(135) In which D.sub.R(n,m) is the diffuse component of the red R channel, S.sub.R(n,m) is the specular component of the red R channel and N.sub.d(n,m) is the corresponding (red R) component of the diffuse normal (shifted to the range [0;1]) for the corresponding principal direction, which for the red R channel in this example is the z-axis corresponding to binary spherical gradient Z and complement Z*. Analogous expressions are used for the green G and blue B color channels.

(136) In the second method of albedo separation, the values of N.sub.d(n,m) are provided by the blurred normals N.sub.blur obtained by blurring the mixed photometric normals N.sub.mixed. For each pixel n, m, Equation (19) is solved to obtain estimates of the diffuse and specular components D.sub.R(n,m), S.sub.R(n,m), and equivalent expressions are solved to obtain the diffuse and specular components from green and blue channels D.sub.G(n,m), D.sub.B(n,m), S.sub.G(n,m), S.sub.B(n,m). Assuming monochromatic specular reflection, an estimate of the specular reflectance .sub.s(n,m) for the pixel n, m may be obtained as the median of the three independently calculated estimates .sub.s(n,m)=median{S.sub.R(n,m), S.sub.G(n,m), S.sub.B(n,m)}. Alternatively, the specular reflectance .sub.s(n,m) may be calculated as a mean, or a weighted mean, of the color channel estimates S.sub.R(n,m), S.sub.G(n,m), S.sub.B(n,m). Finally, an estimate of the diffuse reflectance .sub.d is obtained by pixelwise subtracting the specular reflectance .sub.s from the mixed albedo I.sub.U.

(137) Referring also to FIG. 10, panel (a) shows separated specular reflectance .sub.s(n,m) obtained by solving the linear system as Equation (19), and panel (b) shows the diffuse reflectance .sub.d(n,m) calculated by subtracting the specular reflectance .sub.s(n,m) shown in FIG. 10(a) from the mixed albedo shown in FIG. 8(a).

(138) Comparison to Polarized Linear Gradient Illumination

(139) The utility of the methods of the present specification may be placed in context by comparing it to previous methods based on polarized linear spherical gradients. In polarized light methods, the orientation of polarizers on both light sources and cameras must be meticulously aligned.

(140) Referring also to FIG. 11, acquired diffuse albedo .sub.d and specular albedo .sub.s and photometric normals acquired using binary spherical gradient X, Y, Z illumination according the first method of albedo separation and the first method of separating photometric normals as shown in the top-row of each of panels (a) to (h). The bottom row of each panel shows a corresponding image obtained using polarized linear spherical gradient illumination.

(141) Images for the present methods and for the prior method of Ghosh et al. were obtained for an orange rubber ball (FIGS. 11(a) to 11(d)) and for a human subject (FIGS. 11(e) to 11(h)). The polarized linear spherical gradient illumination images were obtained using the method described by Ghosh et al. (GFT-11). The LED sphere 1was polarised with the latitude-longitude linear polarization pattern proposed by Ghosh et al. for this comparison, and data was acquired for the binary spherical gradients X, Y, Z and complements X*, Y*, Z* with both polarization banks switched on to achieve unpolarized illumination.

(142) It may be observed that the separated albedo and photometric normals acquired using the methods of the present specification using binary spherical gradients X, Y, Z and complements X*, Y*, Z* are qualitatively comparable to those acquired using polarized spherical gradients. Furthermore, the data which needs to be acquired for the methods of the present specification requires half the number of photographs (six) compared to that required with polarized spherical gradients for multiview capture.

(143) Quantitative differences were computed between the acquired maps of the orange ball using both the present methods and the method of Ghosh et al. (GFT-11). The differences are as follows: diffuse albedo=0.064 RMSE (root mean square error); specular albedo=0.02 RMSE, diffuse normal=8.93 angular RMSE, specular normal=14.21 angular RMSE.

(144) The u, v diffuse normal N.sub.u,v obtained using the first method of separating photometric normals was compared against the red diffuse normal obtained with polarized linear spherical gradients, because these normals are qualitatively the most similar in terms of blurring. It may be observed the that the chroma based N.sub.u,v diffuse normal computed according to the present specification is qualitatively slightly softer than that obtained using cross-polarized linear spherical gradient illumination.

(145) In contrast, the specular normal N.sub.spec, N.sub.corr obtained according to the present specification exhibits slightly sharper surface details than that obtained using polarized linear spherical gradients. Without wishing to be bound by theory, this is believed to be because any surface details missing in the calculated diffuse normal N.sub.u,v (for example due to subsurface scattering) is attributed instead to the specular normal N.sub.spec, N.sub.corr as a result of Equations (12) and (13).

(146) Referring also to FIG. 12, a hybrid normal rendering shown in panel (a) is compared against a photograph shown in panel (b).

(147) The rendering of FIG. 12(a) was generated using data acquired with binary spherical gradients X, Y, Z and complements X*, Y*, Z* according to the present specification. The photograph of FIG. 12(b) was obtained using a point light source.

(148) It may be observed that the hybrid normal rendering (FIG. 12(a)) using the estimated diffuse and specular albedo .sub.d, .sub.s and photometric normals N.sub.u,v, N.sub.corr closely approximates the facial appearance in the photograph, with the exception of self-occlusions around the nose and the subject's neck which are not modelled in the rendering (FIG. 12(a)).

(149) Specular Normals and Binary Illumination

(150) It may be noted that besides albedo separation into diffuse and specular components .sub.d, .sub.s, estimating a high quality specular normal using binary spherical gradient X, Y, Z illumination is also a key teaching of the present specification. However, these results are also surprising given the binary nature of the incident illumination using the binary spherical gradients X, Y, Z and complements X*, Y*, Z*.

(151) Unlike previous work which has employed an explicit linear spherical gradient in the incident illumination, for example Ma et al. (MHP-07) or Ghosh et al. (GFT-11), the methods of the present specification exploit an implicit (non-linear) gradient due to Fresnel to estimate the high frequency component of the specular normal field.

(152) The observation of the inventors that Fresnel can be exploited for this purpose even under constant (or binary) illumination is an important teaching of the present specification. Besides being useful for realistic rendering of high frequency specular detail, the estimated specular normals N.sub.spec, N.sub.corr, N.sub.synth may also be employed for embossing mesoscopic surface details on a base geometry, for example as described by Ma et al. (MHP-08) and in NEHAB D., RUSINKIEWICZ S., DAVIS J., RAMAMOORTHI R.: Efficiently combining positions and normals for precise 3d geometry. ACM Trans. Graph. 24, 3 (July 2005), 536-543 (hereinafter NRDR-05).

(153) Arbitrary and Multi-View Capture

(154) In many prior methods for capturing and separating albedo and photometric normals, the relative positioning of a camera used to obtain images is required to be carefully aligned with the illumination conditions, for example polarized linear spherical gradients.

(155) By contrast, provided that the relative position of the camera is known, the camera may be oriented at an arbitrary viewpoint to the first, second and third axes x, y, z with reference to which the binary spherical gradients X, Y, Z and complements X*, Y*, Z* are defined. After capturing the first to sixth images I.sub.X, I.sub.Y, I.sub.Z, I.sub.X*, I.sub.Y*, I.sub.Z* or the seventh and eight images I.sub.G, I.sub.C, the images may be simply transformed to local coordinates of camera (using a 3D rotation), followed by processing as described hereinbefore.

(156) In addition to permitting an arbitrary orientation of the camera viewpoint, this property of methods according to the present specification also permits views to be captured from multiple cameras having different viewpoints simultaneously and using the same illumination conditions. The images for each viewpoint may then simply be rotated to the local frame of reference for albedo and normal separation.

(157) As already defined, the x- and y-axes are relative to x-, y- and z-axes with respect to an image acquired by a camera looking towards the coordinate system origin along the direction =/2, =0, using the spherical polar coordinates used to define the binary spherical gradients X, Y, Z and complements X*, Y*, Z*. The x-axis is parallel to the image plane and corresponds to binary spherical gradient X and complement X*. The y-axis is parallel to the image plane and corresponds to the binary spherical gradient Y and complement Y*. The z-axis is perpendicular to the image plane and corresponds to the binary spherical gradient Z and complement Z*.

(158) For white light illumination, first to sixth images are obtained I.sub.X, I.sub.Y, I.sub.Z, I.sub.X*, I.sub.Y*, I.sub.Z* by a camera, which for the purposes of explanation will be presumed to be pointing towards the origin of the spherical polar coordinates (r, , ) along a direction different from =/2, =0. The first to sixth images I.sub.X, I.sub.Y, I.sub.Z, I.sub.X*, I.sub.Y*, I.sub.Z* need to be rotated to generate local images I.sub.X, I.sub.Y, I.sub.Z, I.sub.X*, I.sub.Y*, I.sub.Z* corresponding to the local axes x, y, z relative to the camera. The rotation method shall be described in relation to obtaining the image I.sub.X corresponding to a local x-axis of the camera. The method is equivalent for rotations to obtain the other local images I.sub.Y, I.sub.Z, I.sub.X*, I.sub.Y*, I.sub.Z*.

(159) The rotation is carried out for each pixel n, m separately, defining vectors for each color channel:

(160) $\begin{array}{cc}R\left(n,m\right)=\left(\begin{array}{c}{R}_X\left(n,m\right)\\ R_Y\left(n,m\right)\\ R_Z\left(n,m\right)\end{array}\right),G\left(n,m\right)=\left(\begin{array}{c}{G}_X\left(n,m\right)\\ G_Y\left(n,m\right)\\ G_Z\left(n,m\right)\end{array}\right),B\left(n,m\right)=\left(\begin{array}{c}{B}_X\left(n,m\right)\\ B_Y\left(n,m\right)\\ B_Z\left(n,m\right)\end{array}\right)& \left(20\right)\end{array}$

(161) For each pixel n, m the initial vectors R, G, B are remapped to vectors F.sub.R, F.sub.G, F.sub.B, by converting components of the vectors R, G, B from a normalised range of 0 to 1, to a normalised range of 1 to 1 for corresponding components of the vectors F.sub.R, F.sub.G, F.sub.B. For example, for the components or F.sub.R:

(162) $\begin{array}{cc}{F}_R\left(n,m\right)=\left(\begin{array}{c}2R_X\left(n,m\right)-1\\ 2R_Y\left(n,m\right)-1\\ 2R_Z\left(n,m\right)-1\end{array}\right)& \left(21\right)\end{array}$
and similar for F.sub.G and F.sub.B. Next, we formulate a 3 by 3 rotation matrix R.sub.V which is simply a standard rotation matrix between the reference direction (0,0,1) in the x, y, z frame used to define the illumination conditions X, Y, Z, X*, Y*, Z* and a view direction V of the camera (which corresponds to (0,0,1) in the local x, y, z frame). Rotated vectors F.sub.R, F.sub.G and F.sub.B are calculated according to:
F.sub.R=R.sub.VF.sub.R, F.sub.G=R.sub.VF.sub.G, F.sub.B=R.sub.VF.sub.B(22)

(163) Followed by re-mapping of the components of F.sub.R, F.sub.G and F.sub.B to the range 0 to 1 to form rotated color vectors R, G, B. Once calculated for each pixel n, m, the components of the rotated color vectors R, G, B may be used to re-construct rotated first to sixth local images I.sub.X, I.sub.Y, I.sub.Z, I.sub.X*, I.sub.Y*, I.sub.Z*. For example, I.sub.X(n,m)={R.sub.X(n,m), G.sub.X(n,m), B.sub.X(n,m)}.

(164) The same rotation may be applied to the spectrally multiplexed seventh and eighth images I.sub.G, I.sub.C to obtain local seventh and eighth images I.sub.G, I.sub.C, by rotation using rotation matrix R.sub.V of the vectors:

(165) $\begin{array}{cc}{F}_7=\left(\begin{array}{c}2G_X\left(n,m\right)-1\\ 2B_Y\left(n,m\right)-1\\ 2R_Z\left(n,m\right)-1\end{array}\right),F_8=\left(\begin{array}{c}2G_{X^{*}}\left(n,m\right)-1\\ 2B_{Y^{*}}\left(n,m\right)-1\\ 2R_{Z^{*}}\left(n,m\right)-1\end{array}\right)& \left(23\right)\end{array}$

(166) In which the exemplary mapping of X, X* to green G, Y, Y* to blue B and Z, Z* to red R used purely for the purpose of providing a definite example.

(167) In this way, an object may be imaged from multiple angles simultaneously, using either six illumination conditions for white illumination, or only two illumination conditions when spectrally multiplexed. The images obtained for each camera may be rotated to images local to that cameras viewpoint V, and subsequently processed as described in this specification.

(168) Third Method of Separating Albedo

(169) A third method of separating albedo into diffuse and specular components .sub.d, .sub.s is the same as the first method, except that instead of determining the specular reflectance .sub.S(n,m) as the median, mean or weighted mean of the three estimates .sub.S(n,m), .sub.Y(n,m), .sub.Z(n,m), according to the third method the specular reflectance .sub.S(n,m) is defined as the mean of only the estimate .sub.X(n,m) obtained using the pair of local images I.sub.X and I.sub.X*, and the estimate .sub.Y(n,m) obtained using the pair of local images I.sub.Y and I.sub.Y*:

(170) $\begin{array}{cc}{}_s\left(n,m\right)=\frac{{}_X\left(n,m\right)+{}_Y\left(n,m\right)}{2}& \left(24\right)\end{array}$

(171) The separation of albedo into diffuse and specular components .sub.d, .sub.s according to the third method may then be used for the first method of separating photometric normals as described hereinbefore.

(172) Third Method of Separating Photometric Normals

(173) A third method of separating photometric normals is the same as the first method, except that instead of performing a color space rotation and calculating the N.sub.u,v diffuse normals, an alternative estimate of the diffuse normals is substituted.

(174) The third method is based on the observation of the inventors that under white illumination, the different color channels of measured skin reflectance (or reflectance of any other colored dielectric material) have the same amount of specular reflectance .sub.s mixed with a spectrally varying amount of diffuse reflectance .sub.d.

(175) A consequence of this observation is that it is possible to estimate a pure diffuse normal using the difference of the brightest and darkest color channels of reflectance. For human skin, the brightest and darkest color channels are red R and blue B channels respectively, and the diffuse normals may be estimated as N.sub.R-B according to:

(176) 0$\begin{array}{cc}{N}_{R-B}\left(n,m\right)=\frac{\begin{array}{c}(R_X\left(n,m\right)-B_{X^{*}}\left(n,m\right),R_Y\left(n,m\right)-B_{Y^{*}}\left(n,m\right),\\ R_Z\left(n,m\right)-B_{Z^{*}}\left(n,m\right))\end{array}}{\begin{array}{c}.Math.(R_X\left(n,m\right)-B_{X^{*}}\left(n,m\right),R_Y\left(n,m\right)-B_{Y^{*}}\left(n,m\right),\\ R_Z\left(n,m\right)-B_{Z^{*}}\left(n,m\right)).Math.\end{array}}& \left(25\right)\end{array}$

(177) Subsequently, specular normals N.sub.spec and corrected specular normals N.sub.corr may be estimated in the same way as the first method, except substituting the normals N.sub.R-B for the color space rotation normals N.sub.u,v.

(178) Referring also to FIG. 13, a comparison is presented between (a) the mixed normals N.sub.mixed, (b) the normals N.sub.u,v obtained by rotation to suv color space according to the first method, and (c) the normals N.sub.R-B obtained by subtracting the darkest color channel from the brightest color channel.

(179) It may be observed that compared to the mixed normal N.sub.mixed, the simple spectral differencing normal N.sub.R-B normal exhibits very similar qualitative softness and blur to the diffuse normal N.sub.u,v computed using the suv-space projection of the first method. However, the proposed spectral differencing normal N.sub.R-B is much simpler and faster to compute. In this way, a spectral differencing step may be used to effectively remove the specular signal, isolating a pure diffuse component

(180) Although described and illustrated with reference to human skin for which the brightest color channel for reflectance is red R and the darkest blue B, it will be appreciated that the specific color channels used for the third method may be varied depending on the object or subject being imaged.

(181) The third method of separating photometric normals may employ albedo components .sub.d, .sub.s obtained using the first or third methods of albedo separation for calculation of the specular normals N.sub.spec.

(182) Fourth Method of Separating Photometric Normals

(183) A fourth method of separating photometric normals is the same as the second method of separating photometric normals, except that instead of calculating the blurred normals N.sub.blur, alternative approximated normals N.sub.app are calculated.

(184) The seventh image I.sub.G(n,m)={R.sub.Z(n,m), G.sub.X(n,m), B.sub.Y(n,m)} and the eighth image I.sub.C(n,m)={R.sub.Z*(n,m), G.sub.X*(n,m), B.sub.Y*(n,m)} may be used to directly compute approximated normals N.sub.app. Firstly, if not already done so, the values R.sub.Z(n,m), G.sub.X(n,m), B.sub.Y(n,m), R.sub.Z*(n,m), G.sub.X*(n,m), B.sub.Y*(n,m) are normalised to the range 0 to 1. Next, we calculate components (x.sub.app(n,m), y.sub.app(n,m), z.sub.app(n,m)) of the approximated normals N.sub.app according to the darker of each binary spherical gradient-complement pair:

(185) $\begin{array}{cc}{x}_{\mathrm{app}}=\{\begin{array}{ccc}1-G_X\left(n,m\right)& \mathrm{if}& G_X\left(n,m\right)G_{X^{*}}\left(n,m\right)\\ -G_{X^{*}}\left(n,m\right)& \mathrm{if}& G_X\left(n,m\right)>G_{X^{*}}\left(n,m\right)\end{array}{y}_{\mathrm{app}}=\{\begin{array}{ccc}1-B_Y\left(n,m\right)& \mathrm{if}& B_Y\left(n,m\right)B_{Y^{*}}\left(n,m\right)\\ -B_Y\left(n,m\right)& \mathrm{if}& B_Y\left(n,m\right)>B_{Y^{*}}\left(n,m\right)\end{array}{z}_{\mathrm{app}}=\{\begin{array}{ccc}1-R_Z\left(n,m\right)& \mathrm{if}& R_Z\left(n,m\right)R_{Z^{*}}\left(n,m\right)\\ -R_{Z^{*}}\left(n,m\right)& \mathrm{if}& R_Z\left(n,m\right)>R_{Z^{*}}\left(n,m\right)\end{array}& \left(26\right)\end{array}$

(186) Subsequently, the normalised approximated normals N.sub.app(n,m) are calculated as:

(187) $\begin{array}{cc}{N}_{\mathrm{app}}\left(n,m\right)=\frac{\left(x_{\mathrm{app}}\left(n,m\right),y_{\mathrm{app}}\left(n,m\right),z_{\mathrm{app}}\left(n,m\right)\right)}{.Math.\left(x_{\mathrm{app}}\left(n,m\right),y_{\mathrm{app}}\left(n,m\right),z_{\mathrm{app}}\left(n,m\right)\right).Math.}& \left(27\right)\end{array}$

(188) The normalised approximated normals N.sub.app(n,m) may provide a more accurate estimation of the diffuse normals than the blurred normals N.sub.blur. With the exception of substituting the blurred normals N.sub.blur with the approximated normals N.sub.app, the fourth method of separating photometric normals is otherwise identical to the second method of separating photometric normals

(189) Although described in relation to the seventh and eight images I.sub.G, I.sub.C, analogous approximated normals N.sub.app may also be calculated for white light illumination using the first to sixth images I.sub.X, I.sub.Y, I.sub.Z, I.sub.X*, I.sub.Y*, I.sub.Z*.

MODIFICATIONS

(190) It will be appreciated that various modifications may be made to the embodiments hereinbefore described. Such modifications may involve equivalent and other features which are already known in the processing of images and/or in the design, manufacture and use of image illuminating and/or capturing apparatuses and component parts thereof and which may be used instead of, or in addition to, features already described herein. Features of one embodiment may be replaced or supplemented by features of another embodiment.

(191) Although specific methods of albedo separation and methods of separating photometric normals have been described, any compatible aspects of these methods may be combined interchangeably:

(192) White Light Illumination

(193) For white light illumination to obtain the first to sixth images I.sub.X, I.sub.Y, I.sub.Z, I.sub.X*, I.sub.Y*, I.sub.Z*, albedo separation into diffuse and specular components .sub.d, .sub.s may be performed according to the first or third methods based on Equations (3) to (5). Any of the described estimates of diffuse normals N.sub.u,v, N.sub.blur, N.sub.R-B or N.sub.app may then be calculated, although the color-space rotation based normals N.sub.u,v or spectral differencing N.sub.R-B normals are preferred. Specular normals N.sub.spec may then be estimated using the weighted mixture method described in relation to Equations (10) to (13), using any of the described estimates for the diffuse normal N.sub.u,v, N.sub.blur, N.sub.R-B, N.sub.app and the diffuse and specular components .sub.d, .sub.s. Corrected specular normals N.sub.corr may then be estimated.

(194) Alternatively, the specular normals N.sub.spec may simply be estimated as being equal to a difference map N.sub. obtained by subtracting one of the described estimates of diffuse normals N.sub.u,v, N.sub.blur, N.sub.R-B or N.sub.app from the mixed normals N.sub.mixed. In other examples, the specular normals N.sub.spec may be estimated as synthetic normals N.sub.synth determined based on the mixed normals N.sub.mixed and the difference map N.sub. as described hereinbefore.

(195) Albedo separation into diffuse and specular components .sub.d, .sub.s for white light may alternatively be performed using the linear system method described in relation to Equations (18) and (19). This approach needs input in the form of an estimate of the diffuse normals, and any of the described estimates of diffuse normals N.sub.u,v, N.sub.blur, N.sub.R-B or N.sub.app may be used. Specular normals N.sub.spec may then be estimated using the weighted mixture method described in relation to Equations (10) to (13), using the diffuse and specular components .sub.d, .sub.s and whichever of the described estimates for the diffuse normal N.sub.u,v, N.sub.blur, N.sub.R-B, N.sub.app was used to calculated the diffuse and specular components .sub.d, .sub.s. Corrected specular normals N.sub.corr may then be estimated.

(196) Alternatively, the specular normals N.sub.spec may simply be estimated as being equal to a difference map N.sub. obtained by subtracting one of the described estimates of diffuse normals N.sub.u,v, N.sub.blur, N.sub.R-B or N.sub.app from the mixed normals N.sub.mixed. In other examples, the specular normals N.sub.spec may be estimated as synthetic normals N.sub.synth determined based on the mixed normals N.sub.mixed and the difference map N.sub. as described hereinbefore.

(197) Spectral Multiplexing

(198) For spectrally multiplexed illumination to obtain the seventh and eighth images I.sub.G, I.sub.C, albedo separation into diffuse and specular components .sub.d, .sub.s for white light may be performed using the linear system method described in relation to Equations (18) and (19). This approach needs input in the form of an estimate of the diffuse normals, and either of the hereinbefore described estimates of diffuse normals N.sub.blur, or N.sub.app may be used for this purpose. The photometric normals N.sub.u,v obtained using suv color space rotation cannot be calculated for the seventh and eighth images I.sub.G, I.sub.C. Similarly, the photometric normals N.sub.R-B (obtained from spectral differencing) cannot be calculated for the seventh and eighth images I.sub.G, I.sub.C.

(199) The specular normals N.sub.spec may simply be estimated as being equal to a difference map N.sub. obtained by subtracting one of the described estimates of diffuse normals N.sub.blur or N.sub.app from the mixed normals N.sub.mixed. In other examples, the specular normals N.sub.spec may be estimated as synthetic normals N.sub.synth determined based on the mixed normals N.sub.mixed and the difference map N.sub. as described hereinbefore.

(200) Subject Motion During Acquisition

(201) The hereinbefore described methods assume no subject motion during acquisition.

(202) However, given the complementary nature of the binary spherical gradient illumination conditions X, Y, Z, X*, Y*, Z*, the motion compensation methods described by Wilson et al. (WGP-10) could be employed, and this document is hereby incorporated by reference in its entirety.

(203) Although claims have been formulated in this application to particular combinations of features, it should be understood that the scope of the disclosure of the present invention also includes any novel features or any novel combination of features disclosed herein either explicitly or implicitly or any generalization thereof, whether or not it relates to the same invention as presently claimed in any claim and whether or not it mitigates any or all of the same technical problems as does the present invention. The applicants hereby give notice that new claims may be formulated to such features and/or combinations of such features during the prosecution of the present application or of any further application derived therefrom.