Method for displaying a content from 4D light field data

Abstract

A method for displaying a content from 4D light field data is described. Such method is executed by an electronic device, and is remarkable in that it comprises changing point of view of the content and/or a focus plane associated with the content, according to viewing rights.

Claims

1. A method for protecting a displaying of a content from a 4D light field data, the method being executed by an electronic device, the method comprises: obtaining said 4D light-field data as well as viewing rights, sending said content for displaying and changing point of view of said content or a focus plane associated with said content, according to said viewing rights associated with a degradation level.

2. The method for protecting the displaying of the content according to claim 1, wherein said changing said point of view of said content or said focus plane associated with said content is done randomly in time.

3. The method for protecting the displaying of the content according to claim 1, wherein said changing said point of view of said content or said changing of said focus plane associated with said content is done based on a random spatial point of a scene associated with the 4D light field data.

4. The method for protecting the displaying of the content according to claim 1, wherein said changing further comprises blurring at least part of said content.

5. The method for protecting the displaying of the content according to claim 4, wherein said blurring comprises adding Gaussian blur to said content.

6. The method for protecting the displaying of the content according to claim 4, wherein said blurring is done randomly in time.

7. The method for protecting the displaying of the content according to claim 4, wherein said blurring is a spatial blurring of said content that is done randomly.

8. The method for protecting the displaying of the content according to claim 1, wherein said content is the 4D light field data.

9. The method for protecting the displaying of the content according to claim 1, wherein said content is a 2D image or 2D video.

10. The method for protecting the displaying of the content according to claim 1, wherein said content is a 3D content or multiviews content.

11. The method for protecting the displaying of the content according to claim 1, wherein a difference value between consecutive changed points of views of said content or consecutive changed focus planes is defined as a function of said viewing rights associated with the degradation level.

12. A non-transitory computer-readable storage medium storing a computer program comprising a set of computer-executable instructions that when said instructions are executed on a processor causes the processor to perform a method for protecting a displaying of a content from a 4D light field data, the method comprises obtaining said 4D light-field data as well as viewing rights, sending said content for displaying, and changing point of view of said content or a focus plane associated with said content, according to said viewing rights associated with a degradation level.

13. A display device for protecting a displaying of a content from a 4D light field data, the display device comprising a processor configured to obtain said 4D light-field data as well as viewing rights, sending said content for displaying and change point of view of said content or a focus plane associated with said content, according to viewing rights associated with a degradation level.

14. The display device for protecting the displaying of the content according to claim 13, wherein said changing said point of view of said content or said focus plane associated with said content is done randomly in time.

15. The display device for protecting the displaying of the content according to claim 13, wherein said changing said point of view of said content or said changing of said focus plane associated with said content is done based on a random spatial point of a scene associated with the 4D light field data.

16. A device for protecting a displaying of a content from a 4D light field data, the device comprising: a processor for obtaining said 4D light-field data as well as viewing rights, sending said content for displaying and changing point of view of said content or a focus plane associated with said content, according to said viewing rights associated with a degradation level.

17. The device of claim 16, wherein said changing said point of view of said content or said focus plane associated with said content is done randomly in time.

18. The device of claim 17, wherein said changing said point of view of said content or said changing of said focus plane associated with said content is done based on a random spatial point of a scene associated with the 4D light field data.

19. The device of claim 16, wherein said changing further comprises blurring at least part of said content.

20. The device of claim 16, wherein a difference value between consecutive changed points of views of said content or consecutive changed focus planes is defined as a function of said viewing rights associated with the degradation level.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The above and other aspects of the invention will become more apparent by the following detailed description of exemplary embodiments thereof with reference to the attached drawings in which:

(2) FIG. 1 presents a flowchart that comprises some steps of a method for displaying according to one embodiment of the disclosure;

(3) FIG. 2 presents a flowchart that comprises some steps of a method for displaying according to another embodiment of the disclosure;

(4) FIG. 3 presents an example of an electronic device that can be used to perform one or several steps of methods disclosed in the present document.

DETAILED DESCRIPTION

(5) FIG. 1 presents a flowchart that comprises some steps of a method for displaying according to one embodiment of the disclosure.

(6) In a step referenced 101, an electronic device (as the one depicted in FIG. 3 of the present document) obtains 4D light-field data as well as viewing rights (or credentials). These viewing rights can be linked to a license that has been obtained after having paid for it.

(7) For example, viewing rights can be associated with a degradation level (such as strong degradation, medium degradation, low degradation, or no degradation) to be applied to the received 4D light-field data.

(8) In a step referenced 102, the electronic device verifies the value of the viewing rights. In the case that no degradations have to be done, the electronic device allows the display device (either a light field display device or a 2D display device) to process the 4D light-field data without restrictions. In the case the electronic device detects that the user should watch a degraded content from the received 4D light-field data, the electronic device controls the display device in such way that the display device has to display a degraded content. In one embodiment of the disclosure, in the case that the display device is 2D display device, the displayed content corresponds to one view extracted from the 4D light-field data. The point of view changes randomly from one view to another possible view, from time to time. For example, in the case that the viewing rights indicate a strong degradation of the displayed content, the displayed content (i.e. the 2D image) is displayed from a view point that changes every two seconds. Hence, the speed at which the point of view is changed is correlated to the value of the viewing rights. Therefore, the more credentials a user has, the more stable the point of view associated with the displayed content is.

(9) In a variant, in the case that the display device is 2D display device, the displayed content (extracted from the 4D light-field data) is associated with a focus plane. The focus plane changes randomly from one possible value to another, from time to time. For example, in the case that the viewing rights indicate a strong degradation of the displayed content, the displayed content (i.e. the 2D image) is displayed with a focal plane that changes every two seconds. Hence, the speed at which the focal plane is changed is correlated to the value of the viewing rights. Therefore, the more credentials a user has, the more stable the focal plane associated with the displayed content is. Obviously, in one embodiment of the invention, the 2D display device can display degraded content in which both the value of the focal plane and the point of view are modified/changed during the display.

(10) In another embodiment of the present principle, when the display device is a light field display device, a similar process for changing the point of views and/or the focal plane of a displayed content is done according to the value of viewing rights. Therefore, the present technique dynamically modifies the points of view during the display of 4D light-field data, when the viewing rights are not sufficient. When the user acquires sufficient viewing rights (e.g. pay-per-view) the process is stopped (or reduced). More precisely, in one embodiment, if a point of view is defined by a focus f (i.e. a pixel in the light field) and a pair of angles of view (, ), then the present technique dynamically adds one additive perturbation (, ) to the pair of angles (, ) of a point of view during the display process. T, the new angle of the point of view is (+, +). The perturbation is a function of time, remarkable by a period and an intensity. The period is the duration before the next update of the point of view along time. The intensity is the maximal value of the perturbation.

(11) Hence, in an embodiment, the perturbations are randomly chosen between zero and the intensity. The period may also be random.

(12) In a variant, the intensity smoothly increases over the time. In a particular case, the intensity may be null at the beginning and for a pre-determined duration. After that, the intensity increases. This can be used for a teasing effect before degrading the user experience.

(13) In a variant, the period smoothly decreases along time. This lets the degraded effect increase along time.

(14) In a variant, the user's chosen point of view is not used at all, the point of view is only determined by the perturbation. In particular, the sequence of point of views can depend from points of interest in the images. In particular, the focus part of the point of view may be perturbed with a term f. Note that the new focus f+f may lie outside the image (in particular for intense perturbation). In this case, it is preferable to choose a new focus f. The choice can be random or chosen among points that appear in each image. Therefore, this embodiment is advantageous in the case when the user does not choose any point of view. When the user later selects a point of view, two cases are possible according to the degradation policy: The same perturbation parameters are applied with respect to the new user point of view. Or the new user point of view is ignored.

(15) It should be noted that, for a given intensity, the visual effect may vary according to the depth of the focus. For a same intensity, the visual impact is more important if the focus is on the background and the impact is less important if the focus is on the foreground.

(16) The displayed content in output of step 102 conveys such modifications.

(17) FIG. 2 presents a flowchart that comprises some steps of a method for displaying according to another embodiment of the disclosure.

(18) In addition to the previous mentioned steps (i.e. the steps 101 and 102), the method for displaying comprises a step referenced 201. The step 201 comprises the blurring of at least part of the content to be displayed.

(19) Usually, the display of a light field content is done as follows: for each pixel p of the displayed image, we have p= m.sub.i p.sub.i where p.sub.i are the pixels corresponding to p in the sub-aperture images i and where m.sub.i is a ponderation value depending from the user chosen focus.

(20) According to the present disclosure, a parameter d.sub.i is added for all pixel. Hence, we have: p= d.sub.i m.sub.i p. When the user owns sufficient viewing rights, for all i, d.sub.i=1 and thus the display/restitution process is unmodified. When the user does not own sufficient viewing rights, d.sub.i varies in range [0;1]. This change introduces a blurring effect on the pixel.

(21) The variation of d.sub.i is determined according to two parameters: a zone Z in the light field: within this zone the modification is active; a fog intensity F: applied within the zone.

(22) An advantage of the present technique is that no additional information nor processing is required in order to create the effect: all the needed visual information is contained in the 4D light-field data. With more traditional content, such as a 2D picture, making the same effect requires additional information (e.g. parameters of a blurring effect) and dedicated processing.

(23) In an embodiment of the disclosure, the zone Z covers the whole 4D light-field data. The value of d.sub.i is randomly chosen in the range of [1-F;1]. This results in a modification of each pixel of the displayed 4D light-field data. The value of F may increase smoothly along the time in order to intensify the degradation. The increase of F may stop at a pre-determine threshold in order to guarantee a minimal user experience. It should be noted that no pre- nor post-processing is required and that the additional operations benefit from knowing optimization of float multiplication. Typically in a set-top-box, there will be no significant additional delay.

(24) In another embodiment, the zone Z does not cover entirely the 4D light-field data. For instance Z may be a circle with center C and radius r. C may be the center of the image and r may be fixed along time, typically one quarter of the image diagonal. Outside Z, d.sub.i=1. Inside Z, the fog effect as defined previously can be applied. Additionally, the intensity F may vary according to the positions of the pixels in Z. For instance, F decreases with the distance to the center of Z. This leads to a stronger modification close to the center and a lesser modification close to the frontier of Z.

(25) In another embodiment, the zone Z itself may vary along time. Typically, the center, the size and the shape may vary along time. In the case of a circle the radius may increase along time up to a predetermined threshold. The center may be chosen randomly. The center may follow a predetermined path within the image. The center may follow one relevant part of the light field (a face, main character, foreground, most rapidly moving object, etc.). The center may also depend from the user: mouse position, eyes focus, etc. The additional delay depends on: the determination of Z, the test to determine if a pixel belongs to Z.

(26) FIG. 3 presents an example of an electronic device that can be used to perform one or more steps of methods disclosed in the present document.

(27) Such device referenced 300 comprises a computing unit (for example a CPU, for Central Processing Unit), referenced 301, and one or more memory units (for example a RAM (for Random Access Memory) block in which intermediate results can be stored temporarily during the execution of instructions a computer program, or a ROM block in which, among other things, computer programs are stored, or an EEPROM (Electrically-Erasable Programmable Read-Only Memory) block, or a flash block) referenced 302. Computer programs are made of instructions that can be executed by the computing unit. Such device 300 can also comprise a dedicated unit, referenced 303, constituting an input-output interface to allow the device 300 to communicate with other devices. In particular, this dedicated unit 303 can be connected with an antenna (in order to perform communication without contacts), or with serial ports (to carry communications contact). It should be noted that the arrows in FIG. 3 signify that the linked unit can exchange data through buses for example together.

(28) In an alternative embodiment, some or all of the steps of the method previously described, can be implemented in hardware in a programmable FPGA (Field Programmable Gate Array) component or ASIC (Application-Specific Integrated Circuit) component.

(29) In an alternative embodiment, some or all of the steps of the method previously described, can be executed on an electronic device comprising memory units and processing units as the one disclosed in the FIG. 3.

(30) In one embodiment of the disclosure, the electronic device depicted in FIG. 3 can be comprised in a light field display device or in a light field acquisition device that are configured to display and/or capture images (i.e. a sampling of a light field). These images are stored on one or more memory units. Hence, these images can be viewed as bit stream data (i.e. a sequence of bits). Obviously, a bit stream can also be converted on byte stream and vice versa.

(31) In one embodiment of the disclosure, the electronic device depicted in FIG. 3 can be comprised in a set-top box, or in a mobile phone.