DUAL-APERTURE ZOOM DIGITAL CAMERA WITH AUTOMATIC ADJUSTABLE TELE FIELD OF VIEW
20220385797 · 2022-12-01
Inventors
- Gal Shabtay (Tel Aviv, IL)
- Ephraim Goldenberg (Tel Aviv, IL)
- Eran Kali (Tel Aviv, IL)
- Noy Cohen (Tel Aviv, IL)
- Gil Avraham (Tel Aviv, IL)
- Ruthy Katz (Tel Aviv, IL)
Cpc classification
H04N23/45
ELECTRICITY
G03B17/17
PHYSICS
H04N23/58
ELECTRICITY
H04N23/951
ELECTRICITY
G02B13/06
PHYSICS
H04N23/55
ELECTRICITY
G03B2217/002
PHYSICS
H04N23/695
ELECTRICITY
H04N23/90
ELECTRICITY
H04N23/69
ELECTRICITY
International classification
Abstract
Digital camera comprising an upright Wide camera configured to provide a Wide image with a Wide image resolution and a folded Tele camera configured to provide a Tele image with a Tele image resolution higher than the Wide image resolution, the Wide and Tele cameras having respective Wide and Tele fields of view FOV.sub.W and FOV.sub.T and respective Wide and Tele image sensors, the digital camera further comprising a rotating OPFE operative to provide a folded optical path between an object or scene and the Tele image sensor, wherein rotation of the OPFE moves FOV.sub.T relative to FOV.sub.W. In some embodiments, a rectangular FOV.sub.T is orthogonal to a rectangular FOV.sub.W. When included in a host device having a user interface that displays FOV.sub.T within FOV.sub.W, the user interface may be used to position FOV.sub.T relative to FOV.sub.W, scan FOV.sub.T across FOV.sub.W and acquire, store and display separate Wide and Tele images, composite Wide plus Tele images and stitched Tele images. The positioning of FOV.sub.T within FOV.sub.W, can be done automatically (autonomously) by continuously tracking an object of interest.
Claims
1. A mobile device, comprising: a Wide camera that provides Wide images with a Wide field of view FOV.sub.W, wherein the Wide camera includes a Wide lens with a Wide lens optical axis and a Wide image sensor; a Tele camera that provides Tele images with a Tele field of view FOV.sub.T smaller than the FOV.sub.W; and a camera controller configured to move FOV.sub.T relative to FOV.sub.W, causing other portions of a scene to become a Tele image.
2. The mobile device of claim 1, wherein the Wide images have a Wide camera resolution, wherein the Tele images have a Tele camera resolution higher than the Wide camera resolution.
3. The mobile device of claim 1, wherein a ratio M=FOV.sub.T/FOV.sub.W is in the range 1/4<M<3/4.
4. The mobile device of claim 1, wherein the camera controller is configured to track an object with FOV.sub.T autonomously.
5. The mobile device of claim 4, wherein the camera controller is configured to use face detection to decide whether to track an object with FOV.sub.T autonomously.
6. The mobile device of claim 4, wherein the autonomous tracking of the object with FOV.sub.T autonomously is done such that the object is as close as possible to a center of the moved FOV.sub.T.
7. The mobile device of claim 1, wherein the camera controller is configured to move FOV.sub.T by ±30 degrees around a zero position.
8. The mobile device of claim 1, wherein the camera controller is configured to record a Wide image and a Tele image during the FOV.sub.T moving and to fuse the Wide image and the Tele image to form a composite image.
9. The mobile device of claim 1, wherein the camera controller is configured to record a Wide image and a plurality of Tele images during the FOV.sub.T moving and to fuse the Wide image and the plurality of Tele images to form a composite image, wherein the Wide image captures an entire scene simultaneously and wherein the plurality of Tele images captures the entire scene consecutively.
10. The mobile device of claim 1, wherein the camera controller is configured to record Wide video streams and Tele video streams during the FOV.sub.T moving.
11. The mobile device of claim 1, wherein in a still mode the camera controller is configured to acquire a scene by both cameras simultaneously, and wherein the Wide camera image provides an entire scene and the Tele camera image provides a part of the scene.
12. The mobile device of claim 1, wherein the Wide camera is an upright camera.
13. The mobile device of claim 1, included in a smartphone.
14. A digital camera, comprising: a Wide camera that provides Wide images with a Wide field of view FOV.sub.W, wherein the Wide camera includes a Wide lens with a Wide lens optical axis and a Wide image sensor, and wherein the Wide camera is configured to recognize an object; a Tele camera that provides Tele images with a Tele field of view FOV.sub.T smaller than the FOV.sub.W, wherein the Tele camera is configured to receive data about the object from the Wide camera and to perform autonomous FOV.sub.T tracking of the object in response to recognition of the object.
15. The digital camera of claim 14, wherein the Wide images have a Wide camera resolution, wherein the Tele images have a Tele camera resolution higher than the Wide camera resolution.
16. The digital camera of claim 14, wherein a ratio M=FOV.sub.T/FOV.sub.W is in the range 1/4<M<3/4.
17. The digital camera of claim 14, configured to use face detection to recognize the object.
18. The digital camera of claim 14, wherein the autonomous FOV.sub.T tracking of the object is done such that the object is as close as possible to a center of FOV.sub.T.
19. The digital camera of claim 14, configured to move FOV.sub.T by ±30 degrees around a zero position.
20. The digital camera of claim 14, configured to record a Wide image and a Tele image during the FOV.sub.T moving and to fuse the Wide image and the Tele image to form a composite image.
21. The digital camera of claim 14, configured to record a Wide image and a plurality of Tele images during the autonomous FOV.sub.T tracking and to fuse the Wide image and the plurality of Tele images to form a composite image, wherein the Wide image captures an entire scene simultaneously and wherein the plurality of Tele images captures the entire scene consecutively.
22. The digital camera of claim 14, configured to record Wide video streams and Tele video streams during the autonomous FOV.sub.T tracking and to fuse the Wide images and Tele images together to form a composite video stream.
23. The digital camera of claim 14, configured to acquire in still mode a scene by both cameras simultaneously, wherein the Wide camera image provides an entire scene and the Tele camera image provides a part of the scene.
24. The digital camera of claim 14, wherein the Wide camera is an upright camera.
25. The digital camera of claim 14, included in a smartphone.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Non-limiting examples of embodiments disclosed herein are described below with reference to figures attached hereto that are listed following this paragraph. Identical structures, elements or parts that appear in more than one figure are generally labeled with a same numeral in all the figures in which they appear. The drawings and descriptions are meant to illuminate and clarify embodiments disclosed herein, and should not be considered limiting in any way.
[0023]
[0024]
[0025]
[0026]
[0027]
[0028]
[0029]
[0030]
[0031]
[0032]
[0033]
[0034]
[0035]
[0036]
[0037]
[0038]
[0039]
[0040]
[0041]
[0042]
[0043]
[0044]
[0045]
[0046]
DETAILED DESCRIPTION
[0047] s
[0048]
[0049] The operation of a camera such as camera 200 is described in more detail in U.S. patent application Ser. No. 14/455,906. In particular, the prism can perform a tilt (rotation) movement 216 around the X-axis as shown. The rotation motion may be caused by a stepping motor 302 shown schematically in
[0050]
[0051] The rotation of prism 212 around the X-axis moves the Tele FOV relative to the Wide FOV, causing other portions of scene 106 to become a “Tele image” with higher resolution. Thus,
[0052] As mentioned, in
[0053]
[0054] Note that a similar FOV.sub.T relative positioning effect to that described above may be obtained by in-plane rotating Tele sensor 214 by 90 degrees and by leaving Wide sensor 208 unchanged from its original position in
[0055] When a dual-aperture camera described above is included for example in a smartphone, the Tele image (i.e. the part of scene 106 viewed and acquired by the Tele camera) is bound by a frame 932 visible on the smartphone screen.
[0056] In still mode, scene 106 is acquired by both cameras, with the Wide camera providing the entire image seen (i.e. the Wide image) and the Tele camera providing the part of scene 106 bound by frame 932. Smartphone 900 further includes, on a front side opposite to the back side, a screen or display 902 displaying a view of scene 106. Screen 902 may display icons or text “A”, “B”, “C”, etc., that provide indications and/or are selectable to perform various operations of the phone and/or the cameras. Such icons or text may be indicative of flash setting, video or stills selection, back or front camera selection, etc. The square boxes surrounding “A”, “B” and “C” are merely illustrative and may have different shape or be removed altogether in some cases. Note that the fact that only three icons are shown is not meant to be limiting, and that more or fewer icons may be displayed and/or selectable at any time during or prior to image acquisition by the cameras and/or during display of acquired images. In an embodiment of the dual-aperture camera as in
[0057] In various embodiments and as described in more detail in PCT/IB2016/056060, smartphone 900 may have a user interface that includes a single camera icon (or “button”) 908 and a “two-camera” button 910. The two-camera button may appear on the screen when the FOV of the scene (FOV.sub.scene) is greater or equal to FOV.sub.T. As described in detail in PCT/IB2016/056060, the user interface displays visually the almost exact FOV.sub.T and enables simple acquisition of the image within FOV.sub.T, thereby providing a Tele image with the highest resolution enabled by the Tele camera. The user interface also enables simultaneous acquisition (with a single input through the user interface, i.e. using two-camera button 910) of a Wide image and a Tele image. Image fusion of the Wide and Tele images or video streams can take place on the capturing device or in a cloud environment.
[0058] The present inventors have determined that, advantageously, a user interface as described above can be used to “drag” frame 932 (and FOV.sub.T) on screen 902 to bring different parts of the scene into FOV.sub.T. That is, the dragging of frame 932 is translated into rotation of the “folded” path prism, such that the higher resolution Tele image “moves” to different parts of the scene. The dragging may be performed by a firm touch of the screen by a finger 920 and movement of the finger across the screen. In
[0059] As described in detail in PCT/IB2016/056060, in terms of image acquisition, a user may press two-camera button 910 to simultaneously acquire two images, the Wide image of scene 106 at its respective (lower) image resolution and the Tele image of region (frame) 932 (or 932′) at its respective (higher) image resolution. The two images may be stored in an on-board storage (such as “camera roll” in an iPhone) and may be displayed or downloaded for further use as known in the art. The user may press single camera button 908 to acquire only the Wide image, which can further be stored, displayed and downloaded for further use. The user may choose for display on screen 902 only the Tele image by, for example, double-tapping or pressing at any point on the screen within frame 932 (or 932′). This action leads to display of the Tele image on the entire screen. The Tele image (only) can then be chosen for acquisition by pressing on single camera button 908. The acquired Tele image can then be stored, displayed and downloaded for further use as above. The two images can also be fused (on the camera hosting device or in a cloud) into a composite image with a portion marked by a frame 932 or 932′ formed by the higher-resolution Tele image, and with a peripheral portion formed by a peripheral portion of the relatively lower resolution Wide image.
[0060] Clearly, frame 932′ (and the Tele FOV) may be dragged to any intermediate position (not shown) between the right and left positions. In other words, the Tele FOV may be moved laterally on the screen to a number of partially overlapping or non-overlapping (but touching) positions, from a right-most position (at a right screen edge 940) to a left-most position (at a left screen edge 942) or vice-versa, and an entire image of the scene may be “stitched” together from the partially overlapping or non-overlapping Tele images. For example, as shown in
[0061] Note that the direction of prism rotation and the consequent movement of FOV.sub.T relative to FOV.sub.W on a smartphone (or any other host device) screen depends on the geometry of the assembly of the dual-aperture camera in the host device. The description above relates to one particular such geometry. In a different geometry, the prism rotation directions and the resulting FOV.sub.T movement may be in opposite directions to those described above.
[0062] The devices, used interface and associated methods disclosed above may be used for automatic movement or “automatic adjustment” of the Tele FOV for e.g. tracking a subject in an autonomous manner. We refer to a camera mode that performs automatic Tele FOV movement to track an object or subject of interest as “autonomous Tele FOV tracking”. The autonomous Tele FOV movement is in response to recognition (through e.g. the smart-phone camera) of the object or subject of interest, and the Tele image focuses on and displays the object or subject of interest. The object recognition may be performed using any of the methods known in the art. An example of autonomous Tele FOV tracking is shown in
[0063]
[0064] While the smartphone shown in
[0065] Wide and Tele images and/or video streams can be recorded during the automatic tracking mode and fused together to form a composite image or composite video stream. This fusion can be applied on the camera hosting device or alternatively, Wide and Tele images or video streams can be uploaded to the cloud for applying this fusion operation. Each composite image may also be formed with FOV.sub.W by scanning with the Tele camera, stitching a plurality of Tele images to provide a “stitched” Tele image, then fusing the stitched Tele image with the Wide image. This is advantageous in that the Wide image captures the entire scene simultaneously, while the Tele images to be stitched together are consecutive, so one can overcome motion or occlusions in the scene if required. The stitching of the Tele images and/or the fusion of the stitched Tele image with the Wide image may also be performed in a cloud.
[0066] While this disclosure has been described in terms of certain embodiments and generally associated methods, alterations and permutations of the embodiments and methods will be apparent to those skilled in the art. The disclosure is to be understood as not limited by the specific embodiments described herein, but only by the scope of the appended claims.