FOCUS PULLING WITH DISTANCE INFORMATION FROM A SECONDARY CAMERA SYSTEM

Abstract

A method for switching focus between objects found simultaneously in an image area from a main camera where the focus of the main camera and a switch between different focus distances is controlled by a remote follow focus unit. A secondary system and a computer system are used to measure the distance to the first and second object simultaneously and continuously. Distance information from the secondary system regarding distance to the first object is used by the follow focus unit as a focus distance to set the focus of the main camera on the first object. Distance information from the secondary system regarding distance to the second object is used by the follow focus unit as a focus distance to switch the focus of the main camera from the first object to the second object. The secondary system and the computer system obtain distance information using a secondary camera system.

Claims

1. A method for switching focus between objects (a, b, c, d, e) found simultaneously in an image area from a main camera (1) used in film production where the focus of said main camera and a switch between different focus distances (d.sub.a, d.sub.b, d.sub.c, d.sub.d, d.sub.e) can be controlled by means of a remote follow focus (2) unit connected to a turning mechanism (3) for controlling the focus of said main camera (1), where a secondary system (4) and a computer system (5) is used for tracking the distance (d.sub.a, d.sub.b) to at least a first and a second object (a, b) by measuring the distance (d.sub.a, d.sub.b) to both said first and said second object (a, b) simultaneously and continuously, where distance information from said secondary system (4) regarding the distance (d.sub.a) to said first object (a) is used by said follow focus unit (3) as a focus distance to set the focus of said main camera (1) on said first object (a), and where the distance information from said secondary system (4) regarding the distance (d.sub.b) to said second object (b) is used by said follow focus unit (3) as a focus distance to switch the focus of said main camera (1) from said first object (a) to said second object (b), characterized in, that said secondary system (4) comprises a secondary camera system (41), such as one camera or a stereo vision camera system, and that said method comprises that said computer system (5) produces said distance information by means of images from said secondary camera system (41).

2-3. (canceled)

4. A method according to claim 1, characterized in, that each object (a, b, c, d, e) is represented by a knob (2a, 2b, 2c, 2d, 2e) on said follow focus unit (2), and that said method comprises selecting an object (b) by causing a switch of focus to said selected object (b) when a knob (2b) representing said selected object (b) is activated.

5. A method according to claim 4, characterized in, that a selector (21) for manual or automatic mode on said follow focus unit (2) can be set to manual switching or automatic switching, and that said method comprises performing an automatic switch of focus when said selector (21) is set to automatic mode, and performing a manual switch of focus when said selector (21) is set to manual mode, that a focus wheel (22) on said follow focus unit (2) is used to switch focus in said manual mode, and that said method comprises providing a tactile response in said focus wheel (22) through which said remote focus unit (2) indicates when the focus distance (d.sub.b) for selected object (b) is approaching and finally has been reached.

6. (canceled)

7. A method according to claim 5, characterized in, that each knob (2a, 2b, 2c, 2d, 2e) for representing an object (a, b, c, d, e) is provided with a tuning wheel (2a, 2b, 2c, 2d, 2e), and that said method comprises providing a manual fine tuning of a focus distance to a selected object by means of the tuning wheel corresponding to the knob representing said selected object.

8. A method according to claim 5, characterized in, said follow focus unit (2) automatically maintaining said selected object in focus by means of information from said continuous distance tracking to said selected object once focus has been set.

9. A method according to claim 1, characterized in, that an object detection algorithm is used to identify and track said objects, that said characteristic features of said objects that are to be tracked are identified and stored as positive classifiers to be used by said object detection algorithm and that characteristic features of objects (f) that are to be ignored, such as objects that are not to be tracked during the filming process or objects in the background of a scene to be filmed, are identified (93a) and stored (93b) as negative classifiers to be used by said object detection algorithm.

10-15. (canceled)

16. A method according to claim 1, characterized in, presenting the image from said main camera (1) in real time by means of a graphic user interface (6), and allowing a user to define said at least first and second object (a, b) by means of said graphic user interface (6).

17. An arrangement (A) for switching focus between objects (a, b, c, d, e) found simultaneously in an image area from a main camera (1) used in film production, said arrangement (A) comprising a remote follow focus unit (2) connected to a turning mechanism (3) adapted to control the focus of said main camera (1), said remote follow focus unit (2) being adapted to enable the control of the focus of said main camera (1) and the switch between different focus distances (d.sub.a, d.sub.b, d.sub.c, d.sub.d, d.sub.e) for said main camera (1), where said arrangement (A) comprises a secondary system (4) and a computer system (5), where said computer system (5) and secondary system (4) is adapted to track the distance to at least a first and a second object (a, b) by measuring the distance (d.sub.a, d.sub.b) to both said first and said second object (a, b) simultaneously and continuously, where said follow focus unit (2) is adapted to receive distance information from said computer system (5) regarding the distance (d.sub.a) to said first object (a) as a focus distance to set the focus of said main camera (1) on said first object (a), and where said follow focus unit (2) is adapted to receive distance information from said computer system (5) regarding the distance (d.sub.b) to said second object (b) as a focus distance to enable the switch of focus of said main camera (1) from said first object (a) to said second object (b), characterized in, that said secondary system (4) comprises a secondary camera system (41), such as one camera or a stereo vision camera system, and that said computer system, (5) is adapted to produce said distance information from images captured by said secondary camera system (41).

18-19. (canceled)

20. An arrangement according to claim 17, characterized in, that said follow focus unit (2) comprises a knob (2a, 2b, 2c, 2d, 2e) adapted to represent each object (a, b, c, d, e), and that said follow focus unit (2) is adapted to switch focus to a selected object as a response to an activation of a knob (2b) representing said selected object (b).

21. An arrangement according to claim 20, characterized in, that said follow focus unit (2) comprises a selector (21) for manual or automatic mode adapted to enable the setting of manual switching or automatic switching, that said follow focus unit (2) is adapted to perform an automatic switch of focus as a response to that said selector (21) is set on automatic mode, and that said follow focus (2) unit is adapted to perform a manual switch of focus as a response to that said selector (21) is set on manual mode, that said follow focus unit (2) comprises a focus wheel (22) adapted to be used to switch focus in said manual mode, that said focus wheel (22) can be marked with markings (22a, 22b, 22c, 22d, 22e) representing a setting on the focus wheel (22), that said follow focus unit (2) is adapted to provide switching of focus from one object (a) to another (b) by manual pulling of said focus wheel (22) from one mark (22a) to another (22b), that said focus wheel (22) comprises said selector (21) for manual or automatic mode, that the focus wheel (22) is adapted to be pressed to toggle said selector (21) between manual and automatic mode, and that said remote follow focus unit (2) comprises an indicator (23) adapted to show if said selector (21) is set to manual or automatic mode.

22-23. (canceled)

24. An arrangement according to claim 21, characterized in, that said follow focus unit (2) is adapted to provide a tactile response in said focus wheel (22) through which said remote focus unit (2) indicates when the focus distance for selected object is approaching and finally has been reached.

25. An arrangement according to claim 21, characterized in, that each knob (2a, 2b, 2c, 2d, 2e) adapted to represent an object (a, b, c, d, e) is provided with a tuning wheel (2a, 2b, 2c, 2d, 2e), and that said remote focus unit 2 is adapted to provide a manual fine tuning of a focus distance to a selected object by means of the tuning wheel (2b) corresponding to the knob (2b) representing said selected object (b).

26. (canceled)

27. An arrangement according to claim 20, characterized in, that said remote follow focus unit (2) has five knobs (2a, 2b, 2c, 2d, 2e) for representing objects (a, b, c, d, e), one knob for each finger on the hand of an operator, and that said knobs (2a, 2b, 2c, 2d, 2e) are positioned on said remote follow focus unit (2) so that each knob can be reached by a finger on the hand of an operator, and that said focus wheel (22) is positioned on said remote follow focus unit (2) so that when said remote follow focus unit (2) is held in one hand to operate said knobs (2a, 2b, 2c, 2d, 2e) for representing objects said focus wheel (22) is easily accessible to the other hand of said operator.

28. (canceled)

29. An arrangement according to claim 21, characterized in, said follow focus unit is adapted to automatically maintaining said selected object (b) in focus by means of information from said continuous distance tracking to said selected object once focus has been set.

30. An arrangement according to claim 17, characterized in, that said computer system (5) is adapted to use an object detection algorithm to identify and track said objects, that said computer system (5) is adapted to identify and store characteristic features of said objects that are to be tracked as positive classifiers to be used by said object detection algorithm, and that said computer system (5) is adapted to identify and store characteristic features of objects that are to be ignored (93), such as objects that are not to be tracked during the filming process or objects in the background of a scene to be filmed, as negative classifiers to be used by said object detection algorithm.

31-36. (canceled)

37. An arrangement according to claim 17, characterized in, that said arrangement comprises a graphic user interface (61), that said computer system (5) is adapted to present the image from said main camera in real time by means of said graphic user interface (61), and that said graphic user interface (61) is adapted to provide the possibility for a user to define said at least first and second object (a, b).

38. A remote follow focus unit characterized in, that said remote follow focus unit is adapted to function as a remote follow focus unit (2) in an arrangement (A) according to claim 17.

39. A computer program product (7), characterized in, that said computer program product (7) comprises computer program code (7a), which, when executed by a computing unit, enables said computing unit to perform the steps of a method according to claim 1.

40. A non-volatile computer readable medium (8), characterized in, that computer program code (7a) according to claim 39 is stored on said computer readable medium (8).

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0083] A method, a system, a computer program product and a computer readable medium according to the present disclosure will now be described in detail with reference to the accompanying drawings, in which:

[0084] FIG. 1 is a schematic and very simplified illustration of a setup for film production,

[0085] FIG. 2 is a schematic, yet more detailed, illustration of a remote follow focus unit,

[0086] FIG. 3 is a flow chart for the initiating steps of a production, and

[0087] FIG. 4 is an exemplifying embodiment of a computer readable medium.

DESCRIPTION OF EMBODIMENTS AS PRESENTLY PREFERRED

[0088] The present disclosure will now be described in more detail with reference to FIG. 1 showing a method for switching focus between objects a, b, c, d, e found simultaneously in an image area from a main camera 1 used in film production where the focus of the main camera 1 and a switch between different focus distances d.sub.a, d.sub.b, d.sub.c, d.sub.d, d.sub.e can be controlled by means of a remote follow focus unit 2 connected to a turning mechanism 3 adapted to control the lens 11 and focus of the main camera 1.

[0089] The remote follow focus unit is shown schematically in FIG. 1, as is other components, and for clarity reference is also made to FIG. 2 where a remote follow focus unit is shown in more detail.

[0090] The method disclosed herein teaches that of a secondary system 4 and a computer system 5 is used for tracking the distance to at least a first and a second object a and b by measuring the distance d.sub.a, d.sub.b to both the first and the second object a, b simultaneously and continuously. In the figure, several objects a, b, c, d, e are illustrated and the method disclosed herein is not limited to the number of objects.

[0091] The distance information from the secondary system 4 regarding the distance d.sub.a to the first object a is used by the follow focus unit 3 as a focus distance to set the focus of the main camera 1 on the first object a, and the distance information regarding the distance d.sub.b to the second object b is used as a focus distance to switch the focus of the main camera 1 from the first object a to the second object b.

[0092] This distance information is thus the information sent to the turning mechanism 3 on the main camera 1 for the setting of the focus control on the lens 11 of the main camera 1.

[0093] The present disclosure teaches that the secondary system 4 comprises a secondary camera system 41 and that the method comprises that the computer system 5 produces the distance information by means of images from the secondary camera system 41.

[0094] It is proposed that one camera or a stereo vision camera system is used for the secondary camera system 41.

[0095] It is proposed that each object a, b, c, d, e is represented by a knob 2a, 2b, 2c, 2d, 2e on the follow focus unit 2, and that the method comprises selecting an object b by causing a switch of focus to the selected object b when a knob 2b representing the selected object b is activated.

[0096] It is also proposed that a selector 21 for manual or automatic mode on the follow focus unit 2 can be set to manual switching or automatic switching, and that the method comprises performing an automatic switch of focus when the selector 21 is set to automatic mode, and performing a manual switch of focus when the selector 21 is set to manual mode.

[0097] When in manual mode a focus wheel 22 on the follow focus unit 2 is used to switch focus from one object to another, such as from the first object a to the second object b. The method comprises providing a tactile response in the focus wheel 22 through which the remote focus unit 2 indicates when the focus distance d.sub.b for selected object b is approaching and finally has been reached.

[0098] The present disclosure teaches that each knob 2a, 2b, 2c, 2d, 2e for representing an object a, b, c, d, e can be provided with a tuning wheel 2a, 2b, 2c, 2d, 2e, and that the method comprises providing a manual fine tuning of a focus distance to a selected object by means of the tuning wheel corresponding to the knob representing the selected object.

[0099] The method may also comprise that the follow focus unit 2 automatically maintains the selected object in focus by means of information from the continuous distance tracking to the selected object once focus has been set.

[0100] The present disclosure teaches that the method comprises the use of an object detection algorithm to identify and track the objects, and that the characteristic features of the objects that are to be tracked are identified and stored as positive classifiers to be used by the object detection algorithm.

[0101] FIG. 3 illustrates a flow chart showing the method of the present disclosure. After a first start 91 it is possible to choose 92 a manual initiation. If manual initiation is chosen then an initiating phase is started where the objects are identified and stored, which initiating phase comprises: [0102] manually identifying an object 92a, and [0103] identifying and storing at least one characteristic feature of the object as a positive classifier to be used by the object detection algorithm 92b,

[0104] where the steps are repeated 92c for each object to be tracked.

[0105] With the purpose of enhancing the accuracy of the object detection algorithm of the present disclosure teaches that it is possible to choose 93 that characteristic features of objects that are to be ignored, such as objects that are not to be tracked during the filming process or objects in the background of a scene to be filmed, are identified and stored as negative classifiers to be used by the object detection algorithm.

[0106] Thus FIG. 3 also shows that the initiating phase comprises the possibility to choose 93: [0107] manually identifying an object to be ignored 93a, and [0108] identifying and storing at least one characteristic feature of the object to be ignored as a negative classifier to be used by the object detection algorithm 93b,

[0109] where the steps are repeated for selected objects to be ignored 93c.

[0110] As illustrated in FIG. 3 it is shown that it is possible to select 94 the use of previous knowledge of an object and its characteristic features by means of a known set of objects 95. This knowledge can for instance come from a catalogue of objects and thereto belonging classifiers through which it is possible to select and store both positive and negative classifiers of objects pertinent to the film production. Such catalogue can for instance hold pictures of actors.

[0111] It is also possible to select 96 the use of a training video with positive and/or negative classifiers to learn to recognize the objects 97 as they vary in shape, scale and rotation by applying a training algorithm.

[0112] The film production 98 can be started, or resumed after being paused 99, without an initiating phase 92 and with the use of previously stored objects 95 and thereto belonging positive and possibly negative classifiers. This knowledge can for instance come from a catalogue of objects and thereto belonging classifiers through which it is possible to select and store both positive and negative classifiers of objects pertinent to the film production.

[0113] With renewed reference to FIGS. 1 and 2 it is shown that a user interface 6 for the use of the inventive method can be provided by presenting the image from the main camera 1 in real time by means of a graphic user interface 61, and allowing a user to define the at least first and second object a, b by means of the graphic user interface 61.

[0114] The present disclosure also relates to an arrangement A for switching focus between objects a, b, c, d, e found simultaneously in an image area from a main camera 1 used in film production.

[0115] The arrangement A comprises a remote follow focus unit 2 connected to a turning mechanism 3 adapted to control the focus of the main camera 1. The remote follow focus unit 2 is adapted to enable the control of the focus of the main camera 1 and the switch between different focus distances d.sub.a, d.sub.b, d.sub.c, d.sub.d, d.sub.e for the main camera.

[0116] The arrangement A comprises a secondary system 4 and a computer system 5 adapted to track the distance to at least a first and a second object a, b by measuring the distance d.sub.a, d.sub.b to both the first and the second object a, b simultaneously and continuously. The number of objects is exemplified by five objects in the figure, the present disclosure is however not limited to a specific number of objects.

[0117] The follow focus unit 2 is adapted to receive distance information from the computer system 5 regarding the distance d.sub.a to the first object a as a focus distance to set the focus of the main camera 1 on the first object a, and to receive distance information from the computer system regarding the distance d.sub.b to the second object b as a focus distance in order to enable the switch of focus of the main camera from the first object a to the second object b.

[0118] The present disclosure specifically teaches that the secondary system 4 comprises a secondary camera system 41, and that the computer system 5 is adapted to produce the distance information from images captured by the secondary camera system 41. It is proposed that the secondary camera system 41 comprises one camera or is a stereo vision camera system.

[0119] The follow focus unit 2 comprises a knob 2a, 2b, 2c, 2d, 2e adapted to represent each object a, b, c, d, e, and the follow focus unit 2 is adapted to switch focus to a selected object b as a response to an activation of a knob 2b representing the selected object b.

[0120] The follow focus unit 2 comprises a selector 21 for manual or automatic mode adapted to enable the setting of manual switching or automatic switching. The follow focus unit 2 is adapted to perform an automatic switch of focus as a response to that the selector 21 is set on automatic mode, and perform a manual switch of focus as a response to that the selector 21 is set on manual mode.

[0121] The follow focus unit 2 comprises a focus wheel 22 adapted to be used to switch focus in the manual mode. The focus wheel 22 can be marked with markings 22a, 22b, 22c, 22d, 22e representing settings on the focus wheel 22 and the switching of focus from one object a to another b is done by manually pulling the focus wheel 22 from one mark 22a to another 22b. Fine tuning can also done manually with the same control.

[0122] The selector 21 for manual or automatic mode can be part of the focus wheel 22, where the focus wheel 22 is pressed to toggle between manual and automatic mode. An indicator 23 on the remote follow focus unit can show if the selector is set to manual or automatic mode. The indicator 23 is in FIG. 2 illustrated as being a ring of light encircling the selector 21 for manual or automatic mode which indicator 23 could indicate manual mode with green light and automatic mode with red light.

[0123] The remote follow focus unit 2 can be adapted to provide a tactile response in the focus wheel 22 through which the remote follow focus unit 2 indicates when the focus distance for selected object is approaching and finally has been reached while switching focus in manual mode.

[0124] It is proposed that each knob 2a, 2b, 2c, 2d, 2e adapted to represent an object can be provided with a tuning wheel 2a, 2b, 2c, 2d, 2e, and that the remote follow focus unit 2 is adapted to provide a manual fine tuning of a focus distance to a selected object b by means of the tuning wheel 2b corresponding to the knob 2b representing the selected object b.

[0125] It is proposed that the remote follow focus unit 2 is a hand held unit with a shape that is ergonomically shaped in order to allow a user to hold and operate the control unit over long periods of time.

[0126] It is also proposed that the remote follow focus unit 2 has five knobs 2a, 2b, 2c, 2d, 2e for representing objects, one knob for each finger on the hand of an operator, and that these knobs are positioned on the remote follow focus unit 2 so that each knob 2a, 2b, 2c, 2d, 2e can be reached by a finger on the hand of an operator.

[0127] It is also proposed that the focus wheel 22 is positioned on the remote follow focus unit 2 so that when the remote follow focus unit 22 is held in one hand to operate the knobs 2a, 2b, 2c, 2d, 2e for representing objects a, b, c, d, e the focus wheel 22 is easily accessible to the other hand of the operator.

[0128] The remote follow focus unit 2 can be adapted to automatically maintain the selected object b in focus by means of information from the continuous distance tracking to the selected object once focus has been set.

[0129] With the purpose of providing a possibility to track different objects it is proposed that the computer system 5 is adapted to use an object detection algorithm to identify and track the objects, and that the computer system 5 is adapted to identify and store characteristic features of the objects that are to be tracked as positive classifiers to be used by the object detection algorithm.

[0130] With reference to FIG. 3, it is shown that the computer system 5 can be adapted to perform an initiating phase 92 where the objects are identified and stored, comprising: [0131] that the computer system is adapted to manual identification of an object 92a, and [0132] that the computer system is adapted to identify and store at least one characteristic feature of the object as a positive classifier to be used by the object detection algorithm 92b,

[0133] and that the computer system is adapted to repeat these steps for each object to be tracked as a part of the initiating phase 92c. [0134] The computer system can also be adapted to identify and store characteristic features of objects that are to be ignored 93, such as objects that are not to be tracked during the filming process or objects in the background of a scene to be filmed, as negative classifiers to be used by the object detection algorithm.

[0135] It is thus proposed that the computer system is adapted to manually identify an object to be ignored, that the computer system is adapted to identify and store at least one characteristic feature of the object to be ignored 93a as a negative classifier to be used by the object detection algorithm 93b, and that the computer system is adapted to repeat these steps for selected objects to be ignored as a part of the initiating phase 93c.

[0136] It is also possible that the computer system is adapted to use previous knowledge of an object and its characteristic features by means of a known set of objects 95, such as a catalogue with pictures of actors.

[0137] It is proposed that the computer system is adapted to use a training video with positive and/or negative classifiers where a training algorithm is applied to learn to recognize the objects as they vary in shape, scale and rotation 97.

[0138] The computer system can be adapted to start the tracking of objects, or resume the tracking of objects after a pause 99 without an initiating phase with the use of previously stored objects and thereto belonging positive and possibly negative classifiers.

[0139] In order to provide a user interface 6 for the inventive arrangement A it is proposed that the arrangement comprises a graphic user interface 61, that the computer system is adapted to present the image from the main camera 1 in real time by means of the graphic user interface 61, and that the graphic user interface 61 is adapted to provide the possibility for a user to define the at least first and second object a, b and to view the results from switching between objects and fine tuning the focus on selected object.

[0140] In FIG. 1, the computing unit 5 is a part of the secondary system 4, however, it should be noted that the computing system 5 may be any kind of computer. It can be a part of the remote follow focus unit 2, it can be a part of the secondary system 4, as illustrated in FIG. 1, or it can be a separate mobile computer unit, such as a lap-top, a smart phone, personal assistant or a stationary computer on the set of the film production.

[0141] The computing unit 5 comprises, or communicates with, any required data storage for the storage of objects, classifiers pertaining to the objects, values representing produced distance information pertaining to objects, catalogues of previously known objects and thereto belonging classifiers and other information required to manage the inventive method and arrangement.

[0142] If the computing unit is not a part of the secondary system 4, then the secondary system 4 and the computing unit 5 are adapted to communicate with each other in order to transmit the images from the stereo camera system 41 to the computing unit 5. If required, this communication is wireless communication.

[0143] If the computing unit 5 is not a part of the remote follow focus unit 2, then the computing unit 5 and the follow focus unit 2 are adapted to communicate wirelessly with each other in order to transmit distance information produced by the computing unit 5 to the remote follow focus unit 2.

[0144] The secondary camera system 41 is adapted to be mounted on the main camera 1, or at least in the vicinity of the main camera 1, in order to always provide images required to produce the correct distance information.

[0145] The graphic user interface 61 can be a on a preview screen of the main camera 1, it can be on a user interface of the computing unit 5, or it can be viewed on a separate screen.

[0146] The remote follow focus unit 2 and the turning mechanism 3 for controlling the focus of the main camera 1 are adapted to communicate wirelessly with each other.

[0147] The present disclosure also relates to a remote follow focus unit 2 according to the inventive arrangement A. Even though it is not described here it should be understood that the remote follow focus unit of the present disclosure can be adapted to control also other features of the main camera, such as aperture and zoom settings.

[0148] The present disclosure also relates to a computer program product 7 comprising computer program code 7a, which, when executed by a computing unit, enables the computing unit to perform the steps of a computing unit 5 in the inventive method.

[0149] The present disclosure also relates to a computer readable medium 8, schematically illustrated as a USB-memory in FIG. 4, upon which computer program code 7a according to the inventive computer program product 7 is stored. It should be understood that the computer readable medium can be any kind of computer readable carrier that can carry data and be read by a computer.

[0150] It will be understood that the present disclosure is not restricted to the aforedescribed and illustrated exemplifying embodiments thereof and that modifications can be made within the scope of the present disclosure as defined by the accompanying Claims.