A SYSTEM AND METHOD FOR SORTING DIGITAL IMAGES

Abstract

A method for sorting digital images, the method comprising the steps of: a) Receiving a digital image acquired with an electronic device associated with a user, the digital image being sent using the electronic device to an electronic image sorting system; b) Generating, using an image analysis portion of the electronic image sorting system, one or more initial identifiers based on the contents of the digital image; c) Sending the one or more initial identifiers to an identifier analysis portion of the electronic image sorting system, wherein the identifier analysis portion includes an electronic database of one or more rules adapted to generate one or more additional identifiers based on the one or more initial identifiers; d) Sending, using the electronic image sorting system, the one or more initial identifiers and/or the one or more additional identifiers, to the electronic device to allow the user to generate one or more selected identifiers from the one or more initial identifiers and/or the one or more additional identifiers; and e) Modifying the one or more rules based on the one or more selected identifiers.

Claims

1. A method for sorting digital images, the method comprising the steps of: a. Receiving a digital image acquired with an electronic device associated with a user, the digital image being sent using the electronic device to an electronic image sorting system; b. Generating, using an image analysis portion of the electronic image sorting system, one or more initial identifiers based on the contents of the digital image; c. Sending the one or more initial identifiers to an identifier analysis portion of the electronic image sorting system, wherein the identifier analysis portion includes an electronic database of one or more rules adapted to generate one or more additional identifiers based on the one or more initial identifiers; d. Sending, using the electronic image sorting system, the one or more initial identifiers and/or the one or more additional identifiers, to the electronic device to allow the user to generate one or more selected identifiers from the one or more initial identifiers and/or the one or more additional identifiers; and e. Modifying the one or more rules based on the one or more selected identifiers.

2. The method according to claim 1 wherein the electronic device is mobile telephone, computing tablet, desktop computer or laptop computer.

3. The method according to claim 1 wherein the electronic image sorting system includes a user interface in which the user may upload the digital image to the electronic image sorting system.

4. The method A method according to claim 1 wherein the image analysis portion includes one or more computer vision units capable of generating the one or more initial identifiers.

5. The method according to claim 1 wherein the image analysis portion generates the one or more initial identifiers based on based on one or more visual cues present in the digital image and/or one or more pieces of information retrieved from one or more information sources external to the electronic image sorting system and/or one or more pieces of information retrieved from metadata associated with the digital image.

6. The method according to claim 1 wherein the one or more rules are stored in an electronic rules database.

7. The method according to claim 1 wherein the identifier analysis portion generates one or more intermediate identifiers prior to the generation of the one or more additional identifiers.

8. The method according to claim 7 wherein the additional identifiers are generated from the intermediate identifiers using fuzzy logic and/or one or more weightings.

9. The method according to claim 8 wherein the one or more weightings include a higher weighting for intermediate identifiers that more closely match the initial identifiers and lower weightings for intermediate identifiers that less closely match the initial identifiers.

10. The method according to claim 1 wherein the one or more additional identifiers and/or the one or more initial identifiers are sent to an electronic interface displayed on a display of the electronic device.

11. The method according to claim 1 wherein at least one of the one or more additional identifiers relates to the human meaning of the digital image.

12. The method according to claim 1 wherein the rules are modified based on the one or more selected identifiers by generating new rules, updating rules with new selected identifiers, modifying the weighting of rules and/or linking or associating two or more rules.

13. The method according to claim 1 wherein the electronic image sorting system is adapted to retrieve digital images from one or more electronic devices based on one or more selected identifiers.

14. A system for sorting digital images, the system comprising: at least one processor, at least one non-transitory computer readable storage medium storing instructions thereon, that, when executed by the at least one processor, cause the system to: a. Receive, from an electronic device associated with a user, a digital image sent using the electronic device to an electronic image sorting system; b. Generate, using an image analysis portion of an electronic image sorting system, one or more initial identifiers based on the contents of a digital image; c. Send, using the electronic image sorting system, the one or more initial identifiers to an identifier analysis portion of the electronic image sorting system, wherein the identifier analysis portion includes an electronic database of one or more rules adapted to generate one or more additional identifiers based on the one or more initial identifiers; d. Send, using the electronic image sorting system, the one or more initial identifiers and/or the one or more additional identifiers, to the electronic device to allow the user to generate one or more selected identifiers from the one or more initial identifiers and/or the one or more additional identifiers; and e. Modify, using the electronic image sorting system, the one or more rules based on the one or more selected identifiers.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0078] Preferred features, embodiments and variations of the invention may be discerned from the following Detailed Description which provides sufficient information for those skilled in the art to perform the invention. The Detailed Description is not to be regarded as limiting the scope of the preceding Summary of the Invention in any way. The Detailed Description will make reference to a number of drawings as follows:

[0079] FIG. 1 illustrates a schematic diagram of a method and system for sorting digital images according to an embodiment of the present invention.

[0080] FIG. 2 illustrates a schematic diagram of a method and system for sorting digital images according to an embodiment of the present invention.

DESCRIPTION OF EMBODIMENTS

[0081] In FIG. 1 there is illustrated a schematic diagram of a system and method for sorting digital images. In this Figure, a user 10 acquires a digital image using an electronic device. In the embodiment of the invention shown in FIG. 1, the electronic device comprises a mobile telephone 11. While it is envisaged that the mobile telephone 11 may include a camera capable of acquiring digital images by taking them using the camera, it will be understood that digital images may be acquired by the mobile telephone 11 in a number of ways, including being downloaded from the Internet, or from within a text message, email message, instant messaging system message and so on.

[0082] Digital images are sent by the user 10 to the electronic image sorting system 13 using the mobile telephone 11. In the embodiment of the invention shown in FIG. 1, the user 10 accesses the electronic image sorting system 13 through an electronic interface 14 provided on the display of the mobile telephone 11. In this embodiment of the invention, it is envisaged that the electronic interface 14 may be provided in the form of an electronic application (i.e. an app) downloaded to the mobile telephone 11. However, it is also envisaged that the user 10 may access the electronic image sorting system 13 through a website or the like.

[0083] In FIG. 1, the user 10 upload a digital image to the electronic image sorting system 13 through the user interface 14. The digital image is received by an image analysis portion 15 of the electronic image sorting system 13, whereupon the image analysis portion 15 (including one or more computer vision units) generates one or more initial identifiers 16 based on the contents of the digital image. The computer vision units in the embodiment of the invention shown in FIG. 1 are a facial recognition/emotion recognition computer vision unit 12a, a colour detection computer vision unit 12b, an age detection computer vision unit 12c, a GPS co-ordinate and location detection computer vision unit 12d, a calendar and date/time detection computer vision unit 12e and an object detection computer vision unit 12f.

[0084] Initial identifiers 16 generated for input to the image analysis portion 17 may include outputs from different categories of computer vision units 12a-12f, metadata associated with the digital image or other sources (such as social media platforms, websites, electronic calendars, electronic location services and the like). For example, the object recognition computer vision unit 12f may output initial identifiers 16 such as chair and table to be input to the image analysis portion 17, while the emotion recognition computer vision unit 12c may output an initial identifier 16 such as smile. Similarly, the colour detection computer vision unit 12b may output an initial identifier 16 such as red.

[0085] Initial identifiers 16 are sent to an identifier analysis portion 17 of the electronic image sorting system 13. The identifier analysis portion 17 compares the initial identifiers against a database of rules in order to generate one or more additional identifiers 18. At least one of the one or more additional identifiers 18 includes an identifier based on the human meaning of the digital image 24.

[0086] In the embodiment of the invention shown in FIG. 1, only the additional identifiers 18 are sent to the user 10 through the electronic interface 14, although it will be understood that, in other embodiments, one or more initial identifiers 16 may also be sent to the user 10. The specific details of the generation of the additional identifiers will be described in more detail later.

[0087] Upon receipt of the additional identifiers 18, the user 10 selects one or more of the additional identifiers 18 that he or she believes best describe the digital image 24. The additional identifiers 17 chosen by the user 10 become the selected identifiers 19. The selected identifiers 19 are associated with the digital image 24 such that a user 10 can retrieve a particular image by searching the electronic image sorting system 13 for selected identifiers 19.

[0088] Once the selected identifiers 19 are entered into the electronic image sorting system 13, the identifier analysis portion 17 updates the database of rules. The database of rules may be updated by creating new rules relating to the selected identifiers 19, by amending existing rules for a particular additional identifier 18 based on whether the user 10 selected or rejected the additional identifiers 18, or linking two or more rules based on the selection by the user 10 of the additional identifiers 18 associated with the rules.

[0089] In FIG. 2 a schematic illustration of an electronic image sorting system 13 according to an embodiment of the present invention is shown. In this Figure, further details of the method and system shown in FIG. 1 are provided.

[0090] Initially, a digital image 24 is uploaded to the electronic image sorting system 13 by a user. The digital image 24 is analysed by the image analysis portion 15 of the system 13 to generate one or more initial identifiers 16. The image analysis portion 15 comprises a plurality of computer vision units adapted to analyse the digital image 24 for certain objects or properties of the digital image 24. For instance, a first computer vision unit 15a detects information, such as written information contained in digital image 24c. A second computer vision unit 15b detects predominant or significant colours in the digital image 24, while a third computer vision unit 15c detects objects.

[0091] A fourth computer vision unit 15d is adapted to recognise scenes (i.e. locations such as a beach, park etc.) while a fifth computer vision unit 15e recognises emotions within a digital image 24. Emotions are determined by detecting the presence of faces in the image 24 and then determining the expressions on the faces. For instance, a smiling face may generate an initial identifier of happy. In a similar vein, a sixth computer vision unit 15f is adapted to identify individuals within a digital image 24 through facial recognition.

[0092] In the embodiment of the invention shown in FIG. 2, initial identifiers 16 may also be generates by the electronic image sorting system 13 accessing a user's electronic personal data stores 20, such as social media accounts, Cloud storage, electronic calendar, location services (such as the current location of the user or their electronic device, or the location at a particular time in the past, such as when a particular photo was taken). One or more initial identifiers 16 may be generated from this information, such as the location at which a digital image was acquired, the event at which the digital image was acquired, and even the people present in the image, or when the image was acquired.

[0093] The initial identifiers 16 are then sent to an identifier analysis portion 17 of the electronic image sorting system 13. The identifier analysis portion 17 comprises a rules database 21 and a rules engine 22.

[0094] When initial identifiers 16 are received by the identifier analysis portion 17, the rules database 21 consults and/or enacts one or more rules that exactly and/or closely match the initial identifiers 16. In the embodiment of the invention illustrated in FIG. 2, intermediate identifiers may be generated based on the identifiers associated with the consulted and/or enacted rules. The intermediate identifiers are generated based on fuzzy logic.

[0095] In this embodiment of the invention, the one or more intermediate identifiers are weighted according to one or more criteria, such as their degree of matching to the initial identifiers 16, the frequency with which they have been used previously by the user and so on. It is envisaged that the more heavily weighted intermediate identifiers (i.e. those more closely matching the initial identifiers, or those used more frequently by a user) may become the additional identifiers 18, while less heavily weighted intermediate identifiers may be discarded.

[0096] It is envisaged that at least one of the one or more additional identifiers 18 will relate to the human meaning of the digital image 24. Thus, it is envisaged that the rules database 21 includes one or more rules that converts or translates an initial identifier 16 into its human meaning. For instance, while the face recognition computer vision unit 15f may recognise individuals within the digital image 24, the rules database 21 may recognise that the individuals in the digital image 24 together have an additional meaning to the user, such as best friends or classmates. The emotions felt by a user when viewing a particular digital image 24 may also be generated as additional identifiers 18 based on one or more rules in the rules database. For example, where the initial identifiers 16 may identify individuals within the digital image 24, the rules database 21 may generate an additional identifier 18 of best friends which may in turn consult and/or enact one or more further rules to generate additional identifiers 18 such as fun, crazy, love or the like. The additional identifiers 18 may be words, or may be emojis, electronic stickers, images or animations that embody the human meaning (such as a love heart for love and so on).

[0097] Preferably, the rules database 21 is federated across the electronic image sorting system 13, such that the same database of rules 21 is consulted for every user of the electronic image sorting system 13. Alternatively, users may be sorted into one or more categories based on one or more factors (age, gender, interests, geographical location etc.) such that only rules associated with, or relevant to, the one or more categories into which the user is sorted are consulted and/or enacted. In these embodiments of the invention, additional identifiers 18 may be weighted based on how closely they match identifiers selected by other users with similar factors to the user. In a specific example, if the user is a teenage female, additional identifiers 18 frequently used by other teenage females in the same city, state or country may be weighted more heavily than additional identifiers frequently used by males over 50, whether or not located in a similar geographical location.

[0098] Once a suitable number of additional identifiers 18 have been generated by the identifier analysis portion 17, the additional identifiers 18 are sent to the user via an electronic interface 14 that, in the embodiment of the invention illustrated in FIG. 2, is displayed on the electronic display of the electronic device associated with the user.

[0099] The user selects one or more of the additional identifiers 18 that he or she feels best described the digital image 24. The additional identifiers 18 selected by the user become the selected identifiers 19, and the selected identifiers 19 are associated with the digital image 24. A user may also manually generate their own selected identifiers 19 in addition to (or instead of) the additional identifiers 18 generated by the electronic image sorting system 13.

[0100] The selected identifiers 19 are electronically associated with the digital image 24 and are stored electronically by the electronic image sorting system 13 in electronic storage 23. It is envisaged that the digital image 24 will not be stored by the electronic image sorting system, but that a low-resolution thumbnail version of the digital image will be stored, along with the selected identifiers 19 and any other relevant metadata associated with the digital image 24.

[0101] Once the selected identifiers 19 have been generated by the user, the identifier analysis portion 17 reviews the selected identifiers 19 and modifies the relevant rules in the rules database 21. The modification of the rules in the rules database 21 may be of any suitable form. For instance, rules may be given greater or less weighting depending on whether an additional identifier 18 relating to that rule was selected as a selected identifier 19. Alternatively, rules may be linked or associated with one another if additional identifiers 18 relating to those rules are selected as selected identifiers 19 in the same digital image 24. Further, rules may be modified if the selected identifiers 19 relating to a particular object or person in an image change. For instance, a user may change their selected identifiers 19 for images containing a former partner, a deceased relative or the like from happy to sad. The rules in the rules database 21 may be modified accordingly when such a change in emotions occurs.

[0102] In other embodiments, new rules may be created based on selected identifiers 19. This may occur if, for instance, new words or expressions are coined, or if the meaning of words or expressions changes over time.

[0103] In the embodiment of the invention shown in FIG. 2, a user may also retrieve images stored across multiple electronic devices (and/or in Cloud storage or the like) using the electronic image sorting system 13. Specifically, a user may enter one or more selected identifiers into the user interface 14 in order to retrieve all images with which the selected identifier is associated. Preferably, the images that may be retrieved in this manner are limited to the user's own images (as opposed to the images of other users with which the same selected identifier is associated), although it is also envisaged that the user may grant permission to other users (such as friends or family) of the electronic image sorting system 13 to access to their images.

[0104] In the present specification and claims (if any), the word comprising and its derivatives including comprises and comprise include each of the stated integers but does not exclude the inclusion of one or more further integers.

[0105] Reference throughout this specification to one embodiment or an embodiment means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearance of the phrases in one embodiment or in an embodiment in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more combinations.

[0106] In compliance with the statute, the invention has been described in language more or less specific to structural or methodical features. It is to be understood that the invention is not limited to specific features shown or described since the means herein described comprises preferred forms of putting the invention into effect. The invention is, therefore, claimed in any of its forms or modifications within the proper scope of the appended claims (if any) appropriately interpreted by those skilled in the art.