METHOD AND ELECTRONIC ARRANGEMENT FOR ITEM MATCHING

Abstract

The present disclosure generally relates to a computer implemented method for finding a best matching item for a user's body part by comparing a geometrical model of the user's body part with a plurality of statistical models for different items intended to interface with the body part. The present disclosure also relates to a corresponding electronic arrangement and a computer program product.

Claims

1. A computer implemented method performed by an electronic arrangement, the electronic arrangement comprising a processing unit arranged in communication with a display screen and a data capturing arrangement, wherein the method comprises the steps of: acquiring, using the data capturing arrangement, a first set of data representative of a surrounding of a user, determining, using the processing unit and based on the acquired first set of data, an area within the surrounding of the user fulfilling a predefined quality metric, acquiring, using the data capturing arrangement and following an indication that the user has moved to the area fulfilling the predefined quality metric, a second set of data, wherein the second set of data comprises data representative of a body part of the user, estimating, using the processing unit and based on the acquired second set of data, a geometric model of the user's body part, determining, using the processing unit, a matching measurement between the estimated geometrical model and each of a plurality of predefined statistical models each relating to different items for the body part, and displaying, using the display screen, a representation of at least one item for the body part having a matching measurement being above a predetermined matching threshold.

2. The method according to claim 1, further comprising at least one of the steps of: noise-filtering, using the processing unit, the second set of data, or forming, using the processing unit, a plurality of outlines of the body part of the user.

3. The method according to claim 1, further comprising the step of: parameterizing, using the processing unit, the model of the body part of the user.

4. The method according to claim 1, wherein the predefined statistical model for one of the plurality of items is associated with a material or manufacturing property for the item.

5. The method according to claim 1, wherein the step of determining the matching measurement comprises applying a machine learning based processing scheme.

6. The method according to claim 1, wherein the predefined quality metric is determined by: identifying at least one of a plurality of predefined object types within the surrounding of the user.

7. The method according to claim 1, further comprising the step of: segmenting the first set of data into a floor plane, the body part and non-related occluding objects.

8. The method according to claim 1, further comprising the step of: providing, using the processing unit and the display screen, realtime movement information to the user to move to the area fulfilling the predefined quality metric.

9. The method according to claim 1, further comprising the step of: providing, using the processing unit and the display screen, realtime instruction information to the user to acquire the second set of data according to a predefined capturing scheme.

10. The method according to claim 1, further comprising the steps of: analyzing, using the processing unit, the second set of data to determine an indication of a quality level of the second set of data, and forming, using the processing unit and if the quality level is below a predefined threshold, a graphical illustration based on the indication of the quality of the second set of data, wherein the graphical illustration is presented at the display screen to influence the user in further acquisition of the second set of data.

11. An electronic arrangement comprising a processing unit arranged in communication with a display screen and a data capturing arrangement, wherein the processing unit is adapted to: acquire, using the data capturing arrangement, a first set of data representative of a surrounding of a user, determine, based on the acquired first set of data, an area within the surrounding of the user fulfilling a predefined quality metric, acquire, using the data capturing arrangement and following an indication that the user has moved to the area fulfilling the predefined quality metric, a second set of data, wherein the second set of data comprises data representative of a body part of the user, estimate, based on the acquired second set of data, a geometric model of the user's body part, determine a matching measurement between the estimated geometrical model and each of a plurality of predefined statistical models each relating to different items for the body part, and display, at the display screen, a representation of at least one item for the body part having a matching measurement being above a predetermined matching threshold.

12. The electronic arrangement according to claim 11, wherein the processing unit is further adapted to: noise-filter the second set of data, or form a plurality of outlines of the body part of the user.

13. The electronic arrangement according to claim 11, wherein the processing unit is further adapted to: parameterize the model of the body part of the user.

14. The electronic arrangement according to claim 11, wherein the processing unit is further adapted to determine the predefined quality metric by: identify at least one of a plurality of predefined object types within the surrounding of the user.

15-17. (canceled)

18. The electronic arrangement according to claim 11, wherein the processing unit is further adapted to: analyze the second set of data to determining an indication of a quality level of the second set of data, and form a graphical illustrating based on the indication of the quality of the second set of data if the quality level is below a predefined threshold, wherein graphical illustrating is presented at the display screen to influence the user in further acquisition of the second set of data.

19. The electronic arrangement according to claim 11, wherein the processing unit comprises at least a first and a second processing element, wherein the first processing element is arranged remotely from the second processing element.

20. The electronic arrangement according to claim 19, wherein the first processing element, the display screen and the data capturing arrangement are comprised with a mobile electronic user device.

21. The electronic arrangement according to claim 19, wherein the second processing element is comprised with a server.

22. The electronic arrangement according to claim 11, wherein the data capturing arrangement comprises at least one of an image sensor, a Lidar arrangement, a radar arrangement, a laser scanner, inertial measurement unit, structured light projector, stereoscopic imaging arrangement or a heat sensor.

23. A computer program product comprising a non-transitory computer readable medium having stored thereon computer program means for operating an electronic arrangement, the electronic arrangement comprising a processing unit arranged in communication with a display screen and a data capturing arrangement, wherein the computer program product comprises: code for acquiring, using the data capturing arrangement, a first set of data representative of a surrounding of a user, code for determining, using the processing unit and based on the acquired first set of data, an area within the surrounding of the user fulfilling a predefined quality metric, code for acquiring, using the data capturing arrangement and following an indication that the user has moved to the area fulfilling the predefined quality metric, a second set of data, wherein the second set of data comprises data representative of a body part of the user, code for estimating, using the processing unit and based on the acquired second set of data, a geometric model of the user's body part, code for determining, using the processing unit, a matching measurement between the estimated geometrical model and each of a plurality of predefined statistical models each relating to different items for the body part, and code for displaying, using the display screen, a representation of at least one item for the body part having a matching measurement being above a predetermined matching threshold.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The various aspects of the present disclosure, including its particular features and advantages, will be readily understood from the following detailed description and the accompanying drawings, in which:

[0043] FIG. 1 schematically illustrates an electronic arrangement according to a currently preferred embodiment of the present disclosure,

[0044] FIGS. 2A and 2B presents an exemplary flow of the steps of performing the method according to a currently preferred embodiment of the present disclosure, and

[0045] FIG. 3 conceptually illustrates a model matching scheme used in conjunction with the present disclosure.

DETAILED DESCRIPTION

[0046] The present disclosure will now be described more fully hereinafter with reference to the accompanying drawings, in which currently preferred embodiments of the present disclosure are shown. This present disclosure may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided for thoroughness and completeness, and fully convey the scope of the present disclosure to the skilled person. Like reference characters refer to like elements throughout. The following examples illustrate the present disclosure and are not intended to limit the same.

[0047] Turning now to the drawings and to FIG. 1 in particular, there is conceptually illustrated an electronic arrangement 100 adapted to match an item 102 to interface with a body part 104 of a user 106. In the example illustrated in FIG. 1, the item 102 is shown as a shoe and the body part 104 is a foot. It may however, as discussed above, be possible make use of the scheme according to the present disclosure to match different items or products (such as e.g. a baseball bat, a hockey stick, a computer mouse, a bike, a chair, glasses, gloves etc.) to any type of body parts (such as e.g. a hand, a head, a torso, an overall body shape, etc.).

[0048] The electronic arrangement 100 is in FIG. 1 illustrated as a client-server implementation comprising a mobile phone 108 operated by the user 106 and a server 110 arranged remotely from the user 106 (not even necessarily within the same country as the user 106). As indicated above, other types of user devices could be possible and fall within the scope of the present disclosure. Such user devices may for example include any device that provides visual feedback to the user while it captures sensor data of the body volume and scene, such as including AR-glasses, VR-headsets, portable computers with screen and sensors etc.

[0049] The server 110 could be a dedicated physical server or a so-called cloud server. The server 110 and the mobile phone 108 are preferably connected with each other using a network connection, such as provided by means of an Internet connection. Any form of wired or wireless network protocol is possible and within the scope of the present disclosure. It should be understood that other types of remote processing implementations are possible, for example including a so-called serverless setup.

[0050] The mobile phone 108 comprises a first processing element 112, a display screen 114 and a data capturing arrangement 116. The data capturing arrangement 116 may in turn comprise one or a plurality of sensors for collecting information relating to the user 106 and to a surrounding of the user 106. Such sensors may for example include an image sensor (i.e. a camera), a Lidar arrangement, a radar arrangement, a laser scanner, inertial measurement unit, structured light projector, stereoscopic imaging arrangement or a heat sensor, etc. Further sensors are of course possible and within the scope of the present disclosure.

[0051] The server 110 in turn comprises a second processing element 118, where the first 112 and the second 118 processing element in combination provides an overall processing functionality, generally referred to as a processing unit. This is specifically relevant as it should be understood that the electronic arrangement in some alternative embodiments may be provided as a single unit implementation, where for example all of the processing functionality could be provided by a single processing unit.

[0052] For reference, the processing unit (and/or processing functionality) may for example be manifested as a general-purpose processor, a graphics processing unit, an application specific processor, a circuit containing processing components, a group of distributed processing components, a group of distributed computers configured for processing, a field programmable gate array (FPGA), etc. The processor may be or include any number of hardware components for conducting data, signal and/or image processing or for executing computer code stored in memory. It may also be possible and within the scope to make use of system-on-chip (SOC) implementations. The memory may be one or more devices for storing data and/or computer code for completing or facilitating the various methods described in the present description. The memory may include volatile memory or non-volatile memory. The memory may include database components, object code components, script components, or any other type of information structure for supporting the various activities of the present description. According to an exemplary embodiment, any distributed or local memory device may be utilized with the systems and methods of this description. According to an exemplary embodiment the memory is communicably connected to the processor (e.g., via a circuit or any other wired, wireless, or network connection) and includes computer code for executing one or more processes described herein.

[0053] During operation of the electronic arrangement 100, with further reference to FIGS. 2A and 2B, the process may for example start by the user 106 operating an application being executed at the mobile phone 108. The application may for example be related to an online store providing different items.

[0054] When initiating the application, for example the camera 116 of the mobile phone 108, possibly in combination with e.g. a Lidar arrangement, will start acquiring, S1, a first set of data that is representative of a scene of a surrounding of the user 106. Based on the acquired first set of data it is possible to determine, S2, an area 202 within the surrounding of the user that is fulfilling a predefined quality metric, such as for example by investigating if there is a suitable flat surface where a following body part scanning could be performed, if the area is sufficiently lit, etc. This determination could for example be performed by the first processing element 112 implementing an image processing scheme, possibly combining the data from the camera 116 and the Lidar arrangement.

[0055] Once a suitable area has been identified, it may in accordance to the present disclosure be possible to instruct the user 106 to move to that specific area 202. Such instructions could be provided using the display screen 114, such as by providing real time movement instructions to the user 116. In some embodiments the movement instructions could be provided by implementing an augmenting reality (AR) functionality, in combination with image data displayed at the display screen 114. As shown in FIG. 2B, such AR instructions could be provided by outlining a portion 204 of the area where the user 106 is to move. It may in some embodiments be advantageous to configure the movement instructions in such a manner that the user 106 applies a desirable pose. As an example, in case the feet of the user 116 is to be (subsequently) scanned, it has shown to be desirable to instruct the user to arrange himself in a standing position.

[0056] When it has been indicated (such as by continuously analyzing image data from the camera 116) that the user 106 has moved to the specific area, the scheme according to the present disclosure proceeds to acquiring, S3, a second set of data, where the second set of data comprises data representative of a body part of the user 106. In this case the feet of the user 116. Also when acquiring the second set of data it may be suitable to instruct the user 106 as to how to acquire the data, again possibly using AR functionality provided in conjunction with the display screen 114. Here it is again possible to continuously analyze the acquired data to see if the user 106 is following the provided instructions or needs to be (in real time) instructed to change his scanning pattern. It is generally desirable to ensure that the user 116 is scanning the body part from at least two, but preferably three sides and possibly more sides of the body part.

[0057] When it has been determined that a sufficient amount of data has been acquired about the body part it is possible to estimate, S4, a geometric model of the user's 106 body part 104. The estimation of the geometric model may for example be performed by combining (and possibly stitching together using an image processing scheme) a large number of images acquired using the camera 116. It is also possible to combine the image data with depth data provided using e.g. the Lidar arrangement (if such sensor functionality is available at the mobile phone) 108. The final geometric model of the user's 106 body part 104 may further be handled by a process for forming a three-dimensional (3D) outline of the body part 104, where the outlined body part 104 is parameterized for further processing.

[0058] The parameterized geometric model of the body part 104 is then compared to each of a plurality of predefined statistical models each relating to different items 102 for the body part 104, in the example provided in FIG. 2B the item is a shoe. As discussed above, a statistical model for an item 102 is not the same as a scanned volume of the item 102. Rather, the statistical model for an item 102 is a combination/correlation of other users' geometrical models for their corresponding body parts. Accordingly, the statistical model for a specific item 102 (such as a specific type of shoe in a specific size) is formed from other users that for example have scanned their feet and then proceeded to purchase that specific shoe in the specific size. The inherent differences between the other user's different body part sizes (length, width, height, etc.) will together provide a probability distribution (in the simplest case a mean and a variance) for the statistical model for the specific item 102. FIG. 3 provides a conceptual and exemplary illustration of an outlined geometrical model 302 of the foot 104 of a user 106 arranged within a statistical model 304 of a shoe 102. The variance for the shoe 102 could be seen as a range for which a foot 104 is likely to be perceived by the user 106 as being a likely fit. The statistical model 304 for a specific item 102 may as such be dynamically built once users have formed geometrical models and then purchased a specific item 102. The more users that purchase the same item 102, the more relevant statistical model for that item 102.

[0059] The comparison between the geometric model 302 of the body part 104 and the statistical model 304 of the item 102 is used for determining, S5, a matching measurement. It is preferred to arrange the matching measurement to penalize a situation where the geometric model 302 of the body part 104 falls outside the statistical model 304 for the item 102. As an example and in relation to a shoe, even in case a length of the foot is considered to be within the variance for the statistical model 304 of the shoe 102, the matching measurement will be heavily penalized in case the width of the foot is considered to be outside of the width for the statistical model 304 of the shoe 102. The matching measurement will in this case be indicated as no fit, bad fit or a low fit number (e.g. between 1-10).

[0060] Once the matching measurements have been determined for the plurality of statistical models, the process proceeds to display, S6, a representation of at least one item 102 for the body part having a matching measurement being above a predetermined matching threshold, such as within a graphical user interface (GUI) provided at the display screen 114 of the mobile phone 108. Accordingly, the predetermined matching threshold is provided to filter out items having a matching measurement that is considered to be too low, ensuring that the user will be presented the matching items that the user is likely to be satisfied with.

[0061] In one embodiment, it could be possible to allow the predetermined matching threshold to be dependent on collected data relating to purchase and return transactions, specifically since returns may be taken an indication that something is to be considered as outside/below the threshold, and lack of returns is an indication that it is within/above the threshold.

[0062] Furthermore, it could be possible to display a list or other form of personal recommendation of items 102 at the display screen 114, having a matching measurement for shoes 102 that indicates at least e.g. a 50% match with the geometric model of the user's 106 feet 104. The list could in some embodiments be correlated with stock inventory, such that only shoes 102 in stock and having a matching of at least 50% is shown to the user 106. It should be understood that 50% match is only an example and can be selected arbitrarily, possibly by the user 106.

[0063] Within the scope of the present disclosure, it is also possible to present a more complete matching between the feet and the shoe, typically based on the matching measurement. For example, it could be possible to display a detailed matching information that indicates where the foot is expected to be best matching, as compared to least matching. As an example, a shoe may be an in comparison good match in relation to length and an in comparison less good match in relation to a width. The user may then take such information into account when determining if to proceed with purchasing a recommended shoe.

[0064] Still further, it could be possible to take some additional information about the user into account, such as for example if the user knows that he/she normally uses a specific shoe, glove, hat, jacket, etc., size. The matching process according to the present disclosure may as such take this prior knowledge into account to reduce the processing needed to find the best matching items 102 for the user 102 as well as possibly getting a better accuracy in the recommendation. Further prior information provided by and/or received about the user 102 may include item brand that the user 102 has previously purchased.

[0065] Furthermore, the control functionality of the present disclosure may be implemented using existing computer processors, or by a special purpose computer processor for an appropriate system, incorporated for this or another purpose, or by a hardwire system. Embodiments within the scope of the present disclosure include program products comprising machine-readable medium for carrying or having machine-executable instructions or data structures stored thereon. Such machine-readable media can be any available media that can be accessed by a general purpose or special purpose computer or other machine with a processor. By way of example, such machine-readable media can comprise RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, solid state drives or other non-volatile flash based storage devices, or any other medium which can be used to carry or store desired program code in the form of machine-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer or other machine with a processor. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a machine, the machine properly views the connection as a machine-readable medium. Thus, any such connection is properly termed a machine-readable medium. Combinations of the above are also included within the scope of machine-readable media. Machine-executable instructions include, for example, instructions and data which cause a general-purpose computer, special purpose computer, or special purpose processing machines to perform a certain function or group of functions.

[0066] Although the figures may show a sequence the order of the steps may differ from what is depicted. Also two or more steps may be performed concurrently or with partial concurrence. Such variation will depend on the software and hardware systems chosen and on designer choice. All such variations are within the scope of the disclosure. Likewise, software implementations could be accomplished with standard programming techniques with rule-based logic and other logic to accomplish the various connection steps, processing steps, comparison steps and decision steps. Additionally, even though the present disclosure has been described with reference to specific exemplifying embodiments thereof, many different alterations, modifications and the like will become apparent for those skilled in the art.

[0067] In addition, variations to the disclosed embodiments can be understood and effected by the skilled addressee in practicing the claimed present disclosure, from a study of the drawings, the disclosure, and the appended claims. Furthermore, in the claims, the word comprising does not exclude other elements or steps, and the indefinite article a or an does not exclude a plurality.