DATA TRANSFORMATION SYSTEM AND METHOD

Abstract

A method of transforming data including receiving data in a first language specific form, converting the data in the first language specific form to a language agnostic form, storing the data in the language agnostic form, converting the data in the language agnostic form to at least one second language specific form and exporting, on demand, the data in at least one of the at least one second language specific form.

Claims

1-87. (canceled)

88. A method of transforming data comprising: receiving unstructured product data describing attributes of an article, said unstructured product data comprising attribute values, which are assignable to a multiplicity of structured product attributes; automatically converting said unstructured product data to structured product data in a form suitable for export to a specific electronic commerce platform, said structured product data comprising, for each attribute of said article, an attribute value associated with said attribute and at least one of a language specific attribute name of said attribute and a language agnostic attribute ID associated with said attribute; and exporting said structured product data to the specific electronic commerce platform in said form suitable for export.

89. The method according to claim 88 and wherein said language agnostic attribute ID is specific to said electronic commerce platform.

90. The method according to claim 88 and wherein said unstructured product data and said structured product data are in the same language.

91. The method according to claim 88 and wherein said unstructured product data and said structured product data are in the different languages.

92. The method according to claim 88 and wherein said automatically converting comprises automatically converting measurements to units of measure applicable to said specific electronic commerce platform.

93. The method according to claim 88 and wherein said automatically converting comprises automatically converting prices to a currency applicable to said specific electronic commerce platform.

94. The method according to claim 88 and wherein said automatically converting comprises utilizing a pricing policy applicable to said specific electronic commerce platform.

95. The method according to claim 88 and wherein said automatically converting said unstructured product data to structured product data in a form suitable for export comprises: analyzing said unstructured product data; and extracting, from said unstructured product data, unstructured product-specific attribute values, which are assignable to corresponding ones of said multiplicity of structured product attributes.

96. The method according to claim 95 and wherein said automatically converting said unstructured product data to structured product data in a form suitable for export also comprises, subsequent to said extracting, assigning each of said unstructured product-specific attribute values to a corresponding one of said structured product attributes as at least one of a value and a value ID.

97. The method according to claim 88 and wherein said receiving said unstructured product data comprises automatically retrieving said unstructured product data from an electronic commerce platform different from said specific electronic commerce platform.

98. A system for transforming data comprising: a data receptacle, receiving unstructured product data describing attributes of an article, said unstructured product data comprising attribute values, which are assignable to a multiplicity of structured product attributes; a data transformer, automatically converting said unstructured product data to structured product data in a form suitable for export to a specific electronic commerce platform, said structured product data comprising, for each attribute of said article, an attribute value associated with said attribute and at least one of a language specific attribute name of said attribute and a language-agnostic attribute ID associated with said attribute; and a marketplace listing exporter, exporting said structured product data to said specific electronic commerce platform in said form suitable for export.

99. The system according to claim 98 and wherein said language agnostic attribute ID is specific to said electronic commerce platform.

100. The system according to claim 98 and wherein said unstructured product data and said structured product data are in the same language.

101. The system according to claim 98 and wherein said unstructured product data and said structured product data are in the different languages.

102. The system according to claim 98 and wherein said data transformer automatically converts measurements to units of measure applicable to said specific electronic commerce platform.

103. The system according to claim 98 and wherein said data transformer automatically converts prices to a currency applicable to said specific electronic commerce platform.

104. The system according to claim 98 and wherein said data transformer utilizes a pricing policy applicable to said specific electronic commerce platform.

105. The system according to claim 98 and wherein said data transformer analyzes said unstructured product data and extracts, from said unstructured product data, unstructured product-specific attribute values, which are assignable to corresponding ones of said multiplicity of structured product attributes.

106. The method according to claim 105 and wherein said data transformer assigns each of said unstructured product-specific attribute values to a corresponding one of said structured product attributes as at least one of a value and a value ID.

107. The system according to claim 98 and wherein said data receptacle automatically retrieves said unstructured product data from an electronic commerce platform different from said specific electronic commerce platform.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present invention will be understood and appreciated more fully from the following detailed description, taken in conjunction with the drawings in which:

[0054] FIG. 1 is a simplified block diagram illustration of a data transformation system, constructed and operative in accordance with a preferred embodiment of the present invention;

[0055] FIG. 2 is a simplified flowchart illustrating some initial steps performed by a seller in cooperation with an embodiment of the system of FIG. 1;

[0056] FIGS. 3A and 3B show two examples of unstructured, language-specific listing data, each of which forms part of a different seller feed file;

[0057] FIG. 4 is a simplified block diagram illustration of Unstructured Product Data Receiving Functionality (UPDRF), which forms part of an embodiment of the system of FIG. 1;

[0058] FIG. 5 is a simplified illustration which illustrates operation of an input data column mapper, which forms part of the Unstructured Product Data Receiving Functionality (UPDRF) of FIG. 4;

[0059] FIG. 6 is a simplified flowchart illustrating operation of the Unstructured Product Data Receiving Functionality (UPDRF) of FIG. 4;

[0060] FIG. 7 is a simplified example of unstructured, language-specific listing data as initially stored in a database, which forms part of an embodiment of the system of FIG. 1;

[0061] FIG. 8 is a simplified flowchart illustrating operation of product data extracting functionality and product matching functionality, which form part of an embodiment of the system of FIG. 1;

[0062] FIG. 9 is a simplified illustration of a category structure table employed in the product data extracting functionality and product matching functionality, which forms part of an embodiment of the system of FIG. 1;

[0063] FIG. 10 is a simplified illustration of an example of operation of product data extracting functionality, which forms part of an embodiment of the system of FIG. 1;

[0064] FIG. 11 is a simplified illustration of a typical category attribute structure table employed by the extraction functionality of FIG. 10;

[0065] FIG. 12 is a simplified illustration of a typical attribute value structure table employed by the extraction functionality of FIG. 10;

[0066] FIG. 13 is a simplified illustration of part of a typical structure translation table employed by an embodiment of the system of FIG. 1;

[0067] FIGS. 14A and 14B are simplified flow charts of two category specific rule sets employed by the extraction functionality of FIG. 10;

[0068] FIG. 15 is a simplified illustration of a typical category specific matrix matching rule;

[0069] FIG. 16 is a simplified flow chart illustrating new product creation and data enrichment operation of an embodiment of the system of FIG. 1;

[0070] FIG. 17 is a simplified illustration of part of a structured product table;

[0071] FIG. 18 is a simplified illustration of part of a structured product data table;

[0072] FIG. 19 is a simplified illustration of part of a typical marketplace specific data specifications table;

[0073] FIG. 20 is a simplified illustration of an example of operation of a language agnostic to language specific product data converting functionality, which forms part of an embodiment of the system of FIG. 1;

[0074] FIG. 21 is a simplified illustration of part of a structure translation table;

[0075] FIG. 22 is a simplified flow chart of a typical product data weight attribute extension and conversion rule;

[0076] FIG. 23 is a simplified illustration of a typical language specific, marketplace specific, product listing creation rule;

[0077] FIG. 24 is a simplified illustration of mapping of categories to a specific marketplace;

[0078] FIG. 25 is a simplified illustration of mapping of attribute-value pairs to a specific marketplace; and

[0079] FIG. 26 is a simplified illustration of a typical language specific, marketplace specific, listing data conversion rule.

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT

[0080] Reference is now made to FIG. 1, which is a simplified block diagram illustration of a data transformation system 100, constructed and operative in accordance with a preferred embodiment of the present invention. FIG. 1 illustrates an embodiment of the present invention in the context of a multi-language electronic commerce environment, it being understood that this and other embodiments of the present invention are not necessary limited to such an environment.

[0081] Conventional electronic commerce platforms typically facilitate listing of products in a semi-structured form. The formats required for listing feeds to the various conventional electronic commerce platforms differ from one to the other and are language specific. Therefore, merchants wishing to list product offerings on a wide range of electronic commerce platforms typically need to list their products separately on each electronic commerce platform and in every desired language.

[0082] It is a particular feature of an embodiment of the present invention that it facilitates importing unstructured product listings in any of a first plurality of languages, converting the imported product listings to a structured language-agnostic form, and subsequently providing the product listings in any of a second plurality of languages. The system of an embodiment of the present invention renders a product listing suitable for listing in a structured multi-language catalog or on any suitable electronic commerce platform, in any language.

[0083] It is appreciated that the second plurality of languages typically also includes the language of the original product listing and that the system of an embodiment of the present invention may be utilized to import an unstructured product listing in a first language and to provide a structured product listing in the same language.

[0084] As seen in FIG. 1, data transformation system 100 typically includes data manipulation functionality 102, interacting with a database 110 which forms part of data transformation system 100. Database 110 preferably includes storage for unstructured product listings, which are typically provided as inputs in a language specific format, as well as structured product listings, which are provided by the data manipulation functionality 102 of data transformation system 100. Structured product listings may be stored in database 110 in both a language agnostic format as well as one or more language specific formats.

[0085] The data manipulation functionality 102 of data transformation system 100 preferably includes at least the following, preferably interfacing with database 110:

[0086] unstructured product data receiving functionality 120, which is described further hereinbelow with reference to FIG. 4 and typically includes a feed uploading tool 122, including an input data mapper 124, and feed file receiving functionality 128, which preferably includes data writing functionality 130, a NEED TO PROCESS ascertainer 132 and a classifier/matcher trigger 134; [0087] product classification/matching functionality 136; [0088] unstructured product data extracting functionality 140 interacting with, inter alia, a dictionary 142; [0089] unstructured language specific to structured language agnostic product data converting functionality 144 interacting with, inter alia, dictionary 142; [0090] new product creation and product data enrichment functionality 146; [0091] language agnostic product data to language specific product data converting functionality 150; [0092] data export functionality 160; and [0093] selected marketplace listing functionality 170.

[0094] These functionalities are explained further hereinbelow with reference to FIGS. 4-25.

[0095] Reference is now made to FIG. 2, which illustrates typical initial operations performed by a seller, employing the system of FIG. 1. As seen in FIG. 2, the seller selects the marketplaces in which he wishes to list his products, here also termed marketplaces for boarding. The seller preferably selects a currency conversion policy for each marketplace, a pricing and markup policy for each marketplace and shipping and return policies for each marketplace. The seller then uploads the seller listing feed file to database 110 in the system of FIG. 1. As seen in FIG. 2, following uploading of the seller listing file, the seller initiates the conversion process of the system of the present invention to convert the unstructured language specific product listings to structured language agnostic product listings, which are then stored in database 110.

[0096] As seen in the illustrated embodiment of FIG. 1, unstructured, language-specific listing data forming part of a seller listing feed file is typically provided to data transformation system 100 via Unstructured Product Data Receiving Functionality (UPDRF) 120. Preferably, data transformation system 100 stores the unstructured, language-specific listing data in database 110. Examples of two different seller listing feed files appear in FIGS. 3A and 3B.

[0097] It is appreciated that each of the examples of seller listing feed files in FIGS. 3A and 3B includes two separate listings for a camera lens. The language of all listings in FIGS. 3A and 3B is English and the listing data in both examples includes free text describing not only the camera lens but also the condition of the lens and price of the lens. In one case shipping data is also included.

[0098] Unstructured Product Data Receiving Functionality (UPDRF) 120 is illustrated generally in FIG. 4 and includes feed uploading tool 122, which receives the unstructured, language-specific listing data, typically in the form of a seller feed file, such as that shown in FIGS. 3A and 3B. Feed uploading tool 122 typically is implemented on a web page associated with a portal of the system of FIG. 1, which also enables the functionality described hereinabove with reference to FIG. 2.

[0099] Feed uploading tool 122 includes, inter alia, input data column mapper 124, which, on the basis of instructions received from the seller, maps the columns in the seller listing feed file to columns in database 110. The input data column mapper 124 provides a mapping instruction output, which is employed by unstructured product data receiving functionality 120 of data transformation system 100 to rearrange the columns in the seller listing feed file to correspond to the columns in database 110 and then uploads the rearranged data to database 110. FIG. 5 illustrates the operation of the input data column mapper 124.

[0100] A simplified flow chart summarizing a preferred process for uploading a seller listing feed file to the database 110 appears in FIG. 6. In addition to the mapping instruction output provided by input data column mapper 124, the seller listing feed file and the mapping instruction output are processed by Feed File Receiving Functionality 128, also forming part of Unstructured Product Data Receiving Functionality (UPDRF) 120.

[0101] Feed File Receiving Functionality 128 includes data writing functionality 130, which writes data from each row in the seller listing feed into a corresponding row in the database 110 in accordance with the mapping instructions generated by input data column mapper 124. The data writing functionality 130 also adds to each row a unique listing key, which is preferably generated by operation of a hash function, such as a conventional MD5 hash function, on the listing data.

[0102] The data writing functionality 130 also preferably adds to each row an update date, indicating the date at which the seller listing data is uploaded to database 110.

[0103] Feed File Receiving Functionality 128 also includes a NEED TO PROCESS ascertainer 132, which ascertains whether a newly loaded listing relates to a listing already present on the database, and if not, lists the newly loaded listing as being subject to a need to process. The NEED TO PROCESS ascertainer 132 compares the unique listing key of the newly added row with the unique listing key of all listings in the database. If no match is found, a NEED TO PROCESS flag appears in a NEED TO PROCESS column of the newly added row.

[0104] Feed File Receiving Functionality 128 additionally includes classifier/matcher trigger 134. Classifier/matcher trigger 134 operates only when the entire seller listing feed file has been uploaded to database 110 and a unique listing key has been generated for each new row.

[0105] Following uploading of the entire seller listing feed file, the classifier/matcher trigger 134 initiates classification/matching functionality 136, which processes rows in which a NEED TO PROCESS flag appears in the NEED TO PROCESS column of a newly added row. Classification/matching functionality 136 attempts to match a new listing with a product which is already in database 110.

[0106] An exemplary updated unstructured language specific listing table stored in database 110, which includes the listing data appearing in the seller listing feed files of FIGS. 3A and 3B, is seen in FIG. 7. It is seen that each row includes, inter alia, a unique listing key, an update date and, where appropriate, a NEED TO PROCESS flag.

[0107] As described hereinabove, following uploading of the entire seller listing feed file, the classifier/matcher trigger 134 initiates classification/matching functionality 136 which processes rows in which a NEED TO PROCESS flag appears.

[0108] Classification/matching functionality 136 is described hereinbelow with respect to FIG. 8, which presents a generalized overview.

[0109] As seen in FIG. 8, classification/matching functionality 136 first checks for the existence of a unique product identifier, such as a UPC code or its equivalent, which consists entirely of numbers, in the new listing. If such a unique product identifier is extracted from the data in the listing description column of a row as stored in database 110 (FIG. 7) from the seller listing feed file and that unique product identifier is already stored in database 110, the product represented by the listing is matched to the product already stored in database 110.

[0110] Should matching using a unique product identifier not be achieved, classification/matching functionality 136 then tries to extract a manufacturer and Manufacturer's Product Number, which may consist of numbers and letters, normally in Latin characters, from the new listing. If a manufacturer and a Manufacturer's Product Number is extracted from the data in the listing description column of a row as stored in database 110 (FIG. 5) from the seller listing feed file and that manufacturer and Manufacturer's Product Number is already stored in database 110, the product represented by the listing is matched to the product already stored in database 110.

[0111] Should matching using a unique product identifier or a manufacturer and Manufacturer's Product Number not be achieved, classification/matching functionality 136 then tries to identify the language of the new listing. This is achieved using conventional techniques, such as can be found at https://pypi.python.org/pypi/langdetect.

[0112] Once the language of the new listing is identified, the listing is classified into a category. Classification/matching functionality 136 classifies the listing into a category by employing language specific functionality, and preferably employing a trained statistical model to classify all of the listing data in each newly added row and to classify the row as relating to one of a plurality of categories in a category structure table, stored in database 110, a simplified example of which appears in FIG. 9.

[0113] Considering FIG. 9, it is seen that the category structure includes an indication of the existence of a parent-child relationship between some of the categories. Thus, for example, categories C1000 and C1002 are related to parent category C100, category C1001 is related to parent category C101 and categories C100 and C101 do not have a parent category. It is appreciated that the classification functionality is operative to classify each newly added row into the most specific category, e.g. a child category, if one exists. The classification functionality is operative to assign a category ID to each row. This category ID is language agnostic.

[0114] Once the category of the listing is ascertained, unstructured product data extracting functionality 140 extracts various attributes and values from the listing data and stores the attributes and values in a listing extracted data language specific cache. An example of such a step is illustrated diagrammatically in FIG. 10.

[0115] From a consideration of FIG. 10, it is seen that various structured language specific attributes and values are extracted from the unstructured language specific listing data by unstructured product data extracting functionality 140. It is also appreciated that each of the extracted attributes and values has a corresponding ID, which is language agnostic. The extracted language specific structured data is preferably stored temporarily in the listing extracted data language specific cache and unstructured language specific to structured language agnostic product data converting functionality 144 provides the corresponding extracted language agnostic structure data, which is preferably temporarily stored in a listing extracted data language agnostic cache.

[0116] Each attribute has an attribute ID. FIG. 11 is an example of a category attribute structure table, stored in database 110, of attributes which are associated with a given category, here Lenses, whose Category ID is C1002 (FIG. 9). It is seen that each attribute has an attribute ID which is language agnostic and is characterized by an attribute type. The attribute types typically include: [0117] a discrete value lista predetermined discrete number of predetermined values; [0118] a numbera number, which may have a great number of values

[0119] Each value has a value ID. FIG. 12 is a simplified example of an attribute value structure table, stored in database 110, of values which are associated with various attributes (FIG. 11).

[0120] It is appreciated that various attributes and values can be applicable to multiple categories although this is rare and is generally assumed not to occur.

[0121] It is appreciated that structures exemplified in the tables of FIGS. 9, 11 and 12 are related to a very large language specific/language agnostic table stored in database 110, which stores product data corresponding to listing descriptions in all relevant languages for categories, attributes and values and their corresponding language agnostic IDs. Dictionary 142 is embodied, inter alia, in a language specific/language agnostic table, a simplified example of which appears in FIG. 13. In practice, structures exemplified in the tables of FIGS. 9, 11 and 12 may be stored in database 110 without a language specific category, attribute or value name.

[0122] Preferably the entire language specific/language agnostic table, which typically includes hundreds of thousands, if not millions, of entries is not used in matching. Instead, only those entries in the ascertained language are used, and of those entries, initially only those entries relating to categories (FIG. 9) are employed in the functionality of FIG. 10. Only once the category of a listing has been ascertained, is matching of attributes and values attempted by using only those entries (FIGS. 11 and 12) in the language specific/language agnostic table (FIG. 13) which relate to the already ascertained language and category.

[0123] Preferably in matching attributes and values, an attempt is first made to match values, using a value table, exemplified in FIG. 13, since in practice most important values are single attribute specific, so that by matching values, an attribute match is also achieved.

[0124] Turning once again to FIG. 10, it is appreciated that there exist various rules which identify various listing data elements, typically values, and associate them with various attributes. For examples 18-55 is recognized by rule as referring to a focal length and F/ refers to a lens aperture. Two typical sets of rules useful in the functionality of FIG. 11 appear in FIGS. 14A and 14B.

[0125] Returning now to FIG. 8, it is seen that a category specific feature match matrix is employed for matching. This matrix defines, for each category, the minimum set of attributes which must be known in order to establish a product match for a product in that category. An example of such a matrix appears in FIG. 15.

[0126] If the attributes and values required to establish a product match are extracted, and a product in the database includes all of the attributes and values specified in the matrix, the product in the new listing is matched to a product in the database.

[0127] If the attributes and values required to establish a product match are not extracted, system 100 adds the listing to database 110 utilizing the new product creation and data enrichment functionality 146.

[0128] Reference is now made to FIG. 16, which is a simplified flow chart illustrating new product creation and data enrichment functionality 146 of system 100.

[0129] As seen in FIG. 16, if a new listing is matched to a product already in the database, all of the attributes and values in the new listing are extracted and stored in a language agnostic structured product table in database 110 in a language agnostic form.

[0130] It is then ascertained whether all of the attribute-value pairs appearing in the new listing appear in the product data as stored in the database. If there are attributes and values in the new listing which do not appear in the product data as stored in the database, these new attributes and values are added to the existing product in database 110.

[0131] If the new listing is not matched to a product already in the database, a new product row is created in the structured product table in database 110. An example of a structured product table appears in FIG. 17. A new product ID is created and all relevant attribute/value pairs are associated therewith and preferably stored in a new Structured Product Data table in database 110, an example of which appears in FIG. 18. Following the creation of the new product ID and new attribute/value pairs, the listing is added to the language agnostic structured product table in database 110 in a language agnostic form.

[0132] As seen in FIG. 16, if the new listing is not classified to a category in database 110, the new listing is discarded and system 100 preferably notifies the seller that the new listing was not able to be classified.

[0133] At this stage, the data in the database 110 is ready for export to one or more marketplaces designated by the seller (FIG. 2). For each marketplace, an initial step is to convert the product data from language agnostic form to a language specific form suitable for that marketplace. FIG. 20 shows an example of the language agnostic product data to language specific product data converting functionality 150 of system 100. Use is preferably made of a structure translation table, an example of which appears in FIG. 21. Additionally, marketplace specific conversion parameters are found in a marketplace specific data specifications table, an example of which is illustrated in FIG. 19.

[0134] Preferably, language agnostic product data to language specific product data converting functionality 150 initially converts attributes from language agnostic form into language specific form suitable for the designated marketplace. Thereafter, language agnostic product data to language specific product data converting functionality 150 translates values which are not numbers. As can be seen in FIG. 20, the process produces as outputs a plurality of attribute-value pairs, in the language specific form suitable for the designated marketplace.

[0135] Since different marketplaces employ different units of measure and currencies, various manipulations of the product data may be carried out by data export functionality 160. Typical rules for such manipulations appear in FIG. 22. Additionally, marketplace specific modifications of product data may be appropriate. FIG. 23 indicates a suitable rule for this purpose.

[0136] As seen in FIG. 23, a typical language specific product listing creation rule is presented for the OZON marketplace in the Russian language. Seven different attribute-value pairs are translated, typically by using the table of FIG. 21 and the translated values. The resulting marketplace language specific product listing is stored in a marketplace language specific product data cache in database 110 and is ready for boarding onto a designated marketplace.

[0137] Preferably, selected marketplace listing functionality 170 employs an API of a designated marketplace to board the listing onto the designated marketplace. Selected marketplace listing functionality 170 of system 100 submits a listing feed in a marketplace specific format and language via the API. The marketplace specific format includes the category mapping and attribute-value pair mapping to the specific marketplace. Examples of this mapping are seen in FIGS. 24 and 25.

[0138] There exist cases wherein this mapping is one to one and cases where the mapping is more complex, such as one to many and/or many to one. In such cases, the attribute-value pairs may be involved in the mapping.

[0139] Once the boarded marketplace category of each listed product has been selected, the attribute value pairs are mapped into the specific attribute value pairs of the boarded marketplace. This typically employs a one-to-one mapping. An example of attribute-value pair mapping appears in FIG. 25. Attribute value pairs in the database which do not exist in the boarded marketplace are typically ignored. Attribute-value pairs which exist in the boarded marketplace but do not exist in the database are also typically ignored but alternatively could be provided by conversions which are carried out in advance. An example of such a conversion appears in FIG. 26.

[0140] A further important step is unifying the marketplace-specific product information with listing information, which is not included in the product data. One example of such listing information is price information. As noted hereinabove with reference to FIG. 2, pricing policy may be marketplace specific, as selected by the seller and is not merely the result of a currency conversion, since different pricing policies may be selected by the seller for different marketplaces.

[0141] Aside from price, the listing information may include seller selected parameters relating to condition of the product, as seen in FIG. 26.

[0142] It will be appreciated by persons skilled in the art that the scope of the present invention is not limited by what has been particularly shown and described hereinabove. Rather the scope of the present invention includes the combinations and subcombinations of the features described hereinabove as well as modifications thereof which are not in the prior art.