A source content routing system is described for distributing source content received from clients such as documents, to translators for performing translation services on the source content. The routing system extracts source content features, which may be represented as vectors. The vectors may be assembled into an input matrix, which may be processed using an artificial neural network, classifier, perceptron, CRF model, and/or the like, to select a translator such as a machine translation system and/or human. The translator provides translation services translation from a source language to a target language, post translation editing, proof reading, quality analysis of a machine, quality analysis of human translation, and/or the like and returns the product to the content routing system or clients.

A source content routing system is described for distributing source content received from clients such as documents, to translators for performing translation services on the source content. The routing system extracts source content features, which may be represented as vectors. The vectors may be assembled into an input matrix, which may be processed using an artificial neural network, classifier, perceptron, CRF model, and/or the like, to select a translator such as a machine translation system and/or human. The translator provides translation services translation from a source language to a target language, post translation editing, proof reading, quality analysis of a machine, quality analysis of human translation, and/or the like and returns the product to the content routing system or clients.

The present disclosure provides a summary generation model training method and apparatus, a device and a storage medium, and relates to the field of computer technologies, and in particular, to the field of artificial intelligence such as natural language processing and deep learning. The summary generation model training method includes: acquiring a document representation corresponding to a document sample; constructing, based on the document representation, a summary representation corresponding to the document representation, the summary representation including a positive summary representation and a negative summary representation; and constructing a total contrastive loss function based on the document representation, the positive summary representation and the negative summary representation, and training a summary generation model based on the total contrastive loss function. The present disclosure may improve accuracy of the summary generation model.

The present disclosure provides a summary generation model training method and apparatus, a device and a storage medium, and relates to the field of computer technologies, and in particular, to the field of artificial intelligence such as natural language processing and deep learning. The summary generation model training method includes: acquiring a document representation corresponding to a document sample; constructing, based on the document representation, a summary representation corresponding to the document representation, the summary representation including a positive summary representation and a negative summary representation; and constructing a total contrastive loss function based on the document representation, the positive summary representation and the negative summary representation, and training a summary generation model based on the total contrastive loss function. The present disclosure may improve accuracy of the summary generation model.

Aspects of the technology employ a machine translation quality prediction (MTQP) model to refine datasets that are used in training machine translation systems. This includes receiving, by a machine translation quality prediction model, a sentence pair of a source sentence and a translated output (802). Then performing feature extraction on the sentence pair using a set of two or more feature extractors, where each feature extractor generates a corresponding feature vector (804). The corresponding feature vectors from the set of feature extractors are concatenated together (806). And the concatenated feature vectors are applied to a feedforward neural network, in which the feedforward neural network generates a machine translation quality prediction score for the translated output (808).

Aspects of the technology employ a machine translation quality prediction (MTQP) model to refine datasets that are used in training machine translation systems. This includes receiving, by a machine translation quality prediction model, a sentence pair of a source sentence and a translated output (802). Then performing feature extraction on the sentence pair using a set of two or more feature extractors, where each feature extractor generates a corresponding feature vector (804). The corresponding feature vectors from the set of feature extractors are concatenated together (806). And the concatenated feature vectors are applied to a feedforward neural network, in which the feedforward neural network generates a machine translation quality prediction score for the translated output (808).

The method for identifying noise samples, includes: obtaining an original sample set; obtaining a target sample set by adding masks to original training corpora in the original sample set using a preset adjustment rule; performing mask prediction on a plurality of target training corpora in the target sample set using a pre-trained language model to obtain a first mask prediction character corresponding to each target training corpus; matching the first mask prediction character corresponding to each target training corpus with a preset condition; and according to target training corpora of which first mask prediction characters do not match the preset condition in the target sample set, determining corresponding original training corpora in the original sample set as noise samples.

The method for identifying noise samples, includes: obtaining an original sample set; obtaining a target sample set by adding masks to original training corpora in the original sample set using a preset adjustment rule; performing mask prediction on a plurality of target training corpora in the target sample set using a pre-trained language model to obtain a first mask prediction character corresponding to each target training corpus; matching the first mask prediction character corresponding to each target training corpus with a preset condition; and according to target training corpora of which first mask prediction characters do not match the preset condition in the target sample set, determining corresponding original training corpora in the original sample set as noise samples.

A method includes obtaining n pairs of translation sentences of a source language and a target language, where each of the n pairs of translation sentences includes a source language sentence and a target language sentence that are translations of each other, extracting a source language segment from each source language sentence in the n pairs of translation sentences using an extraction rule of the source language, extracting a target language segment from each target language sentence in the n pairs of translation sentences, and generating an extraction rule of the target language based on n target language segments extracted from n target language sentences.

A method includes obtaining n pairs of translation sentences of a source language and a target language, where each of the n pairs of translation sentences includes a source language sentence and a target language sentence that are translations of each other, extracting a source language segment from each source language sentence in the n pairs of translation sentences using an extraction rule of the source language, extracting a target language segment from each target language sentence in the n pairs of translation sentences, and generating an extraction rule of the target language based on n target language segments extracted from n target language sentences.