METHOD AND DEVICE FOR MONITORING A FILLING AND/OR CLOSING INSTALLATION AND/OR POST-PROCESSING INSTALLATION

Abstract

The invention relates to a method and a device for monitoring a filling and/or closing installation and/or a post-processing installation, in particular for the pharmaceutical industry, wherein an image of a transport, infeed and/or outfeed region (2, 3, 6) of the filling and/or closing and/or post-processing installation is taken using a camera system (10) and, wherein on the basis of the image, by use of an artificial intelligence model (AI model) (120) that is trained to detect primary packaging means in the transport, infeed and/or outfeed region (2, 3, 6) and to classify detected primary packaging means, it is determined in which image positions primary packaging means are present, and detected primary packaging means are assigned to a class. The invention further relates to a filling and/or closing installation and/or post-processing installation and to a computer program for monitoring a filling and/or closing installation and/or post-processing installation.

Claims

1. A method for monitoring a filling and/or closing and/or post-processing installation, in particular for the pharmaceutical industry, wherein an image of a transport, infeed and/or outfeed region of the filling and/or closing and/or post-processing installation is taken using a camera system and, wherein on the basis of the image, by use of an artificial intelligence model (AI model) that is trained to detect primary packaging means in the transport, infeed and/or outfeed region and to classify detected primary packaging means, it is determined in which image positions primary packaging means are present, and detected primary packaging means are assigned to a class.

2. The method according to claim 1, wherein primary packaging means are classified on the basis of their type, and/or primary packaging means of a type are classified on the basis of their orientation.

3. The method according to claim 1, wherein an output of the AI model is evaluated by use of a rule-based algorithm for the purpose of determining disturbances, and preferably a determined disturbance is classified and/or prioritised, wherein in particular it is specified on the basis of the classification how the determined disturbance is handled and/or specified on the basis of a prioritisation whether-and if so, when-the determined disturbance is handled.

4. The method according to claim 3, wherein a position of a determined disturbance in the transport, infeed and/or outfeed region is identified.

5. The method according to claim 1, wherein a transport, infeed and/or outfeed region having a transport means and/or sorting means, and/or a transport, infeed and/or outfeed region at which primary packaging means are provided or deposited in an unordered manner or ordered in a matrix, is monitored.

6. The method according to claim 1, wherein the camera system is arranged above the transport, infeed and/or outfeed region, offset from the transport, infeed and/or outfeed region in such a manner that a primary air supply to the transport, infeed and/or outfeed region is not disturbed by the camera system, an optical axis of the camera system being inclined with respect to a vertical axis.

7. The method according to claim 1, wherein a determined disturbance is handled using a manipulator, the manipulator being movable by means of a machine controller, by means of a decentralized manipulator controller and/or by means of a manually operable controller for the purpose of handling the determined disturbance.

8. A device for monitoring a filling and/or closing installation and/or post-processing installation, in particular for the pharmaceutical industry, comprising a camera system configured to take an image of a transport, infeed and/or outfeed region of the filling and/or closing installation and/or post-processing installation, and a computing unit comprising an artificial intelligence model (AI model) that is trained to detect primary packaging means in the transport, infeed and/or outfeed region and to classify detected primary packaging means, the computing unit being configured to determine on the basis of the image, by use of the AI model, in which image positions primary packaging means are present, and to assign detected primary packaging means to a class.

9. The device according to claim 8, wherein the AI model is trained to classify primary packaging means on the basis of their type, and/or primary packaging means of a type on the basis of their orientation.

10. The device according to claim 8, wherein the computing unit is configured to evaluate an output of the AI model by use of a rule-based algorithm for the purpose of determining disturbances, and preferably to classify and/or prioritise a determined disturbance by use of the rule-based algorithm, the computing unit being in particular configured to specify, on the basis of a classification, how the determined disturbance is to be handled, and/or to specify, on the basis of a prioritisation, whether, and if so, when, the determined disturbance is to be handled.

11. The device according to claim 8, wherein an optical axis of the camera system is inclined with respect to a vertical axis, such that the camera system can be arranged above the transport, infeed and/or outfeed region, offset from the monitored transport, infeed and/or outfeed region in such a manner that a primary air supply to the transport, infeed and/or outfeed region is not disturbed.

12. The device according to claim 8, wherein a manipulator is provided, which is configured to handle a determined disturbance by means of a central machine controller, a decentralised manipulator controller and/or by means of a manually operable controller.

13. A filling and/or closing installation and/or post-processing installation comprising a transport, infeed and/or outfeed region and a device according to claim 8, the filling and/or closing installation and/or post-processing installation in particular comprising an isolator housing in which the transport, infeed and/or outfeed region is arranged.

14. A computer program comprising instructions that, when the program is executed by a computing unit, cause the latter to determine, on the basis of an image of a transport, infeed and/or outfeed region of a filling and/or closing and/or post-processing installation, by use of an artificial intelligence model (AI model) that is trained to detect primary packaging means in the transport, infeed and/or outfeed region and to classify detected primary packaging means, in which image positions primary packaging means are present, and to assign detected primary packaging means to a class.

15. The computer program according to claim 14, comprising instructions that, when the program is executed by the computing unit, cause the latter to determine, on the basis of an output of the AI model, by use of a rule-based algorithm, whether there is a disturbance present.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Further advantages and aspects of the invention will become apparent from the claims and from the description of exemplary embodiments of the invention, which are explained below with reference to the figures, in which:

[0048] FIG. 1: shows schematically a device for monitoring a filling and/or closing installation and/or post-processing installation with an infeed region;

[0049] FIG. 2: shows schematically a filling and/or closing installation and/or post-processing installation with a plurality of disturbances;

[0050] FIG. 3: shows schematically a nest in a transport, infeed and/or outfeed region of a filling and/or closing installation and/or post-processing installation with two disturbances;

[0051] FIG. 4: shows schematically a tray in a transport, infeed and/or outfeed region of a filling and/or closing installation and/or post-processing installation with a plurality of disturbances; and

[0052] FIG. 5: shows schematically a device for determining and handling disturbances in a filling and/or closing installation and/or post-processing installation with an infeed region.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

[0053] FIG. 1 shows schematically a device 1 for monitoring a filling and/or closing and/or post-processing installation, in particular for the pharmaceutical industry, comprising a transport, infeed and/or outfeed region 2. The transport, infeed and/or outfeed region 2 shown in FIG. 1 has a plurality of linearly running tracks 20, 21, 22, 23 along which primary packaging means, not shown in FIG. 1, in particular stoppers or filled or not yet filled containers, are conveyed. In other embodiments, instead of the tracks 20, 21, 22, 23, additional or other elements are provided by means of or along which the primary packaging means are conveyed, supplied or collected.

[0054] The shown device 1 for monitoring comprises a camera system 10 and a computing unit 12.

[0055] The camera system 10 is configured to take an image or a sequence of images of the transport, infeed and/or outfeed region 2. In the exemplary embodiment shown, an optical axis 100 of the camera system 10 is inclined with respect to a vertical axis I, the camera system 10 being arranged above the transport, infeed and/or outfeed region 2, offset from it in such a manner that a primary air supply 5 to the transport, infeed and/or outfeed region 2 from above, indicated schematically by arrows, is not disturbed by the camera system 10.

[0056] The computing unit 12 comprises an AI model 120 that is trained to detect primary packaging means in the transport, infeed and/or outfeed region 2 and to classify detected primary packaging means.

[0057] By means of the computing unit 12, an image taken by the camera system 10 can be analysed in order to determine on the basis of the image, by use of the AI model 120, in which image positions primary packaging means are present, and to assign detected primary packaging means to a class.

[0058] For classification of the primary packaging means, it is provided in an embodiment that different types of primary packaging means can be detected, with a class being allocated to each type. Alternatively or additionally, in an embodiment, primary packaging means of a type are classified on the basis of their orientation. In an embodiment, in this regard primary packaging means that are fed correctly oriented, overturned in the transport direction, overturned transversely to the transport direction, twisted and/or upside down are differentiated for classification.

[0059] The computing unit 12 shown further comprises a rule-based algorithm 122. The rule-based algorithm 122 is used, based on the output of the AI model 120, to determine whether there is a disturbance present.

[0060] In an embodiment, a determined disturbance is also classified, i.e. assigned to a class, with classes being defined in advance by an expert, according to the application. In an embodiment it is then determined, on the basis of the classification, how the determined disturbance is to be handled.

[0061] Additionally or alternatively, in an embodiment, the rule-based algorithm 122 is used to prioritise a determined disturbance, i.e. a priority value is assigned to the detected disturbance, with priority values for detected disturbances being defined in advance by an expert, according to the application. On the basis of the prioritisation, it is then determined in an embodiment whether and, if so, when the determined disturbance is to be handled. An objective of the prioritisation in this case is to avoid machine stoppages.

[0062] Depending on the specific application, class and/or priority value of the detected disturbance, appropriate measures can then be taken. In the exemplary embodiment shown, a transmission of the data to a memory unit 13 is provided, and information relating to the detected disturbance can be electronically logged in the memory unit 13.

[0063] In an embodiment, a signal is emitted to an operator by which signal the disturbance is communicated visually, acoustically and/or by other means to the operator. The operator can then take appropriate measures to handle the disturbance. In an embodiment, the disturbance is handled manually by the operator. In an embodiment, manual handling is effected in an isolator by means of gloved intervention.

[0064] In addition, in an embodiment, as shown schematically by a dashed arrow, data relating to the detected disturbance is transmitted to a machine controller 140. In an embodiment, the machine controller 140 in this case serves as an interface to a manipulator, not shown in FIG. 1, the manipulator being movable, by means of the machine controller 140, by means of an additional manipulator controller or by means of a manually operable controller, for the purpose of handling the determined disturbance.

[0065] FIG. 2 shows schematically an infeed region 2 that has a plurality of linearly running tracks 20, 21, 22, 23, 24 for primary packaging means 4 in the form of stoppers. The tracks 20, 21, 22, 23, 24 are each delimited laterally. The dimensions of the tracks 20, 21, 22, 23, 24 are selected so that the primary packaging means 4 can each be moved in succession along the tracks 20, 21, 22, 23, 24. In this case, a plurality of disturbances in the infeed region 2 are shown schematically in FIG. 2.

[0066] The disturbances shown are incorrectly oriented primary packaging means 40 and a gap 42, the gap 42 being caused, for example, by a primary packaging means 43 blocking movement in the track 20. In detail, shown in the uppermost track 20 (as viewed in the plane of the drawing), are a primary packaging means 40 that is rotated by 180? or fed-in upside down, and a primary packaging means 41 that is tilted in the direction of the path. A gap 42 is shown in the second track 21 from the top. Shown in the middle track 22 are a primary packaging means 41 overturned in the direction of the path, and a primary packaging means 44 oriented transversely to the direction of the path. A primary packaging means 40 oriented transversely to the direction of the path is likewise shown in the second track 23 from the bottom. The lowermost track 24 is completely correctly filled.

[0067] The primary packaging means 4, 40 present are detected by the computing unit 12 (cf. FIG. 1) by use of the AI model 120, and assigned to a class. The result can be output in a format suitable for further processing.

[0068] In an embodiment, an output of the AI model is subsequently evaluated by use of the rule-based algorithm 122 in order to determine disturbances, i.e. to determine incorrectly oriented primary packaging means 40, 41, 44 and/or to determine gaps 42.

[0069] In an embodiment, a determined disturbance is additionally classified and prioritised using the rule-based algorithm 122. In an embodiment, the rule-based algorithm 122 can be used to determine whether the disturbance is a gap 42, caused by a blockage, or an incorrectly oriented primary packaging means 40, 41, 44. Furthermore, when the primary packaging means are conveyed to the right in the plane of the drawing, in one exemplary embodiment the disturbance marked by a circle in FIG. 2 is assigned a higher priority value than other disturbances, since the incorrectly oriented primary packaging means 41 is fed to a subsequent processing step earlier than other incorrectly oriented primary packaging means 40, 41, 44, and rectification is therefore a priority. However, other prioritisations are also conceivable. For example, in an embodiment, the gap 42 is assigned a higher priority value than a defectively oriented primary packaging means 40, because of the blocking primary packaging means 43.

[0070] FIG. 3 shows schematically a transport, infeed and/or outfeed region 3, comprising a nest 30 from which primary packaging means 4 are taken or into which primary packaging means 4 are placed, the primary packaging means 4 being provided or deposited in an ordered manner in a matrix. The primary packaging means 4 shown in FIG. 3 are syringes; in other embodiments other primary packaging means 4 are provided or deposited in nested form. In the case of the situation shown in FIG. 3, two disturbances are present, each disturbance being an overlying primary packaging means 45.

[0071] The disturbance due to the overlying primary packaging means 45 is detected in each case by the computing unit 12 (cf. FIG. 1) by use of the AI model 120 and the rule-based algorithm 122, and in an embodiment is additionally classified and prioritised using the rule-based algorithm 122. On the basis of the class assigned to the primary packaging means by the AI model, in an embodiment in this case it can be detected, using the rule-based algorithm 122, whether a disturbance has occurred, for example due to a missing primary packaging means or due to an overlying primary packaging means. In an embodiment in this case, operating states can be taken into account, i.e. the algorithm 122 can differentiate between a process-workflow-related empty space in the nest 30 and a disturbance-related empty space.

[0072] FIG. 4 shows schematically a transport, infeed and/or outfeed region 6, comprising a spatially delimited tray 60 on which primary packaging means 4 are provided or deposited in an unordered manner in the exemplary embodiment shown. The primary packaging means 4 shown in FIG. 4 are containers in the form of vials; in other embodiments, other primary packaging means 4 are provided or deposited on a tray 60. In the case of the situation shown in FIG. 4, three disturbances are present, namely two overturned, and thus incorrectly oriented, primary packaging means 41, and one overlying primary packaging means 45.

[0073] The disturbances are detected by the computing unit 12 (cf. FIG. 1) by use of the AI model 120 and the rule-based algorithm 122, and in an embodiment are additionally classified and prioritised using the rule-based algorithm 122.

[0074] The disturbances shown in FIGS. 2 to 4 are for illustrative purposes only. As is obvious to the person skilled in the art, depending on the embodiment of the transport, infeed and/or outfeed region 2, 3, 6, additional and/or other disturbances may occur and/or fewer or more disturbances than those shown may occur at a point in time.

[0075] FIG. 5 shows schematically a device 1 for monitoring a filling and/or closing installation and/or post-processing installation, comprising a transport, infeed and/or outfeed region 2, using a camera system 10 and a computer unit 12 and for automated handling of a determined disturbance using a manipulator 14.

[0076] The camera system 10 is configured to take an image or a sequence of images of the transport, infeed and/or outfeed region 2. In the exemplary embodiment shown, an optical axis 100 of the camera system 10 is inclined with respect to a vertical axis I, the camera system 10 being arranged above the transport, infeed and/or outfeed region 2, offset from it in such a manner that a primary air supply 5 to the transport, infeed and/or outfeed region 2 from above, indicated schematically by arrows, is not disturbed by the camera system 10.

[0077] The computing unit 12 is configured to detect a disturbance on the basis of the image taken by the camera system, by use of an AI model 120 and using a rule-based algorithm 122. In an embodiment, it is additionally provided to classify the detected disturbance using the rule-based algorithm 122, i.e. to assign it to a class and to prioritise it, i.e. to assign a priority value to the determined disturbance.

[0078] The computing unit 12 is also configured to locate the disturbance, i.e. to identify a position of the determined disturbance in the transport, infeed and/or outfeed region 2. For this purpose, in the exemplary embodiment shown, on the basis of a coordinate transformation 124, the position of the disturbance detected in the image plane of the camera system 10 and the dimensions and orientation of the object to be manipulated in order to handle the disturbance, in particular a primary packaging means, are transformed into a coordinate system of the manipulator 14 and/or an operator using a suitable mathematical model 126.

[0079] In an embodiment, the transformed data of the position of the disturbance, as well as data relating to the primary packaging means concerned, such as its dimension and/or orientation, andif availablea classification and/or prioritisation of the detected disturbance, are transmitted to a machine controller 140. In the exemplary embodiment shown, the machine controller 140 is in communication with a computing unit 16 that plans a path for a movement of the manipulator 14 to handle the disturbance, based on the data determined by the computing unit 12 and in consideration of interference contours 160. In the exemplary embodiment shown, the computing unit 16 is formed separately from the computing unit 12 of the monitoring system. In other embodiments, the computing units 12, 16 are formed together.

[0080] In an embodiment, a path along which the manipulator 14 is moved for collision-free handling of the determined disturbance is planned in such a way that travel paths of the manipulator 14 that influence the primary air supply 5 to the primary packaging means and/or to the components of the filling and/or closing installation and/or post-processing installation contacting the primary packaging means are minimised.

[0081] In an embodiment, the path is planned using an algorithm optimized for pharmaceutical compliance, wherein criteria for pharmaceutical compliance are in particular selected from a group comprising minimization of a time for which the manipulator is arranged above primary packaging means, minimization of a time for which the manipulator is arranged above components of the filling and/or closing installation and/or post-processing installation which are in contact with primary packaging means, flow optimization of a movement and/or speed profile, minimization of a rotation of axes of the manipulator, minimization of a movement of a handling system of the manipulator above the primary packaging means and/or the components of the filling and/or closing and/or post-processing installation which are in contact with primary packaging means, in particular minimization of a gripper movement above the primary packaging means, and minimization of an impact surface in the primary air supply.

[0082] The individual criteria in this case are to be weighted, or further criteria are to be added, depending on the application.

[0083] In the exemplary embodiment shown, a monitor 18 is also In the exemplary embodiment shown, a monitor 18 is also provided, which is part of the device 1 or is in communication therewith for data exchange. In an embodiment it is provided that a planned path can be visualized on the monitor 18 in a simulated environment. In other words, the monitor 18 shows a digital twin of the manipulator 14 and its real environment. In an embodiment, the planned path is visualized here on the monitor 18 before the movement of the manipulator 14 is carried out for interactive correction and/or approval. For a correction of the planned path, in an embodiment it is provided that the operator can change individual path points, wherein for this purpose in an embodiment a touch-sensitive monitor 18 is provided. Alternatively or additionally, for a correction in an embodiment, a repetition of the path planning using the computing unit 16 can be triggered by an operator without changing the criteria or their weighting. In other embodiments, it is possible for an operator to optionally repeat the path planning by changing the criteria and/or their weighting. In yet other embodiments, no operator interaction is necessary for approval of the planned path. In an embodiment, an evaluation of the quality of the planned path is performed, wherein no interaction of an operator is required if a defined threshold value is exceeded in the evaluation. In an embodiment, the path planning is first automatically repeated if the value drops below the threshold value, and an interaction of an operator is only required if the value drops below the threshold value again. Alternatively or additionally, a real movement of the manipulator is visualized on the monitor in the simulated environment.

[0084] In the exemplary embodiment shown, a manually operable control unit 142 is also provided, the manipulator 14 being movable by an operator using the controller 142 in order to handle a disturbance.

[0085] In an embodiment, the path along which the manipulator 14 is moved autonomously or using the controller 142 is electronically logged, in particular in the memory unit 13. Data to be logged relating to the path, the cause of a movement of the manipulator 14, primary packaging means moved using the manipulator or the like can be suitably specified in this case by a person skilled in the art, depending on the application. In an embodiment, the time period and/or the extent of coverage over which the manipulator was moved in the primary air supply 5 is logged. In an embodiment, the logged data comprise a video file of the movement visualised on the monitor 18, i.e. a video file of the digital twin. The video file allows an operator to easily evaluate the movement performed.

[0086] The exemplary embodiments shown are merely examples, and numerous variations are conceivable, it being determined on the basis of an image of a transport, infeed and/or outfeed region 2, 3, 6, by use of an AI model 120 that is trained to detect disturbances in the transport, infeed and/or outfeed region 2, 3, 6, whether there is a disturbance present in the transport, infeed and/or outfeed region 2, 3, 6.

[0087] In the case of a filling and/or closing installation and/or post-processing installation that has an isolator housing, in an embodiment the image in this case is taken using a camera system arranged in the isolator housing.