SYSTEMS AND METHODS FOR OBJECT PROCESSING USING GRIPPERS FOR OBJECTS WITH LOW POSE AUTHORITY

Abstract

An end-effector is disclosed for a programmable motion device. The end effector includes a body that includes a vacuum portion through which a vacuum is applied to an object, and a gripping portion that is adapted to engage the object, the vacuum portion being generally orthogonal to the gripping portion.

Claims

1. An end-effector for a programmable motion device, said end effector comprising a body that includes a vacuum portion through which a vacuum from a vacuum source is applied to an object, and a gripping portion that is adapted to engage the object, said vacuum portion being generally orthogonal to the gripping portion.

2. The end-effector as claimed in claim 1, wherein the vacuum portion is provided within a lid containment portion for facilitating retention of a lid of the object.

3. The end-effector as claimed in claim 1, wherein the vacuum portion further includes raised ridges that facilitate distribution of vacuum to a surface of the object.

4. The end-effector as claimed in claim 1, wherein the gripping portion includes no-slip surfaces.

5. The end-effector as claimed in claim 1, wherein the gripping portion includes barbs

6. The end-effector as claimed in claim 5, wherein at least one of the barbs is elongated.

7. The end-effector as claimed in claim 1, wherein the vacuum source is a high flow vacuum providing a vacuum pressure of no more than about 25,000 Pascals below atmospheric, with a maximum air flow rate of, for example at least about 100 cubic feet per minute

8. The end-effector as claimed in claim 7, wherein the vacuum portion includes a first area that provides vacuum to the object within a first enclosed area, at least one second area that provides a non-sealed contact between the vacuum and the object a second non-sealed area, and third area that provides a non-sealed contact between the vacuum and the object a third non-sealed area that permits more non-sealed vacuum flow than at the second non-sealed area.

9. An end-effector for a programmable motion device, said end-effector comprising a lid containment portion for facilitating retention of a lid of an object, and a gripping portion that is adapted to engage the object, said lid containment portion being generally orthogonal to the gripping portion.

10. The end-effector as claimed in claim 9, wherein the lid containment portion further includes a vacuum portion through which a vacuum is applied from a vacuum source to the object.

11. The end-effector as claimed in claim 10, wherein the vacuum source is a high flow vacuum providing a vacuum pressure of no more than about 25,000 Pascals below atmospheric, with a maximum air flow rate of, for example at least about 100 cubic feet per minute.

12. The end-effector as claimed in claim 11, wherein the vacuum portion includes a first area that provides vacuum to the object within a first enclosed area, at least one second area that provides a non-sealed contact between the vacuum and the object a second non-sealed area, and third area that provides a non-sealed contact between the vacuum and the object a third non-sealed area that permits more non-sealed vacuum flow than at the second non-sealed area.

13. The end-effector as claimed in claim 10, wherein the vacuum portion further includes raised ridges that facilitate distribution of vacuum to a surface of the object.

14. The end-effector as claimed in claim 9, wherein the end-effector further includes a coupling for connecting to an end-effector connection portion of the programmable motion device.

15. The end-effector as claimed in claim 9, wherein the end-effector is a vacuum end-effector.

16. The end-effector as claimed in claim 9, wherein the gripping portion includes no-slip surfaces.

17. The end-effector as claimed in claim 9, wherein the gripping portion includes barbs.

18. The end-effector as claimed in claim 17, wherein at least one of the barbs is elongated.

19. A programmable motion device for use in an object processing system, said programmable motion device comprising an end-effector with a lid containment portion for facilitating retention of a lid of an object, and a gripping portion that is adapted to engage the object, said lid containment portion being generally orthogonal to the gripping portion.

20. The programmable motion device as claimed in claim 19, wherein the lid containment portion further includes a vacuum portion through which a vacuum is applied from a vacuum source to the object.

21. The programmable motion device as claimed in claim 20, wherein the vacuum source is a high flow vacuum providing a vacuum pressure of no more than about 25,000 Pascals below atmospheric, with a maximum air flow rate of, for example at least about 100 cubic feet per minute.

22. The programmable motion device as claimed in claim 21, wherein the vacuum portion includes a first area that provides vacuum to the object within a first enclosed area, at least one second area that provides a non-sealed contact between the vacuum and the object a second non-sealed area, and third area that provides a non-sealed contact between the vacuum and the object a third non-sealed area that permits more non-sealed vacuum flow than at the second non-sealed area.

23. The programmable motion device as claimed in claim 20, wherein the vacuum portion further includes raised ridges that facilitate distribution of vacuum to a surface of the object.

24. The programmable motion device as claimed in claim 19, wherein the end-effector further includes a coupling for connecting to an end-effector connection portion of the programmable motion device.

25. The programmable motion device as claimed in claim 19, wherein the gripping portion includes no-slip surfaces.

26. The programmable motion device as claimed in claim 19, wherein the gripping portion includes barbs.

27. An object processing system for processing objects, said object processing system comprising a programmable motion device with an end-effector, a control system for identifying potentially openable portions of the object, and an end-effector with a lid containment portion for facilitating retention of a lid of an object, and a gripping portion that is adapted to engage the object.

28. The object processing system as claimed in claim 27, wherein the lid containment portion further includes a vacuum portion through which a vacuum is applied from a vacuum source to the object.

29. The object processing system as claimed in claim 28, wherein the vacuum source is a high flow vacuum providing a vacuum pressure of no more than about 25,000 Pascals below atmospheric, with a maximum air flow rate of, for example at least about 100 cubic feet per minute.

30. The object processing system as claimed in claim 29, wherein the vacuum portion includes a first area that provides vacuum to the object within a first enclosed area, at least one second area that provides a non-sealed contact between the vacuum and the object a second non-sealed area, and third area that provides a non-sealed contact between the vacuum and the object a third non-sealed area that permits more non-sealed vacuum flow than at the second non-sealed area.

31. The object processing system as claimed in claim 28, wherein the vacuum portion further includes raised ridges that facilitate distribution of vacuum to a surface of the object.

32. The object processing system as claimed in claim 27, wherein the end-effector further includes a coupling for connecting to an end-effector connection portion of the programmable motion device.

33. The object processing system as claimed in claim 27, wherein the gripping portion includes no-slip surfaces.

34. The object processing system as claimed in claim 27, wherein the gripping portion includes barbs.

35. A method of processing objects comprising identifying potentially openable portions of an object, and grasping the object with an end-effector including a lid containment portion for facilitating retention of a lid of an object, and a gripping portion that is adapted to engage the object.

36. The method as claimed in claim 35, wherein the method further comprises applying a vacuum from a vacuum source through the lid containment area.

37. The method as claimed in claim 36, wherein the vacuum source is a high flow vacuum providing a vacuum pressure of no more than about 25,000 Pascals below atmospheric, with a maximum air flow rate of, for example at least about 100 cubic feet per minute.

38. The method as claimed in claim 37, wherein the vacuum portion includes a first area that provides vacuum to the object within a first enclosed area, at least one second area that provides a non-sealed contact between the vacuum and the object a second non-sealed area, and third area that provides a non-sealed contact between the vacuum and the object a third non-sealed area that permits more non-sealed vacuum flow than at the second non-sealed area.

39. The method as claimed in claim 35, wherein method further includes coupling the end effector to a programmable motion device.

40. The method as claimed in claim 35, wherein the identifying potentially openable portions of an object involves locating areas of the object that may include a lid flap.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The following description may be further understood with reference to the accompanying drawings in which:

[0015] FIG. 1 shows an illustrative diagrammatic view of an end-effector of a programmable motion device prior to grasping an object in accordance with a system of the prior art;

[0016] FIG. 2 shows an illustrative diagrammatic view of the end-effector of FIG. 1 while holding the object;

[0017] FIG. 3 shows an illustrative diagrammatic view of the end-effector of FIG. 1 while holding the object from a different grasp location;

[0018] FIG. 4 shows an illustrative diagrammatic view of the end-effector of FIG. 1 while holding the object from a further grasp location;

[0019] FIG. 5 shows an illustrative diagrammatic view of an end-effector of a programmable motion device prior in accordance with an aspect of the present invention;

[0020] FIG. 6 shows an illustrative diagrammatic end view of the end-effector of FIG. 5;

[0021] FIG. 7 shows an illustrative diagrammatic side view of the end-effector of FIG. 5;

[0022] FIG. 8 shows an illustrative diagrammatic bottom view of the end-effector of FIG. 5;

[0023] FIG. 9 shows an illustrative diagrammatic rear view of the end-effector of FIG. 5;

[0024] FIG. 10 shows an illustrative diagrammatic underside enlarged view of the end-effector of FIG. 5;

[0025] FIG. 11 shows an illustrative diagrammatic further enlarged view of the end-effector of FIG. 5;

[0026] FIG. 12 shows an illustrative diagrammatic enlarged end view of the end-effector of FIG. 5;

[0027] FIG. 13 shows an illustrative diagrammatic enlarged underside view of the end-effector of FIG. 5;

[0028] FIG. 14 shows an illustrative diagrammatic enlarged rear view of the end-effector of FIG. 5;

[0029] FIG. 15 shows an illustrative diagrammatic enlarged side sectional view of the end-effector of FIG. 5 taken along line 15-15 in FIG. 14;

[0030] FIG. 16 shows an illustrative diagrammatic enlarged side view of the end-effector of FIG. 5;

[0031] FIGS. 17A and 17B show illustrative diagrammatic views of an end-effector of FIG. 5 approaching a box to be grasped (FIG. 17A) and grasping the box (FIG. 17B);

[0032] FIGS. 18A and 18B show illustrative diagrammatic views of an end-effector of FIG. 5 approaching a differently oriented box to be grasped (FIG. 18A) and grasping the differently oriented box (FIG. 18B);

[0033] FIG. 19 shows an illustrative diagrammatic view of an object processing system with an end-effector of FIG. 5 in accordance with an aspect of the present invention;

[0034] FIG. 20 shows an illustrative diagrammatic enlarged view of a portion of the system of FIG. 19 showing a box being lifted from a bin;

[0035] FIG. 21 shows an illustrative diagrammatic enlarged view of a further portion of the system of FIG. 19 showing a box being placed into a box;

[0036] FIG. 22 shows an illustrative diagrammatic view of enlarged view of the end-effector in accordance with a further aspect of the present invention that includes barbs;

[0037] FIG. 23 shows an illustrative diagrammatic view of enlarged end view of the end-effector FIG. 22; and

[0038] FIGS. 24A and 24B show illustrative diagrammatic views of an end-effector of FIG. 22 approaching a box to be grasped (FIG. 24A) and grasping the box (FIG. 24B);

[0039] The drawings are shown for illustrative purposes only.

DETAILED DESCRIPTION

[0040] In accordance with an aspect, the invention provides an end-effector that may be used with a programmable motion device in an object processing system for processing objects having low pose authority. FIG. 5 for example, shows at 30 a portion of an articulated arm 32 of a programmable motion device that includes an end-effector 34 in accordance with an aspect of the invention. FIG. 5 shows an isometric view of the end-effector 34, which includes a first portion 36 by which a vacuum is applied to an object and that facilitates containing a lid of an object as discussed further below. The end-effector 34 further includes a second portion 38 that may engage a side of an object as also discussed in more detail below.

[0041] FIG. 6 shows a front view of the end-effector 34 with the second portion 38 facing forward. The end-effector 34 may be rotated by rotating the end-effector connection portion 33 (to which the end-effector 34 is connected) of the articulated arm 32. The first portion 36 and the second portion 38 of the end-effector 34 may be generally mutually orthogonal. FIG. 7 shows a side view of the end-effector 34 showing the general relative positional/orientation relationship between the first portion 36 and the second portion 38 of the end-effector 34. FIG. 8 shows a bottom view of the end-effector 34 with the first portion 36 facing downward, and FIG. 9 shows a rear view of the end-effector 34.

[0042] FIG. 10 shows an enlarged view of the first portion 36 of the end-effector 34, including a vacuum opening 40 coupled to a vacuum source. The vacuum source, for example, may be provided by a blower (with a vacuum pressure that is higher than a vacuum pressure provided by the ejector). The vacuum source therefore, may provide a vacuum pressure of no more than about 25,000 Pascals below atmospheric, with a maximum air flow rate of, for example at least about 100 cubic feet per minute (e.g., 130-140 cubic feet per minute). The first portion 36 further includes ridges 42 (some of which cross the opening 40 as shown at 45), and the ridges facilitate distribution of the vacuum over a contact surface of the object. At least one ridge may be formed as part of a closed area 43 over the vacuum opening 40, while one or more other ridges may be open over the vacuum opening 40 as shown at 47. In this way, a portion of the vacuum opening may include a closed area of vacuum (e.g., 43), while in other areas the ridges provide vacuum flow (e.g., at 45) in a first direction (along the ridges), while in further areas, the ridge(s) may further provide vacuum flow (e.g., at 47) in a second direction (through the opening in the ridge) that is orthogonal with respect to the first direction. In this way, some portion(s) may provide more seal against the object than other portions, with the closed area providing the most seal, the ridge areas 45 providing less of a seal (due to vacuum flow along the ridges), and the ridge opening shown at 47 providing even less of a seal (due to vacuum flow along the ridges through the opening at 47). Variable levels of seal and air flow may therefore be provided in different areas at the interface of the first portion 36 and the box being grasped, permitting lighter boxes to be held by the seal, while heavier boxes are further held by the high flow vacuum.

[0043] As also shown in FIG. 10, the second portion 38 of the end-effector 34 includes non-slip surfaces 44 formed, for example, of rubber, cork, adhesive, or electro-static material etc. FIG. 11 shows an enlarged view of the first portion 36 of the end-effector 34 of FIG. 10, and FIG. 12 shows an enlarged front view of the second portion 38 of the end-effector 34 of FIG. 10. A foam or rubber seal 41 may encircle the ridges 42 and aperture 40 on the underside of the first portion 36 to facilitate the application of a vacuum to an object. The height of the foam or rubber seal 41 may be at least as high as the height of the ridges 42.

[0044] FIG. 13 shows a more detailed underside view of the end-effector 34 showing the ridges 36 of the first portion 36, and the non-slip surfaces 44 of the second portion 38, as well as outer wall 50 of the first portion 36 and outer wall 52 of the second portion 38. FIG. 14 shows a rear view of the end-effector 34 of FIG. 13 showing the walls 50, 52, as well as a coupling member 54 for connecting to the end-effector connection portion 33 discussed above. FIG. 15 shows a side-sectional view of the end-effector 34 of FIG. 14 taken along a center of the end-effector 34 shown at 15-15. FIG. 16 shows a side view of the end-effector 34 of FIG. 14.

[0045] The object processing system (as discussed in further detail below with reference to FIG. 19) may include one or more perception units for viewing objects. The perception units are in communication (e.g., wirelessly) to one or more computer processing systems, in part, for identifying grasp locations. The system initially identifies whether the object is any of a category of objects having potentially poor pose authority. For example, objects that appear to be box-shaped or cube-shaped are analyzed for containing potentially exposed or lose flaps. FIG. 17A for example, shows an object 60 that is cube-shaped and includes three flaps 62 (two are shown) that are attached to a top 64 and that overlap a box base 66. The end-effector 34 may be selected to grasp the object 60 such that the first portion of the end-effector 34 contacts the base 66 and at least a portion of one of the flaps 62 along a short side 67 of the box 60 as shown in FIG. 17B. The second portion 38 of the end-effector 34 contacts the underside of the base 66.

[0046] FIG. 18A also shows the object 60 that is cube-shaped and includes three flaps 62 (again, two are shown) that are attached to a top 64 and that overlap a box base 66. The end-effector 34 may similarly be selected to grasp the object 60 such that the first portion of the end-effector 34 contacts the base 66 and at least a portion of one of the flaps 62 along a long side 69 of the box 60 as shown in FIG. 18B. The second portion 38 of the end-effector 34 contacts the underside of the base 66. Additionally, identifying indicia 70 and/or 72 on the box (as shown in FIGS. 17A and 18A) may be identified by the object processing system. The identifying indicia may be used, in combination with a warehouse manifest, to determine a destination location for each box.

[0047] The system may therefore include one or more perception units located on or near an infeed conveyor for identifying indicia on an exterior of each of the bins, providing perception data from which the contents of the bin may be identified, and then knowing its relative position on the conveyor, track its location. It is assumed, in accordance with an aspect, that the bins of objects are marked in one or more places on their exterior with a visually distinctive mark such as a barcode (e.g., providing a UPC code), QR code, or radio-frequency identification (RFID) tag or mailing label so that they may be sufficiently identified with a scanner for processing. The type of marking depends on the type of scanning system used, but may include 1D or 2D code symbologies. Multiple symbologies or labeling approaches may be employed. The types of scanners employed are assumed to be compatible with the marking approach. The marking, e.g. by barcode, RFID tag, mailing label or other means, encodes a identifying indicia (e.g., a symbol string), which is typically a string of letters and/or numbers. The symbol string uniquely associates the vendor bin with a specific set of homogenous objects.

[0048] On the selected infeed conveyor at the object processing station, the perception system assists (using the central control system 100—e.g., one or more computer processing systems) the programmable motion device including the end-effector in locating and grasping an object in the infeed bin. In accordance with further aspects, each object may also be marked with a visually distinctive mark, again such as a barcode (e.g., providing a UPC code), QR code, or radio-frequency identification (RFID) tag or mailing label so that they may be sufficiently identified with a scanner for processing. The type of marking depends on the type of scanning system used, but may include 1D or 2D code symbologies. Again, multiple symbologies or labeling approaches may be employed on each object.

[0049] With reference to FIG. 19, the object processing system 80 may include a perception system 82 that looks down in the object processing station and perceives perception data from one or more objects on an infeed conveyor 84. Objects to be processed may arrive on the infeed conveyor 84 in bins 86. Objects may be processed by placing each object into a designated destination location box 90 that my run along either of two output conveyors 92, 94. A vacuum source 33 provides the high flow vacuum to the end-effector. The perception system 82 is mounted above a bin of objects to be processed, and the perception system 82 may include (on the underside thereof), a camera, a depth sensor and lights. A combination of 2D and 3D (depth) data may be acquired. The depth sensor may provide depth information that may be used together with the camera image data to determine depth information regarding possible flaps on the various objects in view. The lights may be used to remove shadows and to facilitate the identification of edges of objects, and may be all on during use, or may be illuminated in accordance with a desired sequence to assist in object and flap identification. The system uses this imagery and a variety of algorithms to generate a set of candidate grasp locations for the objects that include a flap in the bin as discussed in more detail below.

[0050] The system will identify candidate grasp locations that include a portion of a flap on one or more objects, and may not try to yet identify a grasp location for the object that is partially obscured by other objects. Candidate grasp locations may be indicated using a 3D model of the robot end effector placed in the location where the actual end effector would go to use as a grasp location. Grasp locations may be considered good, for example, if they are close to the center of mass of the object to provide greater stability during grasp and transport, and/or if they avoid places on an object such as caps, seams etc. where a good vacuum seal might not be available.

[0051] If an object cannot be fully perceived by the detection system, the perception system considers the object to be two different objects, and may propose more than one candidate grasps of such two different objects. If the system executes a grasp at either of these bad grasp locations, then it will either fail to acquire the object due to a bad grasp point where a vacuum seal will not occur, or it will acquire the object at a grasp location that is very far from the center of mass of the object, thereby inducing a great deal of instability during any attempted transport. Each of these results is undesirable.

[0052] If a bad grasp location is experienced, the system may remember that location for the associated object. By identifying good and bad grasp locations, a correlation is established between features in the 2D/3D images and the idea of good or bad grasp locations. Using this data and these correlations as input to machine learning algorithms, the system may eventually learn, for each image presented to it, where to best grasp an object, and where to avoid grasping an object.

[0053] With further reference to FIG. 20, an object 60 may be removed from a bin 86, and with reference to FIG. 21, the object 60 may be placed into the appropriate box 90. The conveyors 84, 92, 94 may include bi-direction sections to aid in turning, or may be provided as straight conveyors that are all within a reach of the articulated arm 32. Additional sensors may be provided along the conveyors to identify bins and boxes, and to track their locations.

[0054] Again, the operations of the system described above are coordinated with a central control system 100 that again communicates (e.g., wirelessly) with the articulated arm, the perception units, as well as in-feed conveyor and output conveyors. This system determines from symbol strings the UPC associated with a vendor bin, as well as the outbound destination for each object. The central control system 100 is comprised of one or more workstations or central processing units (CPUs). For example, the correspondence between UPCs or mailing labels, and outbound destinations is maintained by a central control system in a database called a manifest. The central control system maintains the manifest by communicating with a warehouse management system (WMS). The manifest provides the outbound destination for each in-bound object.

[0055] FIG. 22 shows an underside view of an end-effector 134 in accordance with a further aspect of the invention that includes a first portion 136 including a vacuum opening 140 coupled to a vacuum source and through which a vacuum may be provided to an object. The first portion 136 further includes ridges 142 (some of which cross the opening 140), and the ridges facilitate distribution of the vacuum over a contact surface of the object. As also shown in FIG. 23, the second portion 138 of the end-effector 134 includes non-slip surfaces 144 formed, for example, of rubber, cork, adhesive, or electro-static material etc. as well as rows of angle barbs 150, that are angled upward toward the first portion 136 to engage a side of a box. FIG. 22 shows an enlarged view of the first portion 136 of the end-effector 134, and FIG. 23 shows an enlarged front view of the second portion 138 of the end-effector 134. A foam or rubber seal 141 may encircle the ridges 142 and aperture 140 on the underside of the first portion 136 to facilitate the application of a vacuum to an object. The height of the foam or rubber seal 141 may be at least as high as the height of the ridges 142.

[0056] FIG. 24A shows the end-effector 134 approaching a box such that the ridges 142 on the first portion 136 will engage a top 160 of the box, and both the non-slip surfaces 144 and the rows of barbs 150 will engage both a portion of a flap 162 as well as a side wall 164 of a base 166 of the box as further shown in FIG. 24B. In this way, a box having low pose authority may even be grasped with the first portion 136 contacting the top 160 of the box since the second portion 138 may sufficiently engage the side wall 164 of the base 166. In accordance with certain aspects, the vacuum source may be switchable to change to a source of positive air pressure that is pushed from the source to the apertures to urge an object away from the contact surface in accordance with certain aspects of the invention.

[0057] Those skilled in the art will appreciate that numerous modifications and variations may be made to the above disclosed embodiments without departing from the spirit and scope of the present invention.