I come from Belgium, and I work for a company active in the field of aerospace design and manufacturing.
I would like to know your opinion/advice about this:
assuming I am able to understand/apply the techniques taught by @jeremy in DL1 and DL2 (and ML1 ), how difficult is it to build a model which:
inputs a 3D numerical representation of a mechanical part (such as the one shown below; note that this is not just a 2D image, this is a 3D model that you can turn around, zoom in and everything and is built using a graphical interface such as Catia V5)
inputs also additional metadata such as the material (in this case a particular grade of aluminum)
ouputs, say, the cost of the part
based on a dataset of about 20 000 examples
?
This is the input (the 3D representation of the mechanical part):
The resulting part is entirely based on the 3D representation above and looks something like this:
Here is a short video showing the general principle of the manufacturing process involved, which is called āmachiningā:
I found this repo: 3D Machine Learning useful, but I am not sure where to begin in practice or whether my application is realistic considering the size of my dataset or the state of the art as of today in 3D machine learning.
Would you have any thoughts/suggestions/experiences regarding 3D machine learning that you could share with me please?
Looks like the rib on the leading edge of some small planeā¦
Anyways, I am unsure what your application is. What are you really trying to do? Find the cost or supposed cost of parts? For what purpose? Better quoting, SCM?
Are you looking for machining alternatives? topology optimization? etc?
this is an interesting topic. I wonder if the 3d drawing of the part is absolutely necessary. Because when you say, various (standardized) instructions regarding the manufacturing process, you mean the tool paths ?
Maybe you could try to feed the model the tool path for each slice of the part ? Iām just thinking out loud. You should try to define what is actually driving the cost of a part and only feed this kind of data and not overburden the model with un-necessary data. For example, a milling operation i guess on a CNC machine would cost the same whether itās cutting 30 mm in a straight line or in a curve line ? Then if we simplify to the extreme, is cost function of milling time and volume of the initial part only ? I donāt know, Iām just guessingā¦
Thatās right! This is the rib of a movable leading edge (aka āslatā) of a commercial aircraft.
In fact there can be numerous applications based on the 3D representations + CNN. On top of my head there is:
build a model predicting the cost in order to respond more quickly to RFQ (requests for quote) from potential clients (they usually send us a bunch of 3D models and we have to come up with our best estimate of our cost)
then we could also do model interpretation (as in ML1) in order to uncover what are the main drivers of our costs.
another application would be to do classification in order to detect design errors in the model before machining the real part
Tool path optimization is a big topic as well, in order to improve the surface quality of the real part for instance.
Unfortunately, I cannot be of more help with respect to a ML/DL approach to this problem. It would take some time for me to think of an approach which could work. Reason being is that most of what you want to do can already be done using native V5 approaches (design errors, tool paths, etc).
For costs, a simple bounding box around the geometry gives you stock size estimate, and you already know the material. From there it should be āsimpleā math for material needs. Production costs/time will be handled via the tool path/machining simulations.
I guess I would really have to change the way I look at it to give a better solution.
No I didnāt mean the tool path (so I removed this line in my original post in order to avoid confusion). But youāre right, using the tool path could also be one way of optimizing the manufacturing process overall using deep learning.
I like your idea of slicing. It would be like a 2D image with lots of channels, therefore we could use standard CNN architectures maybe? And Iām wondering if there exist some kind of pre-processing tool to prepare the input to the model in this way.
The important thing I didnāt mention is that depending on the size, shape and complexity of the part, it will be manufactured on a different machine, at different cutting speeds, in a different country etc. This is another reason why I think itās important to take the 3D representation into account.
Youāre 100% correct, and this is indeed how weāve been working for years.
However thereās only so much you can do with these approaches and many people feel dissatisfied by all the manual work that must be done. This is very inefficient in many ways.
This recent article on thegradient.pub gave a nice overview of current approaches that try to use Deep Learning in 3-dimensional space.
Maybe you could use something like PointNet++ to create 3D Embeddings of your models. So each model is encoded as a vector, which you then can use together with other metadata for Regression using either classical methods like random forests or you could try newer deep learning approaches.
The material cost seems to depend only on material and the bounding box of the object. I donāt see why you would need DL for that
The complex part of the cost comes from 1. Where to manufacture (sounds a lot like classification) and 2. How long does the appropriate machine need to manufacture (sounds a lot like regression)
Now the question is how to input your 3d models. I believe you have a cad file or mesh of it right?
Whether you want to use 3D convolutions or 2D convolutions with a lot of input channels, you will have to get your mesh in a 3d voxel grid somehow. This grid consisting of voxels (volume pixels) could save for each voxel something like
0 or 1, is there any part of the model intersecting this grid cell (at least by some amount) (occupancy grid)
amount of intersection with the model maybe?
Now you could throw on your 3d convs or view this grid slice-wise and go with 2ds.
The question on what works well there may probably be answered by papers in the medical field. I bet thereās a lot of work going on in classifying volumetric data.
Another thing that might work is rendering the object model from a few distinct points of view and input multiple of those synthetic images to a standard 2d CNN, combining the CNN outputs and do your classification + regression outputs from there. This way you will probably be a lot closer to what is done in the course. (Multi view representation)
Edit: the article above actually mentions pretty much all of the ways Iām suggesting to handle your input, I annotated it accordingly so you can make an easy reference to the paragraphs in the article
For many (very many?) tasks, 3D can be a red herring. Processing appropriate 2D renders often meet requirements without any need to complicate models with another dimension.Certainly Iād start with 2D before seeing if I needed to go 3D.
Indeed I have CAD files, which I could convert to other common types of 3D representations, like a mesh.
Ok I think I get the voxel grid idea. But one thing I worry about is that the mechanical parts can have different sizes and also very different aspect ratios (from a roughly cubed shape a few centimeters wide, say, to long, narrow parts several meters long), whereas in the medical field I believe sizes and aspect ratios are much more consistent within one application. What do you think?
Your last idea (creating a simple 2D image showing the part under various viewpoints) is nice but has the problem that in the case of long, narrow parts, you have to zoom out quite a bit in order to show the part in its entirety, therefore you canāt see the details of the part anymore. Do you see what I mean?
I agree it shouldnāt be more complicated than it needs to be. But as I mentioned above, I worry that 2D images showing the part under various viewpoints can be a problem in case of long, narrow parts.
I see, such aspect ratios could indeed be a porblem if you wanted to render your 2d images.
I think it would be easier to account for this with the voxel grid approach as you could just take a fixed grid size and stretch the object accordingly. If you input the aspect ratios separate you might still be able to learn with that.
Iāll just make a guess and say that those aspect ratios alone will probably already give a good rough estimate on which machine to use. Isnāt this aspect ratio a major factor in this decision anyways?
Maybe you might even have success if you stretch the objects in your rendered images combined with inputting the real aspect ratios. This way you can get information on the object surface from your images and maybe compensate for the āwrongā shape presented there.
I think it might still be worth to try out this approach, after all you will probably have a much easier implementation than with 3D grids. And you will probably have to train a lot shorter to find out if something works since you can utilize those pretrainrd nets for that 2d data. Iām not sure if there is a well pre trained net for 3d data, but I doubt that you will have such āeasyā success as you have with ImageNet pretrained nets.
Correct, aspect ratio and overall size are major drivers of the decision regarding which machine to use. Another major driver is the complexity of the part. Some parts are designed to be built using 3-axis machines (meaning, the cutting tool can only move up/down and along a horizontal plane), more complex parts require more sophisticated machines with 5-axis (the cutting tool can move and rotate in any direction in space).
But youāre right, that sounds like a reasonable approach to try right now!
), how difficult is it to build a model which:
