I have been reading up a little on parameter prediction networks (networks that predict the weight matrices for other networks) in this paper and this paper. A very simplified example of this would include two networks:

- a base network with one weight matrix containing a very large number of parameters (call it W).
- an auxiliary network which outputs a matrix that is the same shape as W

The output from the auxiliary network replaces the weight matrix in the base network. Both networks are trained at the same time. In Keras this would look like one network, potentially with multiple inputs and outputs.

Just for fun, I wanted to see if I could write up a parameter prediction network in Keras. Assuming I train the base network and auxiliary network at the same time, I wasnâ€™t sure how to tie them together. More specifically, I can grab the output tensor of the auxiliary network. I need to take that tensor and use it as the weight matrix in a Keras layer. I looked around at Keras and while it appears possible to set the initial values of a weight matrix it doesnâ€™t appear possible to wrap a layer around an existing tensor and use it as the layerâ€™s weight matrix. Iâ€™m also unsure if the shape of the output tensor will be acceptable for the shape of the tensor for the weight matrix in the used in the base network.

Assuming end to end training, is it possible to grab the output tensor from one layer or model in Keras and use it as a weight matrix in another layer? Assuming I can somehow load a tensor into a layer will I run into tensor shape issues? Any help with this would be much appreciated.