Hi All,
I have been doing the course for the last couple of months and loving it.
whilst i can follow the examples in the course, i thought i would go and try and create my own deep learning problem - as recommended by Jeremy. off piste in other words.
whilst the actual problem might take a while to explain, what i am trying to do is create a 1D conv input layer to then train on my data.
my aim is to associate a certain array with a certain ‘decision’ in my neural net. the input is codified as a list of 6 numbers, which the neural net must return a decision based on their order. This ‘essentially’ sorts the order.
I have spent a good 5hrs trying to nut out what i am doing wrong, and believe its something to do with
- my array shapes
- my input or encoding layer
- my use of numpy or python arrays
heres a sample of my data and what i am trying to do.
[[array([2, 3, 4, 5, 0, 1]), array([2, 3, 1, 0, 4, 5]), array([0, 1, 2, 3, 4, 5])]
encoding
array([4, 1, 0])
essentially i want the neural net to learn that certain arrays require a certain step to order to the new array.
I have tried this approach but to no success
vocab_size = 7
model = Sequential([
Convolution1D(nb_filter=32, filter_length=3, border_mode='valid', input_dim=128, input_length=6),
-- Convolution1D(nb_filter=32, filter_length=3, border_mode='valid', input_shape = (6, 1), input_length=6),
Dense (100, activation="relu"),
Dense (vocab_size, activation='sigmoid')
])
when i try to fit the data
model.compile(loss='binary_crossentropy', optimizer=Adam(), metrics=['accuracy'])
trn[:2]
array([[2, 3, 4, 5, 0, 1],
[2, 3, 1, 0, 4, 5]])
trn_labels[:2]
array([4, 1])
model.fit(trn, trn_labels, batch_size=4, nb_epoch=4)
ValueError: Error when checking model input: expected convolution1d_input_6 to have 3 dimensions, but got array with shape (61, 6)
My full data that i will be stepping through and training my net on is:
'[[array([2, 3, 4, 5, 0, 1]), array([2, 3, 1, 0, 4, 5]), array([0, 1, 2, 3, 4, 5])]
[array([1, 0, 2, 3, 5, 4]), array([5, 4, 2, 3, 0, 1]), array([0, 1, 2, 3, 4, 5]), array([0, 1, 4, 5, 3, 2]), array([0, 1, 2, 3, 4, 5])]
[array([1, 0, 3, 2, 4, 5]), array([1, 0, 4, 5, 2, 3]), array([5, 4, 1, 0, 2, 3]), array([5, 4, 2, 3, 0, 1]), array([5, 4, 0, 1, 3, 2]), array([5, 4, 2, 3, 0, 1]), array([0, 1, 2, 3, 4, 5])]
[array([2, 3, 5, 4, 1, 0]), array([0, 1, 5, 4, 2, 3]), array([0, 1, 2, 3, 4, 5])]
[array([5, 4, 2, 3, 0, 1]), array([5, 4, 0, 1, 3, 2]), array([2, 3, 0, 1, 5, 4]), array([2, 3, 5, 4, 1, 0]), array([4, 5, 2, 3, 1, 0]), array([4, 5, 0, 1, 2, 3]), array([4, 5, 2, 3, 1, 0]), array([0, 1, 2, 3, 4, 5])]
[array([2, 3, 5, 4, 1, 0]), array([5, 4, 3, 2, 1, 0]), array([2, 3, 5, 4, 1, 0]), array([2, 3, 0, 1, 5, 4]), array([1, 0, 2, 3, 5, 4]), array([5, 4, 2, 3, 0, 1]), array([0, 1, 2, 3, 4, 5])]
[array([4, 5, 2, 3, 1, 0]), array([0, 1, 2, 3, 4, 5])]
[array([3, 2, 5, 4, 0, 1]), array([3, 2, 0, 1, 4, 5]), array([5, 4, 0, 1, 3, 2]), array([2, 3, 0, 1, 5, 4]), array([2, 3, 4, 5, 0, 1]), array([1, 0, 4, 5, 2, 3]), array([1, 0, 2, 3, 5, 4]), array([5, 4, 2, 3, 0, 1]), array([1, 0, 2, 3, 5, 4]), array([4, 5, 2, 3, 1, 0]), array([0, 1, 2, 3, 4, 5])]
[array([1, 0, 3, 2, 4, 5]), array([4, 5, 3, 2, 0, 1]), array([0, 1, 3, 2, 5, 4]), array([2, 3, 0, 1, 5, 4]), array([2, 3, 4, 5, 0, 1]), array([2, 3, 1, 0, 4, 5]), array([0, 1, 2, 3, 4, 5])]
[array([2, 3, 0, 1, 5, 4]), array([1, 0, 2, 3, 5, 4]), array([4, 5, 2, 3, 1, 0]), array([4, 5, 1, 0, 3, 2]), array([2, 3, 1, 0, 4, 5]), array([1, 0, 3, 2, 4, 5]), array([3, 2, 0, 1, 4, 5]), array([0, 1, 2, 3, 4, 5])]]`
and my validation data is
[array([4, 1, 0]) array([6, 6, 3, 4, 0]) array([3, 1, 3, 3, 4, 6, 0]) array([5, 3, 0])
array([3, 5, 3, 1, 4, 3, 5, 0]) array([2, 1, 4, 1, 6, 6, 0]) array([5, 0])
array([3, 5, 5, 4, 5, 3, 6, 5, 5, 5, 0]) array([6, 6, 1, 4, 4, 1, 0])
array([1, 5, 3, 5, 2, 2, 2, 0])]
Any advice would be appreciated on how to approach this problem. This is a simple test case, before i step up the scale of what i am trying to achieve.
Thanks a heap in advance.