Neuroevolution and creating new algorithms

Deep Learning
1

Good evening from London!

This may be a bit of a different post and perhaps not 100% fitting for this category, but I felt it was the best option pending future redirection.
My formal background is in Neuroscience and Clinical Medicine and I’ve had an interest in developments in AI for some time, though never on the level that I can actually implement the algorithms myself (I’m currently working through part 1 of the 2019 course and highly enjoying it. I’m on my second run through and focusing on implementing the notebooks).

I understand that the primary focus of the practical course is to, as its name suggests, become proficient with the current best practical methods of AI and applying it to problems in various domains.
My main interest that drove me to undertake the course was in understanding artificial intelligence and using it as a solid grounding to hopefully later go on to develop my own additions to the field as well as solve real world problems as opposed to get a job or anything like that.

Partly inspired by my interest in biology, I’ve been developing abstract, intuitive ideas of how to produce intelligent systems through evolutionary means and an arrangement of different higher level ideas.
I’ve been spending time looking into the vast expanse of AI literature to try and piece together existing elements and algorithms to see if this could come together into something that I can actually test. This led me to discover the work of Kenneth O’Stanley and his relatively recent review paper on Neuroevolution (https://www.nature.com/articles/s42256-018-0006-z). I was really excited to see the opinion of a top researcher suggest the use of neural networks in combination with evolutionary methods seems like a potential future route to general intelligence architectures.

The question I wanted to pose on this forum is whether anyone knows of any great resources for learning how to transform abstract intuitive ideas into algorithms or proof of concepts. At one point in course 1 when discussing Geoffrey Hinton, Jeremy mentions how intuition is the source of novel algorithms, it doesn’t come from Maths. What would be the process for translating such intuitive ideas? Or are there any good places I can present my ideas to more technically proficient individuals to discuss them?

Many thanks to anyone who takes the time to read this message.

2 Likes
2

Hi jackbabs,

I’m from a neuroscience background too.
I find this kind of topic particularly interesting.

To answer your question, it is to me really simple. Take a paper and transform it into code. Start with Alexnet. See if you can implement the architecture. I think it would be easier with keras if you know it.
But if you really want to transform an idea into a mathematical formula, you have no choice but learn a minimal amount of maths. It’s a language and you need to learn it if you don’t.
A good analogy i think is music. To write down a musical tune, you must know the solfege.
It just my opinion on the matter.

We can discuss neuroevolution in more details if you want as i disagree on some points.

Have a nice day.

1 Like
3

Hi there,

Thanks for your response! I’ve been using the Deep learning with Python book by Chollet to get familiar with Keras alongside the fastai course so I’ll do my best to apply that knowledge to the Alexnet paper as you suggest.
I’d very much enjoy discussing Neuroevolution and hearing your point of view. Another very interesting paper regarding a new version of NEAT for autoML was recently published as well which I’d say is worth a look.

All the best